Study guide
13.1 Scheduling Tasks for Data Loading and Processing
SailPoint Certified IdentityIQ Engineer Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: The IdentityIQ Engineer Exam
1.1 Exam Format, Official Blueprint, and Study Strategy
2 Chapter 2: Infrastructure, Installation, and Extended Attributes
2.1 IdentityIQ Infrastructure Components 2.2 Installation Steps and Procedures 2.3 Creating Extended Attributes and the Objects That Support Them
3 Chapter 3: Build, Deployment, Patching, Access, and Email
3.1 Ongoing Build and Deployment 3.2 Installing Patches and Performing Upgrades 3.3 Configuring End-User Access 3.4 Email Server Settings and Email Redirection
4 Chapter 4: Lifecycle Manager Configuration, Provisioning Policies, and Workflows
4.1 Configuring Lifecycle Manager 4.2 Identity Provisioning Policies 4.3 Configuring LCM Workflows
5 Chapter 5: Lifecycle Events, QuickLinks, Batch Requests, and Rapid Setup
5.1 Lifecycle Events and Their Workflows 5.2 QuickLinks and QuickLink Populations 5.3 Configuring and Using Batch Requests 5.4 Rapid Setup in Lifecycle Events and Identity Operations
6 Chapter 6: Certification Campaigns and Certification Rules
6.1 Configuring and Scheduling Certification Campaigns 6.2 The Purpose of Common Certification Rules
7 Chapter 7: Policies and Policy Violations
7.1 Configuring and Leveraging Policies 7.2 Responding to Policy Violations
8 Chapter 8: Reporting, Advanced Analytics, and Auditing
8.1 Executing and Leveraging Reports 8.2 Leveraging Advanced Analytics 8.3 Configuring and Using Auditing
9 Chapter 9: Rules, Rule Libraries, and the SailPoint API
9.1 The Purpose and Use of Rule Libraries 9.2 Rule Input and Output Arguments 9.3 Common SailPoint API Objects and Methods
10 Chapter 10: Workflow Development
10.1 Workflow Variables, Transitions, and Step Conditions 10.2 Workflow Steps That Generate Work Items 10.3 The Different Workflow Step Usages 10.4 Leveraging Sub-Workflows
11 Chapter 11: Email Templates, Branding, and Localization
11.1 Customizing and Using Email Templates 11.2 Branding the SailPoint User Interface 11.3 Localizing the SailPoint UI
12 Chapter 12: Application Definitions, Connector Settings, and Rapid Setup Aggregation
12.1 Configuring Application Definitions 12.2 Configuring Common Connector Settings 12.3 Configuring Rapid Setup Aggregation Settings
13 Chapter 13: Data-Loading Tasks and Application and Connector Rules
13.1 Scheduling Tasks for Data Loading and Processing 13.2 Common Application Rule Types 13.3 Common Connector Rule Types
14 Chapter 14: Debugging with log4j, Syslog, and the IdentityIQ Console
14.1 Configuring and Leveraging log4j 14.2 Configuring and Leveraging Syslog 14.3 Leveraging the IdentityIQ Console for Debugging
15 Chapter 15: Debug Pages and the Administrator Console
15.1 Leveraging the Debug Pages for Debugging 15.2 Leveraging the Administrator Console for Debugging
16 Chapter 16: Role Model and Identity Attribute Mappings
16.1 Default Role Models and Role Types 16.2 Configuring Roles and Role Options 16.3 Configuring Identity Attribute Mappings
17 Chapter 17: Data Objects, Relationships, and Group Types
17.1 Common IdentityIQ Data Objects 17.2 Relationships Between Common Data Object Models 17.3 The Difference Between Group Types
13.1 Scheduling Tasks for Data Loading and Processing
Key Takeaways Explain This Topic AI
Aggregate authoritative sources first, then other applications and account groups, and then run Identity Refresh.
Aggregation marks changed identities as needing refresh by default, and the refresh option "Refresh only identities marked as needing refresh during aggregation" (off by default) uses that flag.
The Sequential Task Launcher runs tasks in a fixed order, with an optional stop on error, instead of guessing start times.
Detect deleted accounts has a Maximum deleted accounts safety limit; above it, no accounts are deleted.
Delta aggregation reads only changes and needs connector support; a periodic full aggregation and full refresh are still recommended.
Last updated: September 2026
Scheduling Tasks for Data Loading and Processing
Objective 5.4 asks you to schedule various tasks to achieve desired data loading and processing outcomes . Tasks are defined on Setup > Tasks from task types (templates). Their results appear on Task Results , and schedules on Scheduled Tasks .
The Data-Loading Pipeline
Account Aggregation of authoritative sources (HR, contractor systems) creates and updates identities.
Account Aggregation of other applications creates and updates Links and correlates them to identities.
Account Group Aggregation loads groups and application objects as ManagedAttributes, for the catalog, group certifications, and descriptions and owners.
Identity Refresh promotes attributes, assigns and detects roles, checks policies, processes lifecycle events, synchronizes attributes, and more.
Follow-on tasks as needed, such as Effective Access Indexing , Refresh Role Indexes , and Full Text Index Refresh .
Authoritative first matters because non-authoritative accounts correlate to identities that must already exist .
Account Aggregation: Options to Know
Option Why it matters
Applications to scan One task can aggregate several applications. Sequential Execution – Terminate on Error forces the listed order and stops at the first error.
- Creation rule for new identities An IdentityCreation-type rule applied when an account creates an identity
- Only create links if they can be correlated Update existing identities but never create new ones
Refresh assigned and detected roles , Check active policies , Refresh the identity risk scorecards , Maintain identity histories Refresh-type work during aggregation. SailPoint does not recommend role refresh here if Identity Refresh manages correlation or provisioning, because it can cause unintended or incomplete provisioning.
Enable Delta Aggregation Read only changes; needs connector support
Detect deleted accounts + Maximum deleted accounts Removes Links for accounts that disappeared. If more than the maximum would be deleted, none are. This is a safety net against an empty or truncated feed.
- Disable optimization of unchanged accounts Forces full processing of unchanged accounts
- Promote managed attributes Adds newly seen entitlement values to the catalog
- Disable marking the identity as needing refresh Leave it off to support delta refresh
Enable Partitioning , Objects per partition (default 1,000), Loss Limit Scale out, and bound the rework after a sudden termination
Terminate when maximum number of errors is exceeded Stops a runaway task
Actions to include in the task result Detailed actions such as Correlate New Account, Create New Identity, Correlate Reassign, and Remove Account
If activity targets are used, run Target Aggregation after account aggregation, because account aggregation removes targets.
Account Group Aggregation
It aggregates groups and application objects (only where a group schema exists), with options for delta aggregation, detect deleted account groups , the description attribute and locale, a Group Aggregation Refresh rule (for example, to set owners), classification promotion, partitioning, and error limits. When scanning a single application, you can filter object types .
Delta Identity Refresh
Aggregation sets needsRefresh on identities it changes. This is on by default .
In Identity Refresh, choose Refresh only identities marked as needing refresh during aggregation , which is off by default . The task clears the flag unless Do not reset the needing refresh marker is selected. Use that option when several segmented refresh tasks each need the flag.
Only aggregation sets the flag. LCM requests and other changes do not, so SailPoint recommends periodic full refreshes , such as an hourly delta with a daily full, or a daily delta with a weekly full.
Timed triggers , such as a pre-offboarding workflow the day before a termination date, need a full refresh with Process Events , because the identity may not have changed in any aggregation.
Sequencing and Scheduling
Sequential Task Launcher runs a list of tasks in order . It has a timeout per task, optional log statements, and Cease execution if one of the executing tasks encounters an error . It removes the guesswork of scheduling by estimated run time.
Schedules: right-click a task and choose Schedule . Give it a unique name, a first execution time (or Run Now ), and a frequency. Tasks run at the scheduled time in the time zone where they were scheduled .
Hosts: a task's Host field can pin it to a host, or a comma-separated list where the first host with an active heartbeat runs it (section 2.1).
Predefined tasks are not templates. Editing them changes the shipped task. Create new tasks with New Task .
Other Tasks Worth Knowing
Run Rule runs an arbitrary rule with name/value arguments. The rule should return a status string, and anything other than "Success" produces a failed task result. It is useful for scheduled clean-up or reconciliation scripts.
Refresh Role Indexes must run before role searching works.
Full Text Index Refresh builds the LCM full-text indexes (section 4.1).
Perform Identity Request Maintenance prunes old identity request objects and scans unverified access requests to check whether provisioning completed.
Check Expired Work Items and Check Expired Mitigations handle escalations and expired exceptions.
Prune Identity Cubes deletes identities with no accounts and no important references. Managers, identities with capabilities, owners, and protected identities are kept.
A Typical Schedule
When Sequence
Every 2 hours (business days) HR delta aggregation, then AD delta aggregation, then delta Identity Refresh (attributes, roles, Process Events)
Nightly Full aggregations of all applications, then Account Group Aggregation, then full Identity Refresh (policies, risk, histories), then Perform Identity Request Maintenance
Weekly Perform Maintenance pruning, role index refresh, and a full refresh with scorecards
Pair these with email task alerts on failure (section 3.4) and the Administrator Console (section 15.2) for monitoring.