Job scheduler is a common mechanism that we use in our day to day life. Few examples are,
- Setting an alarm on mobile phones.
- Adding an event to the Google calendar.
- Setting up a timer in a microwave oven.
- Scheduling a task in the Windows operating system using Windows Task Scheduler.
To keep it simple, we have a task/job and we schedule it for a particular time for the system to perform.
What is Job Scheduler?
Job Scheduler in Pega is an internal background process operating on the server/node that runs business logic on a periodic/recursive basis.
In earlier versions of Pega, agents were used for handling all the background processes. As part of optimization, Pega has introduced the Job scheduler and Queue processor as a replacement of agents starting from version 8.
Consider we have a requirement to send the report of transactions to the chief officer by 9 PM every day. Below are the basic steps that we would follow,
- Write an activity rule in Pega to browse the transaction for the current day.
- Implement logic to send the email.
- Invoke the above-configured activity at 9 pm daily to process and send the email.
Steps to implement looks fine, but how do we invoke this task everyday at 9 pm?😳
We definitely can’t hire persons and ask them to send emails every day at 9 pm. And also, the execution does not need to happen onscreen which will definitely have an impact in terms of performance.
To overcome all these, we can configure a Job Scheduler to send emails at 9 pm daily which takes care of its execution as a background process to enhance performance.
How to configure Job Scheduler?
- Job Schedulers are instances of class RULE-ASYNC-JOBSCHEDULER.
- We should have pzJobSchedulerAdministrator privilege to create and manage Job Scheduler rules. To view the list of job schedulers in the application, make sure we have the pzJobSchedulerObserver privilege.
- Job schedulers are versioned rules and can be seen under the SysAdmin category in Records explorer.
Please go through our Requestor types in Pega article to know about Requestor types and how does ASYNCPROCESSOR/ System Runtime Context play a vital role in Job scheduler execution.
- When we open any job scheduler rule, we can see a banner displaying the below message.
It’s required to ensure that either ASYNCPROCESSOR requestor type or System Runtime Context has access to the configured Job scheduler. Failing which will not run the Job scheduler on configured time.
We can classify Job scheduler configuration into three sections,
- Node configuration
- Scheduling logic
- Business process
Node Configuration
Before starting the node configuration, let’s understand few basic details of Node.
What is a Node?
Each environment has a server that hosts the PRPC rules engine. Each of these servers is referred to as Node and a node will always be pointing to a database.
What is Multi Node environment?
In most of the cases, one node won’t be sufficient enough to handle throughput and we end up installing multiple servers with different port numbers. Even if the system has multiple servers/nodes, it will be pointing to the same database.
What is Node Classification?
Introducing multiple nodes will optimize the throughput? Definitely No. We need to classify & use them effectively.
Pega has classified nodes namely Web Users, Background processing, BIX, Search, etc. Node classification helps us segregate nodes by their purpose. Proper classification is needed to have the maximum throughput.
We can update the type of node anytime in the system. The process to update the node type depends on the application server that we use.
For Example, If our application server is Apache Tomcat, to modify the node type of any existing node, we need to update the setenv.bat file in the tomcat/bin directory with -DNodeType=<node-type> JVM arguments.
- Hope now we are clear on what is a node and how nodes are classified in Pega. Coming back to our configuration, each Job scheduler should be tagged to the node types on which it should be running.
- A single Job Scheduler can be configured to run on multiple node types. But if the execution is going to be too heavy or more frequent, then consider configuring it in multiple node types. Else configure it to run on one node type (Background Processing – Recommended)
- Background processing is one node type in the system & it can have multiple nodes under it. The below option enables us to select if it should run on only one node of type background processing or on all nodes that are tagged under background processing type.
Scheduling logic
- The next section to configure is the schedule in which the Job scheduler should run in the background.
- Each scheduler can be configured to run in one of the below 3 ways.
Startup Schedule
- We can configure a job scheduler to notify the system administrator whenever a node of any type comes online (Startup) in the system.
Periodic Schedule
- We can configure a job scheduler to purge the resolved work objects once in every 5 hours.
Recurring Schedule – Daily
- We can configure a job scheduler to send the transactional emails daily at 9 AM on all weekdays.
Recurring Schedule – Weekly
- We can configure a job scheduler to send a NewsLetter to employees on weekends.
Recurring Schedule – Monthly
- We can configure a job scheduler to run on the last working day of every month to process the salary and send emails to employees.
Recurring Schedule – Yearly
- We can configure a job scheduler to send new year wishes every year January 1st at 12 AM.
Business Process
- Once we are done with the scheduling logic, the next configuration would be the business process. Let’s quickly see how the job scheduler execution happens at run time to understand this configuration better.
- This section has 2 fields to configure.
- Context
- Class & Activity
Context
- This field defines the application context for the scheduler rule.
- We can configure a job scheduler to reuse the same access group mapped in the ASYNCPROCESSOR requestor type or we can map an access group specific to the job scheduler for its execution.
- In both the scenarios we must ensure that the business process (Activity) mapped in the job scheduler is accessible with the access group mentioned.
Class & Activity
- As we discussed earlier, the job scheduler performs a task at the background on the scheduled time. This field defines the task that the scheduler must perform.
- As we do in our normal implementations, we write activity to perform the required task and map it to this field for the scheduler to execute it from the back end.
- We can even pass parameters to the activity from the job scheduler rule. In most cases, these parameters will be a static value.
Business Scenario
OSP organization has a claims application to process the reimbursement request submitted by Customers. The system creates a work object for each claim request submitted in the system and tracks it using different work statuses. Now requirement says that the summary of the claim requests which are successfully resolved should be sent to the client for internal auditing purpose on a daily basis.
Implementation
- As we do in our normal implementation, let’s first code the business logic of sending out the summary on resolved claims using an activity rule.
- Once the business logic is ready, we can create the job scheduler rule and configure the node types and its schedule.
- Once the scheduler configuration is done, we need to configure the ASYNCPROCESSOR requestor type to map the access group that can invoke the configured scheduler. Failing which will not run our job scheduler rule.
- As soon as we create and save the Job scheduler rule and ASYNCPROCESSOR requestor type, the system marks an entry into Data-SystemOperations-JobManagement-JobInfo & makes the job scheduler ready for its execution.
Error handling in Job Scheduler
- Error handling is an important factor in Job scheduler as this helps in analyzing the root cause in case of failure.
- In our scenario, the scheduler business logic might fail for 3 reasons,
- While fetching the list of resolved claim cases [Database connectivity issue, etc].
- When trying to send email [SMTP issues, etc.]
- Infrastructure issue when executing scheduler.
- If we observe that a particular job scheduler fails with its execution, we can temporarily disable it from the background process. Once the root cause is analyzed and fixed, it can then be enabled back.
How to trace & debug Job Schedulers?
- We can trace a Job scheduler from the job landing page in the admin studio. We will be seeing the events in the tracer only if the scheduler runs in the background.
- Since the job scheduler is all about running a stand-alone business logic on the scheduled interval, we can also debug it by running the activity in the job scheduler rule stand-alone.
- The job scheduler has an enhanced version of tracking execution statistics.
- A detailed summary of Job schedulers in the system can be seen from the Admin studio.
- Job schedulers by default keep track of all the history of execution in the pr_perf_stats table. It is possible for us to view the past history of execution along with its status for each of the job schedulers from the admin studio landing page.
- If we want to turn off the behavior of the system keeping track of all the previous execution summary, we can update the below environmental variable in the prconfig.xml to false or by creating DSS for the same. By default, this environment variable will be set to true.
<env name="usage/collectjobschedulerdata" value="false" />
Replacing an agent with a Job Scheduler
The common question that everyone has in terms of agents and job schedulers are,
- Will my Agent rule still work in v8 – YES.
- Should I replace all my agents with job schedulers – Partially YES
- Going forward should I create job schedulers to schedule tasks – YES
When we say partial Yes for replacing agents with job schedulers, how do we do that? 🧐
- Disable all the agent rule that are to be replaced with job schedulers.
- Make sure the agent schedules are updated accordingly. If not, update it manually.
- Now configure a job scheduler for the same purpose by following the steps discussed above.
- Update the ASYNCPROCESSOR requestor type to include the required access groups from the BATCH requestor type in v8.1, 8.2, 8.3, or add the application to System Runtime Context starting from v 8.4.
A complete impact analysis is required before replacing the agents with the job scheduler if the application is already live and running with agents.
Pointers to Remember
- Job schedulers will not have any agent schedules created rather it directly executes the job scheduler rule on the scheduled time.
- Job schedulers are easy to configure and manage.
- Unlike agents, we create separate job schedulers for each business process which gives us a lot of control in terms of configuration.
- The job scheduler has a lot more advanced scheduling options when compared to agents.
- Job schedulers play way better in terms of maintaining execution statistics and success rates.
- Job schedulers look user friendly when compared to agents.
- Starting from Pega version 8, applications should be added to System Runtime Context to enable Job scheduler for processing.
We are now at the end of the post. We hope you got the basics of job scheduler along with its configuration & troubleshooting. We wanted the article to be up-to-date and didn’t cover much about agents. If you feel you want us to publish an article about agents, then feel free to comment on this post or raise your request here.
We are glad to publish the article requested by @ Mukkram on “Job Schedulers”. If you want us to publish any articles of your choice, please fill out this form & we will make sure it gets published.
Stay tuned for more posts on Background Processing😊
I have a question
How to enable one particular agent to run on our desired given node?
In the Agent rule, across each agent, we will have an option to associate it to a node type. We can use that configuration for each agent to specify the node type.
One more limitation I see is, there is no way to modify time in production. It requires to package and deploy for changing schedules as it’s a locked rule instance. with agents we can change this in agent schedules.
We would not say that as a limitation rather we would say that as best practices. The only reason why Pega maintains agent schedules for agent processing is to ensure that one BATCH requestor types support multiple agents across the application.
But we developers started updating date-time or execution mode in the agent schedule directly which is something that Pega does not recommend.
Hence this restriction in Job scheduler is not a limitation, it just ensures the good code quality.
Agreed. But still I would think it as a limitation. Any business might want the ability to change their schedules for business activities.
Like what time mails should be send etc.,
Atleast the rule should have ability to refer a dynamic value from a data page.
Nice post on job scheduler
Thank you so much @shivakumar
Happy Learning from OSP😊
I have a job scheduler that will fetch records from DB then process and update some flag in DB. If my job scheduler is configured to run on multiple nodes, how can i make sure that, records picked by Job scheduler turned on node-1 should not pick Job scheduler turned on node-2 (Do we need to handle this as part of our activity or is there anything we can do).
With the actual Job scheduler configuration, it’s not achievable. We can handle this wisely in two ways,
1) We can make our job scheduler to run on only one node (Anyone associated node) & adjust the scheduler frequency to get the maximum throughput.
2) We can have logic to browse the records from DB using Job scheduler and can queue it to the queue processor for execution. Queue processor in v8 executes the business logic using data flow and has an inbuilt mechanism to partition the records between nodes.
Hi, I too have the same question which Kumar had raised regarding the usage of
“Run on all Associated nodes” .
Can you give us one real time scenario where we can use “Run on all Associated nodes” in Job scheduler.
Nice explanation guys. Does job scheduler a replacement to advanced agent guys? It will be good guys if u tell or compare std and adv agent with this guys as we are most used to that agent guys. And please post about queue processor and I really need property qualifier guys. Please help.
Thank you so much and soon we will post about Queue Processor.
Can you please start a discussion in our OSP Forum with the scenario or explanation that you want about Property qualifiers?
Happy Learning from OSP 😊
I actually wanted to understand property qualifier end to end guys. Now am hardcoding labels like % guys as I do not understand property qualifier usage. I felt guys it might be a good article in this Blog guys so suggested for article.
Can you also please tell me guys if job scheduler is replaced for adv agent guys?
That is the article description that we started off with. The job scheduler is not a replacement of Advanced agents, it’s just an enhanced version of advanced agents whereas queue processor is an enhanced version of the standard agents.
It’s highly recommended to use Job scheduler and queue processor in v 8 for any background processing to have maximum throughput.
Hi Team,
I’m not able to see the past history of execution for my job scheduler. Can you please provide the exact steps to view the statistics of job scheduler.
Regards,
Azhar
It seems the collection of historical data is turned off in your application. Please turn it on using either of the below approaches and let us know,
1) Update prconfig.xml with the environmental variable
2) Create DSS usage/collectjobschedulerdata in the Pega-Engine Ruleset with value as true & restart the server.
I’m able to see records in the pr_perf_stats table. We haven’t turnoff this feature. Can you please let me know where exactly we will find this Statistics in Admin Studio.
In Admin Studio, the Jobs landing page will show you the list of job schedulers in the system along with the information about each scheduler. The column Executions on the landing page will be a hyperlink that will show you the execution history of the particular job scheduler.
Also please make sure you have admin access with full privilege or add pzJobSchedulerAdministrator and pzJobSchedulerObserver privilege to your access role and verify.
Got it…Thank you so Much
Hi Team,
For Advanced Agent, we had “Run this Agent only on single node at a time” but it is missing for job scheduler. Any info on this.
Regards,
Azhar
Pega has replaced that option in Job scheduler as Anyone associated nodes.
This ensures that the node which wakes up first starts with the execution.
But these two options are different right
Run this Agent only on single node at a time – Agents will run on multiple nodes but Two Agents should not run at same time.
Any One Associated Node – It will run only on a single node and ignore all other nodes.
Yes, you are correct. Anyone Associated node actually does it in other ways. But there is special handling in Pega which makes it work similar to Run the agent only on single-node at a time.
When the configured node type has 3 nodes and if the scheduler is configured to run on one node, system automatically selects the other nodes in the cluster of specified type when the existing node is down/offline. This capability makes it work similar to the option in the agent rule.
Hi Team,
Except separating each business process and statistics there is no much difference between Advanced Agent and Job Scheduler right. Can you pls provide some additional info.
1)Job schedulers are easy to configure and manage
– All configurations are same as Advanced Agents(Same Associating Nodes, Same Scheduling, Same Context)
2)Unlike agents, we create separate job schedulers for each business process which gives us a lot of control in terms of configuration.
-The only extra control we got is on the Context, earlier it was common context for all agents and now we have it separately.
3)The job scheduler has a lot more advanced scheduling options when compared to agents.
-Only change we can see is for “multiple times a day” option, in agents we should provide in seconds but here it is in minutes and hours
One of the major advantages of using Job scheduler is on the statistics & the success rates that it provides for each execution. In earlier advanced agents, if an agent fails it’s hard to debug the root cause.
With the job scheduler, the optimization from advanced agents is very minimal which is not the case with the queue processor. Queue processor is totally different from standard agents in terms of execution. Queue processor performs x20 times better than standard agents (Proven data from Pega).
Stay tuned for our article on Queue Processing.
Thanks for the quick response
Hi Team,
In Admin Studio, we had flexibility to start or stop agents at node level(Agent will be configured on 4 nodes but for some reason you can stop it on a single node and other three nodes will be running). But for Job Schedulers we don’t have enable or disable option at node level. Any Idea how to handle this.
Regards,
Azhar
Should a new AsyncProcessor requester type be created for a new job scheduler or should the Asyncprocessor available by default should be modified ?
Starting from v8, it will be available by default. We just need to update the access group or the application context in the requestor type.
Note: Pega never recommends us to create requestor types on our own.
can you please expand each step of the activities you configured and explain them in detail, so that it will be easy for the learners who are new to this topic?
Sure we will follow that going forward in our future posts. Thanks for taking time to comment a valid ask.
Happy Learning from OSP 😊
Do we have an equivalent of Queue-For-An-Agent with jobscheduler? What are the alternate options you see?
Job schedulers are schedule driven and are the replacement of Advanced agents in v8. Only event-driven entries (Queue Processor/ Standard Agents) can be triggered based on a real-time event. Hence Job Schedulers (or) Advanced Agent can’t be queued using any of the activity methods.
Let us know if this answers your question.
How to create a job scheduler to run on every 30 second ?
It seems to be a restriction in the Job Scheduler rule.
Please let us know your requirement and we will try to solution it in the optimal way.
I have created a job scheduler to perform one action and my application is hosted on multi node system.
Let’s say Node 1 and Node 2 and I would like Job Scheduler to be run on only Node 1.
In the Job Scheduler configuration, I have selected the Runs on as “Any one associated node”.
Here my question is, in which node my Job Scheduler would run?? Node 1 or Node 2??
Kindly let me know how to restrict a Job Scheduler to be run only on Node 1.
Dear OSP Team,
Can I have any update on this request??
Batch Requestor works similarly to the Browser Requestor. When you log in through Browser, Pega identifies the web nodes based on the balancer and creates your requestor in any one of the nodes. In the same way, if you have 2 Background processing nodes, Pega creates a requestor in any one of the nodes (Random) during every Job Scheduler execution.
To make it run on one node consistently out of two available nodes, Please add your background processing node to a secondary node type (Say Custom1) and configure your Job Scheduler to run on the newly configured node type.
A node can be tagged to more than one node type. Please refer to our article to see how we can associate a node to multiple node types.
Hi Team, Thank you for all the efforts you guy made. I have one business requirement. I want my job scheduler to run at 10am and 5pm daily. Can I achieve this using 1 job scheduler? I see in “multiple times a day” we can only configure hours, not calendar time.
Afcreating creating the Job Scheduler I could not see the entry in Data-SystemOperations-JobManagement-JobInfo.Can you please explain on this part.
Diff/Specific Adv between JobSchedule and Advance Agent
Hi Team,
I have a requirement to schedule a task on June 1st of every year. How can I achieve it using job schedulers?
Hi Harish,
I think you got it already but in case you are still looking you can configure it by selecting the option Yearly and First day of June.
Thanks,
Durga Prasad
if i want to change the scheduler time in production how can i do that because job schedular associated with version
when you say
its recommended to use Job scheduler and queue processor in v 8 for any background processing to have maximum throughput.
can you say how performence of job scheduler increased wr.t advanced agent …?
what backing internally to give maximum throughput.
for QP we know kafka was introduced but for job scheduler ….?
Thank you sir, nice explanation please upload the data transform concept.
Hi,
Thanks for the explanation. Can you explain when jobs are created for a job scheduler…for example, if i configure interval as monthly and 5th of every month. When will system create jobs..starting day of every month or how does it work. And Is there any option where we can stop the job scheduler from creating further jobs as per the interval.
Can you please help in telling difference between job scheduler and advanced agents in terms of performance like how job scheduler has better performance than agent when there is almost no difference in both?
Also, as we know the importance of thread count in queue processor making its processing faster than standard agent, how we can increase thread count for job scheduler and how will it work as compared to thread in queue processor by avoid race condition also?
How to check the Audit history of particular Job Scheduler because some time we need to track who enable/run it?