Data Page – its Configuration, Usage & Execution

Before getting into the discussion of the data page, let’s explore a few key terminologies for better understanding.

Memory

Memory is just like a human brain to store information/data.

We will have a few friends who will tell us what they did for us at the age of 2 when we are at the age of 30+😲. They tend to have good memory power. On the other hand, we will have a few friends who will forget what we did for them last week😞.

In the same way, System has two types of memory using which we can access data.

Permanent Memory

This type of memory will store the data in a storage space, which can be retrieved whenever required. Typically, it is a database [Oracle, SQL, Postgresql].

Temporary Memory

Temporary memory will hold the data only for a specific time period.
Rules can be referred from temporary memory [Rule Cache] & Data can be referred from temporary memory [Clipboard].

Why Temporary Memory?

Pega uses the Clipboard Page as temporary memory. Let’s compare the below two diagrams to see the usage of temporary memory [Clipboard] in terms of data retrieval.

When there is no temporary memory, every time when information is required, a DB call happens to fetch the data which is very expensive.

When there is a temporary memory, the first time DB call happens to fetch the required information and sets the data in the temporary memory [Clipboard]. Subsequent calls will refer to the data directly from the clipboard.

What is a Page?

Let’s take a famous book Ponniyin Selvan written by Kalki. This book has approximately 2300 pages. Each page in the book has the actual information/data about the story. In the same way, each page in the clipboard has the actual data needed for processing. Pega has categorized pages in the clipboard into different types,

• User Pages – Top-level clipboard pages created using activity rule. These pages are specific to thread.
• Data Pages – Our hero of the post. We will see this later 😎.
• Linked Property Pages – pages created through linked properties. These pages are specific to thread.
• System Pages – pages created by Pega to maintain system & session related information. We can refer these pages, but, we can’t create any System page.

User pages, Linked property pages & system pages are gone from memory in the following situations,

• When the user logs-off the session.
• When the requestor session times out.
• When the thread is closed or becomes in-active [not applicable for System pages].

Why Data Page?

Few data in the application will be required more frequently. It will be good if we maintain a single copy of data that can be referred whenever needed.

Can the User page satisfy this need? – No 😒

User pages are specific to thread and become inactive when we close the thread. Hence Pega provides Data Pages, which can be reused across a thread, a requestor or multiple requestors (node level) based on the configuration.

What is a Data Page?

• Data page is a clipboard page that can be shared across a thread, a requestor or multiple requestors (node level).
• Data page rule is an instance of Rule-Declare-Pages and is available under Data Model in Records Explorer.
• Data pages are loaded when it’s first referenced during execution.
• Data pages can be parameterized.
• Data pages can be reloaded/expired based on the configuration defined.
• Data Pages always starts with the prefix “D_“ or “Declare_”(legacy). Data pages are informally called as DPage.

How to Configure a Data Page?

• Similar to other rules in Pega, the New Rule form of Data Page accepts Name, Class & Ruleset Version.

• Data Page configuration is simple when you know the actual purpose of each of the fields. Let’s quickly see the usage of each,

Definition

This tab of the Data page rule has 3 main sections,

• Data page definition
• Data sources
• Post load processing

Data page definition

This section of the data page has most of the configurations which describes how the data page is in clipboard.

Structure

• This defines the mode of the data page. Possible modes of a data page can be Page & List [Page List].
• The list structure data page provides an additional option to configure keyed access. Keyed page access is one of the data access patterns available in Pega which can be seen in detail in a separate post.

Object type

Each page in the clipboard will have a class context. For Example, the OperatorID page is of class Data-Admin-Operator-ID, pxRequestor page is of class Code-Pega-Requestor. In the same way, each data page in the clipboard belongs to a class and this field holds that class name.

Mode

This field decides where exactly the data page will be shown in the clipboard. It can be Read-Only, Editable & Savable.

• Read-Only– The data page of this mode will be used only for referring data. We can’t update the information on this data page. It can be modified only during loading or in Post processing activity. Read-Only data pages can be seen under the Data Pages category in the clipboard.

• Editable – The data page of this mode can be used for referring the data as well as it can be updated on the fly using Activity, Data Transforms, etc. This mode of data page works similar to User Pages and can be seen under the User pages category in the clipboard.

• Savable– The data page of this mode can be used to persist information directly into the database. We can explore the Savable Data page in detail in a separate post. These modes of data pages are similar to User Pages and can be seen under the User pages category in the clipboard.

High Throughput

This option is applicable only if the mode of the data page is Read-only.

We can improve clipboard performance in data page loading by enabling this option. Enabling this is recommended when the data is going to be static and used repeatedly. For Example, a data page to hold the list of Countries.

When this option is enabled, the below feature will not be supported in Data Page.

• Declarative rules.
• Page messages.
• Complex property references [Property-Ref].
• Saving pages to a database.

Lightweight clipboard mode

This option is applicable only if the mode of the data page is Editable.

We can improve clipboard performance in data page loading by using the Lightweight clipboard mode option.

When this option is enabled, the below feature will not be supported in Data Page.

• Properties referencing a data page.
• Linked properties.
• Change tracking for property value updates in the user interface

Scope

Let’s see the below architecture before defining scope.

The scope of a data page can be Node or Requestor or Thread.

Node

• If a data page is defined with node-level scope, then all the requestors & its related threads (BROWSER, APP, BATCH, PORTAL) running in the same node can share the same data page. 
• Node level data pages can only be of mode Read-Only.

Requestor

• If a data page is defined with requestor level scope, the threads active in that requestor can access the same data page. 
• Requestor level data pages can be of mode Read-only, Editable and Savable.

Thread

• Thread level data pages are similar to user pages. If a data page (having case-related information) is defined with thread-level scope, it can be accessed multiple times by the same thread without having it to reload.
• Thread level data pages can be of mode Read-only, Editable and Savable.

For Example: Assume that you have launched a Case manager portal and opened multiple cases in a series of tabs. The cases opened in each tab runs in a new thread. If a data page having case-related information is defined with Thread level, It can be reused by the same case multiple times. If a data page is defined with the Requestor level, then all the cases opened in each tab can access the same data page.

Data sources

This section gives us an option to configure the actual source of our data page.

• We can select any of the available sources based on the structure of the page and load the data page.
• We can also have multiple sources for a data page and decide on those conditionally using a when rule.
• Only Activity of type Load Data Page can be used as a source of data page when Activity is selected as the source type. The activity type can be changed under the security tab of the activity rule.

Post load processing

• We can’t ensure that we always have the required data readily available to us. There might be a situation where we should manipulate information based on the results.
• Post loading activity that we specify here will be used to perform any custom manipulations on the results.

Load Management

• This tab of the data page rule is more important as the configurations available here ensures that the data is up-to-date.
• Load management is only applicable when the mode of the data page is read-only.

Page Management

• This option is most likely used by the developers when they want to manually remove/flush the data page from the clipboard.

• We can refer the below-mentioned function in an activity/data transform to remove all instances of the data page from the clipboard irrespective of the parameters.
  
  @(Pega-RULES:DeclarePages).pzDeleteAllInstancesOfDeclarativePage(tools, “D_XXXXXXXXX”)

Load authorization

• This section is only applicable if the scope of the data page is Node.

• Requestor and Thread level data pages use the application context of the logged-in user. Node level data page is outside of the requestor context and hence the application context will be decided based on the access group provided in this field.

Refresh Strategy

This section enables us to define a mechanism using which the data on the page remains up-to-date.

Reload once per interaction

• Enabling this option will freshly load the data page whenever it is referred. Careful consideration is required before enabling as this may cause a huge performance issue.
• This option is available only when the data page is of scope Thread or Requestor.
• When selected, the system ignores any values in the Reload if older than and Do not reload when fields at run time.

Do not reload when

• We can also control the refresh strategy of the data page using a when rule.

With this configuration, during data page reference, the system validates if the customer is in-active. If yes, the data page will not be refreshed else, it will be refreshed.

• This option is available only when the data page is of scope Thread or Requestor.

Reload if older than

• We can make a data page reload based on the time duration in Days, Hours, Minutes and Seconds.

• When the data page runs for the first time, it sets the cache with the expiration time based on the configuration.

• The next consecutive data page calls will check if the expiration time has reached or the page is stale. If yes then reloads the page, else fetch the values from the existing page.

Page limits

• This option is most recommended when the data page is of mode read-only with scope as Requestor or node.

Limit to a single data page

• This option is only available when the data page has at least one parameter defined. This can be seen in the later part of the post when we discuss the parameters tab.

Clear pages after non-use

• This option when selected forces the system to delete the clipboard instances of the data page after an interval passes with no access. Subsequent attempts by a requestor to access the data pages cause the pages to reload.
• The time interval until it waits is defined in the DSS DeclarePages/DefaultIdleTimeSeconds. This value will be used to expire the page. By default, DSS value is set to 86400 seconds or one day.
• we can adjust the value of the DSS based on our needs.

Parameters

Similar to other rules in Pega, data pages can be parameterized. Parameters defined in the data page define the actual data on the page.

Parameter values can be used in conjunction with the settings limit to a single data page in the load management tab.

• When the limit to a single data page is enabled, the system only maintains the latest instance of the data page in the clipboard.

• When the limit to a single data page is not enabled, the system maintains all the instances of the data page in the clipboard.

Let’s create a data page for a real time business scenario to understand it better.

Business Scenario:

Consider OSP Organization has an HR Onboarding application. Candidates can apply for a job via the company’s site or via Naukri. Candidate details are available in OSP’s database if candidates use Company’s site and in the external system if they use the Naukri site to apply for a job.

Let’s now see how we can configure a data page to fetch candidate information conditionally based on the application source.

Implementation

D_CandidateDetail – To get the Candidate detail from either local data table or from an external source via a connector.

Step 1: Goto Records Explorer -> Data Model -> DataPage -> Create

Step 2:  We can change the data page definition if needed. As per our requirement, cases are created for each candidate, hence, the Scope should be Thread and Structure should be page. Since we are just going to display the candidate details, we can choose the Mode as Read-only.

Step 3: Configure the data source to get a candidate detail from the candidate table.  To do that, we can use the lookup option and provide the class name with the key parameter passed.

Step 4: Once the data page to load data from the Data table is configured, we can add another source to load data from service.

Step 5: Create Data Source provided in each Source is to configure the new Source. In our scenario, we can create a Connect Rest by configuring it. Creating a connect rest will be covered in our future post.

Once the Connector rule is ready, we can choose the connect rule and configure its Request and Response Data transform as mentioned below.

Note the highlighted classes, since the class of the connector and class of the data page are different, it’s mandatory to provide Response data transform to map the data from the int layer to the Data layer.

Execution

Login as HR and select a Candidate ID to view the information. The Source of the profile selected here will be conditionally used in the data page.

Since the selected source is the OSP site, the data page loads the result using a lookup.

Unit Testing

Data page can be unit tested directly by running it stand-alone in the designer studio. We can also trace a data page run to see the DB operations, cache hits, etc.

Now we are at the end of the post. We have covered most of the basic concepts of data pages in this post. We can explore the below features of the data page in our continuation post.

• Error handling in Data Page.
• Asynchronous Data Page.
• Declarative page cache.
• Savable Data Page.
• Keyed page access in Data Page.

Stay tuned for our continuation post 😎