Genpact Cora Knowledge Center

Support

Configuring Elasticsearch Service

V9.4

Overview

The search feature provides Google-like search capabilities. When search is enabled, users can retrieve any data related to processes and cases. 

The search feature uses the Elasticsearch engine. The Elasticsearch cluster needs to be set up and running before you can configure the search feature on Cora SeQuence.

Configuring search involves several procedures, including defining the index (retrievable data), setting search permissions, and enabling search on the system. 

NOTE
Cora SeQuence now supports Elasticsearch for AWS with AWS Identity and Access Management (IAM) authentication (key/secret) only.

The Search feature is available only for Premium packages.

Search configuration steps


StepsLocationPerformed by 
1Define retrievable dataJSON file
  • Cora SeQuence Administrator
  • DBA
2Enable change trackingCora SeQuence database
3Create connection stringAdministration site
  • Architect
  • Tech Lead
  • Developer
4Set up indexing jobsAdministration site
5Enable search in FlowtimePowerShell function




1. Define retrievable data

Create a JSON for each solution or workflow that you want to enable search for. The JSON file defines the fields of the Cora SeQuence database that will be indexed by Elasticsearch and can be retrieved by user's search. 

View an example of a JSON file.

Consider the following factors when you create the JSON file:

  • The database change-tracking mechanism uses primary keys to point to rows that have changed. To make sure that changing data is correctly tracked, the keyFields value in the JSON file must contain only primary key columns. 
  • If you want custom data to be retrievable, you need to add the custom table's fields to the JSON.
  • To ensure that users can retrieve data from workflows that have been already closed, make sure that the hasClosedTable property is set to "true".
  • If you do not want to index all the data in a table, you can use the Conditions section to filter out data. To do that, specify the column that should be filtered out, its value, data type, and condition type, such as gt, lt, or eq. For example, if you want to index only the data that was created from a certain date and on, you can set the creation date as a condition.

 

2. Enable change tracking

To make sure that the Elasticsearch index is up to date with the latest data, you need to enable change tracking in the Cora SeQuence database. Change tracking enables applications to track changes to data and access information that is related to the changes. 

For details on how to configure change tracking, see this page.

3. Create the connection string

Configure the connection to the Elasticsearch service. You set up one connection per index. In this step, you also define who will be able to search the specific index. 

Prerequisites

Before you create the connection string, make sure that:

  • The Elasticsearch cluster is setup.
  • You have the Elasticsearch connection details, such as:
    • Connection type
    • End Points
    • Required credentials
  • You have the relevant JSON file.
  • You know which teams or users should be allowed to search the data defined in the JSON. 

Procedure

  1. Go to Administration > Global Settings > Elastic Search Connections, and click Add New Record.
  2. Enter a name for the connection string.
    This is the name displayed in the list of connections.
  3. Under Alias, enter a meaningful name for the search index.
    This is the name that will display in the Search bar on Flowtime. 
  4. Click Set Permissions, and then select one of the following:
    1. Assign Everyone: To allow anyone to search this index.
    2. Edit Assigned: To set search permissions to specific teams or users.
  5. Select the connection type. 
  6. Enter a meaningful name under Index Name.
    This is the name that you use to view log data and analytics about the solution's index in Kibana.
  7. Click the ellipsis button to open the Index Configuration screen.
  8. In the Index Configuration screen, paste the content from the JSON file that you created for this index. 
  9. Under End Point, enter the connection to the Elasticsearch cluster. 
  10. If your connection requires credentials, select the credential type and enter a credential name.
  11. Click Add.

4. Set up the indexing jobs

Data indexing is performed by a JES job. You need to set up two jobs:

  • Full load: Creates the Elasticsearch index based on the JSON definitions. This job copies the relevant data from the Cora SeQuence database to the Elasticsearch database.

NOTE
Depending on the size of your database, this job can take some time. To avoid performance issues, configure it to run during the weekend or during low-peak hours.

  • Incremental load: This job starts to run right after the full load job completes. This job makes sure that any changes to the Cora SeQuence database are synced to the Elasticsearch database.


Prerequisites

Before you create the indexing jobs, make sure that:

  • Change tracking was enabled in the database.
  • The Elasticsearch connection has been setup.

Configure the full load indexing job

  1. Go to Administration > Global Settings > Jobs Management, and click Add Job.
  2. Select ElasticSearch Full Load Data Collector, and then click Create.
  3. On the job definition screen, define the following:
    • Name: Enter a meaningful name for the full load job.
    • Scaling: Set the required scaling settings. (Scaling is available only for Cora SeQuence V9.3 and later versions.)
      For more details, see this article.
    • Job is enabled: Select this option only after you complete the workflow, or if you want to run the job for testing purposes.
  4. Click Next.
  5. On the Command tab, set the following:
    • Connection to ElasticSearch (mandatory): Select the relevant connection string.  
    • Batch Size (mandatory): You can enter up to 10000.
      Batch Size is the number of records that are synced at one time. Depending on the complexity of the cases, you may consider to set a smaller batch size.
  6. Click Next.
  7. On the Advanced Options tab, set the following options: 
    • Set the job to execute every 2 minutes. 
    • Select the "If a job is currently running, do not start the next job..." checkbox.
    • If not defined, job will run with default options.  
  8. Click Finish.

Configure the incremental indexing job

Configuring the incremental job is similar to configuring the full load job. Just note the following settings: 

  • Job Type: ElasticSearch Incremental Load Data Collector
  • Batch Size (mandatory): You can enter up to 10000.
    Batch Size is the number of changes introduced to the database.

For the detailed procedure, see the Configure the full load indexing job section above.

If you need to reset the full-load index synchronization, see this article.

5. Enable search in Flowtime

You need to enable search to display the Search bar on Flowtime. 

After enabled, the Search bar displays only if:

  • There is at least one workflow or solution indexed.
  • Your user, or the team you belong to, has permissions to access the index.

You run a PowerShell function to enable or disable the search feature. You can also use this function to set the total number of retrieved records for each search query. 

Procedure

  1. Run the Set-CoraSeQuenceQuickSearchConfiguration function on every server that hosts the Flowtime application. 
  2. To define the total number of retrieved records, set the PageSize parameter.

V9.3

Overview

The search feature provides Google-like search capabilities. When search is enabled, users can retrieve any data related to processes and cases. 

The search feature uses the Elasticsearch engine. The Elasticsearch cluster needs to be set up and running before you can configure the search feature on Cora SeQuence.

Configuring search involves several procedures, including defining the index (retrievable data), setting search permissions, and enabling search on the system. 

Search configuration steps


StepsLocationPerformed by 
1Define retrievable dataJSON file
  • Cora SeQuence Administrator
  • DBA
2Enable change trackingCora SeQuence database
3Create connection stringAdministration site
  • Architect
  • Tech Lead
  • Developer
4Set up indexing jobsAdministration site
5Enable search in FlowtimePowerShell function


NOTE
The Search feature is available only for Premium packages.

1. Define retrievable data

Create a JSON for each solution or workflow that you want to enable search for. The JSON file defines the fields of the Cora SeQuence database that will be indexed by Elasticsearch and can be retrieved by user's search. 

View an example of a JSON file.

Consider the following factors when you create the JSON file:

  • The database change-tracking mechanism uses primary keys to point to rows that have changed. To make sure that changing data is correctly tracked, the keyFields value in the JSON file must contain only primary key columns. 
  • If you want custom data to be retrievable, you need to add the custom table's fields to the JSON.
  • To ensure that users can retrieve data from workflows that have been already closed, make sure that the hasClosedTable property is set to "true".
  • If you do not want to index all the data in a table, you can use the Conditions section to filter out data. To do that, specify the column that should be filtered out, its value, data type, and condition type, such as gt, lt, or eq. For example, if you want to index only the data that was created from a certain date and on, you can set the creation date as a condition.

 

2. Enable change tracking

To make sure that the Elasticsearch index is up to date with the latest data, you need to enable change tracking in the Cora SeQuence database. Change tracking enables applications to track changes to data and access information that is related to the changes. 

For details on how to configure change tracking, see this page.

3. Create the connection string

Configure the connection to the Elasticsearch service. You set up one connection per index. In this step, you also define who will be able to search the specific index. 

Prerequisites

Before you create the connection string, make sure that:

  • The Elasticsearch cluster is setup.
  • You have the Elasticsearch connection details, such as:
    • Connection type
    • End Points
    • Required credentials
  • You have the relevant JSON file.
  • You know which teams or users should be allowed to search the data defined in the JSON. 

Procedure

  1. Go to Administration > Global Settings > Elastic Search Connections, and click Add New Record.
  2. Enter a name for the connection string.
    This is the name displayed in the list of connections.
  3. Under Alias, enter a meaningful name for the search index.
    This is the name that will display in the Search bar on Flowtime. 
  4. Click Set Permissions, and then select one of the following:
    1. Assign Everyone: To allow anyone to search this index.
    2. Edit Assigned: To set search permissions to specific teams or users.
  5. Select the connection type. 
  6. Enter a meaningful name under Index Name.
    This is the name that you use to view log data and analytics about the solution's index in Kibana.
  7. Click the ellipsis button to open the Index Configuration screen.
  8. In the Index Configuration screen, paste the content from the JSON file that you created for this index. 
  9. Under End Point, enter the connection to the Elasticsearch cluster. 
  10. If your connection requires credentials, select the credential type and enter a credential name.
  11. Click Add.

4. Set up the indexing jobs

Data indexing is performed by a JES job. You need to set up two jobs:

  • Full load: Creates the Elasticsearch index based on the JSON definitions. This job copies the relevant data from the Cora SeQuence database to the Elasticsearch database.

NOTE
Depending on the size of your database, this job can take some time. To avoid performance issues, configure it to run during the weekend or during low-peak hours.

  • Incremental load: This job starts to run right after the full load job completes. This job makes sure that any changes to the Cora SeQuence database are synced to the Elasticsearch database.


Prerequisites

Before you create the indexing jobs, make sure that:

  • Change tracking was enabled in the database.
  • The Elasticsearch connection has been setup.

Configure the full load indexing job

  1. Go to Administration > Global Settings > Jobs Management, and click Add Job.
  2. Select ElasticSearch Full Load Data Collector, and then click Create.
  3. On the job definition screen, define the following:
    • Name: Enter a meaningful name for the full load job.
    • Scaling: Set the required scaling settings. (Scaling is available only for Cora SeQuence V9.3 and later versions.)
      For more details, see this article.
    • Job is enabled: Select this option only after you complete the workflow, or if you want to run the job for testing purposes.
  4. Click Next.
  5. On the Command tab, set the following:
    • Connection to ElasticSearch (mandatory): Select the relevant connection string.  
    • Batch Size (mandatory): You can enter up to 10000.
      Batch Size is the number of records that are synced at one time. Depending on the complexity of the cases, you may consider to set a smaller batch size.
  6. Click Next.
  7. On the Advanced Options tab, set the following options: 
    • Set the job to execute every 2 minutes. 
    • Select the "If a job is currently running, do not start the next job..." checkbox.
    • If not defined, job will run with default options.  
  8. Click Finish.

Configure the incremental indexing job

Configuring the incremental job is similar to configuring the full load job. Just note the following settings: 

  • Job Type: ElasticSearch Incremental Load Data Collector
  • Batch Size (mandatory): You can enter up to 10000.
    Batch Size is the number of changes introduced to the database.

For the detailed procedure, see the Configure the full load indexing job section above.

If you need to reset the full-load index synchronization, see this article.

5. Enable search in Flowtime

You need to enable search to display the Search bar on Flowtime. 

After enabled, the Search bar displays only if:

  • There is at least one workflow or solution indexed.
  • Your user, or the team you belong to, has permissions to access the index.

You run a PowerShell function to enable or disable the search feature. You can also use this function to set the total number of retrieved records for each search query. 

Procedure

  1. Run the Set-CoraSeQuenceQuickSearchConfiguration function on every server that hosts the Flowtime application. 
  2. To define the total number of retrieved records, set the PageSize parameter.


V9.2

Overview

The Case Search feature provides Google-like search capabilities. When the Case Search feature is enabled for a Hot Operations solution, users can search for any data related to the solution's cases. 

The search feature is enabled by the integration of Cora SeQuence with Elasticsearch.

Configuring search involves several procedures, including defining the searchable data, creating the search index, setting up indexing jobs, and connecting the the Hot Operations solution to the index. 

Configuring the Case Search feature involves the following steps:


StepsLocationPerformed by Documentation
1Define searchable fieldsJSON file
  • Cora SeQuence Administrator
  • DBA
Current article
2Enable change trackingCora SeQuence database
3Create connection stringAdministration site
  • Architect
  • Tech Lead
  • Developer

Setting Up Search for a Hot Operations Solution
4Set up indexing jobsAdministration site
5Enable search in FlowtimePowerShell function
6Select a search providerAdministration site

This article covers the first two configuration steps. The remaining steps are covered in this article.

Define retrievable data

Create a JSON for each Hot Operations solution for which you want to enable case search. The JSON file defines the fields of the Cora SeQuence database that will be indexed by Elasticsearch and can be retrieved by user's search. 

View an example of a JSON file.

If your implementation includes custom data that should be retrievable, you need to add the custom table's fields to the JSON.

Enable change tracking

To make sure that the Elasticsearch index is up to date with the latest data, you need to enable change tracking in the Cora SeQuence database. Change tracking enables applications to track changes to data and access information that is related to the changes. 

For details on how to configure change tracking, see this page.

Next steps

To continue the search configuration for a specific solution, see this article.