Improve data extraction performance from database data source

In order to achieve better processing time, the thread size should be managed. 

 

Threads are objects within a process that run instructions. Threads allow concurrent operations within a process so that a process can run different parts of the request simultaneously.

In Smarten, by default, the database query executes in single thread. So, when you want to parallelize the query process in multiple threads, you can set a parameter for number of threads for a dataset query operation. If you allocate multiple threads, you can reduce the processing time for executing the queries.

In Smarten, number of threads can be specified in the datasource profile. Based on the number of threads set in Smarten, when you create a dataset, the Smarten engine can do automatic partition within and divide the query among these partitions and thus process the request in multiple threads to improve query performance.

 

To set the number of threads follow the below steps.

Steps:

1. Create a new datasource profile (From the Top right menu, select New -> Dataset ) or open an already existing datasource profile (created from datasource option)

 

 

NUMBER OF THREADS OPTION IN DATASOURCE PROFILE

 

The option to specify the thread numbers will be there. Specify the number of threads and click OK.

 

PS: Multithreading leads to higher utilization of the CPU as well as the target database. You need to select no. of threads to balance your CPU utilization and configuration of database server. 

 

Note: This article is based on Smarten Version 5.0. This may or may not be relevant to the Smarten version you may be using.

Dataset Datasource performance SSDP data extraction threading multi threading Threads