topic RE: Adding priority to large input data in Product Forum

Adding priority to large input data

RaviSingh4 — Mon, 28 Oct 2019 05:35:00 GMT

Hi everyone,
I know how to use OLEDB to fetch large data from excel into a collection but how could I add 10K or more data from excel to a work queue with priority without looping them one by one ?

------------------------------
Ravi Singh
------------------------------

RE: Adding priority to large input data

PratyushGarikap — Fri, 01 Nov 2019 08:47:00 GMT

Once you load the input data into a Collection, you can add a new column in the collection to store Priority. You can set the Priority values based on your business rules, and then load this into the work queue by setting the input parameters appropriately.

Do keep a watch on potential memory issues when working with large amounts of data, in which case you may have to read the input data in batches.

------------------------------
Pratyush Garikapati
ROM Architect
Blue Prism
Asia/Kolkata
------------------------------

RE: Adding priority to large input data

RaviSingh4 — Fri, 01 Nov 2019 14:16:00 GMT

Dear Pratyush
Let's suppose as per business rule we have only 2 priorities, 0 for 5K data and 1 for rest of the 5K data.
Now, the priorities are added successfully in the collection.
But how could I use ADD TO QUEUE action of work queue to successfully add these 2 priorities in the work queue because I can set a single priority at a time with ADD TO QUEUE.
Can we use code stage to select data from different priorities at a time and then use dynamic data item to set the priority input of ADD TO QUEUE action 0 or 1.

------------------------------
Ravi Singh
------------------------------

RE: Adding priority to large input data

sutirtha.gupta — Sat, 02 Nov 2019 12:22:00 GMT

Hi Pratush,

If possible can you fetch the data using oledb based on priority. It will solve your problem.

Regrads

Sutirtha Gupta

TCS

------Original Message------

Dear Pratyush
Let's suppose as per business rule we have only 2 priorities, 0 for 5K data and 1 for rest of the 5K data.
Now, the priorities are added successfully in the collection.
But how could I use ADD TO QUEUE action of work queue to successfully add these 2 priorities in the work queue because I can set a single priority at a time with ADD TO QUEUE.
Can we use code stage to select data from different priorities at a time and then use dynamic data item to set the priority input of ADD TO QUEUE action 0 or 1.

------------------------------
Ravi Singh
------------------------------

RE: Adding priority to large input data

RaviSingh4 — Sat, 02 Nov 2019 12:34:00 GMT

Hi Sutirtha
Can you please elaborate the design layout ?

Regards,
Ravi Singh

------------------------------
Ravi Singh
------------------------------

RE: Adding priority to large input data

PratyushGarikap — Mon, 04 Nov 2019 11:18:00 GMT

You can run 2 SQL queries - first one pulls all the 5K rows for which priority should be 0, and load them into Collection1; second one fetches the 5K rows for which priority should be 1, and load them into Collection2. Now you can load Collection1 into queue with Priority=0 and then Collection2 with Priority=1.

Alternatively, as Sutirtha suggested, you can have Priority column in the input excel itself (someone will have to manually populate this or populate this automatically using BP) which you can use to filter the rows returned the SQL query. This will especially be useful if the Priority logic requires you to recursively search across multiple rows of data.

The scenario you have presented doesn't seem to require a code stage, although it certainly is an alternative if none of the out-of-the-box functionalities are giving you the desired result.

------------------------------
Pratyush Garikapati
ROM Architect
Blue Prism
Asia/Kolkata
------------------------------