topic RE: Looking up data in a 9000 row excel spreadsheet was too slow. in Product Forum

Looking up data in a 9000 row excel spreadsheet was too slow.

Denis__Dennehy — Wed, 10 Sep 2014 17:14:00 GMT

I had a query from Kevin at Aviva - His process needed to do a lookup in an Excel spreadsheet which had 9000 rows. This lookup, required for each item being worked, was taking 10-15 minutes per case. I recommended to Kevin that he should use a Blue Prism work queue for his look up. Create an new process just to load the Excel lookup data into a work queue. This will need to be ran if the look up data ever changes. Then within his process he can use the Internal - Work Queues, Is Item In Queue action to look up the specific key he wanted for the look up data, and they get that data using the Item ID. This solution will increase the speed of his lookup considerably - and is a better solution for multiple robots (rather than them all using Excel).

RE: Looking up data in a 9000 row excel spreadsheet was too slow.

LeonardoCaldas — Wed, 28 Jul 2021 17:19:00 GMT

Hi,

Here are two suggestions.

1) Make sure the spreadsheet is in a local disk, not in a network drive.

2) Try to split the lookup of the Excel spreadsheet until you have a good performance.

------------------------------
Leonardo Caldas
Support Software Engineer
Customer support
America/Chicago
------------------------------

RE: Looking up data in a 9000 row excel spreadsheet was too slow.

krishna.kodimurali9 — Mon, 02 Aug 2021 11:56:00 GMT

Try turning off "Automatic calculation" on the excel during the time manipulation and enable it at the end.

------------------------------
Murali Krishna
RPA Consultant
------------------------------

RE: Looking up data in a 9000 row excel spreadsheet was too slow.

PvD_SE — Mon, 16 Aug 2021 07:01:00 GMT

...Wow! Two new answers on an open question from 2014 (...it was actually more like a suggestion). In that case I won't be shy and throw in my suggestion as well!

Whenever running selects on XL sheets with many rows, I'd use the BP actions for OLEDB. Where filtering and other collection selection actions take minutes to perform or run into OutOfMemory errors that require you to split up the XL in smaller chunks, OLEDB will do the same within seconds.

Note:
Personally, I do not class a 9k row XL as large and a tool as BP should be able to handle this without any errors, memory issues or delays - ever. Unfortunately, experience has uncovered many issues when handling XLs with 15k+ rows and more. But these XLs aren't large in my view either!
We're currently at 6.5 and I certainly hope things have improved in the newer BP versions that we hopefully get to work with soon.

------------------------------
Happy coding!
Paul
Sweden
------------------------------