topic Re: Working with large xlsx files without Chunk in Product Forum

Working with large xlsx files without Chunk

Ernest1 — Mon, 12 Jun 2023 15:40:59 GMT

He everyone,

Regarding to those posts:

https://community.blueprism.com/communities/community-home/digestviewer/viewthread?GroupId=385&MessageKey=16adfb54-82e0-4699-bf49-d1142daeec53&CommunityKey=1e516cfe-4d1f-4de9-a9eb-58d15bf38c81&reply-inline=16adfb54-82e0-4699-bf49-d1142daeec53

https://community.blueprism.com/communities/community-home/digestviewer/viewthread?GroupId=145&MID=1900&CommunityKey=3743dbaa-6766-4a4d-b7ed-9a98b6b1dd01&tab=digestviewer

I have a question. How to work with such large xlsx/csv file in a situation where I need to find one record according to the X criteria, then find another record according to the Y criteria, etc. I have the criteria saved in another xlsx/csv file, which has about 600-1000 rows.

I don't think splitting a file will be a good solution in this case because the row result can be in one or all of the split files.

Could you help?

Re: Working with large xlsx files without Chunk

harish.mogulluri — Mon, 12 Jun 2023 16:19:12 GMT

Hi Marcin Grzelak,

1) create custom DLL's to manipulate large amount of data in run time. In this case you need to write and manipulate data before getting the data in to BP

2) Create macros to filter the data and other manipulation i

3) Apply data manipulation in the excel itself. like Filters, Formulas and other available excel capabilities for this you can use existing excel VBO and build the code on top of it.

if the file is in 1000's I will read the data from the excel and perfrom data manipulation in the code stage in bp itself.

Note: Also latest BLue Prism version 7.2 can handle much more than the previous versions because it is 64 bits

Re: Working with large xlsx files without Chunk

Athiban_Mahamathi — Mon, 12 Jun 2023 23:42:47 GMT

Hello Marcin,

You can achieve this by executing a select query statement via OLEDB. I have attached the OLEDB guide for your reference.

Re: Working with large xlsx files without Chunk

harish.mogulluri — Tue, 13 Jun 2023 02:56:35 GMT

As Other member mentioned OLEDB is also an option in general.

Re: Working with large xlsx files without Chunk

Ernest1 — Tue, 13 Jun 2023 08:11:38 GMT

Hi,

Many thanks for answer for both.

I will check OLEDB, but I need to catch Admin for installation Microsoft Access Database Engine.

All the best.M