topic Re: Merge 2 big CSV Files in Product Forum

Merge 2 big CSV Files

LaurentAxa — Tue, 26 Aug 2025 15:33:19 GMT

Hello,

My robot need to download from a website a large CSV File and add all the rows from this file to an existing CSV file.

Any idea if I can find an object or a code able to do that ?

I tried with a C# code I found online but when I execute on Blue Prism I have compilation error.

Re: Merge 2 big CSV Files

Mohamad_Dakkouri — Tue, 26 Aug 2025 20:44:35 GMT

Hi @LaurentAxa

We can help you.

Could you just tell me if you want to read the CSV file inside a collection, or merge it directly without going through a collection?

Regards

Re: Merge 2 big CSV Files

hossein.azimi — Wed, 27 Aug 2025 04:10:24 GMT

Hello LaurentAxa

One of the methods I would try is as follows:

When the CSV can fit in memory and you don’t mind a couple of seconds of processing:

-> Download

●　If the URL is publicly reachable (or uses your Windows auth):
Use Utility – HTTP > HTTP Request (GET) → save response to a temp file (e.g., C:\Temp\new.csv).

○　Set Use Default Credentials = True if you need NTLM/AD.
○　Add headers/cookies if needed (Authorization, cookie, etc.).

-> Append (skip header)
●　Use Utility – File Management:

○　Read Lines From File → C:\Temp\new.csv → collection NewLines
○　If you want to skip header in new.csv: loop from row 2, else from row 1.
○　In loop, call Append Line To Text File targeting C:\Data\existing.csv.

Note: this reads the full file into a collection first—fine for small/medium CSVs, not great for multi-GB.

Regards

Hossein Azimi (APJ Team)