System out of memory exception

[PoojaLuthra]

Hi, In my process I am using 3 collections simultaneously one of which is picking up data from excel and taking a range as a Collection. I am traversing two of the collections using Utility- Collection Manipulation. The Excel sheet that I am taking as a Collection; the Collection is taking more than 3000 rows and 4 columns. The no of items to be retrieved from the excel varies upto 40. However, upon running the same, I encountered with the error as has been attached. Please guide me as to what issue is occurring and how it can be resolved. I have already altered my SQL Server Database limit to unrestricted.

[John__Carter]

Please speak to your Lead Developer for advice on development technique. It could be that Utility - Collection Manipulation isn't the most effective method, you should use a Loop stage to iterate through a collection and calculation stages to extract field values. The utility makes use code stages to convert a collection into a .Net datatable, and it could be that if you are doing this repeatedly you are consuming too much of the PC's memory. Also check that logging is minimised to reduce the amount of data being sent to the DB.

[PrasadM]

Hi John, I have a requirement of comparing the data between two excel files which I used to do using collection with loops but ""system out of memory"" was thrown for more than 500 records, so used OLEDB and sql statements to query to the Excel2 and running a loop for collection (data from Excel 1), it works fine for 3000 records if the data increases say 4000 , I am again facing System out of memory, is there and way where I can free memory in the process ? will GC.collect in code block work please help . Thanks

[PrasadM]

HI Will enabling stag logging will slow down my process ?

[Anonymous]

We recently had a similar error upon attempting to iterate through a rather large collection. In addition, it was taking a significant amount of time to complete, up to 3 hours! After trying a few alternatives, we ended up using a Code Block, which not only alleviated the aforementioned memory issue, but also allowed it's completion within 3-4 seconds.

[Anonymous]

We recently had a similar error upon attempting to iterate through a rather large collection. In addition, it was taking a significant amount of time to complete, up to 3 hours! After trying a few alternatives, we ended up using a Code Block, which not only alleviated the aforementioned memory issue, but also allowed it's completion within 3-4 seconds.

[Anonymous]

We recently had a similar error upon attempting to iterate through a rather large collection. In addition, it was taking a significant amount of time to complete, up to 3 hours! After trying a few alternatives, we ended up using a Code Block, which not only alleviated the aforementioned memory issue, but also allowed it's completion within 3-4 seconds.

[DeeptiGupta1]

Hi Josh we are also facing the similar issue can you please provide some details how you have fixed this

[DeeptiGupta1]

Hi Josh we are also facing the similar issue can you please provide some details how you have fixed this