Web API Services - System.OutOfMemoryException - File Upload

[PeterLacken]

Hi all,

Is there a known limit to the size of file that can be uploaded using the Web Services API with a Body Content setting of "Single File"?

I am finding it works great for files around 4 MB but I get a OutOfMemoryException for larger files.  I have not found the boundary just yet but so far as low as 14 MB will cause the crash. 

Thanks.


------------------------------
PeterLacken
Ren Røros Intelligent Automation
Peter.Lacken@rria.no
http://www.rria.no
------------------------------

[ErinConnerley]

More than likely this is simply a limitation of the code which is streaming the file (could be a low memory machine... but I honestly doubt that) (could also be an api limitation but then you'd see strict adhering to a strict limit) I believe Web Services API is simply using the Utility - HTTP VBO so you could look in there for how it handles uploads and see if anything could be improved with code efficiency and or if there is any custom code on the particular service you are using you should be able to see ... most of it... from the Web Service itself in edit mode.

------------------------------
Erin Connerley
Automation Consultant
Agilify Automation
America/Indiana/Indianapolis
------------------------------

[ewilson]

@Peter Lacken,

This is an issue with the amount of available memory on the machine where you're running the process. A System.OutOfMemory exception is raised by the OS, not Blue Prism, when insufficient memory is available. This could be related to the actual upload attempt (trying to read the file into memory) or from BP or some other process trying to log information at the time. Below is a link to a KB article on the support site about some steps you can take to troubleshoot this type of issue, but in general I'd say increase the memory on the that machine/VM if possible.

https://portal.blueprism.com/customer-support/support-center#/path/Operational-Support/Performance/Routine-Maintenance/1141009962/How-do-I-avoid-Out-Of-Memory-issues.htm

Cheers,
​

​

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

[PeterLacken]

Hi and Thanks for your replies Erin and Eric,

I have solved the problem by not passing the Binary as a parameter to the REST API as "Single File" but isntead writing a code stage that reads the binary in and performs the POST as a C# code snippet.

The machine I am running on is a top spec Microsoft Server 32 GB memory so doubtful it is a resource problem.   I think it more likely relates to the memory usage within Blue Prism when larger parameters are passed from process to object and then to Web API.  As mentioned these files are not large but much larger than a regular String / Double.  It is unfortunate I have to perform such a work around and Single File REST API is limited to the type of solution I can use it for but also great I can extend Blue Prism wiht C# to find a working solution.

------------------------------
PeterLacken
Ren Røros Intelligent Automation
Peter.Lacken@rria.no
http://www.rria.no
------------------------------

[AndreyKudinov]

There seems to be two issues here:
1) Blueprism is 32 bit and inherently has OOM issues even on moderate data sets. 
2) WebAPI is trying to fit everything into memory, possibly encoding binary to base64 in the process, which is far from ideal. ​

I never bothered to use webapi. http vbo is enough.

------------------------------
Andrey Kudinov
Project Manager
MobileTelesystems PJSC
Europe/Moscow
------------------------------

[ManishRawat]

Hi Peter,

Can you share the C# code snippet used to do the Job? I am facing a similar kind of issue while sending files using API call.

Thanks in advance!!

------------------------------
Manish Rawat
Project Manager
Mercer
New Delhi
------------------------------