Memory Management in Long-Running Blue Prism Processes

[SouravSaha]

I was working with some long-running Blue Prism processes recently and noticed that memory usage can increase massively over time, which sometimes impacts performance. And that is where memory management becomes so important.

Here are a few best practices that can be performed for better memory management:

1. Close applications and release system resources when not in use.
2. Use Cleanup stages inside loops to free memory.
3. Avoid keeping unnecessary data in memory.
4. Process data in batches instead of loading everything at once.
5. Use Work Queues for large datasets rather than holding large collections in memory.
6. Load only the required range or rows from Excel files.
7. Split files into manageable chunks to process if possible.
8. Regularly monitor Digital Worker memory usage during UAT and hypercare.
9. Restart workers in scheduled downtime if required.

Would love to know how you handle high memory consumption in long-running automations.

[david.l.morris]

Unless there is a memory leak (which there is not in most versions of Blue Prism), the longer the session runs really should have no impact on memory issues. If the memory is increasing over time, it definitely is something to look into, but that does not sound like a normal issue. We've had sessions run for days straight with no issues. We normally cut it off so it doesn't run that long just in case though. The real problem I've seen is that the session log of a single session being too big can make it impossible to view the session log from inside Blue Prism.

I'll add a few things as well. Note that I think these are only important if you actually run into memory issues. I don't think it's necessary for most automations. I just want to be clear that people reading this stuff should not be doing it all the time. It is not worth the extra effort in complex designs until you actually have a problem or anticipate one based on past issues.

1. In processes and objects, for any large collections, put them on the Main Page/Initialise page and make them global and then re-use those same global collections so that the same collections get overwritten as each new operation is performed on them. This can make readability worse and can make it more likely to have logical issues if you aren't careful, but it is a solid way to reduce memory usage. This is especially important in the Collection Manipulation object and any other object where large collections might be inputted or outputted. You can also then have an action that can be called which specifically clears the collections in the object.

2. Limit the number of object instances that are created by avoiding nested calls to objects if you are having memory issues. To be clear, I'm not saying you should avoid nested object calls all the time; in fact this is a very useful thing to do, but it can be avoided in certain automations.

3. If using surface automation, be careful about how many images get saved into the application model. You would have to do quite a few to cause a memory issue, but it's possible. You can mitigate this by making the images as small as possible. The way I do this is by making the window I'm spying as small as possible first (assuming the application window is resizable), and then I spy the window with surface automation. Of course some apps won't work for this, but I have found that many do. The smaller the image, the smaller amount of memory used by that image when the object is loaded into memory.

4. Finally, there is no reason to have a really long running session. Even just using Scheduler, you can set up the Schedule and tasks in such a way that the automation can run for a while and stop and then start back up again. We use CTWO which makes this super easy. But if you have to only use Scheduler, what you can do is use the Stop After Time input to a process so it doesn't run forever. Let's say you tell it to stop running after 4 hours. Then you set up the task to run every 4.5 hours or something like that. This will cause the memory used to be dumped every 4 hours regardless of how bad the memory management is in the automation.

[greggoodrem]

Great list and some I've not thought about so thank you!

A few that I've come across in the past which I think has been improved with newer versions of Blue Prism and Microsoft UIA Framework is spying in the correct modes. So if you are building on a Browser app obviously try and use Browser mode. It was known in the past that UIA on Browser automations was more memory hungry than using UIA due to it loading and managing each element of a webpage

Stage Logging is kept to a minimum normally Errors Only works well however more complex processes might require Key Stages. (These can be changed at all levels of the process, in the stage, in the object or Process or on the actual resource)

Obviously make sure archiving of your process logs is also happening regularly as this will slow down if you have a large SQL Database.

Also get rid of bloatware such as Xbox, Skype or Solitaire the bots don't get time to socialise or play games!

[greggoodrem]

@david.l.morris I liked the minimising down the windows for surface automation I've added a product idea recently that might save you time if it is ever built.
Redaction Tool - SS&C Blue Prism Community