Performance Issues in Blue Prism

[RaimondvandeSteeg]

Hello Community,

We are currently experiencing severe performance issues in our Blue Prism environments and hope someone here has encountered similar challenges and can share a solution. The issues are as follows:

Performance Issues in the Control Room:

• Dragging processes is extremely slow.
• Input fields (e.g., when starting a process) load very slowly.
• Callback messages are not received, leading to pop-ups.
• Logs cannot be accessed.

Problems with Processes and Objects:

• Opening and saving processes and objects takes an excessively long time.

Difference Between Test and Production Environments:

• Our test environment is approximately 10 times slower than production, while production itself is barely usable. This makes working in the test environment particularly challenging.

Actions Taken So Far:

• Tables in the database have been cleared, including the BPASESSIONLOG table.
• We are now retaining only the last 14 days of data in the BPASESSIONLOG table, but this has had no noticeable effect.
• Housekeeping scripts have been run as recommended by Blue Prism, but without any improvement.
• We contacted Blue Prism support, and they advised continuing to use the standard housekeeping scripts.

Question to the Community:
Has anyone experienced similar performance issues, particularly in the Control Room or when opening and saving processes and objects? If so, what steps or solutions have been effective? Are there specific optimizations in configuration, database, or infrastructure that we might be overlooking? Additionally, does anyone have tips for addressing the slowness in both the test and production environments?

Any help or suggestions would be greatly appreciated. Thank you in advance!

[Brigianakopec]

Could you please provide your BP version?

Have you tried using App Inspector?

https://digitalexchange.blueprism.com/dx/entry/3439/solution/app-inspector-xbp

[RaimondvandeSteeg]

Thank you for your suggestion.

We haven’t used the App Inspector yet, but we’ll take a closer look and evaluate the results.

Currently, we’re running Blue Prism version 7.3.0.

Thank you again for your advice!

[david.l.morris]

We have had some significant slowness issues, but that was with 6.10.1. We also ran the App Inspector recently and started doing more database maintenance than before. I do think this can have an impact, but it really makes no sense whatsoever that this would affect local behavior of Blue Prism. I see that the items you listed sound like they all involve network communication, so it is quite possible for you that it's a database issue. That could be that one or more of your tables have grown too huge, which App Inspector can help you identify. But you should also check on the database server to see how hard it is working. Maybe the same for the app server. Another thing to consider trying is to see how Blue Prism functions with a Direct Database connection. That may be a temporary solution for you in your Test environment at least. Note that with a Direct Database connection, you cannot retrieve or update credentials from that connection (if the encryption key is stored on the app servers), but it should work just fine for using Control Room and other things that don't require using credentials.

However, our environments (all of them) suddenly got way faster with seemingly no changes made. What I think was happening for us was a security program (or programs) that were slowing down the Blue Prism process (Windows process) based on what it thought Blue Prism was doing. This was for both network traffic and local activity too.

What I would check is to see what processes jump to the top of CPU % in Task Manager, and then ask your security department to see if there might be something interfering with Blue Prism. I should mention that I have absolutely no idea what was changed on our end, but the processes that used to jump to the top of our Task Manager CPU % list suddenly no longer show up for us, so I think our security department disabled them, made them less aggressive, or changed programs.

[Jrwork]

Hello, @RaimondvandeSteeg 
I hope you're doing well!

We went through similar situations, and I will detail our problem and how we improved:

Our servers were very slow, opening a process, running a robot, opening recent activities... it took a long time! Also, many times the connection would drop. Due to all these issues, we conducted a thorough analysis and here are the improvement points:

• Our environment is on Google Cloud, so we did an analysis with the network team regarding the connection ports between the servers and the BP server. The team made a release that we thought was already done, and after that, the action time on BP changed from minutes to seconds.

• We upgraded the memory and CPU on the BP server. We added 8GB of memory to the BP server (note: 8GB of memory, but it could be 4, 12, etc.). You will determine the appropriate amount based on performance analysis in your environment, even if it's on-premises.

• We upgraded our production machines, improving server resources by 50%, and this had an immediate and highly positive effect.

• We stored the BP log tables (check in the Analytics tab).

• We disabled general logging and kept only error logs for the robots (this is the default mode, but there were robots with logging enabled).

• A crucial factor: we used the dev server to view the prod VMs. As a result, there was extreme slowness due to internal blockages. We then opened ports to facilitate the connection of a VM in the dev environment with visibility into the prod environment. This was done by the IT team.

Our version is 7.3, and it's very fast now. If you want more details, I am available for any discussions.