Need help in Optimizing Our Two-Set Process Flow

[Ezio..]

Hey Hi Community,

Hope you can help me optimize a process flow that I am currently working on

We are working on a project with two distinct work queues: Queue 1 and Queue 2. Queue 2 is dependent on the completion of cases in Queue 1 for the majority of requests, though not all. To clarify the setup:

• Queue 1 handles account creation requests via RPA bot in an portal.
• Queue 2 manages document uploads (ranging from 1 to 50 documents per account) in the same portal.

Due to infrastructure constraints, each document upload must be submitted as a separate request hence multiple request in Queue 2. This means one account creation in Queue 1 can correspond to up to 50 document upload requests in Queue 2. Document uploads can only proceed once the account is successfully created.

We have divided our bots into two groups:

• Group 1 (10 bots) focuses on account creation (Queue 1)
• Group 2 (30 bots) handles document uploads (Queue 2)

Additionally, there are cases where accounts are created manually by users, and only document uploads are needed.

So in these two cases, bots check if the account creation request exists and completed in Queue 1. If it does, they proceed with document uploads; if not completed , then they defer document upload requests for one hour before retrying. If no account creation request exists, then bots proceed directly with document uploads.

Our bots received request between 20-hour window daily. On average, Queue 1 processes about 500 cases per day, while Queue 2 handles around 3,000 cases.

Apart from this both type of cases take around 10 minutes , really difficult to reduce it

Currently, we run both bot groups every 20 minutes, which has led to several challenges:

• Frequent spillovers and delays in document upload processing
• High license utilization (around 40-44 licenses)
• Difficulties in managing and maintaining the bots
• Sudden spikes in work queue volume (e.g., from 30 to 2,000 requests)
• queues work on "First come, first served" basis , if we recived 4 doc for particular account at 3 pm and 10 cases at 7 pm , bots will only process few 4 cases first and then process othere cases and pick up remaining 10
•  

Given this complexity, I would greatly appreciate any insights or recommendations on how to streamline this process. or any strategies, such as using active queues,dynamic resource allocation or other optimizations, that could help us improve efficiency, reduce maintenance overhead, and better manage workload spikes?

Thank you in advance community

[greggoodrem]

Although Complex, to me it seems like you could simplify it to have just 1 queue with both the initial account creation and create a nested Collection within your data item to then process the uploads after the initial account creation has been completed. You can change the status of the Item from say Pending to Account Created and then finally Documents uploaded. You can take it from End to End in  the same process and use all your bots without delay. 

[Ezio..]

Thank you for the suggestion @greggoodrem . We were also initially considering this same approach.

However, I believe it might be a bit challenging to manage because the main concern is that both requests do not arrive simultaneously. For example, we often receive account creation request first, and corresponding documents may arrive a few hours later.

Additionally, there are several other scenarios to consider:

• Sometimes, only the account needs to be created without any document upload activity.
• In other cases, the account may already exist, and new documents need to be uploaded against that account.
• And again , main Use case,  where new account needs to be created along with the document upload activity.

Due to these various factors, we have opted to use two separate queues to handle these processes more effectively.

[greggoodrem]

@Ezio.. I see your predicament you could set a deferral if the documents are not present after account creation and they get picked up later if you need a couple of hours.
If there are new documents to be added to your queue you would just mark them with a status to bypass the account creation. Really depends on your process build structure.
Might be worth looking at your process with the end in mind and working back from there, ultimately the bots seem to be doing the work right but the order of work is the issue right?! So if you prioritise that then at least you will see what capacity your workers. I suppose the only other issue is looking at reducing the time per item worked.