Best Practice for Handling Nested Work Queues in Blue Prism

[Omar-Abu-Snineh]

Hello everyone,

I have a Blue Prism process related to the Recruitment and HR department. The process handles multiple recruits/employees, and each recruit has several related tasks.

A senior colleague suggested using two work queues:

1. Parent Queue
   This queue contains the recruit information.

2. Child Queue
   This queue contains the task information related to each recruit.

For example, if I have 5 recruits, and each recruit has 15 tasks, then the process will work with:

• 5 items in the Parent Queue

• 5 × 15 = 75 items in the Child Queue

The loop in the Main Page is currently designed as a nested loop structure, where the process first loops through the Parent Queue items, and then inside each parent item it loops through the related Child Queue items.

Currently, the item keys are structured as follows:

• Parent Queue item key:
  Full Recruit Name - Employee Number

• Child Queue item key:
  Full Recruit Name - Task Name

I managed to build a working solution by directly using actions from the Work Queue object, the Collection object, and many loops. However, this made the process much more complex than expected. The logic became spread across many pages, and even the Main Page became quite complicated.

After a lot of problem-solving and complex logic, the process is now working correctly. However, I would like to ask for advice in case I face a similar process in the future and need to use a nested queue structure again.

Is there any recommended way to link the Parent Queue and Child Queue more cleanly in Blue Prism?

For example:

• Is there a best-practice methodology for handling parent-child queue relationships?

• Is there a better way to design the item keys or tags?

• Should I store a parent reference inside the child queue item data?

• Is there a recommended queue design pattern that can reduce the number of loops and Work Queue actions?

• Would it be better to avoid nested queues completely and use another approach?

I would appreciate any advice, best practices, or design patterns that could help simplify this type of solution in the future, because I would prefer not to go through the same level of complexity again.

Thank you.

[greggoodrem]

Hi Omar,
Interesting question, I think best practice in your case would mainly rely on the dependancy of the 15 sub tasks, what are there relationship to one and other? Would it be daisy chain approach or separate processing?
So is a particular order in which to process these tasks should undertake? Or if they can all be performed simulataneously?

I would generally go with a simple approach in the loader/collector phase of the process and get all the information ready into the selected queue structure for the executor/fulfilment phase of the process. 

One thing to consider is your reporting needs from all the tasks being done or is this the parent element task to keep track of what work items have been completed?

• Is there a best-practice methodology for handling parent-child queue relationships?
  There is an Advanced Work Queues course within the Professional Developer Learning Plan you might be interested in and could answer a lot of these questions. My thoughts are below:

• Is there a better way to design the item keys or tags?
  Depending on the approach you can be quite creative with the item keys I always have a field called Item Key in the Item Data and this is set within a loop prior to the add to queue action and also checks for duplicate items already processed. One of the basics is to make your item key unique however without having any identifiable information in the item key. Remember that the item key is the only data visible even if the work queue is encrypted.

• Should I store a parent reference inside the child queue item data?
  It would make sense to reference in someway the parent like a candidate reference number and then in the child queues having the different steps as a dash number or letter e.g. Parent = 321123 and in the Child Queues could be 321123-A, 321123-B etc

• Is there a recommended queue design pattern that can reduce the number of loops and Work Queue actions?
  Reducing the loops could be as simple and basic as having 15 separate queues populated as soon as the Parent queue item is created (I'm guessing this item will track the progress of the 15 tasks?) then you have your VMs work through the different tasks and when a report is produced it informs what tasks have and havent been completed of the 15 tasks.

• Would it be better to avoid nested queues completely and use another approach?The best solution will always be what is best for that particular process. Its never a one shoe size fits all when it comes to design. Please let us know what route you went down.

[Omar-Abu-Snineh]

Hi Greg,

Thank you very much for your detailed answer and helpful suggestions. I really appreciate it.

Your advice about Item Keys, Parent references, and the different design options was very useful. I'll keep them in mind for future processes.

I also plan to take the Advanced Work Queues course.

Thanks again!

[stepher]

@Omar-Abu-Snineh ,

As @greggoodrem mentions, there are almost always multiple ways to address a challenge, and it feels like I have tried most of them at one time or another.

That said, my general build approach is: (1) one Process dedicated to loading the Blue Prism work queue, (2) one or more Processes that perform the work, and (3) one Process to report results and clean up/reset for the next cycle.

With this in mind, I would likely have only a single work queue (based on the employee/candidate), but break each of your “tasks” into its own component Process. For example, validating right-to-work and validating professional certifications would be separate processes. With fifteen defined tasks, the division between processes seems like it would be clear. You could combine sequential steps in the same target system, but I prefer separating by task.

In order to control the flow of the work units from Process to Process, I use Tags to gate records. At the end of each component Process, it will update the item’s Tag to indicate what was completed or what should happen next. For example, when the loader adds a record, tag it “01 – Item Loaded” (or just “01”). Each downstream process looks for Pending items with its expected Tag. So, the second process might create the shell record in your HCM system: it picks up Pending items tagged “01,” does the work, then replaces “01” with “02 – Record Created” (or “02”) and releases the item back to the queue for the next process. If processing fails, either set the item to Exception to prevent pickup, or change the Tag to a value that routes it to an alternate process.

Caveat 1: We are using Blue Prism Enterprise v7.3.2. I emphasize Enterprise because it supports filtering Work Queue items by Tag value; my understanding is WorkHQ (formerly Blue Prism NextGen) does not.

Caveat 2: Our organization uses C TWO for scheduling. The approach below should work with Blue Prism scheduling, but it will likely be less efficient (for reasons explained at the end).  C TWO allows us to configure each Process ("Task") to start when one or more Pending items exist with a specific Tag. Legacy Blue Prism scheduling did not support Tag-based triggering, so you must run on a fixed schedule and have the Process exit gracefully when no items are available—functional, but less efficient.

There are definitely Pros and Cons to this approach, but it is what I have gotten comfortable doing.  At the end of the day, if what you developed works, then you go with it.  Funny enough, the project I am deploying today uses a parent/child work queue set up, much like yours, but it is based on the header and line information for a document.  It is cumbersome, but I could not think of a better way.  I think the key is that I have 'work' occurring at both the header and line that needs to be managed.  If everything was line-level, I could probably go with a single work queue.  

I am happy to hear counter arguments.

Good luck,

Red