Control room warning

[BenAnderson]

Hi, 

Whilst executing a schedule overnight, we occasionally come in the next morning to find that a process in the schedule has hung/frozen on a particular stage. The stage is different every time and we are confident that it is the network connection dropping, which is causing the process to hang. Unfortunately this is outside of our control, as it is the client's network. 

Is it possible to programmatically check to see whether a warning status is in the control room, perhaps using the session ID, or some script like Powershell? So for example, if we had a mini process which ran in the background which checked the status once in a while, and if it has hung or frozen then take corrective action, or even send an alert? Please see below for a screenshot:

I should mention that I don't have access to the database, as this is hosted in the cloud by a third party.

Many thanks for your help,

Ben

------------------------------
Benjamin Anderson,
Senior Consultant,
Belfast, UK
------------------------------

[Jean-PhilippeCr]

Hi,

I have a similar problem. My processes will often freeze at "Get Data Table" while doing an SQL Query. Restarting the process fixes it and we cannot pinpoint the issue.

Problem is that if we are not monitoring the control room, the process will stay frozen for hours preventing anything else to start.

I would like to be able to set a treshold where the process would either restart or stop.

Hope you get your answer!

------------------------------
Jean-Philippe Crepeau
Director Consulting Expert
CGI
America/New_York
------------------------------

[BenAnderson]

Hi Jean-Philippe, 

We have the same issue with processes, whereby if we are not constantly monitoring the control room and a process freezes then it will stay like that for hours unfortunately. If you have access to your BP database, then you could possibly write a query to check for "session status = warning". I saw something similar on the forum here. 
Depending on how your objects are built, within the Application Modeller you can set a time out after a period of unresponsiveness or inactivity, which can also help. Unfortunately, our processes are running on an unreliable client network which is out of our control. 

Kind regards,
Ben

------------------------------
Benjamin Anderson,
Senior Consultant,
Belfast, UK
------------------------------

[TetsujiJunicho]

Hi,

You might want to use Application Modeller's Global Timeout feature then you can handle that as an exception within your Process and react accordingly.:
http://portal.blueprism.com/customer-support/support-center#/path/1203344572

Hope this helps.

Regards,

Tetsuji

------------------------------
Tetsuji Junicho
Manager of RPA Practice Grp.
Accenture
Asia/Tokyo
------------------------------

[NicholasZejdlik]

There are two ways I know of to check session results programmatically. One is with the Blue Prism database, specifically in the BPASession table. The other is to connect to the robot directly and check the status - http://<NetworkNameOrIP>:8181/status will give you a list of sessions that the resource has executed, and the results.

If you are having a problem with freezing in the process - that is, the session remains active and the robot is stuck - you will want to check into that. Ideally, processes should never remain stuck, and it can become very difficult to determine if a process is frozen in this manner. I once ran a 24x7 process once that would frequently cause Blue Prism to crash, the "fix" was to have it write a date stamp to file a periodically and run a script in Windows Task Scheduler every few minutes to restart Blue Prism if the date stamp in the file became too old. It technically worked, but it felt very wrong and kludgy. The ultimate fix for us was to revise the entire process and how it operated, which prevented the crashing in the first place. I'd recommend a deep dive into why the process is freezing, and try to correct it. Network disconnects could cause a process to terminate, but it shouldn't be causing it to freeze indefinitely.

The other thing to be cognizant of is that if a network disconnect is occurring and the robot is unable to connect to the database server, then it not be able to report its session status to the BPASession table. I am not sure what the behavior would be in this case, if the robot would update the BPASession table after it comes back online, or if the session would remain stuck in the table. For that reason, it may be best in your case to use the URL method to pull the session status out of the robot.

Edit: Since you don't have access to the database directly, you can use AutomateC.exe with /status and supply a session ID to look up. It appears to use the BPASession table to pull the session status, so it's a way to access it without needing to use the database directly.

------------------------------
Nicholas Zejdlik
RPA Developer
------------------------------

[zdenek.kabatek]

Hi, Ben,

if you have more than one Runtime Resource then you can utilize the other resource to check the first one if it is hanging or not. As Nicholas mentioned there are two ways - AutomateC command line or HTTP request. I just did a short test and if you call HTTP Request with URL like this "http://"&[ResourcePCName]&":"&[Port]&"/user name "&[UserName]&"&password "&[Password]&"&status" you will be given in Result data item the something like this USER SET
USER AUTHENTICATED
RESOURCE UNIT
Total running: 0
I am attaching the xml export of my process so that you can import it and see it. Change the Resource name, port, user name and password and it should run.

Regards,

------------------------------
Zdeněk Kabátek
Head of Professional Services
NEOOPS
http://www.neoops.com/
Europe/Prague
------------------------------

[BenAnderson]

Hi guys, 

Thank you very much for the above suggestions. I will definitely look into each of them. 

@Tetsuji Junicho​ Many thanks. I think we have utilised this across some of our objects, but not all. So I'll be able to rule this out as a cause, by making sure they are all set to time out. 

@Nicholas Zejdlik​ Thank you very much for this. It is definitely something worth exploring, and not something we've thought of previously. 

@Zdeněk Kabátek Thank you for attaching that, I'll definitely have a look and see if it's something we could utilise. 

Thanks again, it's a great help, and will hopefully help to resolve this. 

Kind regards, 

Ben

------------------------------
Benjamin Anderson,
Senior Consultant,
Belfast, UK
------------------------------

[GopalBhaire]

I guess the HTTP commands don't show if the process/resource is in Warning state, it only shows that the process is running.

Also there is no direct status in BP DB for warning, It is calculated from Session table by checking lastupdated & warningthreshold.

------------------------------
Gopal Bhaire
Analyst
Accenture
------------------------------

[BenAnderson]

Hi

@GopalBhaire​ Many thanks. Do you think it would be possible if a schedule is running to utilise the "Terminate the session when unrecoverable logging errors occur" check box on the main page of the process (in the Process Information box) ? At least this way if it does end up hanging for whatever reason, then it would move on to the next process in the schedule and not hold everything up.

Many thanks,

Ben


------------------------------
Benjamin Anderson,
Senior Consultant,
Belfast, UK
------------------------------

[GopalBhaire]

I think it is related to BP logging errors not sure if it will work.

------------------------------
Gopal Bhaire
Analyst
Accenture
------------------------------