Scheduler error: Resource XXXX:8001 is offline. But it's not in control room.

[RuneHedegaard]

Hello We got som problems with Scheduler. For some reason it random get stuck after login, and says "Resource UDCBLUERUN7:8001 is offline - retried 3 times" We created a simple process to check resolution from a object utility to test the problem. Below the logs from Recent Activity from control room. I have made some comments to the logs below.   Schedule Completed Z Test  - DIRXLYA (Citrix 7) 2018-05-02 15:10:00Z 2018-05-02 13:09:59Z 2018-05-02 13:11:12Z  Task Completed Login DIRXLYA  2018-05-02 13:09:59Z 2018-05-02 13:10:04Z  Session Completed abbd1ea2-8fad-4102-8601-a34179c7aa7a  2018-05-02 13:10:01Z 2018-05-02 13:10:04Z  Task Completed Bent - Check resolution  2018-05-02 13:10:04Z 2018-05-02 13:11:09Z  Session Completed 53b7b66c-43c7-4939-b167-91e3a560fe4e  2018-05-02 13:11:05Z 2018-05-02 13:11:09Z  Task Completed Logout  2018-05-02 13:11:09Z 2018-05-02 13:11:12Z  Session Completed 1748a191-0f43-408b-bbe4-167188bce5d6  2018-05-02 13:11:10Z 2018-05-02 13:11:12Z         Schedule Terminated Z Test  - DIRXLYA (Citrix 7) 2018-05-02 15:15:00Z 2018-05-02 13:14:59Z 2018-05-02 13:21:04Z  Task Completed Login DIRXLYA  2018-05-02 13:14:59Z 2018-05-02 13:15:03Z  Session Completed 96a2d462-2146-459e-9890-11ea5bed3c18  2018-05-02 13:15:00Z 2018-05-02 13:15:03Z  Task Terminated Bent - Check resolution  2018-05-02 13:15:03Z 2018-05-02 13:18:03Z Task: Bent - Check resolution threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Task Terminated Logout  2018-05-02 13:18:03Z 2018-05-02 13:21:04Z Task: Logout threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times               Schedule Terminated Z Test  - DIRXLYA (Citrix 7) 2018-05-02 15:25:00Z 2018-05-02 13:24:59Z 2018-05-02 13:31:04Z  Task Completed Login DIRXLYA  2018-05-02 13:24:59Z 2018-05-02 13:25:04Z  Session Completed f236b260-0e8d-4439-bf13-9529be7ed871  2018-05-02 13:25:00Z 2018-05-02 13:25:04Z  Task Terminated Bent - Check resolution  2018-05-02 13:25:04Z 2018-05-02 13:28:04Z Task: Bent - Check resolution threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Task Terminated Logout  2018-05-02 13:28:04Z 2018-05-02 13:31:04Z Task: Logout threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Schedule Terminated Z Test  - DIRXLYA (Citrix 7) 2018-05-02 15:35:00Z 2018-05-02 13:34:59Z 2018-05-02 13:41:04Z  Task Completed Login DIRXLYA  2018-05-02 13:34:59Z 2018-05-02 13:35:04Z  Session Completed 47410358-44a3-4a49-b0bf-604eec5e23c2  2018-05-02 13:35:01Z 2018-05-02 13:35:04Z  Task Terminated Bent - Check resolution  2018-05-02 13:35:04Z 2018-05-02 13:38:04Z Task: Bent - Check resolution threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Task Terminated Logout  2018-05-02 13:38:04Z 2018-05-02 13:41:04Z Task: Logout threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times        Comment: We did manuelle run logout (by drag and drop logout process to the resource UDCBLUERUN7:8001 in the control room and waited for det next schedule. Se below       Schedule Completed Z Test  - DIRXLYA (Citrix 7) 2018-05-02 15:45:00Z 2018-05-02 13:45:00Z 2018-05-02 13:46:10Z  Task Completed Login DIRXLYA  2018-05-02 13:45:00Z 2018-05-02 13:45:03Z  Session Completed a0cf5ed3-6d23-4d57-846b-4137163106f3  2018-05-02 13:45:00Z 2018-05-02 13:45:03Z  Task Completed Bent - Check resolution  2018-05-02 13:45:03Z 2018-05-02 13:46:06Z  Session Completed 703fad23-8dd6-4544-9349-33fbc89aa5db  2018-05-02 13:46:04Z 2018-05-02 13:46:06Z  Task Completed Logout  2018-05-02 13:46:06Z 2018-05-02 13:46:10Z  Session Completed c1aa4568-edde-4746-bbb1-a4632e3ff669  2018-05-02 13:46:07Z 2018-05-02 13:46:10Z         Comment: The first run did work fine. But the 2 schedule failed again. But for some reason the logout did not fail. (se below)       Schedule Completed Z Test  - DIRXLYA (Citrix 7) 2018-05-02 15:50:00Z 2018-05-02 13:50:00Z 2018-05-02 13:57:09Z  Task Completed Login DIRXLYA  2018-05-02 13:50:00Z 2018-05-02 13:50:04Z  Session Completed ce0a34fd-0831-4674-a58e-f472f2e16a03  2018-05-02 13:50:01Z 2018-05-02 13:50:04Z  Task Terminated Bent - Check resolution  2018-05-02 13:50:04Z 2018-05-02 13:53:04Z Task: Bent - Check resolution threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Task Completed Logout  2018-05-02 13:53:04Z 2018-05-02 13:57:09Z  Session Completed 2be860d5-6972-43ea-ab84-05adb08cd0c1  2018-05-02 13:57:05Z 2018-05-02 13:57:09Z                Schedule Completed Z Test  - DIRXLYA (Citrix 7) 2018-05-02 16:00:00Z 2018-05-02 14:00:00Z 2018-05-02 14:01:13Z  Task Completed Login DIRXLYA  2018-05-02 14:00:00Z 2018-05-02 14:00:04Z  Session Completed 20399468-456b-487c-8207-8965940fb63c  2018-05-02 14:00:01Z 2018-05-02 14:00:04Z  Task Completed Bent - Check resolution  2018-05-02 14:00:04Z 2018-05-02 14:01:09Z  Session Completed 822b824b-66ef-4be4-897e-19ec803584a4  2018-05-02 14:01:05Z 2018-05-02 14:01:09Z  Task Completed Logout  2018-05-02 14:01:09Z 2018-05-02 14:01:13Z  Session Completed ddfd9075-9ab3-4f10-a91b-79c3844aefbb  2018-05-02 14:01:10Z 2018-05-02 14:01:13Z         Schedule Completed Z Test  - DIRXLYA (Citrix 7) 2018-05-02 16:05:00Z 2018-05-02 14:05:00Z 2018-05-02 14:09:07Z  Task Completed Login DIRXLYA  2018-05-02 14:05:00Z 2018-05-02 14:05:03Z  Session Completed aeb58cff-f34b-4b59-8dcb-51d2a9752932  2018-05-02 14:05:00Z 2018-05-02 14:05:03Z  Task Terminated Bent - Check resolution  2018-05-02 14:05:03Z 2018-05-02 14:08:04Z Task: Bent - Check resolution threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Task Completed Logout  2018-05-02 14:08:04Z 2018-05-02 14:09:07Z  Session Completed 46a152fb-3888-4b8f-8e37-90290e5c70e3  2018-05-02 14:09:04Z 2018-05-02 14:09:07Z         Schedule Completed Z Test  - DIRXLYA (Citrix 7) 2018-05-02 16:10:00Z 2018-05-02 14:09:59Z 2018-05-02 14:14:06Z  Task Completed Login DIRXLYA  2018-05-02 14:09:59Z 2018-05-02 14:10:04Z  Session Completed efd20bd2-1296-41bd-b3c5-1fee4f60f104  2018-05-02 14:10:00Z 2018-05-02 14:10:04Z  Task Terminated Bent - Check resolution  2018-05-02 14:10:04Z 2018-05-02 14:13:04Z Task: Bent - Check resolution threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Task Completed Logout  2018-05-02 14:13:04Z 2018-05-02 14:14:06Z  Session Completed 792886f4-28ee-46f7-b1d6-fbf8e108c3ce  2018-05-02 14:14:04Z 2018-05-02 14:14:06Z         Schedule Completed Z Test  - DIRXLYA (Citrix 7) 2018-05-02 16:15:00Z 2018-05-02 14:14:59Z 2018-05-02 14:19:08Z  Task Completed Login DIRXLYA  2018-05-02 14:14:59Z 2018-05-02 14:15:04Z  Session Completed 842e1616-658e-4fb6-baf5-84b22104734a  2018-05-02 14:15:01Z 2018-05-02 14:15:04Z  Task Terminated Bent - Check resolution  2018-05-02 14:15:04Z 2018-05-02 14:18:04Z Task: Bent - Check resolution threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Task Completed Logout  2018-05-02 14:18:04Z 2018-05-02 14:19:08Z  Session Completed 721e4968-33a1-46a6-b283-c9e69d791c5b  2018-05-02 14:19:05Z 2018-05-02 14:19:08Z         Schedule Completed Z Test  - DIRXLYA (Citrix 7) 2018-05-02 16:20:00Z 2018-05-02 14:19:59Z 2018-05-02 14:27:07Z  Task Completed Login DIRXLYA  2018-05-02 14:19:59Z 2018-05-02 14:20:03Z  Session Completed 51571d7c-6c0b-4969-9ffe-d4e86cd6244f  2018-05-02 14:20:00Z 2018-05-02 14:20:03Z  Task Terminated Bent - Check resolution  2018-05-02 14:20:03Z 2018-05-02 14:23:03Z Task: Bent - Check resolution threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Task Completed Logout  2018-05-02 14:23:03Z 2018-05-02 14:27:07Z  Session Completed c57c7f86-6089-402d-8f10-a5ea353850d6  2018-05-02 14:27:04Z 2018-05-02 14:27:07Z         Comment: For som reason the logout task also failed after some tryes. Se below       Schedule Terminated Z Test  - DIRXLYA (Citrix 7) 2018-05-02 16:30:00Z 2018-05-02 14:29:59Z 2018-05-02 14:38:03Z  Task Completed Login DIRXLYA  2018-05-02 14:29:59Z 2018-05-02 14:30:03Z  Session Completed 99e761e6-d1d0-45b6-b668-9210f7840c5c  2018-05-02 14:30:00Z 2018-05-02 14:30:03Z  Task Terminated Bent - Check resolution  2018-05-02 14:30:03Z 2018-05-02 14:33:03Z Task: Bent - Check resolution threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Task Terminated Logout  2018-05-02 14:33:03Z 2018-05-02 14:38:03Z Task: Logout threw an exception: Resource UDCBLUERUN7:8001 is offline - retried 3 times Comment: Now the login also fails, because of the logout problem. Se blow Schedule Terminated Z Test  - DIRXLYA (Citrix 7)2018-05-02 16:40:00Z2018-05-02 14:39:59Z2018-05-02 14:43:00Z Task Terminated Login DIRXLYA2018-05-02 14:39:59Z2018-05-02 14:43:00Z Task: Login DIRXLYA threw an exception: Resource UDCBLUERUN7 is offline - retried 3 times Schedule Terminated Z Test  - DIRXLYA (Citrix 7)2018-05-02 16:45:00Z2018-05-02 14:44:59Z2018-05-02 14:48:00Z Task Terminated Login DIRXLYA2018-05-02 14:44:59Z2018-05-02 14:48:00Z Task: Login DIRXLYA threw an exception: Resource UDCBLUERUN7 is offline - retried 3 times

[PetriKaukua]

Hello, We have bee investigating this issue couple of months already and the cause seems to be Automate.exe is issuing hard TCP reset to any and all connections initiated by the Scheduler service. Since Blue Prism client (doesn't matter where the client is started, application server or from IC) is a different connection than the scheduler service, the connection is being accepted by the ResourcePC and you are being able to run any and all processes. If you issue RunNow for a schedule then suddenly you get error ""Resource is offline tried x times"" even though you can see that the resource is online, and you can connect it with telnet for example. We have been able to verify this with Appinternals (attached directly .NET components) software in conjunction with PaloAlto and Riverbed monitoring and we have provided Blue Prism support a lot of evidence that it is the Blue Prism itself that causes these disconnections but currently our support tickets are heading nowhere since none of the evidence is being treated seriously and either useless knowledgebase links are provided or completely non-relevant questions are being asked. Perhaps if can get some noise in here too someone at Blue Prism will take this matter seriously.

[SamanthaShaw]

Thank you for your posts.  Please be aware that these forums are not routinely monitored by Support staff, and are primarily here for the user community to interact with each other. Anyone experiencing issues with the software that are suspected to be problems with the software itself, should be raised with Support. Currently all such issues that have already been raised are actively being worked on.  There can be more than one cause for apparently identical errors, especially around connectivity related issues. Customers wishing to escalate any ticket already raised, should refer to the Escalation Procedure on our Customer Support section of the Portal: https://portal.blueprism.com/customer-support/how-get-help-customer-support

[SamanthaShaw]

Just a small footnote - one thing I have noticed in common with some customers experiencing this issue, is that they have combined the Login process in a Schedule alongside other tasks. As the Login process involves establishing a connection with the Login Agent resource, asking it to log the machine in, and then closing this resource down, and then bringing online the normal resource instance before it is in a state to successfully run any other process, it is not best practice to place all these tasks in a single Schedule unless some kind of appropriate length gap is left between the Login and the process to be run, and also suitable checks are made to ensure the normal resource is online and ready to run normal processes. For example, the original poster shows that it is often the task after the Login which fails - this could be because not enough time has been left to ensure the normal resource is online - hence it reporting an offline problem. It is also worth bearing in mind that frequent logging in and out of a resource, relies on this working perfectly every time.  Unless there is a need to switch user, we would suggest it might be better to set up a Schedule just to log all resources in, and then rather than logging in and out between process runs, to alternatively lock and unlock the screen, which still ensures they are password secured when not in use, but a lock and unlock will occur much faster than a login and a logout, and there is less risk of the machine being left in the wrong state.

[TammaAditya1]

Hi all,   Facing the same problem here. Would like to know the latest progress if any, and will keep tracking this thread too.   Thanks to @Petri_Kaukau for informing me. I've also posted something about this here. https://portal.blueprism.com/forums/general-discussion/general-discussi…

[PetriKaukua]

@Sam Fidler: Separating schedules doesn't solve anything since it is either the login agent itself or the resource pc that is offline according to the scheduler when in fact they are both online and connectivity can be proven. Separating schedules just generates the double amount of error messages and nothing else.   Br, Petri Kaukua

[FredrikAdland]

I have the same issue. BP version: 6.2.0.2669 w/ corresponding Login Agent version. Connection type: WCF. Happens with Login as the only task in the Login-Schedule. Creating a ticket now, but I'm posting here as well to give the issue more visibility. Picture attached shows how the error suddenly happens, that the resource is connected and that it fails on the Login Schedule (where Login is the only task).

[KK1]

Hello, is there any update on how to solve this issue? We are facing the same problem. Thanks for any help

[NarendraMundwad]

Hi,   Even we are facing same issues. As the scheduler doesn't work as expected, the whole point of having a scheduler goes for a toss. Someone then needs to manually start the process which is not ideal in non working hours when automated processes work 24hours.   Any update please? Thanks, Narendra

[BjörnGranacher1]

Hi we are experiencing the same problem and have just raised a support request. It has taken much time already + external consulting cost, with no final outcome yet. If anyone was able to solve it it would by highly appreciated if posted here. In our case we have tried various setup to isolate the problem. We use BP 6.2 with .NET Remoting Secure still. Our test schedule is to login (schedule #1) and logout (schedule #2) in endless cycle of 5 or 10 minutes. It works without any issues on our DEV environment. The same RR reconnected to QA or PROD has sporadic failing schedule starts (either the Login or Logout), sometimes it works 3,4, 5 times in a row, then we have the ""offline"" error again.   PROD has most load / schedules. QA has 3 processes running for UAT. DEV has none, only the test of Login and Logout. I hope the implementation is not as such that such a small load will make it fail.    

[Anonymous]

Hi Experts, This is what i am getting, termination reason -> ""Resouce XXX was unavailable : The state given as : InUse"" Note : There are no pending processes , doing it on the local machine Please  help, thanks in advanced.