Control Room suddenly useless - Can start any process on any machine whether it's logged in or not - obvious error and swiftly terminates

BenKirimlidis · ‎06-04-19

Hi, In BP Control room, whole system became suddenly useless today. We have 10 resources, have multiple schedules that run and occaisionally kick off process manually when required. Suddenly today, control room stopped workign normally. Now all process permanently show as logged out. -Using LoginAgent I can login to any resource however, while the session marks itself as a success and completes. The machine is NOT logged into. -Can do this over and over, always a success but the machines status never changes. -Likewise I can manually kick off processes on any machine. Normally, if the machine is not logged into, the process will always terminate immeadiately as it's not on the windows home screen. However now, all process will allow the process to "start". This morning I could not run a process on a machine that wasn't logged into. Then process fails because obviously the machine is not logged into. Have tried - Rebooting all the machines, problem persists - With IT, remoting into resources. This allows the BP listern to start workign normally so i can run processes. But as soon as the machine logs out. The same behavior outlined abvoe resumed. - With the machine remoted into, terminating the BP Listener service and restartign it, then logging out. Does not work. Only way we can currently run our BAU is to have a PC remote int othe Run Time Resource so that the Listener starts working, run processes manually while moving the mouse. Naturally this is completely unacceptable and cannot continue. Been trying to solve the problem all day and have gotten nowhere. Open to any and all suggestions. Thanks in advance.

AmiBarrett · ‎27-04-20

@DavidVizcaíno - Below is the latest version I have, which includes a session check, as you found might be needed. The 20 windows are expected, unfortunately. In normal cases, you might be able to get away with only using the first one or two tsdiscon lines, though.

The only way I know of to launch something hidden, would be via additional code. For example, VB (also below). You could make a .vbs file to launch the bat file, then tie that into your task. I haven't tested it on this particular bat file, so I don't know if the child windows would also be hidden. "@ECHO off" only specifies that it shouldn't output text to the cmd window.

Have you checked that the task is running as administrator? It's required to use tsdiscon.

Dim startInfo As New ProcessStartInfo("unlockme.bat")
startInfo.WindowStyle = ProcessWindowStyle.Hidden

Process.Start(startInfo)

@ECHO off

setlocal EnableDelayedExpansion



for /f "tokens=2,4" %%a in ('QUERY USER ^| FINDSTR ">"') DO (

 set "consoleTemp=%%a"

 set "connectionState=%%b"

 set "consoleType=!consoleTemp:~0,7!"

)



SET consoleTypeCorrect=0

IF "!consoleType!"=="rdp-tcp" SET consoleTypeCorrect=1



SET connectionStateCorrect=1

IF %connectionState%==Active SET connectionStateCorrect=1



set TRUE=%connectionStateCorrect%%consoleTypeCorrect%



IF %TRUE%==11 (

 endlocal

) else (

 start "" tscon 1 /dest:console

 start "" tscon 2 /dest:console

 start "" tscon 3 /dest:console

 start "" tscon 4 /dest:console

 start "" tscon 5 /dest:console

 start "" tscon 6 /dest:console

 start "" tscon 7 /dest:console

 start "" tscon 8 /dest:console

 start "" tscon 9 /dest:console

 start "" tscon 10 /dest:console

 start "" tscon 11 /dest:console

 start "" tscon 12 /dest:console

 start "" tscon 13 /dest:console

 start "" tscon 14 /dest:console

 start "" tscon 15 /dest:console

 start "" tscon 16 /dest:console

 start "" tscon 17 /dest:console

 start "" tscon 18 /dest:console

 start "" tscon 19 /dest:console

 start "" tscon 20 /dest:console

)

------------------------------
Ami Barrett
Sr Product Consultant
Blue Prism
Plano, TX
------------------------------

AmiBarrett · ‎27-04-20

@RaúlHernández - This is specifically if there is no other method setup to remotely connect to the resource(s) than RDP. Ideally, you'd use something like VNC or TeamViewer.

------------------------------
Ami Barrett
Sr Product Consultant
Blue Prism
Plano, TX
------------------------------

RaúlHernández · ‎27-04-20

Hi, Ami.

I understand. So, there is no login agent working here.

Best regards.

------Original Message------

@RaúlHernández - This is specifically if there is no other method setup to remotely connect to the resource(s) than RDP. Ideally, you'd use something like VNC or TeamViewer.

------------------------------
Ami Barrett
Sr Product Consultant
Blue Prism
Plano, TX
------------------------------

AmiBarrett · ‎27-04-20

The RDP logon would be for manual intervention, not automatic login.

------------------------------
Ami Barrett
Sr Product Consultant
Blue Prism
Plano, TX
------------------------------

RaúlHernández · ‎27-04-20

Ok, So, is this situation solved now?

Best regards.

------Original Message------

The RDP logon would be for manual intervention, not automatic login.

------------------------------
Ami Barrett
Sr Product Consultant
Blue Prism
Plano, TX
------------------------------

DavidVizcaíno · ‎27-04-20

@Ami Barrett

So by now, the .bat shows the windows, right? Does the .bat stop launching cmd windows depending on which tscon line got the correct connection? Could you please explain in a more technical way how the .bat actually does the trick?

I've tried your code example with VB with no luck, it throws error 800a0401. I tried with this and the windows still popup.

command = "powershell.exe -nologo -command unlockMe.bat"

set shell = CreateObject("WScript.Shell")

shell.Run command,0

Set oShell = CreateObject ("Wscript.Shell") 
Dim strArgs
strArgs = "cmd /c unlockMe.bat"
oShell.Run strArgs, 0, false

CreateObject("Wscript.Shell").Run "unlockMe.bat", 0, True

I'm running the task with elevated rights.

Question: When you tried this .bat, the windows flashing up didn't represent a problem??

------------------------------
David Vizcaíno
RPA Programmer
EY
America/Bogota
------------------------------

AmiBarrett · ‎27-04-20

The script is meant as a shotgun-blast approach. Even if it hits the correct session number, it will still continue through. 20 was an extreme case that I've seen useful in exactly one deployment. Pretty much every other time only required the first one or two sessions before it had a successful flip.

I was messing around with the VB solution, and was not able to get it to hide the child windows as-is. I found a way to make it hide the child windows in powershell, but it won't hide the main window.

This got me poking around it some more. When I had first built this bat file against some Server 2012r2 VMs, there was a technical need to spawn each tsdiscon in a new window. Seems something's changed along the way, so you can remove the "start """ from each line and contain it into a single window. So yay, no more child windows to mess with. I was then able to fix the VBS file to get the desired effect.

PowerShell:

PowerShell.exe -windowstyle hidden { "-command unlockMe.bat" }

VB:

1) Create file: unlockme.vbs (or whatever you want to call it), and modify the path to the bat file:

CreateObject("Wscript.Shell").Run "c:\tools\unlockme.bat",0,False

2) Modify unlockme.bat

@ECHO off

setlocal EnableDelayedExpansion

for /f "tokens=2,4" %%a in ('QUERY USER ^| FINDSTR ">"') DO (
 set "consoleTemp=%%a"
 set "connectionState=%%b"
 set "consoleType=!consoleTemp:~0,7!"
)

SET consoleTypeCorrect=0

IF "!consoleType!"=="rdp-tcp" SET consoleTypeCorrect=1
SET connectionStateCorrect=1

IF %connectionState%==Active SET connectionStateCorrect=1
set TRUE=%connectionStateCorrect%%consoleTypeCorrect%

IF %TRUE%==11 (
 endlocal
) else (
  tscon 1 /dest:console
  tscon 2 /dest:console
  tscon 3 /dest:console
  tscon 4 /dest:console
  tscon 5 /dest:console
  tscon 6 /dest:console
  tscon 7 /dest:console
  tscon 8 /dest:console
  tscon 9 /dest:console
  tscon 10 /dest:consol
  tscon 11 /dest:console
  tscon 12 /dest:console
  tscon 13 /dest:console
  tscon 14 /dest:console
  tscon 15 /dest:console
  tscon 16 /dest:console
  tscon 17 /dest:console
  tscon 18 /dest:console
  tscon 19 /dest:console
  tscon 20 /dest:console
)

------------------------------
Ami Barrett
Sr Product Consultant
Blue Prism
Plano, TX
------------------------------

DavidVizcaíno · ‎28-04-20

@AmiBarrett

Thank you so much for your help. For some strange reason the session check you mentioned was not working on my test environment. So I implemented some logic into a powershell script to check for the session state and only run the .bat when session state is disconnected (RDP session is closed). Then I modified the .bat as you mentioned on the previous comment, created the VBS linked to the .bat, and ran the VBS accordingly from the powershell script.

Things are working now, I've done two tests:

1) Use RDP to login into resource PC, run a process from control room, close RDP session. Process continued running on console and everything succeeded (as long as I did not interfere with Blue Prism pressing buttons while closing the RDP window)

2) Use scheduler to Login into a resource PC using Login Agent and run a process, RDP to the resource PC while the process is running (process will seem to pause for a few seconds and then starts working again), closed the RDP session to let the process finish.

Neither the powershell script I created for session check, nor the cmd windows popped up.

Thank you again.

------------------------------
David Vizcaíno
RPA Programmer
EY
America/Bogota
------------------------------

AmiBarrett · ‎28-04-20

Awesome! Glad you got it working!

------------------------------
Ami Barrett
Sr Product Consultant
Blue Prism
Plano, TX
------------------------------

SS&C Blue Prism Community

Control Room suddenly useless - Can start any process on any machine whether it's logged in or not - obvious error and swiftly terminates