January 2021 Windows Security Patches - Breaking Single Sign-on to Blue Prism?

[BrentonWestwood]

Hello all.   On any desktop where the January 2022 Windows Security Patches have been installed, I can no longer sign into Blue Prism.   On desktops where the security patch is not installed, everything works as normal.   The patches were installed on 2 of my desktops last night but others have not been patched yet.   We use single sign-on (AD setup).   I get this error:   

Error: Could not connect to '{connection name}'.

SOAP security negotiation with 'http://{appserver}.southernco.com:8187/bpserver' for target 'http:/{appserver}.southernco.com:8187/bpserver' failed. See inner exception for more details.

System.ComponentModel.Win32Exception: Either the client credential was invalid or there was an error collecting the client credentials by the SSPI.
at System.ServiceModel.Security.WindowsSspiNegotiation.GetOutgoingBlob(Byte[] incomingBlob, ChannelBinding channelbinding, ExtendedProtectionPolicy protectionPolicy)
at System.ServiceModel.Security.SspiNegotiationTokenProvider.GetOutgoingBlobProxy.GetOutgoingBlob(ChannelBinding channelBinding)
at System.ServiceModel.Security.RequestSecurityToken.GetBinaryNegotiation()
at System.ServiceModel.Security.WSTrust.Driver.WriteRequestSecurityToken(RequestSecurityToken rst, XmlWriter xmlWriter)
at System.ServiceModel.Security.RequestSecurityToken.OnWriteTo(XmlWriter writer)
at System.ServiceModel.Security.RequestSecurityToken.WriteTo(XmlWriter writer)
at System.ServiceModel.Security.RequestSecurityToken.OnWriteBodyContents(XmlDictionaryWriter writer)
at System.ServiceModel.Channels.BodyWriterMessage.OnWriteBodyContents(XmlDictionaryWriter writer)

------------------------------
Brenton Westwood
Systems Analyst
Southern Company
------------------------------

[RaoulOlsson]

• Tested on Windows Server 2016 (SSO not activated)
  
• Tested all settings from the Knowledge base article. Didn't work.
• Tested all settings on app-server and client. (Just to make sure, didn't work.)
• Also installed patch https://docs.microsoft.com/en-us/windows/release-health/windows-message-center , nothing new. Didn't work.

We decided to pull the update from all our 2016 servers, because they are mainly only BP.

------------------------------
Raoul Olsson
------------------------------

[EricLim]

We've put into place the solution recommended by Blue Prism (setting the Service Principal Name) and it is working for us in our non-production environment.

The only issue encountered was that the team executing the setspn command did it on our Interactive Clients and Runtime Resources - rather than on the Application Server(s).

Thanks to Diane and Brenton for keeping us updated during the initial stages of this email.

------------------------------
Eric Lim
Assistant Director
Australian Government Department of Finance
------------------------------

[paul.anderson · Accepted solution]

A follow-up regarding this matter, relating to the latest Microsoft Windows security update/patch for 'CVE-2022-21907,' which was released on January 11th, 2022.

Customers using Blue Prism with 'Windows Authentication' have reported that Blue Prism Interactive Clients/Runtimes are triggering an additional prompt for credentials; however, when these credentials are entered, it is resulting in the below error: 

•  Windows Authentication connection modes: 'SOAP security negotiation with 'http://XXXXXXXX:8199/bpserver' for target 'http://XXXXXXXXX:8199/bpserver' failed. See inner exception for more details. ---> System.ComponentModel.Win32Exception: Either the client credential was invalid or there was an error collecting the client credentials by the SSP'

Blue Prism has released a solution/fix for this issue in the following KB article on our Support Portal: 

• [Solution] Windows updates from 11th January 2022 causing authentication issues in Blue Prism...

We highly encourage that you speak to your IT team for assistance in applying this fix/solution, and that you first test this solution in a non-production environment. 

The latest article update provides details about the issue, investigation and solution.  Please also check the additional information in the article after solution section, including guidance for customers with complex environments. 

------------------------------
Paul Anderson
Blue Prism
------------------------------

[vinodchinthakin]

Hi
I have uninstalled KB5009557 patch and issue got resolved. But the patch is auto updated on next day, so Issue appears again.
I would like to know can we disable auto updates on windows? is it best practice for Production Env?

------------------------------
vinod chinthakindi
------------------------------

[CarlosCabral]

Our IT Department has setup the SPN at the BP Application Server with Local System account.

But as soon as we installed KB5009543 (KB5009545 or KB5009546) on the BP Resources they started to not work.

So the Fix seems not to work.

------------------------------
Carlos Cabral
IT Security Consultant
Altice Portugal
Europe/London
------------------------------

[diane.sanzone]

Hi Carlos, 
We were able to successfully apply the fix here. Did you confirm the service account being used to run the Blue Prism Service?  In my environment we're using a GSA account, so we had to explicitly list that in the setspn command.  When we first ran it and didn't specify the name of the account running the service, it didn't work for us either.
hope this helps.

------------------------------
Diane Sanzone
------------------------------

[CarlosCabral]

We have the BP Service configured like the default, with the Local System account:

The Fix was applied with this command:
                >Setspn -S HTTP/200000356-APP1.ptportugal-dev.local/BPServer 200000356-APP1

The SPN was registered:

>Setspn -L 200000356-APP1

Registered ServicePrincipalNames for CN=200000356-APP1,OU=CyberRPA,OU=LSB-PIC,OU=DC_PT,DC=ptportugal-dev,DC=local:

        HTTP/200000356-APP1.ptportugal-dev.local/BPServer

But BP Resources or BP Clients with the Jan2022 Patches still do not work.

I will try with a dedicated service account.

Carlos Cabral
Security Analytics, Data Science and RPA Consultant

Altice Portugal

Cyber Security & Privacy (DCY)

Email: carlos-s-cabral@telecom.pt
    Tlm: 966025853
Av. Fontes Pereira de Melo, 38/40
1069-300 LISBOA
meo.pt

AVISO DE CONFIDENCIALIDADE
Esta mensagem e quaisquer ficheiros anexos a ela contêm informação confidencial, propriedade da Altice Portugal e/ou das demais sociedades que com ela se encontrem em relação de domínio, Fundação Altice Portugal e ACS, destinando-se ao uso exclusivo do destinatário. Se não for o destinatário pretendido, não deve usar, distribuir, imprimir ou copiar este e-mail. Se recebeu esta mensagem por engano, por favor informe o emissor e elimine-a imediatamente.
Obrigado

Publico

[CarlosCabral]

We have tried starting the BP Service with an domain account name, but still the FIX did not seem to work.

Since at the Application Server the executable binary is BPServerService.exe and not BPServer.exe should we replace the command:

Setspn -S HTTP/<server_fdqn>/BPServerService <accountname>

Instead of:

Setspn -S HTTP/<server_fdqn>/BPServer <accountname>

Or should we use the connection name "BPDCYPRD" in our case?

Carlos Cabral
Security Analytics, Data Science and RPA Consultant

Altice Portugal

Cyber Security & Privacy (DCY)

Email: carlos-s-cabral@telecom.pt
    Tlm: 966025853
Av. Fontes Pereira de Melo, 38/40
1069-300 LISBOA
meo.pt

AVISO DE CONFIDENCIALIDADE
Esta mensagem e quaisquer ficheiros anexos a ela contêm informação confidencial, propriedade da Altice Portugal e/ou das demais sociedades que com ela se encontrem em relação de domínio, Fundação Altice Portugal e ACS, destinando-se ao uso exclusivo do destinatário. Se não for o destinatário pretendido, não deve usar, distribuir, imprimir ou copiar este e-mail. Se recebeu esta mensagem por engano, por favor informe o emissor e elimine-a imediatamente.
Obrigado

[diane.sanzone]

Hi Carlos,

I see a few things with the strings you sent over.  Here's how we formatted the command:

Setspn –S HTTP/<FQDN>:<PORT>/BPServer <SERVICE_ACCOUNT_NAME>

It looks like the command you used did not include the colon and port number immediately following the FQDN of the server. This is required.  If you can't get your BP port from the error message, check the "connection" on the login screen to see the port used for your instance.  (We have 2 instances running on the same server, so we had to run the command for two different ports on the same machine).

As for the service name, even though the windows service is called Blue Prism Server (or something), we used BPServer in our command and it worked.  We did NOT include the .EXE in the string

For the service account name, we didn't include the domain, however if you have multiple domains maybe you need to include that?  That's an unlikely thing, but I mention it just in case.

All that said, I think your real issue is the missing port. Throw that in and give it a try.  I'm betting it fixes the problem.  If you're still stuck, you can always send your specific details to BP support and ask them to help you create the command.

Good luck and hope this helps!

------------------------------
Diane Sanzone
------------------------------

[diane.sanzone]

Hi Carlos,

I wrote and posted this whole long response to you and it got lost in the inter-ether.  Here's the short version.

I think you need to use BPServer as the executable name (no .EXE) as BP stated in their instructions, but you're missing the port number after the FQDN in your command string. If you can't find your BP port in the error message, open your connection settings and look there. It's an important value in the command - we have 2 instances running on the same server using different ports, so we had to run the command twice on the same server.  Your command should therefore look like this:

Setspn –S HTTP/<FQDN>:<PORT>/BPServer <SERVICE_ACCOUNT_NAME>

Note the colon symbol between the FQDN and the Port.

Hope this helps.  Good luck!

------------------------------
Diane Sanzone
------------------------------