cancel
Showing results for 
Search instead for 
Did you mean: 

Chrome Elements not being recognised

saivikas729
Level 5
Hi, 

We are using Blue prism 6.9 version and Chrome version- 86. We have this one peculiar issue - 

Process is running fine in chrome for 2 to 3 days - but after that while trying to login we are facing below error first  :-ERROR: Internal : AMI error occurred in Wait Start Stage 'Wait Daimler Window' on page 'New - Login' - Not Connected. 

Then all of sudden all the elements which are already spied are not being recognised and wait stages are failing. 
this is happening on completely random days resulting in terminations.

Resource restart is solving this issue for time being but - this keeps happening after 2 or 3 days. 

Please help us in resolving this - 

Regards,
Kolli.

------------------------------
Kolli Sai Vikas
------------------------------
9 REPLIES 9

daniel.leach
Staff
Staff
Hi Kolli,

If you have "Use fixed browser extension port port (31924)" enabled, then a possible reason for the issues you have seen is that the port is not available.

Chrome integration works in such a way that (after a fresh reboot), once a Process is run that interacts with Chrome, a connection is made with Chrome over port 31924. Even after that Process has completed, the connection to port 31924 will still be open and is not disposed of.

 

When performing testing of a Process via Studio/stepping through via Debug mode, the Process is run via the local (private) Resource. As described above, the local Resource would initially open the connection, and that local Resource would then hold on to that connection.

 

If you subsequently attempt to run the Process from Control Room to a Runtime Resource running locally on the same machine, as the connection had already been opened by the local Resource, a new connection cannot then be initiated, which causes a failure.


So, if this is the cause of the issue then the resolution is to restart the listener or restart the Resource machine. This will free the connection and allow the Process to run again via either Control Room or Studio.


Thanks,
Dan

------------------------------
Daniel Leach
Automation Developer
Blue Prism Cloud
Europe/London
------------------------------

Hi, 

Thanks so much for taking time and explaining things clearly. 

I have few points to add - 

1. Our Current process is divided into two sub processes - Here First part is being run without any issues - then second part which uses same flow to login into chrome is Failing. 

Can this happen because we have this enabled? "Use fixed browser extension port port (31924)"

I agree - restarting resource or Listener is working but - we do we have to do this manually everyday? or is there any workaround for this? 
Because if we don't restart machine for 3 to 4 day's there is a chance that process would fail due to above error !

Please share your thought's on this - 

Thanks 



------------------------------
Kolli Sai Vikas
------------------------------

Hi,

You're very welcome. The short answer is: yes it is possible that this is the cause of the issue. But I would advise to monitor the situation.

Just to reiterate, you would only have to restart after you have switched between running the Process from the Control Room and running it from the resource itself (i.e. logging into Blue Prism on the resource and running in debug). Having said that, it is good practice to restart resources regularly to flush temp files, so doing so daily as a matter of course is a good idea.

If you don't want to restart resources manually, one option is to use Windows task scheduler to automate the restarts.

Thanks,
Dan



------------------------------
Daniel Leach
Automation Developer
Blue Prism Cloud
Europe/London
------------------------------

Hi, 

Thanks so much for the explanation and quick response.  
I came across exact similar issue on different thread - Explained by Anna twomey.

Link :https://community.blueprism.com/communities/community-home/digestviewer/view-question?ContributedContentKey=858ea2a8-046b-4265-b333-25e506297eed&CommunityKey=3743dbaa-6766-4a4d-b7ed-9a98b6b1dd01&tab=digestviewer#6956abd2-2889-4a43-8c7e-1ecf17519be4

Hi @AnnaTwomey

Could you please share your thought's on how to resolve this?  Experiencing the same exact Issue -
We are using KILL Chrome action and subsequent process is failing. 

As shared in the other post - We can shut down chrome using CTRL+SHIFT+W. 
But I am wondering if  there is any generic way to shut down chrome - instead of making it portal specific to pass global send keys?

Hi @daniel.leach Please do share your thought's on this as well, Thanks for amazing support.

Regards,
Kolli Sai Vikas.





​​​​​

------------------------------
Kolli Sai Vikas
------------------------------

The best way I have found to close Chrome so that it has as few problems as possible is to click the three dots at the top right and then click "Exit" near the bottom of the context menu that pops up. I started doing this with MS Edge as well. It seems that using CTRL+SHIFT+W and clicking the Close button on the window and the Terminate action in Blue Prism etc. etc. do not close Chrome as fully as using the method I'm referring to.

What we did was build a reusable object/action that we call "Close MS Edge Gracefully" in order to close Edge...well...gracefully, and it uses the same technique as the screenshots show below except in Edge. You could use a reusable action like that in all of your processes to ensure that Chrome or whatever browser is closed correctly at the end. I would personally avoid using Kill Process except as a last resort in a retry loop.

Screenshots:
12264.png

12265.png

I see that others have already suggested considering whether Fixed Port mode is your issue or not. I've spent an obscene amount of time troubleshooting Chrome and MS Edge Browser Extension issues. I've posted on a bunch of other threads regarding my thoughts on things to try. I'm assuming you've already read through some of those threads. But, in any case, here's some quick thoughts.

First, I agree with Daniel that now Browser Connectivity issues with Fixed Port Mode are usually due to someone switching back and forth on the same machine between running in Control Room and running in DEBUG mode (having Process/Object Studio open).

Second, I think it is important to note that while Blue Prism's browser automation has gotten significantly more stable over the last year or so, I've seen some environments just have issues with it such as you're describing. From what I've found, v6.10.1 and v7.0 are both very stable, and I just plain cannot get v7.0 to fail in browser mode at all so it seems quite stable at least from my local Blue Prism environment. So, something you could always consider is setting up a Test environment where you install a newer version of Blue Prism and runs some tests there with your automations to see if you still encounter the same issues.

Third, you could always consider trying to turn off Fixed Port Mode. I seem to always visit the Community in the evening when I'm tired so I may have overlooked this being discussed already in this thread. =) If you tried Ephemeral Port Mode, you would potentially find that browser automation would fail less often if at all. However, note that Fixed Port Mode and Ephemeral Port Mode do not function the same in all scenarios, especially when it comes to development and such. There's limitations to both. Anyway, we're all here for you if you decided to try out that route and can give some suggestions.

Last, I cannot be sure of this and it might just be superstition talking, but I tend to like using only a specific browser per machine if possible. So, if you're using Chrome and Edge and Firefox or just two of them on the same machine, I'd recommend using just one of them if you can, and then if you pair that with closing Chrome the way I suggested above, I think you'll have fewer issues.



------------------------------
Dave Morris
Cano Ai
Atlanta, GA
------------------------------

Dave Morris, 3Ci at Southern Company

Hi, 

Thanks for all your input's. Closing chrome Gracefully :) as you suggested - Looks like a great idea and I already developed object and deployed it. 
Hopefully this resolves this - I am not able to find the exact reason for this Issue. 

We do have several other procesess in different VM  that uses Kill Chrome action but going fine without much issues. but this VM is causing trouble - 
Please see the exact issue below and provide your Input's please. thanks 

ISSUE :- processes built using objects that had been identified in chrome work correctly but, intermittently (after about 3-5 times through the process it just drops the objects and will no longer find the fields.  The problem will resolve if Blue Prism is restarted but, will happen again after a few times through.  This is happening on multiple processes in same robot.   is it due to the use of the Kill Command?  The process uses the kill command to end Chrome then later reopens Chrome for another part of the process.  

Regards
Kolli Sai Vikas


------------------------------
Kolli Sai Vikas
------------------------------

I feel like we are discussing a few different issues here.

Regarding the issue of:
ISSUE :- processes built using objects that had been identified in chrome work correctly but, intermittently (after about 3-5 times through the process it just drops the objects and will no longer find the fields.

This sounds like what has been mentioned above, an issue with Blue Prism communicating with the browser extension which is done through, by default, an ephemeral port, and typically made more resilient by changing this setting in Blue Prism System Settings to be a static port. Look for something like this in the settings:
12272.pngand check the box if it isn't already checked, restart the machines and see if you still encounter this issue.

This is a known issue reported in the support portal:
12273.png
Somewhat related to this issue is that if you ever, on the same runtime resource and or development machine, open studio locally, studio launches its own connection to the browser. It never disposes of it, so if from there you try to run something against Chrome from the Control Room, it will typically not be able to connect to and talk to Chrome. (and or vice versa) Refer to the explanation provided by @daniel.leach on this thread: THIS THREAD which I think explains the WHY rather eloquently.
12274.png




Separately, regarding using 'kill' on, well, really anything, is that many applications don't like it because it is a forced stop very similar to ending a task forcefully through the task manager in windows. Secondly, using the native Blue Prism 'Utility - Environment' kill action isn't quite reliable because if you look at the code:
Try
    for each p as System.Diagnostics.Process in System.Diagnostics.Process.GetProcessesByName(Process_Name)
    p.kill
    next
Catch ex As Exception
End Try​
It retrieves each p (process) with a name of (input process name) and tells it to kill it in a loop as there may be multiple instances of a process launched. Eg. launch chrome or edge manually and open task manager, I'll bet you find numerous process with differing PID's listed.

This part in an of itself isn't terrible, however, saying kill and verifying said kill actually killed it are two different things, so you really want this code to additionally have a check that the kill actually killed the thing it said it was going to kill, probably a thread sleep in between each process in the loop to wait for Windows to dispose of the process as well and move on to the next one in the loop. (waiting for it to exit is a valid .NET method and has exited is a valid .NET property of the System.Diagnostics.Process class that can be used... System.Diagnostics.Process MSDN Documentation)  If you were really being paranoid you'd also check has exited before calling kill because calling kill on something that is 'being' killed will throw an exception. 

Next up, this same class actually has methods called close and close main window which close out of applications significantly more gracefully when they listen to this call from .NET / Windows (I can tell you Edge and Chrome at least both listen to this and exit more gracefully and with this solution programmatically)  

Next up, if you look at Blue Prism's internal code again you'll notice the try catch simply catches, does nothing, and ends the code... the issue here is lets say you have 5 instances of edge running and the code encounters and issue killing instance # 2 of 5, it will just stop, won't proceed to try and kill #3, #4, and #5. 

If you wanted to be REALLY thorough you'd built a code stage that had a timeout, checked for existing instances, tried to close them gracefully, gave a grace period, then forcefully killed them, then at the end of everything it would check again how many instances still exist, and ​if any exist keep looping for the duration of the timeout attempting to close and kill. Finally after the timeout if no instances remain, it was successful, if any do remain, you can report that out through a success flag and a message and choose whether or not to throw an exception. (keep in mind there are a few reasons something can fail to kill and these reasons are valid reasons.)

An example rough draft thrown together of what I mean would look something like this:
'Preset Message and Success
Message = vbNullString
Success = False

Dim TimeOutWatch As Stopwatch
TimeOutWatch = New Stopwatch

Try
	'Catch Bad Input Data
	If String.IsNullOrWhiteSpace(Process_Name) Then
		Throw New Exception("Process Name to kill was blank.")
	End If
	If Time_Out = TimeSpan.Zero Then
		Throw New Exception("Your entered Time Out timespan is set to 0.")
	End If

    'Attempt a cleaner close first.
	'Only execute Close if there is something to Close
	'the following line will count existing processes with Process_Name
	If Process.GetProcessesByName(Process_Name).Length > 0 Then
		'Start the stopwatch to count for timeout condition
		TimeOutWatch.Start()
		'Loop through all the processes of Process_Name
		For Each p As Process In Process.GetProcessesByName(Process_Name)
			'Check if kill attempt time has elapsed past the requested Time_Out
			If TimeOutWatch.ElapsedMilliseconds <= CLng(Time_Out.TotalMilliseconds) Then				
	            'Only attempt to Close the process if it is alive/running
				'This if statement checks this condition
				If Not p.HasExited() Then
                    Try
					    p.CloseMainWindow()
						p.Close()
					Catch ex As InvalidOperationException
					    'This means that the process has already exited
					    Message = (Message + vbCrLf + "The Process has already exited: " + vbCrLf + ex.Message).Trim()
				    Catch ex As NotSupportedException
					    'This means that the process is remote
					    Message = ((Message + vbCrLf + "Cannot Close Remote Process: " + vbCrLf + ex.Message).Trim())
					Finally
					    'Do Nada
					End Try
				End If
                Try
        		    'This will wait a maximum of 5 seconds for the process to exit, the '5000' is milliseconds
                    p.WaitForExit(5000)
                    Thread.Sleep(100)
		    	Catch exc As Exception
				Finally
					'clean the resources used by the process for the Garbage Collector to be able to clear it
					p.Dispose()
                    Thread.Sleep(100)
                End Try				
			Else
				'This means the elapsed time during the attempt to Close the process has elapsed past the requested Time_Out
				Exit For
			End If
		Next
	End If

    Thread.Sleep(1000)
	
	'Try to catch stragglers with a little bit uglier 'kill'
	'Only execute kill if there is something to kill
	'the following line will count existing processes with Process_Name
	If Process.GetProcessesByName(Process_Name).Length > 0 Then
		'Loop through all the processes of Process_Name
		For Each p As Process In Process.GetProcessesByName(Process_Name)
			'Check if kill attempt time has elapsed past the requested Time_Out
			If TimeOutWatch.ElapsedMilliseconds <= CLng(Time_Out.TotalMilliseconds) Then				
	            'Only attempt to Kill the process if it is alive/running
				'This if statement checks this condition
				If Not p.HasExited() Then
                    Try
					    p.Kill()
				    Catch ex As System.ComponentModel.Win32Exception
					    'This means that the process was terminating or can't be terminated - we'll double check later
					    Message = (Message + vbCrLf + "The process was terminating or cannot be terminated: " + vbCrLf + ex.Message).Trim()
				    Catch ex As InvalidOperationException
					    'This means that the process has already exited
					    Message = (Message + vbCrLf + "The Process has already exited: " + vbCrLf + ex.Message).Trim()
				    Catch ex As NotSupportedException
					    'This means that the process is remote
					    Message = ((Message + vbCrLf + "Cannot Kill Remote Process: " + vbCrLf + ex.Message).Trim())
					Finally
					    'Do Nada
					End Try
				End If
                Try
				    'This will wait a maximum of 5 seconds for the process to exit, the '5000' is milliseconds
                    p.WaitForExit(5000)
                    Thread.Sleep(100)
				Catch exc As Exception
				Finally
					'clean the resources used by the process for the Garbage Collector to be able to clear it
					p.Dispose()
                    Thread.Sleep(100)
                End Try			
			Else
				'This means the elapsed time during the attempt to kill the process has elapsed past the requested Time_Out
				Exit For
			End If
		Next
	End If
	
Catch ex As Exception
	Message = (Message + vbCrLf + ex.Message).Trim()
Finally
	'Continue to wait for requested timeout to elapse if we haven't established success
	While TimeOutWatch.ElapsedMilliseconds < CInt(Time_Out.TotalMilliseconds) And Not Success
		If Process.GetProcessesByName(Process_Name).Length <= 0 Then
			Success = True
		End If
	End While
	'Final Success Check before exiting
	If Not String.IsNullOrWhiteSpace(Process_Name) AndAlso Not Time_Out = TimeSpan.Zero AndAlso Process.GetProcessesByName(Process_Name).Length <= 0 Then
		Success = True
        Message = vbNullString
	Else If Not String.IsNullOrWhiteSpace(Process_Name) AndAlso Not Time_Out = TimeSpan.Zero AndAlso Process.GetProcessesByName(Process_Name).Length > 0 Then
		Success = False
		Message = (Message + vbCrLf + "Remaining # of [" + Process_Name + "] to kill: [" + Process.GetProcessesByName(Process_Name).Length.ToString + "]").Trim()
	End If
End Try​



Now... separately from this is the fact that when you open and close Chrome or chromium based browsers, if you don't specify the profile to use it will use the default profile and keep track of the 'crash' status of the browser in the profile file (a JSON file stored in Local AppData). If you tried your best in whichever method you prefer to gracefully close but it is still launching and popping up a message from the browser, this could potentially block spying. This can be solved by editing the profile file before launching Chrome... something like this:
Dim preferencesFilePath As String
preferencesFilePath = Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData) + "\Microsoft\Chrome\User Data\Default\Preferences"
If System.IO.File.Exists(preferencesFilePath) Then
    Dim preferencesText As String
    preferencesText = System.IO.File.ReadAllText(preferencesFilePath)
    preferencesText = System.Text.RegularExpressions.Regex.Replace(preferencesText,"(?<=,""exit_type"":"")(.*)(?="",)","Normal")
    preferencesText = System.Text.RegularExpressions.Regex.Replace(preferencesText,"(?<=,""exited_cleanly"":)(.*)(?=,)","true")
    System.IO.File.WriteAllText(preferencesFilePath,preferencesText)
End If​
​In this case the example is not really parsing the JSON preferences file but just finding the section it cares about with regex and replacing it with a value the browser understands as 'closed cleanly'

------------------------------
Erin Connerley
Automation Consultant
Agilify Automation
America/Indiana/Indianapolis
------------------------------

Hi, 

Thanks so much for taking the time and explaning all the issues - clearly. I implemented Closing Chrome -through clicking exit and using CTRL+SHFT+W in case exit fails - Monitoring the Process now -

I will for sure go through all the steps above and make changes if issue still persists - thanks again. :)

------------------------------
Kolli Sai Vikas
------------------------------

Hi @Kolli Sai Vikas, did you manage to solve the issue?

I I'm facing the same situation: ​an MS Edge object works fine for a few items (can be 5, can be 15), then  suddenly stops recognizing elements spied using the Web mode (UIA elements still work fine) or finding them takes much more time (minutes instead of seconds). 

For now the only solution I found is to restart a machine (or kill the automate.exe process and start it again).
I've tried changing the Edge settings - switch off 'Startup boost' and 'Continue running background extension when Edge is closed' as advised in this thread:
Solution: The port establishing the connection to the browser extension cannot be opened. | Blue Prism Product
I've also changed the BP port setting to work with ephemeral ports instead of fixed one. 
But it didn't fix the issue.

I'd be really grateful for any tips how to overcome this issue.

Regards,
Jan Skarbek

------------------------------
Jan Skarbek
------------------------------