07-02-23 07:34 AM
Answered! Go to Answer.
07-02-23 08:28 AM
Hi @Abhijeet Kate,
Thanks for reaching out to the community. Please find the below details which can help you:
Initialise Page:
Global Code Options Within Initialise:
Private m_Certificates As New Dictionary(Of String, X509Certificate2)
Private Function GetCertificate(ByVal subject As String) As X509Certificate2
Dim store As New X509Store(StoreName.My, StoreLocation.LocalMachine)
Try
store.Open(OpenFlags.ReadOnly)
Dim certs As X509Certificate2Collection = store.Certificates.Find(X509FindType.FindBySubjectName, subject, True)
For Each cert As X509Certificate2 In certs
Return cert
Next
Finally
store.Close()
End Try
Return Nothing
End Function
Function ToDictionary(webResponse As HttpWebResponse ) As Dictionary(Of String, String)
Dim dict = New Dictionary(Of String, String)()
For index As Integer = 0 To webResponse.Headers.Count - 1
If webResponse.Headers.Keys.Count -1 >= index Then
dict.Add(webResponse.Headers.Keys(index), webResponse.Headers(index) )
End if
Next
Return dict
End Function
Code Options Within Initialise:
Setup Code Stage Within Initialise:
System.Net.ServicePointManager.Expect100Continue = False
HTTP Request File Action:
Code Stage Within HTTP Request File Action:
Try
	Dim request As WebRequest = WebRequest.Create(addressURL)
	If forcePreAuth Then
	   ' Sometimes a web server will require the authorisation header in the initial request
	   ' In which case we have to add the basic authorization header manually.
	   Dim bytes() As Byte = System.Text.Encoding.UTF8.GetBytes(String.Format("{0}:{1}",username,password))
	   Dim base64 As String = Convert.ToBase64String(bytes)
	   request.Headers.Add("Authorization", "Basic " & base64)
	Else
	  If Not String.IsNullOrEmpty(username) AndAlso Not String.IsNullOrEmpty(password) Then
		request.Credentials = New NetworkCredential(username,password)
	  End If
	End If
    ' Be aware that if you use forcePreAuth as well as UseBearerToken then forcePreAuth gets priority.
	If Use_Bearer_Token And Not forcePreAuth Then
		request.Headers.Add("Authorization","Bearer " & Bearer_Token)
	End If
	If useProxy Then
		Dim proxyURI As New Uri(proxyURL)
		Dim proxy As New WebProxy(proxyURI, True)
		Dim proxyCred As New NetworkCredential(proxyUsername, proxyPassword)
		Dim credCache As New CredentialCache()
		credCache.Add(proxyURI, "Basic", proxyCred)
		proxy.UseDefaultCredentials = False
		proxy.Credentials = credCache
		request.Proxy = proxy
	End If
	' Set the request timeout. Need to see if the user specified 'Infinite' (a value of -1 according to 
	' https://docs.microsoft.com/en-us/dotnet/api/system.net.webrequest.timeout?view=net-5.0)
	If ((requestTimeout = -1) Or (requestTimeout > 0)) Then
		request.Timeout = requestTimeout
	Else 
		request.Timeout = 100000 ' Default
	End If
	request.Method = method
	request.ContentType = contentType
	Dim httpRequest As HttpWebRequest = TryCast(request, HttpWebRequest)
	If httpRequest IsNot Nothing Then
	  If Not String.IsNullOrEmpty(accept) Then
		httpRequest.Accept = accept
	  End If
	  If Not String.IsNullOrEmpty(certID) Then
		httpRequest.ClientCertificates.Add(m_Certificates(certID))
	  End If
	  ' Set the AllowAutoRedirect attribute.
	  httpRequest.AllowAutoRedirect = allowAutoRedirects
	  httpRequest.MaximumAutomaticRedirections = maxRedirects
	End If
	For Each r As DataRow In headers.Rows
		For Each c As DataColumn In headers.Columns
			Dim columnName As String = c.ColumnName
			Dim val As String = r(columnName).ToString
			' This check ensures that we only include a single Authorization header in the request header collection.
			If Not columnName.ToUpper().Equals("AUTHORIZATION") Then
				request.Headers.Add(columnName,val)
			Else
				If Not forcePreAuth And Not Use_Bearer_Token Then
					request.Headers.Add(columnName,val)					
				End If
			End If
		Next
		Exit For ' Only one row is allowed
	Next
	' Normal stream writer, we aren't encoding a file etc.
	If Not usingFile Then
		If Not String.IsNullOrEmpty(body) Then
		  Dim requestStream As IO.Stream = request.GetRequestStream()
		  Using sw As New IO.StreamWriter(requestStream, New Text.UTF8Encoding(False))
			sw.Write(body)
		  End Using
		End If
	Else If usingFile Then
		' Prep the request body
		Dim byteArray = File.ReadAllBytes( body )
		Dim requestStream As IO.Stream = request.GetRequestStream()
		requestStream.Write(byteArray,0, byteArray.Length)
	End If
	Using response As WebResponse = request.GetResponse()
		' Need to cast the WebResponse to an HttpWebResponse so we can get the StatusCode and check to see if compression was used.
		Dim httpResponse As HttpWebResponse = CType(response, HttpWebResponse)
		resultCode = Convert.ToInt32(httpResponse.StatusCode)
		resultHeadersJson = JsonConvert.SerializeObject(ToDictionary(response))
		Dim responseStream As IO.Stream = response.GetResponseStream()
        Dim lWriter As New IO.FileStream(FilePath, IO.FileMode.Create)
		' Was compression used on the data?
		If (httpResponse.ContentEncoding.ToLower().Contains("gzip"))
			responseStream = New System.IO.Compression.GZipStream(responseStream, System.IO.Compression.CompressionMode.Decompress)
		ElseIf (httpResponse.ContentEncoding.ToLower().Contains("deflate"))
			responseStream = New System.IO.Compression.DeflateStream(responseStream, System.IO.Compression.CompressionMode.Decompress)            
		End If
		Dim lLength As Long
		Dim lBytes(256) As Byte
		Do
			lLength = responseStream.Read(lBytes, 0, lBytes.Length)
			lWriter.Write(lBytes, 0, lLength)
		Loop While lLength > 0
		lWriter.Close()
		responseStream.Close()
		resultData = "done"
	End Using
Catch e As WebException
	If Not e.Response Is Nothing Then
		' Cast the WebResponse to an HttpWebResponse so we can get the StatusCode
		Dim httpResponse As HttpWebResponse = CType(e.Response, HttpWebResponse)
		resultCode = Convert.ToInt32(httpResponse.StatusCode)
		
		' Get the HTTP response message.
		resultData = e.Message
	Else
		resultData = "Exception: " & e.Message
		If (e.InnerException.Message.Length > 0) Then
			resultData = resultData & vbCRLF & "Inner Exception: " & e.InnerException.Message
		End If
	End If
End Try
07-02-23 08:28 AM
Hi @Abhijeet Kate,
Thanks for reaching out to the community. Please find the below details which can help you:
Initialise Page:
Global Code Options Within Initialise:
Private m_Certificates As New Dictionary(Of String, X509Certificate2)
Private Function GetCertificate(ByVal subject As String) As X509Certificate2
Dim store As New X509Store(StoreName.My, StoreLocation.LocalMachine)
Try
store.Open(OpenFlags.ReadOnly)
Dim certs As X509Certificate2Collection = store.Certificates.Find(X509FindType.FindBySubjectName, subject, True)
For Each cert As X509Certificate2 In certs
Return cert
Next
Finally
store.Close()
End Try
Return Nothing
End Function
Function ToDictionary(webResponse As HttpWebResponse ) As Dictionary(Of String, String)
Dim dict = New Dictionary(Of String, String)()
For index As Integer = 0 To webResponse.Headers.Count - 1
If webResponse.Headers.Keys.Count -1 >= index Then
dict.Add(webResponse.Headers.Keys(index), webResponse.Headers(index) )
End if
Next
Return dict
End Function
Code Options Within Initialise:
Setup Code Stage Within Initialise:
System.Net.ServicePointManager.Expect100Continue = False
HTTP Request File Action:
Code Stage Within HTTP Request File Action:
Try
	Dim request As WebRequest = WebRequest.Create(addressURL)
	If forcePreAuth Then
	   ' Sometimes a web server will require the authorisation header in the initial request
	   ' In which case we have to add the basic authorization header manually.
	   Dim bytes() As Byte = System.Text.Encoding.UTF8.GetBytes(String.Format("{0}:{1}",username,password))
	   Dim base64 As String = Convert.ToBase64String(bytes)
	   request.Headers.Add("Authorization", "Basic " & base64)
	Else
	  If Not String.IsNullOrEmpty(username) AndAlso Not String.IsNullOrEmpty(password) Then
		request.Credentials = New NetworkCredential(username,password)
	  End If
	End If
    ' Be aware that if you use forcePreAuth as well as UseBearerToken then forcePreAuth gets priority.
	If Use_Bearer_Token And Not forcePreAuth Then
		request.Headers.Add("Authorization","Bearer " & Bearer_Token)
	End If
	If useProxy Then
		Dim proxyURI As New Uri(proxyURL)
		Dim proxy As New WebProxy(proxyURI, True)
		Dim proxyCred As New NetworkCredential(proxyUsername, proxyPassword)
		Dim credCache As New CredentialCache()
		credCache.Add(proxyURI, "Basic", proxyCred)
		proxy.UseDefaultCredentials = False
		proxy.Credentials = credCache
		request.Proxy = proxy
	End If
	' Set the request timeout. Need to see if the user specified 'Infinite' (a value of -1 according to 
	' https://docs.microsoft.com/en-us/dotnet/api/system.net.webrequest.timeout?view=net-5.0)
	If ((requestTimeout = -1) Or (requestTimeout > 0)) Then
		request.Timeout = requestTimeout
	Else 
		request.Timeout = 100000 ' Default
	End If
	request.Method = method
	request.ContentType = contentType
	Dim httpRequest As HttpWebRequest = TryCast(request, HttpWebRequest)
	If httpRequest IsNot Nothing Then
	  If Not String.IsNullOrEmpty(accept) Then
		httpRequest.Accept = accept
	  End If
	  If Not String.IsNullOrEmpty(certID) Then
		httpRequest.ClientCertificates.Add(m_Certificates(certID))
	  End If
	  ' Set the AllowAutoRedirect attribute.
	  httpRequest.AllowAutoRedirect = allowAutoRedirects
	  httpRequest.MaximumAutomaticRedirections = maxRedirects
	End If
	For Each r As DataRow In headers.Rows
		For Each c As DataColumn In headers.Columns
			Dim columnName As String = c.ColumnName
			Dim val As String = r(columnName).ToString
			' This check ensures that we only include a single Authorization header in the request header collection.
			If Not columnName.ToUpper().Equals("AUTHORIZATION") Then
				request.Headers.Add(columnName,val)
			Else
				If Not forcePreAuth And Not Use_Bearer_Token Then
					request.Headers.Add(columnName,val)					
				End If
			End If
		Next
		Exit For ' Only one row is allowed
	Next
	' Normal stream writer, we aren't encoding a file etc.
	If Not usingFile Then
		If Not String.IsNullOrEmpty(body) Then
		  Dim requestStream As IO.Stream = request.GetRequestStream()
		  Using sw As New IO.StreamWriter(requestStream, New Text.UTF8Encoding(False))
			sw.Write(body)
		  End Using
		End If
	Else If usingFile Then
		' Prep the request body
		Dim byteArray = File.ReadAllBytes( body )
		Dim requestStream As IO.Stream = request.GetRequestStream()
		requestStream.Write(byteArray,0, byteArray.Length)
	End If
	Using response As WebResponse = request.GetResponse()
		' Need to cast the WebResponse to an HttpWebResponse so we can get the StatusCode and check to see if compression was used.
		Dim httpResponse As HttpWebResponse = CType(response, HttpWebResponse)
		resultCode = Convert.ToInt32(httpResponse.StatusCode)
		resultHeadersJson = JsonConvert.SerializeObject(ToDictionary(response))
		Dim responseStream As IO.Stream = response.GetResponseStream()
        Dim lWriter As New IO.FileStream(FilePath, IO.FileMode.Create)
		' Was compression used on the data?
		If (httpResponse.ContentEncoding.ToLower().Contains("gzip"))
			responseStream = New System.IO.Compression.GZipStream(responseStream, System.IO.Compression.CompressionMode.Decompress)
		ElseIf (httpResponse.ContentEncoding.ToLower().Contains("deflate"))
			responseStream = New System.IO.Compression.DeflateStream(responseStream, System.IO.Compression.CompressionMode.Decompress)            
		End If
		Dim lLength As Long
		Dim lBytes(256) As Byte
		Do
			lLength = responseStream.Read(lBytes, 0, lBytes.Length)
			lWriter.Write(lBytes, 0, lLength)
		Loop While lLength > 0
		lWriter.Close()
		responseStream.Close()
		resultData = "done"
	End Using
Catch e As WebException
	If Not e.Response Is Nothing Then
		' Cast the WebResponse to an HttpWebResponse so we can get the StatusCode
		Dim httpResponse As HttpWebResponse = CType(e.Response, HttpWebResponse)
		resultCode = Convert.ToInt32(httpResponse.StatusCode)
		
		' Get the HTTP response message.
		resultData = e.Message
	Else
		resultData = "Exception: " & e.Message
		If (e.InnerException.Message.Length > 0) Then
			resultData = resultData & vbCRLF & "Inner Exception: " & e.InnerException.Message
		End If
	End If
End Try
07-02-23 11:21 AM
08-02-23 06:14 AM
08-02-23 03:37 PM
09-02-23 11:57 AM
Yes. I am using same action created using code shared above and able to download document , however PDF is not getting opened, when i try to open downloaded pdf file it just give me message that, "Adobe acrobat reader could not open file because it is either not supported file type or because the file has been damaged", tried opening in browser too but no luck. I observed you downloaded BP document when you showed example in this thread above, just wanted to check if you also got the same issue. 
thanks 
------------------------------
Abhijeet Kate
RPA Specialist
Accenture India Pvt. Ltd
Pune
------------------------------
10-02-23 05:29 PM
Can you try opening the PDF in a text editor and send a screenshot of what it looks like? I'm wondering is perhaps the file contents are encoded and for some reason the code is not decoding it or something like that. One other thing, have you tried comparing the file size of the VBO downloaded file to that of the file when you download it via the browser to se if there's a size difference?
Cheers,
13-02-23 06:07 AM
@ewilson , Thanks for your response.
Yes i can see files downloaded form API has size only 1 KB, however, same files downloaded through UI are around 2 MB in size.
below is data which can be seen when i opened one of the files in notepad, sorry couldn't attach screenshot due to Constraints.
{"company_number":"04260035","barcode":"XB7C6IPS","significant_date":null,"significant_date_type":"","category":"officers","pages":1,"filename":"04260035_tm01_2022-07-01","created_at":"2022-07-01T14:14:06.120255437Z","etag":"","links":{"self":"https://document-api.company-information.service.gov.uk/document/GpIQOkVX5_y_d-Qe2Ij4GFgVjP57kcVHWixwNHhhcaU","document":"https://document-api.company-information.service.gov.uk/document/GpIQOkVX5_y_d-Qe2Ij4GFgVjP57kcVHWixwNHhhcaU/content"},"resources":{"application/pdf":{"content_length":83915}}}
Thanks
Abhijeet
13-02-23 12:09 PM
That's interesting. That is most definitely not PDF content. That is JSON metadata. What's the API endpoint that you're calling on the Company House API? It seems the JSON metadata provides you with a link to where you can get the content of the file if you look at the element titled document. I suspect you need to issue another call directly to that URL to get the actual contents.
Also, what's the HTTP response code to this first request. It seems like it should be a redirect code like an HTTP 302.
Cheers,
