07-02-23 07:34 AM
Answered! Go to Answer.
07-02-23 08:28 AM
Hi @Abhijeet Kate,
Thanks for reaching out to the community. Please find the below details which can help you:
Initialise Page:
Global Code Options Within Initialise:
Private m_Certificates As New Dictionary(Of String, X509Certificate2)
Private Function GetCertificate(ByVal subject As String) As X509Certificate2
Dim store As New X509Store(StoreName.My, StoreLocation.LocalMachine)
Try
store.Open(OpenFlags.ReadOnly)
Dim certs As X509Certificate2Collection = store.Certificates.Find(X509FindType.FindBySubjectName, subject, True)
For Each cert As X509Certificate2 In certs
Return cert
Next
Finally
store.Close()
End Try
Return Nothing
End Function
Function ToDictionary(webResponse As HttpWebResponse ) As Dictionary(Of String, String)
Dim dict = New Dictionary(Of String, String)()
For index As Integer = 0 To webResponse.Headers.Count - 1
If webResponse.Headers.Keys.Count -1 >= index Then
dict.Add(webResponse.Headers.Keys(index), webResponse.Headers(index) )
End if
Next
Return dict
End Function
Code Options Within Initialise:
Setup Code Stage Within Initialise:
System.Net.ServicePointManager.Expect100Continue = False
HTTP Request File Action:
Code Stage Within HTTP Request File Action:
Try
Dim request As WebRequest = WebRequest.Create(addressURL)
If forcePreAuth Then
' Sometimes a web server will require the authorisation header in the initial request
' In which case we have to add the basic authorization header manually.
Dim bytes() As Byte = System.Text.Encoding.UTF8.GetBytes(String.Format("{0}:{1}",username,password))
Dim base64 As String = Convert.ToBase64String(bytes)
request.Headers.Add("Authorization", "Basic " & base64)
Else
If Not String.IsNullOrEmpty(username) AndAlso Not String.IsNullOrEmpty(password) Then
request.Credentials = New NetworkCredential(username,password)
End If
End If
' Be aware that if you use forcePreAuth as well as UseBearerToken then forcePreAuth gets priority.
If Use_Bearer_Token And Not forcePreAuth Then
request.Headers.Add("Authorization","Bearer " & Bearer_Token)
End If
If useProxy Then
Dim proxyURI As New Uri(proxyURL)
Dim proxy As New WebProxy(proxyURI, True)
Dim proxyCred As New NetworkCredential(proxyUsername, proxyPassword)
Dim credCache As New CredentialCache()
credCache.Add(proxyURI, "Basic", proxyCred)
proxy.UseDefaultCredentials = False
proxy.Credentials = credCache
request.Proxy = proxy
End If
' Set the request timeout. Need to see if the user specified 'Infinite' (a value of -1 according to
' https://docs.microsoft.com/en-us/dotnet/api/system.net.webrequest.timeout?view=net-5.0)
If ((requestTimeout = -1) Or (requestTimeout > 0)) Then
request.Timeout = requestTimeout
Else
request.Timeout = 100000 ' Default
End If
request.Method = method
request.ContentType = contentType
Dim httpRequest As HttpWebRequest = TryCast(request, HttpWebRequest)
If httpRequest IsNot Nothing Then
If Not String.IsNullOrEmpty(accept) Then
httpRequest.Accept = accept
End If
If Not String.IsNullOrEmpty(certID) Then
httpRequest.ClientCertificates.Add(m_Certificates(certID))
End If
' Set the AllowAutoRedirect attribute.
httpRequest.AllowAutoRedirect = allowAutoRedirects
httpRequest.MaximumAutomaticRedirections = maxRedirects
End If
For Each r As DataRow In headers.Rows
For Each c As DataColumn In headers.Columns
Dim columnName As String = c.ColumnName
Dim val As String = r(columnName).ToString
' This check ensures that we only include a single Authorization header in the request header collection.
If Not columnName.ToUpper().Equals("AUTHORIZATION") Then
request.Headers.Add(columnName,val)
Else
If Not forcePreAuth And Not Use_Bearer_Token Then
request.Headers.Add(columnName,val)
End If
End If
Next
Exit For ' Only one row is allowed
Next
' Normal stream writer, we aren't encoding a file etc.
If Not usingFile Then
If Not String.IsNullOrEmpty(body) Then
Dim requestStream As IO.Stream = request.GetRequestStream()
Using sw As New IO.StreamWriter(requestStream, New Text.UTF8Encoding(False))
sw.Write(body)
End Using
End If
Else If usingFile Then
' Prep the request body
Dim byteArray = File.ReadAllBytes( body )
Dim requestStream As IO.Stream = request.GetRequestStream()
requestStream.Write(byteArray,0, byteArray.Length)
End If
Using response As WebResponse = request.GetResponse()
' Need to cast the WebResponse to an HttpWebResponse so we can get the StatusCode and check to see if compression was used.
Dim httpResponse As HttpWebResponse = CType(response, HttpWebResponse)
resultCode = Convert.ToInt32(httpResponse.StatusCode)
resultHeadersJson = JsonConvert.SerializeObject(ToDictionary(response))
Dim responseStream As IO.Stream = response.GetResponseStream()
Dim lWriter As New IO.FileStream(FilePath, IO.FileMode.Create)
' Was compression used on the data?
If (httpResponse.ContentEncoding.ToLower().Contains("gzip"))
responseStream = New System.IO.Compression.GZipStream(responseStream, System.IO.Compression.CompressionMode.Decompress)
ElseIf (httpResponse.ContentEncoding.ToLower().Contains("deflate"))
responseStream = New System.IO.Compression.DeflateStream(responseStream, System.IO.Compression.CompressionMode.Decompress)
End If
Dim lLength As Long
Dim lBytes(256) As Byte
Do
lLength = responseStream.Read(lBytes, 0, lBytes.Length)
lWriter.Write(lBytes, 0, lLength)
Loop While lLength > 0
lWriter.Close()
responseStream.Close()
resultData = "done"
End Using
Catch e As WebException
If Not e.Response Is Nothing Then
' Cast the WebResponse to an HttpWebResponse so we can get the StatusCode
Dim httpResponse As HttpWebResponse = CType(e.Response, HttpWebResponse)
resultCode = Convert.ToInt32(httpResponse.StatusCode)
' Get the HTTP response message.
resultData = e.Message
Else
resultData = "Exception: " & e.Message
If (e.InnerException.Message.Length > 0) Then
resultData = resultData & vbCRLF & "Inner Exception: " & e.InnerException.Message
End If
End If
End Try
07-02-23 08:28 AM
Hi @Abhijeet Kate,
Thanks for reaching out to the community. Please find the below details which can help you:
Initialise Page:
Global Code Options Within Initialise:
Private m_Certificates As New Dictionary(Of String, X509Certificate2)
Private Function GetCertificate(ByVal subject As String) As X509Certificate2
Dim store As New X509Store(StoreName.My, StoreLocation.LocalMachine)
Try
store.Open(OpenFlags.ReadOnly)
Dim certs As X509Certificate2Collection = store.Certificates.Find(X509FindType.FindBySubjectName, subject, True)
For Each cert As X509Certificate2 In certs
Return cert
Next
Finally
store.Close()
End Try
Return Nothing
End Function
Function ToDictionary(webResponse As HttpWebResponse ) As Dictionary(Of String, String)
Dim dict = New Dictionary(Of String, String)()
For index As Integer = 0 To webResponse.Headers.Count - 1
If webResponse.Headers.Keys.Count -1 >= index Then
dict.Add(webResponse.Headers.Keys(index), webResponse.Headers(index) )
End if
Next
Return dict
End Function
Code Options Within Initialise:
Setup Code Stage Within Initialise:
System.Net.ServicePointManager.Expect100Continue = False
HTTP Request File Action:
Code Stage Within HTTP Request File Action:
Try
Dim request As WebRequest = WebRequest.Create(addressURL)
If forcePreAuth Then
' Sometimes a web server will require the authorisation header in the initial request
' In which case we have to add the basic authorization header manually.
Dim bytes() As Byte = System.Text.Encoding.UTF8.GetBytes(String.Format("{0}:{1}",username,password))
Dim base64 As String = Convert.ToBase64String(bytes)
request.Headers.Add("Authorization", "Basic " & base64)
Else
If Not String.IsNullOrEmpty(username) AndAlso Not String.IsNullOrEmpty(password) Then
request.Credentials = New NetworkCredential(username,password)
End If
End If
' Be aware that if you use forcePreAuth as well as UseBearerToken then forcePreAuth gets priority.
If Use_Bearer_Token And Not forcePreAuth Then
request.Headers.Add("Authorization","Bearer " & Bearer_Token)
End If
If useProxy Then
Dim proxyURI As New Uri(proxyURL)
Dim proxy As New WebProxy(proxyURI, True)
Dim proxyCred As New NetworkCredential(proxyUsername, proxyPassword)
Dim credCache As New CredentialCache()
credCache.Add(proxyURI, "Basic", proxyCred)
proxy.UseDefaultCredentials = False
proxy.Credentials = credCache
request.Proxy = proxy
End If
' Set the request timeout. Need to see if the user specified 'Infinite' (a value of -1 according to
' https://docs.microsoft.com/en-us/dotnet/api/system.net.webrequest.timeout?view=net-5.0)
If ((requestTimeout = -1) Or (requestTimeout > 0)) Then
request.Timeout = requestTimeout
Else
request.Timeout = 100000 ' Default
End If
request.Method = method
request.ContentType = contentType
Dim httpRequest As HttpWebRequest = TryCast(request, HttpWebRequest)
If httpRequest IsNot Nothing Then
If Not String.IsNullOrEmpty(accept) Then
httpRequest.Accept = accept
End If
If Not String.IsNullOrEmpty(certID) Then
httpRequest.ClientCertificates.Add(m_Certificates(certID))
End If
' Set the AllowAutoRedirect attribute.
httpRequest.AllowAutoRedirect = allowAutoRedirects
httpRequest.MaximumAutomaticRedirections = maxRedirects
End If
For Each r As DataRow In headers.Rows
For Each c As DataColumn In headers.Columns
Dim columnName As String = c.ColumnName
Dim val As String = r(columnName).ToString
' This check ensures that we only include a single Authorization header in the request header collection.
If Not columnName.ToUpper().Equals("AUTHORIZATION") Then
request.Headers.Add(columnName,val)
Else
If Not forcePreAuth And Not Use_Bearer_Token Then
request.Headers.Add(columnName,val)
End If
End If
Next
Exit For ' Only one row is allowed
Next
' Normal stream writer, we aren't encoding a file etc.
If Not usingFile Then
If Not String.IsNullOrEmpty(body) Then
Dim requestStream As IO.Stream = request.GetRequestStream()
Using sw As New IO.StreamWriter(requestStream, New Text.UTF8Encoding(False))
sw.Write(body)
End Using
End If
Else If usingFile Then
' Prep the request body
Dim byteArray = File.ReadAllBytes( body )
Dim requestStream As IO.Stream = request.GetRequestStream()
requestStream.Write(byteArray,0, byteArray.Length)
End If
Using response As WebResponse = request.GetResponse()
' Need to cast the WebResponse to an HttpWebResponse so we can get the StatusCode and check to see if compression was used.
Dim httpResponse As HttpWebResponse = CType(response, HttpWebResponse)
resultCode = Convert.ToInt32(httpResponse.StatusCode)
resultHeadersJson = JsonConvert.SerializeObject(ToDictionary(response))
Dim responseStream As IO.Stream = response.GetResponseStream()
Dim lWriter As New IO.FileStream(FilePath, IO.FileMode.Create)
' Was compression used on the data?
If (httpResponse.ContentEncoding.ToLower().Contains("gzip"))
responseStream = New System.IO.Compression.GZipStream(responseStream, System.IO.Compression.CompressionMode.Decompress)
ElseIf (httpResponse.ContentEncoding.ToLower().Contains("deflate"))
responseStream = New System.IO.Compression.DeflateStream(responseStream, System.IO.Compression.CompressionMode.Decompress)
End If
Dim lLength As Long
Dim lBytes(256) As Byte
Do
lLength = responseStream.Read(lBytes, 0, lBytes.Length)
lWriter.Write(lBytes, 0, lLength)
Loop While lLength > 0
lWriter.Close()
responseStream.Close()
resultData = "done"
End Using
Catch e As WebException
If Not e.Response Is Nothing Then
' Cast the WebResponse to an HttpWebResponse so we can get the StatusCode
Dim httpResponse As HttpWebResponse = CType(e.Response, HttpWebResponse)
resultCode = Convert.ToInt32(httpResponse.StatusCode)
' Get the HTTP response message.
resultData = e.Message
Else
resultData = "Exception: " & e.Message
If (e.InnerException.Message.Length > 0) Then
resultData = resultData & vbCRLF & "Inner Exception: " & e.InnerException.Message
End If
End If
End Try
07-02-23 11:21 AM
08-02-23 06:14 AM
08-02-23 03:37 PM
09-02-23 11:57 AM
Yes. I am using same action created using code shared above and able to download document , however PDF is not getting opened, when i try to open downloaded pdf file it just give me message that, "Adobe acrobat reader could not open file because it is either not supported file type or because the file has been damaged", tried opening in browser too but no luck. I observed you downloaded BP document when you showed example in this thread above, just wanted to check if you also got the same issue.
thanks
------------------------------
Abhijeet Kate
RPA Specialist
Accenture India Pvt. Ltd
Pune
------------------------------
10-02-23 05:29 PM
Can you try opening the PDF in a text editor and send a screenshot of what it looks like? I'm wondering is perhaps the file contents are encoded and for some reason the code is not decoding it or something like that. One other thing, have you tried comparing the file size of the VBO downloaded file to that of the file when you download it via the browser to se if there's a size difference?
Cheers,
13-02-23 06:07 AM
@ewilson , Thanks for your response.
Yes i can see files downloaded form API has size only 1 KB, however, same files downloaded through UI are around 2 MB in size.
below is data which can be seen when i opened one of the files in notepad, sorry couldn't attach screenshot due to Constraints.
{"company_number":"04260035","barcode":"XB7C6IPS","significant_date":null,"significant_date_type":"","category":"officers","pages":1,"filename":"04260035_tm01_2022-07-01","created_at":"2022-07-01T14:14:06.120255437Z","etag":"","links":{"self":"https://document-api.company-information.service.gov.uk/document/GpIQOkVX5_y_d-Qe2Ij4GFgVjP57kcVHWixwNHhhcaU","document":"https://document-api.company-information.service.gov.uk/document/GpIQOkVX5_y_d-Qe2Ij4GFgVjP57kcVHWixwNHhhcaU/content"},"resources":{"application/pdf":{"content_length":83915}}}
Thanks
Abhijeet
13-02-23 12:09 PM
That's interesting. That is most definitely not PDF content. That is JSON metadata. What's the API endpoint that you're calling on the Company House API? It seems the JSON metadata provides you with a link to where you can get the content of the file if you look at the element titled document. I suspect you need to issue another call directly to that URL to get the actual contents.
Also, what's the HTTP response code to this first request. It seems like it should be a redirect code like an HTTP 302.
Cheers,