AWS S3 API Calls fails when objects has special characters or spaces in the name

[EslamGhandour]

Greetings,

I am using the last AWS S3 object from DXchange and all the prequisite objects as well are up to date.

I managed to use the main actions such as uploading, downlading and so. However, they only work if the object name has no space or special characters. 
According to AWS documentation, this scenario needs a special handling. It makes complete sense to do so as it will be affecting the URL and it has to be encoded. 

I am not good with custom codes, but I understood that there is no encoding happening. which leads to the problem I'm facing. 
Is there any possibility to get any kind of help to modify the code to make it encode the special characters?

I've been trying to solve that issue for over than a week. I tried to build my own webservice action but I got stuck with AWS Signature. 

Main goal is to download files which can be achieved by GetObject action. 

I would appreciate any kind of help. Thanks in advance!

------------------------------
Eslam Ghandour
------------------------------

[ewilson]

Hi @EslamGhandour,

When you say "object name" you're referring to the bucket name, correct? ​If so, it's pretty easy to URL encode the data before you pass it into the specific S3 action on the VBO. There's another VBO available on the DX called Utility - HTTP. You may already have it installed. If so, check to see if you have the version that includes the action titled URL Encode. If you do, you can pass your bucket name into that action and it will return the URL encoded version of the string. Then you can pass that into the specific S3 action.

If you don't have a version of the Utility - HTTP VBO with the URL Encode action, you can download the latest version here.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

[EslamGhandour]

Hi @ewilson,

Thank your taking the time to help!

By "object name" I meant the file that I need to download from S3. As you are probably I ware, I need to pass the bucket name and the key of the file that needs to be downloaded. The bucket has no special characters, the file names do. 

Considering this example /Bucket-Name/FirstFolder/SecondFolder/File to download 10:20:30.pdf

I did try using the URL Encode action​ in the Utility - HTTP on FirstFolder/SecondFolder/File to download 10:20:30.pdf since it's the key for the object to be downloaded. However, this still did not work. 
It also encoded the "/" which doesn't seea problem for the request either ways as I can download without issues if the name of the file as FileToDownload (with no special characters)

Do you think this is the correct approach to change only that or are you suggesting using it on the full AWSEndpoint?
I tried to use it on the full AWSEndpoint and I received an error  when I passed it to the HTTP Request File action.



Looking forward for your answer.

------------------------------
Eslam Ghandour
------------------------------

[ewilson]

@EslamGhandour,

If you open the Utility - HTTP VBO and go to the URL Encode action you'll find a single Code stage on the page. Go into the actual code of that stage and change the following line of code:

Encoded_URL = HttpUtility.UrlEncode( URL )​

to this

Encoded_URL = HttpUtility.UrlPathEncode( URL )

This will encode the file name, but it will leave the "/" untouched.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

[EslamGhandour]

Hi @ewilson,

Thank you very much for your fast replies and your support. I really appreciate that.

On the other hand, as this is for sure a very valuable information and it did achieve the task of encoding only the path, sadly it did not solve the original issue. 

I'm still facing the main issue which is the http request still failing to go through. It appears to me that it is something within the code stage in HTTP Request File action.

I'm facing a dead-end here and I can't think of any other reason.

I am sure that the problem is related to the key being passed as it works fine when there are no special characters in it. Additionally, I tried the same request with the key that has special characters using postman and it works fine as well. Taking in consideration that it works with postman's automatic encoding and when I disable it and encode the special characters manually in the URL.

It appears to me that I know the problem and why it's failing but I have no clue on how to solve it.

I am sorry for any inconvenience and I just want to thank you so much for helping!

------------------------------
Eslam Ghandour
------------------------------

[ewilson]

@EslamGhandour,

Ok, so you're able to execute the actions (ex. GetObject) and everything works fine as long as the specific object doesn't include spaces or other special characters in the name. As soon as the object includes spaces, the request fails. Is that an accurate description of the issue? When the request fails, do you receive a specific error response message?

Cheers,
​

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

[EslamGhandour]

@ewilson​,

Yes, this is an accurate description. 

The error I receive is a 403 forbidden.

At first I thought it might be related to permissions but that was cleared out as I already can download other files using the same GetObject action. As well as I can download the those files with space using WinScp.

------------------------------
Eslam Ghandour
------------------------------

[ewilson]

@EslamGhandour,

Ok, I've figured out what the issue is. Deep inside the VBO title BluePrism AWS REST API ​Utility there is a line that performs URL encoding of the request path. The problem appears to be that this is causing AWS to view the calculated signature on the request as invalid. I'm trying to understand why this step was added before I make any changes though. Stay tuned.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

[ewilson · Accepted solution]

@EslamGhandour,

We just pushed version 2.2 of the BluePrism AWS REST API Utility VBO to the DX.​ This addresses the issue you are seeing with files that include spaces in their name. It turns out there was a double URL encoding scenario going on.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

[EslamGhandour]

Hi @ewilson.

This is great, it did indeed solve the space issue. Thank you very much for your input and the release!

However, I am still facing the same issue due to the colon. Taking the same previous example FirstFolder/SecondFolder/File to download 10:20:30.pdf it is still failing because of ":"
I tested the new version with multiple scenarios and the space for sure is working and I tested it not only in GetObject but also PutObject. 
My assumption is that the characters mentioned in AWS documentation (screenshot in original message) that needs special handling are not being specially handled. The colon is one of them.

I don't really understand what AWS means by special handling. Do you think this is something  that can be handled in a way on process level or you will have to find a solution and make another version of it? 

Looking forward for your reply and thanks again for the great release!

------------------------------
Eslam Ghandour
------------------------------