Browse Source

[raz] Fix non-ascii directory creation in S3 (#3144)

* [raz] Unquote path fully for GET operation with non-ascii chars

- Creating a directory with non-ascii characters was throwing a signature mismatch error.
- On further investigation, found out that there was a signature mismatch for the GET requests which go through the boto2 client implementation in Hue, and the signed headers from RAZ are not matching what S3 was generating to verify them.
- This is only happening for GET requests, so most probably all GET method operations must be failing with this error.
- Next steps were to see why is this signature mismatch happening for the path, is it the path which we sent to RAZ for signed header generation or to S3 side request for actual S3 operation where S3 verifies the header?
- After narrowing down the issue, found out that we need to fully unquote the path before sending to RAZ so that signed headers are sent for correct path and S3 can verify them correctly. We didn't touch the path sent to S3 side.
Harsh Gupta 2 years ago
parent
commit
2c810b3edb
1 changed files with 9 additions and 0 deletions
  1. 9 0
      desktop/core/src/desktop/lib/raz/raz_client.py

+ 9 - 0
desktop/core/src/desktop/lib/raz/raz_client.py

@@ -241,6 +241,15 @@ class RazClient(object):
 
 
   def _make_s3_request(self, request_data, request_headers, method, params, headers, url_params, endpoint, resource_path, data=None):
+
+    # In GET operations with non-ascii chars, only the non-ascii part is URL encoded.
+    # We need to unquote the path fully before making a signed request for RAZ.
+    if method == 'GET' and 'prefix' in url_params and '%' in url_params['prefix']:
+      if isinstance(url_params['prefix'], unicode) and sys.version_info[0] < 3:
+        url_params['prefix'] = url_params['prefix'].encode()
+
+      url_params['prefix'] = lib_urlunquote(url_params['prefix'])
+
     allparams = [raz_signer.StringListStringMapProto(key=key, value=[val]) for key, val in url_params.items()]
     allparams.extend([raz_signer.StringListStringMapProto(key=key, value=[val]) for key, val in params.items()])
     headers = [raz_signer.StringStringMapProto(key=key, value=val) for key, val in headers.items()]