Преглед изворни кода

HUE-8758 [bquery] Load credentials via json text instead of file

Romain пре 5 година
родитељ
комит
a26094172e

+ 2 - 1
desktop/core/src/desktop/lib/connectors/types.py

@@ -462,7 +462,8 @@ CONNECTOR_TYPES = [
     'dialect': 'bigquery',
     'interface': 'sqlalchemy',
     'settings': [
-      {'name': 'url', 'value': 'bigquery://projectName/datasetName'},
+      {'name': 'url', 'value': 'bigquery://project-XXXXXX/dataset_name'},
+      {'name': 'credentials_json', 'value': '{"type": "service_account", ...}'}
     ],
     'category': 'editor',
     'description': '',

+ 7 - 0
desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py

@@ -80,8 +80,10 @@ from notebook.models import escape_rows
 if sys.version_info[0] > 2:
   from urllib.parse import quote_plus as urllib_quote_plus
   from past.builtins import long
+  from io import StringIO
 else:
   from urllib import quote_plus as urllib_quote_plus
+  from cStringIO import StringIO
 
 
 ENGINES = {}
@@ -177,6 +179,11 @@ class SqlAlchemyApi(Api):
         'username': self.user.username
       })
 
+    if self.options.get('credentials_json'):
+      self.options['credentials_info'] = json.loads(
+          self.options.pop('credentials_json')
+      )
+
     options = self.options.copy()
     options.pop('session', None)
     options.pop('url', None)

+ 2 - 2
docs/designs/sql/hiveserver-ha.md

@@ -1,7 +1,7 @@
 
 # HiveServer HA discovery and failover
 
-A list of the available HiveServers [can be read](https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/fault-tolerance/content/configuring_hiveserver2_high_availability_using_zookeeper.html) from Apache Zookeeper. This enables dynamic configuration as well as managing fail overs.
+A list of the available HiveServers [can be read](https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/fault-tolerance/content/configuring_hiveserver2_high_availability_using_zookeeper.html) from Apache Zookeeper. This enables dynamic configuration as well as managing failovers.
 
 ## Discovery only
 
@@ -89,7 +89,7 @@ Questions:
 
 ### Manual HA
 
-On possibiliy would be to require a user Hue page refresh or close of connection and clear the caches in the call. This would be for an immediate solution for the short term.
+One possibiliy would be to require a user Hue page refresh or close of connection and clear the caches in the call. This would be for an immediate solution for the short term.
 
 Advantage:
 

+ 9 - 4
docs/docs-site/content/administrator/configuration/connectors/_index.md

@@ -531,15 +531,20 @@ Then give Hue the information about the database source:
     [[[bigquery]]]
        name = BigQuery
        interface=sqlalchemy
-       options='{"url": "bigquery://projectName"}'
+       options='{"url": "bigquery://project-XXXXXX", "credentials_json": "{\"type\": \"service_account\", ...}"}'
 
-To restrict to only one dataset:
+Where to get the names? In the 'Resources' panel of Big Query UI:
 
-       options='{"url": "bigquery://projectName/datasetName"}'
+* Project name, e.g. project-XXXXXX, bigquery-public-data..., is the first level
+* Dataset name, e.g. austin_bikeshare, is the second level
+
+To restrict to one dataset:
+
+       options='{"url": "bigquery://project-XXXXXX"/dataset_name"}'
 
 Supporting additional [connection parameters](https://github.com/mxmzdlv/pybigquery#connection-string-parameters):
 
-      options='{"url": "bigquery://", "credentials_path": "/etc/conf/hue/demo-4a0e4e08d81a.json"}'
+      options='{"url": "bigquery://", "use_query_cache": "true"}'
 
 
 ### Apache Drill