Bläddra i källkod

HUE-8887 [hive] Adding LLAP Interpreter to Hue.ini and PseudoDistributed.ini.tmpl (#913)

* adding in llap interpreter

* LLAP Static Server => Port should be thrift

* Updating Documentation (plus cleanup)
jkm 6 år sedan
förälder
incheckning
6bc477be26

+ 1 - 1
apps/beeswax/src/beeswax/server/dbms.py

@@ -122,7 +122,7 @@ def get_query_server_config(name='beeswax', server=None, cluster=None):
         zk.stop()
       else:
         LOG.debug("Zookeeper Discovery not enabled, reverting to config values")
-        cache.set("llap", json.dumps({"host": LLAP_SERVER_HOST.get(), "port": LLAP_SERVER_PORT.get()}), CACHE_TIMEOUT.get())
+        cache.set("llap", json.dumps({"host": LLAP_SERVER_HOST.get(), "port": LLAP_SERVER_THRIFT_PORT.get()}), CACHE_TIMEOUT.get())
     activeEndpoint = json.loads(cache.get("llap"))
   elif name != 'hms' and name != 'impala':
     activeEndpoint = cache.get("hiveserver2")

+ 5 - 3
desktop/conf.dist/hue.ini

@@ -891,6 +891,10 @@
     # [[[hive]]]
     #   name=Hive
     #   interface=hiveserver2
+    
+    # [[[llap]]]
+    #   name=LLAP
+    #   interface=hiveserver2
 
     # [[[impala]]]
     #   name=Impala
@@ -1171,9 +1175,7 @@
   # To keep the load on zookeeper to a minimum
   # ---- we cache the LLAP activeEndpoint for the cache_timeout period
   # ---- we cache the hiveserver2 endpoint for the length of session
-  # configurations to set the cache name, the cache path, and the time between zookeeper checks
-  ## caches_hive_discovery_key = hive_discovery
-  ## caches_hive_discovery_key_path = /tmp/hive_discovery_cache
+  # configurations to set the time between zookeeper checks
   ## cache_timeout = 60
 
   # Host where Hive Metastore Server (HMS) is running.

+ 5 - 3
desktop/conf/pseudo-distributed.ini.tmpl

@@ -877,6 +877,10 @@
     #   name=Hive
     #   interface=hiveserver2
 
+    # [[[llap]]]
+    #   name=LLAP
+    #   interface=hiveserver2
+
     # [[[impala]]]
     #   name=Impala
     #   interface=hiveserver2
@@ -1156,9 +1160,7 @@
   # To keep the load on zookeeper to a minimum
   # ---- we cache the LLAP activeEndpoint for the cache_timeout period
   # ---- we cache the hiveserver2 endpoint for the length of session
-  # configurations to set the cache name, the cache path, and the time between zookeeper checks
-  ## caches_hive_discovery_key = hive_discovery
-  ## caches_hive_discovery_key_path = /tmp/hive_discovery_cache
+  # configurations to set the time between zookeeper checks
   ## cache_timeout = 60
   
   # Host where Hive Metastore Server (HMS) is running.

+ 37 - 0
docs/admin-manual/manual.md

@@ -263,6 +263,43 @@ The download feature in the file browser can be disabled separately with the fol
       # Port where HiveServer2 Thrift server runs on.
       ## hive_server_port=10000
 
+### LLAP
+    LLAP is added by enabling the following settings:
+        [notebook]
+            [[interpreters]]
+                [[[llap]]]
+                #   name=LLAP
+                #   interface=hiveserver2
+        
+        [beeswax]
+        # When the LLAP interpreter is added, there are 2 ways to enable connectivity (direct configuration or service discovery)
+            # Direct Configuration
+            ## llap_server_host = localhost
+            ## llap_server_port = 10500
+            ## llap_server_thrift_port = 10501
+        
+            # Service Discovery
+            ## hive_discovery_llap = true
+            ## hive_discovery_llap_ha = false
+            # Shortcuts to finding LLAP znode Key
+            # Non-HA - hiveserver-interactive-site - hive.server2.zookeeper.namespace ex hive2 = /hive2
+            # HA-NonKerberized - <llap_app_name>_llap ex app name llap0 = /llap0_llap
+            # HA-Kerberized - <llap_app_name>_llap-sasl ex app name llap0 = /llap0_llap-sasl
+            ## hive_discovery_llap_znode = /hiveserver2-hive2
+
+**Service Discovery**
+
+When setup, Hue will query zookeeper to find an enabled hiveserver2 or llap endpoint.
+
+        [beeswax]
+            ## hive_discovery_llap = true
+            ## hive_discovery_hs2 = true
+
+In order to prevent spamming zookeeper, HiveServer2 is cached for the life of the process and llap is cached based on the following setting:
+
+        [beeswax]
+            cache_timeout = 60
+
 **Tez**
 
 Requires support for sending multiple queries when using Tez (instead of a maximum of just one at the time). You can turn it on with this setting: