瀏覽代碼

HUE-8741 [doc] Add info about SparkSQL Livy connector

Romain Rigaux 6 年之前
父節點
當前提交
d0a3897
共有 5 個文件被更改,包括 38 次插入5 次删除
  1. 1 4
      apps/spark/java/README.rst
  2. 4 0
      desktop/conf.dist/hue.ini
  3. 4 0
      desktop/conf/pseudo-distributed.ini.tmpl
  4. 24 1
      docs/admin-manual/manual.md
  5. 5 0
      docs/sdk/sdk.md

+ 1 - 4
apps/spark/java/README.rst

@@ -1,9 +1,6 @@
 Welcome to Livy
 Welcome to Livy
 ===============
 ===============
 
 
-Livy is an open source REST interface for interacting with Apache Spark from anywhere. 
+Livy is an open source REST interface for interacting with Apache Spark from anywhere.
 
 
 Livy was originally created in Hue and then moved to its own project http://livy.io.
 Livy was originally created in Hue and then moved to its own project http://livy.io.
-
-And read more about Spark in Hue on http://gethue.com/spark/.
-

+ 4 - 0
desktop/conf.dist/hue.ini

@@ -854,6 +854,10 @@
     #   name=SparkSql
     #   name=SparkSql
     #   interface=hiveserver2
     #   interface=hiveserver2
 
 
+    [[[sparksql]]]
+      name=SparkSql
+      interface=livy
+
     [[[spark]]]
     [[[spark]]]
       name=Scala
       name=Scala
       interface=livy
       interface=livy

+ 4 - 0
desktop/conf/pseudo-distributed.ini.tmpl

@@ -856,6 +856,10 @@
     #   name=SparkSql
     #   name=SparkSql
     #   interface=hiveserver2
     #   interface=hiveserver2
 
 
+    [[[sparksql]]]
+      name=SparkSql
+      interface=livy
+
     [[[spark]]]
     [[[spark]]]
       name=Scala
       name=Scala
       interface=livy
       interface=livy

+ 24 - 1
docs/admin-manual/manual.md

@@ -310,6 +310,20 @@ Same as Presto.
 
 
 ### Spark SQL
 ### Spark SQL
 
 
+Via [Apache Livy](https://livy.incubator.apache.org/) (recommended):
+
+    [[[sparksql]]]
+      name=SparkSql
+      interface=livy
+
+    ...
+
+    [spark]
+      # The Livy Server URL.
+      livy_server_url=http://localhost:8998
+
+Via native HiveServer2 API:
+
     [[[sparksql]]]
     [[[sparksql]]]
       name=SparkSql
       name=SparkSql
       interface=hiveserver2
       interface=hiveserver2
@@ -466,13 +480,16 @@ Before adding your interpreter’s JDBC configurations to hue.ini, verify that t
 You can leave out the username and password in the JDBC options, and Hue will instead prompt the user for a username and password. This allows administrators to provide access to JDBC sources without granting all Hue users the same access.
 You can leave out the username and password in the JDBC options, and Hue will instead prompt the user for a username and password. This allows administrators to provide access to JDBC sources without granting all Hue users the same access.
 
 
 
 
-
 ### Spark
 ### Spark
 
 
     [[[pyspark]]]
     [[[pyspark]]]
       name=PySpark
       name=PySpark
       interface=livy
       interface=livy
 
 
+    [[[sparksql]]]
+      name=SparkSql
+      interface=livy
+
     [[[spark]]]
     [[[spark]]]
       name=Scala
       name=Scala
       interface=livy
       interface=livy
@@ -481,6 +498,12 @@ You can leave out the username and password in the JDBC options, and Hue will in
         name=R
         name=R
         interface=livy
         interface=livy
 
 
+    ...
+
+    [spark]
+      # The Livy Server URL.
+      livy_server_url=http://localhost:8998
+
 ### Pig
 ### Pig
 
 
     [[[pig]]]
     [[[pig]]]

+ 5 - 0
docs/sdk/sdk.md

@@ -71,7 +71,12 @@ MapReduce, Pig, Java, Shell, Sqoop, DistCp [Oozie connector](https://github.com/
 
 
 ### Spark / Livy
 ### Spark / Livy
 
 
+Based on the [Livy REST API](https://livy.incubator.apache.org/docs/latest/rest-api.html)
+
 * [Notebook connector](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/spark_shell.py)
 * [Notebook connector](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/spark_shell.py)
+  * PySpark
+  * Scala
+  * Spark SQL
 * [Batch connector](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/spark_batch.py)
 * [Batch connector](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/spark_batch.py)
 
 
 # Dashboard
 # Dashboard