Pārlūkot izejas kodu

HUE-8741 [doc] Add info about SparkSQL Livy connector

Romain Rigaux 6 gadi atpakaļ
vecāks
revīzija
d0a3897

+ 1 - 4
apps/spark/java/README.rst

@@ -1,9 +1,6 @@
 Welcome to Livy
 ===============
 
-Livy is an open source REST interface for interacting with Apache Spark from anywhere. 
+Livy is an open source REST interface for interacting with Apache Spark from anywhere.
 
 Livy was originally created in Hue and then moved to its own project http://livy.io.
-
-And read more about Spark in Hue on http://gethue.com/spark/.
-

+ 4 - 0
desktop/conf.dist/hue.ini

@@ -854,6 +854,10 @@
     #   name=SparkSql
     #   interface=hiveserver2
 
+    [[[sparksql]]]
+      name=SparkSql
+      interface=livy
+
     [[[spark]]]
       name=Scala
       interface=livy

+ 4 - 0
desktop/conf/pseudo-distributed.ini.tmpl

@@ -856,6 +856,10 @@
     #   name=SparkSql
     #   interface=hiveserver2
 
+    [[[sparksql]]]
+      name=SparkSql
+      interface=livy
+
     [[[spark]]]
       name=Scala
       interface=livy

+ 24 - 1
docs/admin-manual/manual.md

@@ -310,6 +310,20 @@ Same as Presto.
 
 ### Spark SQL
 
+Via [Apache Livy](https://livy.incubator.apache.org/) (recommended):
+
+    [[[sparksql]]]
+      name=SparkSql
+      interface=livy
+
+    ...
+
+    [spark]
+      # The Livy Server URL.
+      livy_server_url=http://localhost:8998
+
+Via native HiveServer2 API:
+
     [[[sparksql]]]
       name=SparkSql
       interface=hiveserver2
@@ -466,13 +480,16 @@ Before adding your interpreter’s JDBC configurations to hue.ini, verify that t
 You can leave out the username and password in the JDBC options, and Hue will instead prompt the user for a username and password. This allows administrators to provide access to JDBC sources without granting all Hue users the same access.
 
 
-
 ### Spark
 
     [[[pyspark]]]
       name=PySpark
       interface=livy
 
+    [[[sparksql]]]
+      name=SparkSql
+      interface=livy
+
     [[[spark]]]
       name=Scala
       interface=livy
@@ -481,6 +498,12 @@ You can leave out the username and password in the JDBC options, and Hue will in
         name=R
         interface=livy
 
+    ...
+
+    [spark]
+      # The Livy Server URL.
+      livy_server_url=http://localhost:8998
+
 ### Pig
 
     [[[pig]]]

+ 5 - 0
docs/sdk/sdk.md

@@ -71,7 +71,12 @@ MapReduce, Pig, Java, Shell, Sqoop, DistCp [Oozie connector](https://github.com/
 
 ### Spark / Livy
 
+Based on the [Livy REST API](https://livy.incubator.apache.org/docs/latest/rest-api.html)
+
 * [Notebook connector](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/spark_shell.py)
+  * PySpark
+  * Scala
+  * Spark SQL
 * [Batch connector](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/spark_batch.py)
 
 # Dashboard