浏览代码

HUE-8741 [doc] Clean-up of the end user guide

Romain Rigaux 6 年之前
父节点
当前提交
e9ebc51
共有 3 个文件被更改,包括 163 次插入185 次删除
  1. 53 5
      docs/admin-manual/manual.md
  2. 15 23
      docs/sdk/sdk.md
  3. 95 157
      docs/user-guide/user-guide.md

+ 53 - 5
docs/admin-manual/manual.md

@@ -215,6 +215,34 @@ It is available in Editor or Notebook mode and focuses on SQL. Dialects can be a
           interface=sqlalchemy
           options='{"url": "mysql://root:root@localhost:3306/hue"}'
 
+
+Download and export options with limited scalability can be limited in the number of rows or bytes transferred using the following options respectively in your hue.ini:
+
+<pre>
+  [beeswax]
+  # A limit to the number of rows that can be downloaded from a query before it is truncated.
+  # A value of -1 means there will be no limit.
+  download_row_limit=-1
+
+  # A limit to the number of bytes that can be downloaded from a query before it is truncated.
+  # A value of -1 means there will be no limit.
+  download_bytes_limit=-1
+</pre>
+
+In addition, it is possible to disable the download and export feature in the editor, dashboard, as well as in the file browser with the following option in your hue.ini:
+<pre>
+  [desktop]
+  # Global setting to allow or disable end user downloads in all Hue.
+  # e.g. Query result in Editors and Dashboards, file in File Browser...
+  enable_download=false
+</pre>
+
+The download feature in the file browser can be disabled separately with the following options in your hue.ini:
+<pre>
+[filebrowser]
+show_download_button=false
+</pre>
+
 ### Impala
 
     [impala]
@@ -245,6 +273,7 @@ Requires support for sending multiple queries when using Tez (instead of a maxim
     [beeswax]
     max_number_of_sessions=10
 
+
 ### MySQL
 
 Recommended way:
@@ -290,6 +319,27 @@ The Presto JDBC client driver is maintained by the Presto Team and can be downlo
 
 ### PostgreSQL
 
+First, in your hue.ini file, you will need to add the relevant database connection information under the librdbms section:
+
+    [librdbms]
+      [[databases]]
+        [[[postgresql]]]
+        nice_name=PostgreSQL
+        name=music
+        engine=postgresql_psycopg2
+        port=5432
+        user=hue
+        password=hue
+        options={}
+
+Secondly, we need to add a new interpreter to the notebook app. This will allow the new database type to be registered as a snippet-type in the Notebook app. For query editors that use a Django-compatible database, the name in the brackets should match the database configuration name in the librdbms section (e.g. – postgresql). The interface will be set to rdbms. This tells Hue to use the librdbms driver and corresponding connection information to connect to the database. For example, with the above postgresql connection configuration in the librdbms section, we can add a PostgreSQL interpreter with the following notebook configuration:
+
+    [notebook]
+      [[interpreters]]
+        [[[postgresql]]]
+        name=PostgreSQL
+        interface=rdbms
+
 ### AWS Athena
 
 Same as Presto.
@@ -1451,11 +1501,9 @@ import groups from an LDAP directory.
 
 #### Limit users can login only if they are belong to one of listed LDAP groups
 
-<pre>
-  [desktop]
-  [[ldap]]
-  login_groups=ldap_grp1,ldap_grp2,ldap_grp3
-</pre>
+    [desktop]
+    [[ldap]]
+    login_groups=ldap_grp1,ldap_grp2,ldap_grp3
 
 #### Importing Groups from an LDAP Directory
 

+ 15 - 23
docs/sdk/sdk.md

@@ -19,25 +19,24 @@
 
 # Concept
 
-Hue is generic and let's you integrate with other analytics systems so that for example
-your users can explore data with other databases.
-In addition, whole new apps can also be created in order to provide end user solutions.
+Hue service integration is generic and lets you integrate with other analytics systems. Here is a list of the main APIs:
 
-# Editor / Notebook
+* Connect or create SQL connectors to any Database
+* List any files from any filesytem
+* List any job or query list
+* Integrate with a Data Catalog
 
-The goal of the Editor is to open-up data to more users by making self service querying easy and productive.
+REST APIs are not all public yet but this is work in progress in [HUE-1450](https://issues.cloudera.org/browse/HUE-1450).
+Upcoming APIs: how to add new vizualizations, new SQL grammar and highlighting, workflow systems.
+In addition, whole new apps can also be created in order to provide end to end solutions.
 
-It is available in Editor or Notebook mode and will be integrated with the Dashboard soon. The Editor focuses on Apache Hive and Apache Impala but is also compatible with:
+# Editor / Notebook
 
-* Any SQL databases: MySQL, SparkSQL, Oracle, Apache Phoenix, Apache Presto, Apache Drill, Apache Kylin, PostgreSQL, Redshift, BigQuery…
-* MapReduce
-* Spark
-* Pig
-* Solr SQL
+They provide SQL integration with any database via several connectors (native, JDBC, SQL Alchemy...).
 
-Other modes like MapReduce, Java, Shell, Sqoop are also available. Here is a list of the [https://github.com/cloudera/hue/tree/master/desktop/libs/notebook/src/notebook/connectors](existing connectors).
+Other modes like MapReduce, Java, Shell, Sqoop are also available. Here is a list of the [existing connectors](https://github.com/cloudera/hue/tree/master/desktop/libs/notebook/src/notebook/connectors).
 
-Connectors are pluggable and can new engines can be supported. Feel free to comment on the [https://groups.google.com/a/cloudera.org/forum/#!forum/hue-user](Hue list) of [https://github.com/cloudera/hue/issues](github) about it.
+Connectors are pluggable and can new engines can be supported. Feel free to comment on the [Hue list](https://groups.google.com/a/cloudera.org/forum/#!forum/hue-user) of [github](https://github.com/cloudera/hue/issues) about it.
 
 ## SQL
 
@@ -867,18 +866,11 @@ Django, running on the WSGI container/web server (typically CherryPy), manages
 the url dispatch, executes application logic code, and puts together the views
 from their templates.  Django uses a database (typically sqlite)
 to manage session data, and Hue applications can use it as well
-for their "models".  (For example, the JobDesigner application stores
-job designs in the database.)
+for their "models".  (For example, the saved Editor stores
+saved queries in the database.)
 
 In addition to the web server, some Hue applications run
-daemon processes "on the side".  For example, Spark runs a daemon
-("livy_server") that keeps track of the Spark shells of the user. Running
-a separate process for applications is the preferred
-way to manage long-running tasks that you may wish
-to co-exist with web page rendering.  The web "views"
-typically communicate with these side daemons
-by using Thrift (e.g., for Hive query execution) or by exchanging state
-through the database.
+daemon processes "on the side". Some examples are the `Celery Task Server`, `Celery Beat`.
 
 ### Interacting with Hadoop
 

+ 95 - 157
docs/user-guide/user-guide.md

@@ -39,7 +39,6 @@ When you found your data, the Editor's autocomplete is extremely powerful as the
 
 Each app of Hue can be extended to support your own languages or apps as detailed in the [SDK](../sdk/sdk.html).
 
-
 ## Interface
 
 The layout simplifies the interface and is now single page app, and this makes things snappier and unifies the apps together.
@@ -58,7 +57,7 @@ From top to bottom we have:
 Learn more on the [The Hue 4 user interface in detail](http://gethue.com/the-hue-4-user-interface-in-detail/).
 
 
-### Top search
+## Top search
 
 The new search bar is always accessible on the top of screen, and it offers a document search and metadata search too if Hue is configured to access a metadata server like Cloudera Navigator.
 
@@ -85,90 +84,22 @@ Example of searches:
 
 Learn more on the [Search and Tagging](https://blog.cloudera.com/blog/2017/05/new-in-cloudera-enterprise-5-11-hue-data-search-and-tagging/).
 
-### Left assist
+## Left assist
 
 Data where you need it when you need it
 
 You can now find your Hue documents, HDFS and S3 files and more in the left assist panel, right-clicking items will show a list of actions, you can also drag-and-drop a file to get the path in your editor and more.
 
-### Right assist
+## Right assist
 This assistant content depends on the context of the application selected and will display the current tables or available UDFs.
 
-### Sample popup
+## Sample popup
 
 This popup offers a quick way to see sample of the data and other statistics on databases, tables, and columns. You can open the popup from the SQL Assist or with a right-click on any SQL object (table, column, function…). In this release, it also opens faster and caches the data.
 
-### Charting
-
-These visualizations are convenient for plotting chronological data or when subsets of rows have the same attribute: they will be stacked together.
-
-* Pie
-* Bar/Line with pivot
-* Timeline
-* Scattered plot
-* Maps (Marker and Gradient)
-
-## Data Importer
-
-The goal of the importer is to allow ad hoc queries on data not yet in the clusters thereby expedite self-service analytics.
-
-If you want to import your own data instead of installing the sample
-tables, open the importer from the left menu or from the little `+` in the left assist.
-
-If you've ever struggled with creating new SQL tables from files, you'll be happy to learn that this is now much easier. The wizard has been revamped to two simple steps and also offers more formats. Now users just need to:
-
-1. Select a source type
-2. Select the type of object for the destination
-
-And that's it!
-
-To learn more, watch the video on [Data Import Wizard](http://gethue.com/import-data-to-be-queried-via-the-self-service-drag-drop-create-table-wizard/).
-
-### SQL Tables
-
-Although you can create tables by executing the appropriate Hive HQL DDL
-query commands, it is easier to create a table using the create table wizard.
-
-**From a File**
-
-If you've ever struggled with creating new SQL tables from files, you'll be happy to learn that this is now much easier. With the latest Hue release, you can now create these in an ad hoc way and thereby expedite self-service analytics. The wizard has been revamped to two simple steps and also offers more formats. Now users just need to:
-
-1. In the Importer Manager selects source from a 'File'
-1. Select the type of table
-
-Files can be dragged & dropped, selected from HDFS or S3 (if configured), and their formats are automatically detected. The wizard also assists when performing advanced functionalities like table partitioning, Kudu tables, and nested types.
-
-
-**Manually**
-
-1.  In the Importer Manager selects 'Manually'
-2.  Follow the instructions in the wizard to create the table. The basic
-    steps are:
-    -   Name the table.
-    -   Choose the record format.
-    -   Configure record serialization by specifying delimiters for
-        columns, collections, and map keys.
-    -   Choose the file format.
-    -   Specify the location for your table's data.
-    -   Specify the columns, providing a name and selecting the type for
-        each column.
-    -   Specify partition columns, providing a name and selecting the
-        type for each column.
-
-
-### Indexing
-
-In the past, indexing data into Solr to then explore it with a [Dynamic Dashboard](http://gethue.com/search-dashboards/) has been quite difficult. The task involved writing a Solr schema and a Morphlines file then submitting a job to YARN to do the indexing. Often times getting this correct for non trivial imports could take a few days of work. Now with Hue's new feature you can start your YARN indexing job in minutes. This tutorial offers a step by step guide on how to do it.
-
-[Read more about it here](http://gethue.com/easy-indexing-of-data-into-solr/).
-
-### Traditional Databases
-
-Read more about [ingesting data from traditional databases](http://gethue.com/importing-data-from-traditional-databases-into-hdfshive-in-just-a-few-clicks/).
-
 ## Documents
 
-Similarly to Google Doc, queries, workflows... can be saved and shared with other.
+Similarly to Google Document, queries, workflows... can be saved and shared with other users.
 
 ### Sharing
 
@@ -184,7 +115,8 @@ The language is automatically detected from the Browser or OS. English, Spanish,
 
 The language can be manual set by a user in the "My Profile" page. Please go to My Profile > Step2 Profile and Groups > Language Preference and choose the language you want.
 
-# Editors / Notebook
+
+# SQL Editors / Notebook
 The goal of Hue's Editor is to make data querying easy and productive.
 
 It focuses on SQL but also supports job submissions. It comes with an intelligent autocomplete, search & tagging of data and query assistance.
@@ -194,28 +126,24 @@ next to its name so that it becomes the default editor and the landing page when
 
 First, in your hue.ini file, you will need to add the relevant database connection information under the librdbms section:
 
-<pre>
-[librdbms]
-  [[databases]]
-    [[[postgresql]]]
-    nice_name=PostgreSQL
-    name=music
-    engine=postgresql_psycopg2
-    port=5432
-    user=hue
-    password=hue
-    options={}
-</pre>
+    [librdbms]
+      [[databases]]
+        [[[postgresql]]]
+        nice_name=PostgreSQL
+        name=music
+        engine=postgresql_psycopg2
+        port=5432
+        user=hue
+        password=hue
+        options={}
 
 Secondly, we need to add a new interpreter to the notebook app. This will allow the new database type to be registered as a snippet-type in the Notebook app. For query editors that use a Django-compatible database, the name in the brackets should match the database configuration name in the librdbms section (e.g. – postgresql). The interface will be set to rdbms. This tells Hue to use the librdbms driver and corresponding connection information to connect to the database. For example, with the above postgresql connection configuration in the librdbms section, we can add a PostgreSQL interpreter with the following notebook configuration:
 
-<pre>
-[notebook]
-  [[interpreters]]
-    [[[postgresql]]]
-    name=PostgreSQL
-    interface=rdbms
-</pre>
+    [notebook]
+      [[interpreters]]
+        [[[postgresql]]]
+        name=PostgreSQL
+        interface=rdbms
 
 ## Concepts
 ### Running Queries
@@ -260,31 +188,6 @@ To get things started, press the export icon, the bottom last element of the act
 1.  Export to a file on your cluster's file systems. This exports the results to a single file. In the export icon, choose Export and then First XXX.
 2.  Download to your computer as a CSV or XLS. This exports the results to a single file in comma-separated values or Microsoft Office Excel format. In the export icon, choose Download as CSV or Download as XLS.
 
-Download and export options with limited scalability can be limited in the number of rows or bytes transferred using the following options respectively in your hue.ini:
-<pre>
-  [beeswax]
-  # A limit to the number of rows that can be downloaded from a query before it is truncated.
-  # A value of -1 means there will be no limit.
-  download_row_limit=-1
-
-  # A limit to the number of bytes that can be downloaded from a query before it is truncated.
-  # A value of -1 means there will be no limit.
-  download_bytes_limit=-1
-</pre>
-
-In addition, it is possible to disable the download and export feature in the editor, dashboard, as well as in the file browser with the following option in your hue.ini:
-<pre>
-  [desktop]
-  # Global setting to allow or disable end user downloads in all Hue.
-  # e.g. Query result in Editors and Dashboards, file in File Browser...
-  enable_download=false
-</pre>
-
-The download feature in the file browser can be disabled separately with the following options in your hue.ini:
-<pre>
-[filebrowser]
-show_download_button=false
-</pre>
 
 <a id="advancedQuerySettings"></a>
 ### Advanced Query Settings
@@ -409,6 +312,7 @@ For values that are not textual, omit the quotes.
 <pre>
 select * from boolean_table where boolean_column = ${boolean_column}
 </pre>
+
 ### Syntax checker
 
 A little red underline will display the incorrect syntax so that the query can be fixed before submitting. A right click offers suggestions.
@@ -418,6 +322,17 @@ A little red underline will display the incorrect syntax so that the query can b
 Read more about the [Query Assistant with Navigator Optimizer Integration
 ](https://blog.cloudera.com/blog/2017/08/new-in-cloudera-enterprise-5-12-hue-4-interface-and-query-assistant/).
 
+### Charting
+
+These visualizations are convenient for plotting chronological data or when subsets of rows have the same attribute: they will be stacked together.
+
+* Pie
+* Bar/Line with pivot
+* Timeline
+* Scattered plot
+* Maps (Marker and Gradient)
+
+Read more about extending [charts](../sdk/sdk.html).
 
 ### Risk Alerts
 
@@ -431,9 +346,9 @@ While editing, Hue will run your queries through Navigator Optimizer in the back
 
 Turns a list of semi-colon separated queries into an interactive presentation. It is great for doing demos or basic reporting.
 
-## SQL
+## SQL Databases
 
-Use the query editor with any [JDBC](http://gethue.com/custom-sql-query-editors/) or Django-compatible database.
+Use the query editor with any database.
 
 ### Hive
 ### Impala
@@ -471,7 +386,7 @@ Apache Kylin is an open-source online analytical processing (OLAP) engine.
 See how to configure the [Kylin Query Editor](http://gethue.com/using-hue-to-interact-with-apache-kylin/).
 
 ### Others
-Extend with JDBC or your own [connectors](../sdk/sdk.html#sql).
+Extend with SQL Alchemy, JDBC or build your own [connectors](../sdk/sdk.html#sql).
 
 ## Jobs
 
@@ -609,8 +524,63 @@ Make sure that the Notebook and interpreters are set in the hue.ini, and Livy is
       interface=livy
 </pre>
 
-## SDK
-Read more about extending [connectors](../sdk/sdk.html#spark-livy).
+# Data Importer
+
+The goal of the importer is to allow ad hoc queries on data not yet in the clusters thereby expedite self-service analytics.
+
+If you want to import your own data instead of installing the sample
+tables, open the importer from the left menu or from the little `+` in the left assist.
+
+If you've ever struggled with creating new SQL tables from files, you'll be happy to learn that this is now much easier. The wizard has been revamped to two simple steps and also offers more formats. Now users just need to:
+
+1. Select a source type
+2. Select the type of object for the destination
+
+And that's it!
+
+To learn more, watch the video on [Data Import Wizard](http://gethue.com/import-data-to-be-queried-via-the-self-service-drag-drop-create-table-wizard/).
+
+## SQL Tables
+
+Although you can create tables by executing the appropriate Hive HQL DDL
+query commands, it is easier to create a table using the create table wizard.
+
+**From a File**
+
+If you've ever struggled with creating new SQL tables from files, you'll be happy to learn that this is now much easier. With the latest Hue release, you can now create these in an ad hoc way and thereby expedite self-service analytics. The wizard has been revamped to two simple steps and also offers more formats. Now users just need to:
+
+1. In the Importer Manager selects source from a 'File'
+1. Select the type of table
+
+Files can be dragged & dropped, selected from HDFS or S3 (if configured), and their formats are automatically detected. The wizard also assists when performing advanced functionalities like table partitioning, Kudu tables, and nested types.
+
+
+**Manually**
+
+1.  In the Importer Manager selects 'Manually'
+2.  Follow the instructions in the wizard to create the table. The basic
+    steps are:
+    -   Name the table.
+    -   Choose the record format.
+    -   Configure record serialization by specifying delimiters for
+        columns, collections, and map keys.
+    -   Choose the file format.
+    -   Specify the location for your table's data.
+    -   Specify the columns, providing a name and selecting the type for
+        each column.
+    -   Specify partition columns, providing a name and selecting the
+        type for each column.
+
+
+## Indexing
+
+In the past, indexing data into Solr to then explore it with a [Dynamic Dashboard](http://gethue.com/search-dashboards/) has been quite difficult. The task involved writing a Solr schema and a Morphlines file then submitting a job to YARN to do the indexing. Often times getting this correct for non trivial imports could take a few days of work. Now with Hue's new feature you can start your YARN indexing job in minutes. This tutorial offers a step by step guide on how to do it.
+
+[Read more about it here](http://gethue.com/easy-indexing-of-data-into-solr/).
+
+## Traditional Databases
+
+Read more about [ingesting data from traditional databases](http://gethue.com/importing-data-from-traditional-databases-into-hdfshive-in-just-a-few-clicks/).
 
 
 # Dashboards
@@ -933,41 +903,9 @@ column.
     job you want to view. This shows the **Job** page for the job, with
     the recent tasks associated with the job are displayed in the
     **Tasks** tab.
-2.  Click the **Metadata** tab to view the metadata for this job.
+2.  Click the **Logs** tab to view the logs for this job.
 3.  Click the **Counters** tab to view the counter metrics for the job.
 
-**To view details about the tasks associated with the job:**
-
-1.  In the **Job** window, click the **View All Tasks** link at the
-    right just above the **Recent Tasks** list. This lists all the tasks
-    associated with the job.
-2.  Click **Attempts** to the right of a task to view the attempts for
-    that task.
-
-**To view information about an individual task:**
-
-1.  In the **Job** window, click the **View** link to the right of the
-    task. The attempts associated with the task are displayed.
-2.  Click the **Metadata** tab to view metadata for this task. The
-    metadata associated with the task is displayed.
-3.  To view the Hadoop counters for a task, click the **Counters** tab.
-    The counters associated with the task are displayed.
-4.  To return to the **Job** window for this job, click the job number
-    in the status panel at the left of the window.
-
-**To view details about a task attempt:**
-
-1.  In the **Job Task** window, click the **View** link to the right of
-    the task attempt. The metadata associated with the attempt is
-    displayed under the **Metadata** tab.
-2.  To view the Hadoop counters for the task attempt, click the
-    **Counters** tab. The counters associated with the attempt are
-    displayed.
-3.  To view the logs associated with the task attempt, click the
-    **Logs** tab. The logs associated with the task attempt are
-    displayed.
-4.  To return to the list of tasks for the current job, click the task
-    number in the status panel at the left of the window.
 
 ### Types
 #### YARN (Spark, MapReduce)