Browse Source

HUE-2890 [doc] Clean-up of the manual

Romain Rigaux 8 years ago
parent
commit
afde680ec9
1 changed files with 71 additions and 210 deletions
  1. 71 210
      docs/manual.txt

+ 71 - 210
docs/manual.txt

@@ -5,22 +5,15 @@ Introduction
 ------------
 ------------
 
 
 Hue is a graphical user interface to operate and develop applications for
 Hue is a graphical user interface to operate and develop applications for
-Apache Hadoop. Hue applications are collected into a desktop-style environment
-and delivered as a Web application, requiring no additional installation for
-individual users.
+performing self-service data analytics. Hue applications are delivered as a Web application,
+requiring no additional installation for individual users.
 
 
-This guide describes how to install and configure a Hue tarball. For
-information about installing Hue packages, see
-http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/cdh5ig_cdh_hue_configure.html[Installing Hue].
+This guide describes how to install and configure a Hue tarball or packages.
 
 
 There is also a companion SDK guide that describes how to develop
 There is also a companion SDK guide that describes how to develop
 new Hue applications:
 new Hue applications:
 link:sdk/sdk.html[Hue SDK Documentation]
 link:sdk/sdk.html[Hue SDK Documentation]
 
 
-.Conventions Used in this Guide:
-* Commands that must be run with +root+ permission have a +#+ command prompt.
-* Commands that do not require +root+ permission have a +$+ command prompt.
-
 
 
 Hue Installation Instructions
 Hue Installation Instructions
 -----------------------------
 -----------------------------
@@ -51,29 +44,6 @@ certain development libraries be installed on your system. To install from the
 tarball, you must have the following installed:
 tarball, you must have the following installed:
 
 
 .Required Dependencies
 .Required Dependencies
-[grid="rows"]
-```~
-Redhat,Ubuntu 10.04,Ubuntu 12.04/14.04
-~~~~~~~~~~
-gcc,gcc,gcc
-g++,g++,g++
-libxml2-devel,libxml2-dev,libxml2-dev
-libxslt-devel,libxslt-dev,libxslt-dev
-cyrus-sasl-devel,libsasl2-dev,libsasl2-dev
-cyrus-sasl-gssapi,libsasl2-modules-gssapi-mit,libsasl2-modules-gssapi-mit
-mysql-devel,libmysqlclient-dev,libmysqlclient-dev
-python-devel,python-dev,python-dev
-python-setuptools,python-setuptools,python-setuptools
-sqlite-devel,libsqlite3-dev,libsqlite3-dev
-ant,ant,ant
-libsasl2-dev,cyrus-sasl-devel,libsasl2-dev
-libsasl2-modules-gssapi-mit,cyrus-sasl-gssapi,libsasl2-modules-gssapi-mit
-libkrb5-dev,krb5-devel,libkrb5-dev
-libtidy-0.99-0,libtidy,libtidy-0.99-0 (For unit tests only)
-mvn,mvn (From maven2 package or tarball),mvn (From maven2/maven3 package or tarball)
-openldap-dev / libldap2-dev,openldap-devel,libldap2-dev
-~~~~~~~~~~
-
 The full list is here: https://github.com/cloudera/hue#development-prerequisites
 The full list is here: https://github.com/cloudera/hue#development-prerequisites
 
 
 Build
 Build
@@ -81,13 +51,10 @@ Build
 
 
 Configure `$PREFIX` with the path where you want to install Hue by running:
 Configure `$PREFIX` with the path where you want to install Hue by running:
 
 
-  $ PREFIX=/usr/share make install
-  $ cd /usr/share/hue
+  PREFIX=/usr/share make install
+  cd /usr/share/hue
 
 
 You can install Hue anywhere on your system, and run Hue as a non-root user.
 You can install Hue anywhere on your system, and run Hue as a non-root user.
-The Shell application needs root privileges to launch various sub-processes as
-the logged in users. `$USER` should be the group of the user running Hue (same as
-its username).
 
 
 It is a good practice to create a new user for Hue and either install Hue in
 It is a good practice to create a new user for Hue and either install Hue in
 that user's home directory, or in a directory within `/usr/share`.
 that user's home directory, or in a directory within `/usr/share`.
@@ -103,9 +70,9 @@ A: Due to the use of absolute paths by some Python packages, you must run a
 series of commands if you move your Hue installation. In the new location, run:
 series of commands if you move your Hue installation. In the new location, run:
 
 
 ----
 ----
-$ rm app.reg
-$ rm -r build
-$ make apps
+rm app.reg
+rm -r build
+make apps
 ----
 ----
 
 
 .Q: Why does "make install" compile other pieces of software?
 .Q: Why does "make install" compile other pieces of software?
@@ -125,22 +92,14 @@ are already setup (that way Hue can talk to them).
 
 
 .Dependency
 .Dependency
 [grid="rows"]
 [grid="rows"]
-`-----------------------------------------------.------------------------
-  Component      Applications                       Notes
--------------------------------------------------------------------------
-  HDFS        Core, Filebrowser                  HDFS access through WebHdfs or HttpFS
-  MR1         JobBrowser, JobDesigner, Beeswax   Job information access through hue-plugins
-  MR2/YARN    JobBrowser, JobDesigner, Beeswax   Job information access through hue-plugins
-  Oozie       JobDesigner, Oozie                 Oozie access through REST API
-  Hive        Beeswax                            Requires HiveServer2
-  HBase       HBase Browser                      Requires Thrift 1 service
-  Pig         Pig Editor                         Requires Oozie
-  Sqoop2      Sqoop Editor                       Requires Sqoop2 server
-  Search      Search                             Requires Solr server
-  Impala      Impala Editor                      Requires an Impalad
-  ZooKeeper   ZooKeeper Browser                  Requires ZooKeeper server and REST server
-  Spark       Spark Editor                       Requires Spark Jobserver
---------------------------------------------------------------------------------------
+`-----------------------------------------------------------------------
+  Component      Applications                       
+------------------------------------------------------------------------
+  Editor      SQL (Hive, Impala, any database...), Pig, Spark...
+  Browsers    YARN, Oozie, Impala, HBase, Livy
+  Scheduler   Oozie
+  Dashboard   Solr, SQL (Impala, Hive...)
+------------------------------------------------------------------------
 
 
 
 
 Hadoop Configuration
 Hadoop Configuration
@@ -185,48 +144,6 @@ Also add this in `httpfs-site.xml` which might be in `/etc/hadoop-httpfs/conf`.
     </property>
     </property>
 
 
 
 
-Configure MapReduce 0.23+ (MR2)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Hue is using only standard Hadoop APIs.
-
-
-Configure MapReduce 0.20 (MR1)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Hue communicates with the JobTracker via the Hue plugins, which is a jar
-file that you place in your MapReduce `lib` directory.
-
-If you JobTracker and Hue are located on the same host, copy it over.
-
-    $ cd /usr/share/hue
-    $ cp desktop/libs/hadoop/java-lib/hue-plugins-*.jar /usr/lib/hadoop-0.20-mapreduce/lib
-
-If you JobTracker runs on a different host, you need to `scp` the Hue plugins
-jar to the JobTracker host.
-
-Then add this to your `mapred-site.xml` and *restart* your JobTracker.
-Depending on your setup, your `mapred-site.xml` might be in `/etc/hadoop/conf`.
-
-    <property>
-      <name>jobtracker.thrift.address</name>
-      <value>0.0.0.0:9290</value>
-    </property>
-    <property>
-      <name>mapred.jobtracker.plugins</name>
-      <value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value>
-      <description>Comma-separated list of jobtracker plug-ins to be activated.</description>
-    </property>
-
-You can confirm that the plugins are running correctly by tailing the daemon
-logs:
-
-    $ tail --lines=500 /var/log/hadoop-0.20/hadoop*jobtracker*.log | grep ThriftPlugin
-    2009-09-28 16:30:44,337 INFO org.apache.hadoop.thriftfs.ThriftPluginServer: Starting Thrift server
-    2009-09-28 16:30:44,419 INFO org.apache.hadoop.thriftfs.ThriftPluginServer:
-    Thrift server listening on 0.0.0.0:9290
-
-
 Configure Oozie
 Configure Oozie
 ^^^^^^^^^^^^^^^
 ^^^^^^^^^^^^^^^
 
 
@@ -244,55 +161,6 @@ your `oozie-site.xml` (even in a non-secure cluster), and restart Oozie:
     </property>
     </property>
 
 
 
 
-
-Further Hadoop Configuration and Caveats
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-`HADOOP_CLASSPATH` Caveat
-^^^^^^^^^^^^^^^^^^^^^^^^^
-
-If you are setting `$HADOOP_CLASSPATH` in your `hadoop-env.sh`, be sure
-to set it in such a way that user-specified options are preserved.  For example:
-
-Correct:
-
-    # HADOOP_CLASSPATH=<your_additions>:$HADOOP_CLASSPATH
-
-Incorrect:
-
-    # HADOOP_CLASSPATH=<your_additions>
-
-This enables certain components of Hue to add to
-Hadoop's classpath using the environment variable.
-
-`hadoop.tmp.dir`
-^^^^^^^^^^^^^^^^
-
-If your users are likely to be submitting jobs both using Hue and from the
-same machine via the command line interface, they will be doing so as the `hue`
-user if they're using Hue and via their own user account on the command line.
-This leads to some contention on the directory specified by `hadoop.tmp.dir`,
-which defaults to `/tmp/hadoop-${user.name}`. Specifically, `hadoop.tmp.dir`
-is used to unpack jars in `bin/hadoop jar`. One work around to this is
-to set `hadoop.tmp.dir` to `/tmp/hadoop-${user.name}-${hue.suffix}` in the
-core-site.xml file:
-
-    <property>
-      <name>hadoop.tmp.dir</name>
-      <value>/tmp/hadoop-${user.name}${hue.suffix}</value>
-    </property>
-
-Unfortunately, when the variable is unset, you'll end up
-with directories named `/tmp/hadoop-user_name-${hue.suffix}` in
-`/tmp`.  Despite that, Hue will still work.
-
-IMPORTANT: The Beeswax server writes into a local directory on the Hue machine
-that is specified by `hadoop.tmp.dir` to unpack its jars. That directory
-needs to be writable by the `hue` user, which is the default user who starts
-Beeswax Server, or else Beeswax server will not start. You may also make that
-directory world-writable.
-
-
 Configuring Your Firewall for Hue
 Configuring Your Firewall for Hue
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 
@@ -305,7 +173,7 @@ to interact with the system.
 Hive Configuration
 Hive Configuration
 ~~~~~~~~~~~~~~~~~~
 ~~~~~~~~~~~~~~~~~~
 
 
-Hue's Beeswax application helps you use Hive to query your data.
+Hue's Hive SQL Editor application helps you use Hive to query your data.
 It depends on a Hive Server 2 running in the cluster. Please read
 It depends on a Hive Server 2 running in the cluster. Please read
 this section to ensure a proper integration.
 this section to ensure a proper integration.
 
 
@@ -347,7 +215,7 @@ and provides help and information on the default values.
 ============================================================
 ============================================================
 To view the current configuration from within Hue, open:
 To view the current configuration from within Hue, open:
 
 
-    http://<hue>/dump_config
+    http://<hue>/hue/dump_config
 ============================================================
 ============================================================
 
 
 [TIP]
 [TIP]
@@ -403,23 +271,15 @@ link:sdk/sdk.html[Hue SDK Documentation].
 Configuring Hue for SSL
 Configuring Hue for SSL
 ^^^^^^^^^^^^^^^^^^^^^^^
 ^^^^^^^^^^^^^^^^^^^^^^^
 
 
-You can configure Hue to serve over HTTPS.  To do so, you must install
-"pyOpenSSL" within Hue's context and configure your keys.
-
-To install `pyOpenSSL`, from the root of your Hue installation path,
-do the following steps:
-
-1. Run this command:
-
-    $ ./build/env/bin/easy_install pyOpenSSL
+You can configure Hue to serve over HTTPS.
 
 
-2. Configure Hue to use your private key by adding the following
+1. Configure Hue to use your private key by adding the following
 options to the `hue.ini` configuration file:
 options to the `hue.ini` configuration file:
 
 
     ssl_certificate=/path/to/certificate
     ssl_certificate=/path/to/certificate
     ssl_private_key=/path/to/key
     ssl_private_key=/path/to/key
 
 
-3. Ideally, you would have an appropriate key signed by a Certificate Authority.
+2. Ideally, you would have an appropriate key signed by a Certificate Authority.
 If you're just testing, you can create a self-signed key using the `openssl`
 If you're just testing, you can create a self-signed key using the `openssl`
 command that may be installed on your system:
 command that may be installed on your system:
 
 
@@ -447,7 +307,7 @@ the `hue.ini` configuration file.
 HDFS Cluster
 HDFS Cluster
 ^^^^^^^^^^^^
 ^^^^^^^^^^^^
 
 
-Hue only support one HDFS cluster currently. That cluster should be defined
+Hue supports one HDFS cluster. That cluster should be defined
 under the `[[[default]]]` sub-section.
 under the `[[[default]]]` sub-section.
 
 
 fs_defaultfs::
 fs_defaultfs::
@@ -458,55 +318,40 @@ webhdfs_url::
   You can also set this to be the HttpFS url. The default value is the HTTP
   You can also set this to be the HttpFS url. The default value is the HTTP
   port on the NameNode.
   port on the NameNode.
 
 
-hadoop_hdfs_home::
-  This is the home of your Hadoop HDFS installation. It is the
-  root of the Hadoop untarred directory, or usually
-  `/usr/lib/hadoop`.
-
-hadoop_bin::
-  Use this as the HDFS Hadoop launcher script, which is usually
-  `/usr/bin/hadoop`.
-
 hadoop_conf_dir::
 hadoop_conf_dir::
   This is the configuration directory of the HDFS, typically
   This is the configuration directory of the HDFS, typically
   `/etc/hadoop/conf`.
   `/etc/hadoop/conf`.
 
 
 
 
-MapReduce (MR1) Cluster
-^^^^^^^^^^^^^^^^^^^^^^^
-
-Hue only support one MapReduce cluster currently. That cluster should be defined
-under the `[[[default]]]` sub-section. Note that JobBrowser only works with MR1.
-
-jobtracker_host::
-  The host running the JobTracker. In a secured environment, this needs to
-  be the FQDN of the JobTracker host, and match the "host" portion of the
-  `mapred' Kerberos principal full name.
-
-jobtracker_port::
-  The port for the JobTracker IPC service.
-
-submit_to::
-  If your Oozie is configured with to talk to a 0.20 MapReduce service, then
-  set this to `true`. Hue will be submitting jobs to this MapReduce cluster.
-
-
 Yarn (MR2) Cluster
 Yarn (MR2) Cluster
 ^^^^^^^^^^^^^^^^^^
 ^^^^^^^^^^^^^^^^^^
 
 
-Hue only support one Yarn cluster currently. That cluster should be defined
-under the `[[[default]]]` sub-section.
+Hue supports one or two Yarn clusters (two for HA). These clusters should be defined
+under the `[[[default]]]` and `[[[ha]]]` sub-sections.
 
 
 resourcemanager_host::
 resourcemanager_host::
   The host running the ResourceManager.
   The host running the ResourceManager.
 
 
 resourcemanager_port::
 resourcemanager_port::
-  The port for the ResourceManager IPC service.
+  The port for the ResourceManager REST service.
+
+logical_name::
+  NameNode logical name.
 
 
 submit_to::
 submit_to::
-  If your Oozie is configured with to talk to a Yarn cluster, then
-  set this to `true`. Hue will be submitting jobs to this Yarn cluster.
-  But note that JobBrowser will not be able to show MR2 jobs.
+   To enable the section, set to True.
+
+
+Impala Configuration
+~~~~~~~~~~~~~~~~~~~~
+
+In the `[impala]` section of the configuration file, you can
+_optionally_ specify the following:
+
+server_host::
+  The hostname or IP that the Impala Server should bind to. By
+  default it binds to `localhost`, and therefore only serves local
+  IPC clients.
 
 
 
 
 Hive Configuration
 Hive Configuration
@@ -520,16 +365,13 @@ beeswax_server_host::
   default it binds to `localhost`, and therefore only serves local
   default it binds to `localhost`, and therefore only serves local
   IPC clients.
   IPC clients.
 
 
-hive_home_dir::
-  The base directory of your Hive installation.
-
 hive_conf_dir::
 hive_conf_dir::
   The directory containing your `hive-site.xml` Hive
   The directory containing your `hive-site.xml` Hive
   configuration file.
   configuration file.
 
 
 
 
-JobDesigner and Oozie Configuration
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Oozie Configuration
+~~~~~~~~~~~~~~~~~~~
 
 
 In the `[liboozie]` section of the configuration file, you should
 In the `[liboozie]` section of the configuration file, you should
 specify:
 specify:
@@ -539,6 +381,26 @@ oozie_url::
   environment variable for Oozie.
   environment variable for Oozie.
 
 
 
 
+Solr Configuration
+~~~~~~~~~~~~~~~~~~
+
+In the `[search]` section of the configuration file, you should
+specify:
+
+solr_url::
+  The URL of the Solr service.
+
+
+HBase Configuration
+~~~~~~~~~~~~~~~~~~~
+
+In the `[hbase]` section of the configuration file, you should
+specify:
+
+hbase_clusters::
+  Comma-separated list of HBase Thrift servers for clusters in the format of "(name|host:port)".
+
+
 UserAdmin Configuration
 UserAdmin Configuration
 ~~~~~~~~~~~~~~~~~~~~~~~
 ~~~~~~~~~~~~~~~~~~~~~~~
 
 
@@ -558,7 +420,7 @@ Configuration Validation
 Hue can detect certain invalid configuration.
 Hue can detect certain invalid configuration.
 
 
 To view the configuration of a running Hue instance, navigate to
 To view the configuration of a running Hue instance, navigate to
-`http://myserver:8888/dump_config`, also accessible through the About
+`http://myserver:8888/hue/dump_config`, also accessible through the About
 application.
 application.
 
 
 
 
@@ -568,7 +430,7 @@ Starting Hue from the Tarball
 After your cluster is running with the plugins enabled, you can start Hue on
 After your cluster is running with the plugins enabled, you can start Hue on
 your Hue Server by running:
 your Hue Server by running:
 
 
-    # build/env/bin/supervisor
+    build/env/bin/supervisor
 
 
 This will start several subprocesses, corresponding to the different Hue
 This will start several subprocesses, corresponding to the different Hue
 components. Your Hue installation is now running.
 components. Your Hue installation is now running.
@@ -584,9 +446,6 @@ section for more details about managing and operating a Hue installation.
 Hue Processes
 Hue Processes
 ~~~~~~~~~~~~~
 ~~~~~~~~~~~~~
 
 
-Process User
-^^^^^^^^^^^^
-Filebrowser requires Hue to be running as the 'hue' user.
 
 
 Process Hierarchy
 Process Hierarchy
 ^^^^^^^^^^^^^^^^^
 ^^^^^^^^^^^^^^^^^
@@ -637,7 +496,7 @@ Viewing Recent Log Messages Online
 
 
 In addition to logging `INFO` level messages to the `logs` directory, the Hue
 In addition to logging `INFO` level messages to the `logs` directory, the Hue
 web server keeps a small buffer of log messages at all levels in memory. You can
 web server keeps a small buffer of log messages at all levels in memory. You can
-view these logs by visiting `http://myserver:8888/logs`. The `DEBUG` level
+view these logs by visiting `http://myserver:8888/hue/logs`. The `DEBUG` level
 messages shown can sometimes be helpful in troubleshooting issues.
 messages shown can sometimes be helpful in troubleshooting issues.
 
 
 
 
@@ -752,9 +611,11 @@ The Help application guides users through the various installed applications.
 Supported Browsers
 Supported Browsers
 ~~~~~~~~~~~~~~~~~~
 ~~~~~~~~~~~~~~~~~~
 
 
-* Windows: Chrome, Firefox 3.6+, Internet Explorer 9+, Safari 5+
-* Linux : Chrome, Firefox 3.6+
-* Mac: Chrome, Firefox 3.6+, Safari 5+
+The two latest LTS versions of each browsers.
+* IE/Edge
+* Safari
+* Chrome
+* Firefox
 
 
 Feedback
 Feedback
 ~~~~~~~~
 ~~~~~~~~