Эх сурвалжийг харах

HUE-8815 [doc] Add more configuration content and prepare new TOC

Romain 6 жил өмнө
parent
commit
31b1803404

+ 2 - 2
docs/docs-site/config.toml

@@ -29,7 +29,7 @@ home = [ "HTML", "RSS", "JSON"]
   disableSearch = false
   # Javascript and CSS cache are automatically busted when new version of site is generated.
   # Set this to true to disable this behavior (some proxies don't handle well this optimization)
-  disableAssetsBusting = false
+  disableAssetsBusting = true
   # Set this to true to disable copy-to-clipboard button for inline code.
   disableInlineCopyToClipBoard = false
   # A title for shortcuts in menu is set by default. Set this to true to disable it.
@@ -60,4 +60,4 @@ weight = 11
 name = "<i class='fas fa-fw fa-bookmark'></i> gethue.com"
 identifier = "hugodoc"
 url = "http://gethue.com"
-weight = 20
+weight = 20

+ 1 - 1
docs/docs-site/content/administrator/administration/_index.md

@@ -5,4 +5,4 @@ draft: false
 weight: 4
 ---
 
-This sections details the "ops" part of Hue, e.g. how to monitor the servers, manage users and permissions.
+This sections details the "ops" part of Hue, e.g. how to monitor the servers, manage users and permissions.

+ 166 - 1
docs/docs-site/content/administrator/administration/database.md

@@ -39,6 +39,16 @@ another node for backup. It is recommended that you back it up on a regular
 schedule, and also that you back it up before any upgrade to a new version of
 Hue.
 
+### Clean up the database
+
+When the database has too many entries in certain tables, it will cause performance issue. Now Hue config check will help superuser to find this issue. Login as superuser and go to “Hue Administration”, this sample screenshot will be displayed in the quick start wizard when the tables have too many entries.
+
+Run following clean up command:
+
+  cd /opt/cloudera/parcels/CDH/lib/hue # Hue home directory
+  ./build/env/bin/hue desktop_document_cleanup
+
+
 ### Configuring to Access Another Database
 
 Although SQLite is the default database type, some advanced users may prefer
@@ -100,4 +110,159 @@ To configure Hue to store data in MySQL:
     /usr/share/hue/build/env/bin/hue loaddata $temporary-file-containing-dumped-data.json
     </pre>
 
-Your system is now configured and you can start the Hue server as normal.
+Your system is now configured and you can start the Hue server as normal.
+
+### Migrate the Hue Database
+Note: Hue Custom Databases includes database-specific pages on how to migrate from an old to a new database. This page summarizes across supported database types.
+When you change Hue databases, you can migrate the existing data to your new database. If the data is dispensable, there is no need to migrate.
+
+The Hue database stores things like user accounts, SQL queries, and Oozie workflows, and you may have accounts, queries, and workflows worth saving. See How to Populate the Hue Database.
+
+Dump Database
+
+Note: Refresh the page to ensure that the Hue service is stopped: Service Stopped icon.
+Select Actions > Dump Database and click Dump Database. The file is written to /tmp/hue_database_dump.json on the host of the Hue server.
+
+Log on to the host of the Hue server in a command-line terminal. You can find the hostname on the Dump Database window and at Hue > Hosts.
+Edit /tmp/hue_database_dump.json by removing all objects with useradmin.userprofile in the model field. For example:
+
+Count number of objects
+
+    grep -c useradmin.userprofile /tmp/hue_database_dump.json
+    vi /tmp/hue_database_dump.json
+    {
+      "pk": 1,
+      "model": "useradmin.userprofile",
+      "fields": {
+        "last_activity": "2016-10-03T10:06:13",
+        "creation_method": "HUE",
+        "first_login": false,
+        "user": 1,
+        "home_directory": "/user/admin"
+      }
+    },
+
+Connect New Database
+
+Set the appropriate database parameters :
+
+    Hue Database Type: MySQL or PostgreSQL or Oracle
+    Hue Database Hostname: <fqdn of host with database server>
+    Hue Database Port: 3306 or 5432 or 1521
+    Hue Database Username: <hue database username>
+    Hue Database Password: <hue database password>
+    Hue Database Name: <hue database name or SID>
+
+Oracle users only should add support for a multithreaded environment:
+
+Add support for a multithreaded environment by setting Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini:
+    [desktop]
+    [[database]]
+    options={"threaded":True}
+
+In the Hue Web UI, click the home icon Hue Home icon to ensure that all documents were migrated.
+
+    MariaDB / MySQL
+    Synchronize Database in Cloudera Manager.
+    Log on to MySQL:
+    mysql -u root -p
+    Enter password: <root password>
+
+Drop the foreign key constraint from the hue.auth_permission table:
+
+Execute the following statement to find the content_type_id_refs_id_<value> in the CONSTRAINT clause of the CREATE TABLE statement for the hue.auth_permission table:
+
+    SHOW CREATE TABLE hue.auth_permission;
+
+This SHOW CREATE TABLE statement produces output similar to the following:
+
+    |  auth_permission | CREATE TABLE 'auth_permission' (
+      'id' int(11) NOT NULL AUTO-INCREMENT,
+      'name' varchar(50) NOT NULL,
+      'content_type_id' int(11) NOT NULL,
+      'CODENAME' VARCHAR(100) NOT NULL,
+      PRIMARY KEY ('id'),
+      UNIQUE KEY 'content_type_id' ('content_type_id', 'codename'),
+      KEY 'auth_permission_37ef4eb4' ('content_type_id'),
+      CONSTRAINT 'content_type_id_refs_id_d043b34a' FOREIGN KEY ('content_type_id')
+    REFERENCES 'django_content_type' ('id')
+    ) ENGINE=InnoDB AUTO_INCREMENT=229 DEFAULT CHARSET=utf8 |
+
+Then execute the following statement to drop the foreign key constraint:
+
+    ALTER TABLE hue.auth_permission DROP FOREIGN KEY
+    content_type_id_refs_id_<value>;
+
+For example, if you used the above output from the SHOW CREATE TABLE statement, you would use the following ALTER TABLE statement:
+
+    ALTER TABLE hue.auth_permission DROP FOREIGN KEY
+    content_type_id_refs_id_d043b34a;
+
+Delete the contents of django_content_type:
+
+    DELETE FROM hue.django_content_type;
+
+Load Database.
+
+    Add the foreign key, content_type_id, to auth_permission:
+
+    ALTER TABLE hue.auth_permission ADD FOREIGN KEY (content_type_id) REFERENCES django_content_type (id);
+
+#### PostgreSQL
+
+Log on to PostgreSQL:
+    psql -h localhost -U hue -d hue
+
+Password for user hue:
+
+Drop the foreign key constraint from auth_permission:
+
+    \d auth_permission;
+    ALTER TABLE auth_permission DROP CONSTRAINT content_type_id_refs_id_<id value>;
+
+Delete the contents of django_content_type:
+
+    TRUNCATE django_content_type CASCADE;
+
+Load Database.
+
+    Add the foreign key, content_type_id, to auth_permission:
+    ALTER TABLE auth_permission ADD FOREIGN KEY (content_type_id) REFERENCES django_content_type(id) DEFERRABLE INITIALLY DEFERRED;
+
+#### Oracle
+
+Oracle users should delete all content from the Oracle tables after synchronizing and before loading:
+
+Log on to Oracle:
+
+    su - oracle
+    sqlplus / as sysdba
+
+Grant a quota to the tablespace where tables are created (the default is SYSTEM). For example:
+
+    ALTER USER hue quota 100m on system;
+
+Log on as the hue:
+
+    sqlplus hue/<hue password>
+
+Create a spool script that creates a delete script to clean the content of all tables.
+
+    vi spool_statements.ddl
+
+Save in spool_statements.ddl (which generates delete_from_tables.ddl)
+
+    spool delete_from_tables.ddl
+    set pagesize 100;
+    SELECT 'DELETE FROM ' || table_name || ';' FROM user_tables;
+    commit;
+    spool off
+    quit
+
+Run both scripts:
+
+    -- Create delete_from_tables.ddl
+    sqlplus hue/<your hue password> < spool_statements.ddl
+
+    -- Run delete_from_tables.ddl
+    sqlplus hue/<your hue password> < delete_from_tables.ddl

+ 6 - 122
docs/docs-site/content/administrator/administration/operations.md

@@ -111,27 +111,6 @@ The below metrics of most concern to us are displayed on the page:
 One of the most useful ones are the percentiles of response time of requests and the count of active users.
 Admins can either filter a particular property in all the metrics or select a particular metric for all properties
 
-## Process Hierarchy
-
-A script called `supervisor` manages all Hue processes. The supervisor is a
-watchdog process -- its only purpose is to spawn and monitor other processes.
-A standard Hue installation starts and monitors the following processes:
-
-* `runcpserver` - a web server based on CherryPy that provides the core web
-functionality of Hue
-
-If you have installed other applications into your Hue instance, you may see
-other daemons running under the supervisor as well.
-
-You can see the supervised processes running in the output of `ps -f -u hue`:
-
-    UID        PID  PPID  C STIME TTY          TIME CMD
-    hue       8685  8679  0 Aug05 ?        00:01:39 /usr/share/hue/build/env/bin/python /usr/share/hue/build/env/bin/desktop runcpserver
-
-Note that the supervisor automatically restarts these processes if they fail for
-any reason. If the processes fail repeatedly within a short time, the supervisor
-itself shuts down.
-
 ## Logging
 
 The Hue logs are found in `/var/log/hue`, or in a `logs` directory under your
@@ -162,11 +141,11 @@ messages shown can sometimes be helpful in troubleshooting issues.
 ## Troubleshooting
 
 To troubleshoot why Hue is slow or consuming high memory, admin can enable instrumentation by setting the `instrumentation` flag to True.
- <pre>
-[desktop]
-instrumentation=true
-</pre>
- If `django_debug_mode` is enabled, instrumentation is automatically enabled. This flag appends the response time and the total peak memory used since Hue started for every logged request.
+
+    [desktop]
+    instrumentation=true
+
+If `django_debug_mode` is enabled, instrumentation is automatically enabled. This flag appends the response time and the total peak memory used since Hue started for every logged request.
 
 ### Instrumentation enabled
 
@@ -182,99 +161,4 @@ instrumentation=true
 
 ## Database
 
-Hue requires a SQL database to store small amounts of data, including user
-account information as well as history of job submissions and Hive queries.
-By default, Hue is configured to use the embedded database SQLite for this
-purpose, and should require no configuration or management by the administrator.
-However, MySQL is the recommended database to use. This section contains
-instructions for configuring Hue to access MySQL and other databases.
-
-### Inspecting the Database
-
-The default SQLite database used by Hue is located in: `/usr/share/hue/desktop/desktop.db`.
-You can inspect this database from the command line using the `sqlite3`
-program or typing `/usr/share/hue/build/env/bin/hue dbshell'. For example:
-
-    sqlite3 /usr/share/hue/desktop/desktop.db
-    SQLite version 3.6.22
-    Enter ".help" for instructions
-    Enter SQL statements terminated with a ";"
-    sqlite> select username from auth_user;
-    admin
-    test
-    sample
-    sqlite>
-
-It is strongly recommended that you avoid making any modifications to the
-database directly using SQLite, though this trick can be useful for management
-or troubleshooting.
-
-### Backing up the Database
-
-If you use the default SQLite database, then copy the `desktop.db` file to
-another node for backup. It is recommended that you back it up on a regular
-schedule, and also that you back it up before any upgrade to a new version of
-Hue.
-
-### Configuring to Access Another Database
-
-Although SQLite is the default database type, some advanced users may prefer
-to have Hue access an alternate database type. Note that if you elect to
-configure Hue to use an external database, upgrades may require more manual
-steps in the future.
-
-The following instructions are for MySQL, though you can also configure Hue to
-work with other common databases such as PostgreSQL and Oracle.
-
-<div class="note">
-Note that Hue has only been tested with SQLite and MySQL database backends.
-</div>
-
-
-### Configuring to Store Data in MySQL
-
-To configure Hue to store data in MySQL:
-
-1. Create a new database in MySQL and grant privileges to a Hue user to manage
-   this database.
-
-    <pre>
-    mysql> create database hue;
-    Query OK, 1 row affected (0.01 sec)
-    mysql> grant all on hue.* to 'hue'@'localhost' identified by 'secretpassword';
-    Query OK, 0 rows affected (0.00 sec)
-    </pre>
-
-2. Shut down Hue if it is running.
-
-3. To migrate your existing data to MySQL, use the following command to dump the
-   existing database data to a text file. Note that using the ".json" extension
-   is required.
-
-    <pre>
-    /usr/share/hue/build/env/bin/hue dumpdata > $some-temporary-file.json
-    </pre>
-
-4. Open the `hue.ini` file in a text editor. Directly below the
-   `[[database]]` line, add the following options (and modify accordingly for
-   your MySQL setup):
-
-    <pre>
-    host=localhost
-    port=3306
-    engine=mysql
-    user=hue
-    password=secretpassword
-    name=hue
-    </pre>
-
-5. As the Hue user, configure Hue to load the existing data and create the
-   necessary database tables:
-
-    <pre>
-    /usr/share/hue/build/env/bin/hue syncdb --noinput
-    mysql -uhue -psecretpassword -e "DELETE FROM hue.django_content_type;"
-    /usr/share/hue/build/env/bin/hue loaddata $temporary-file-containing-dumped-data.json
-    </pre>
-
-Your system is now configured and you can start the Hue server as normal.
+See the dedicated [Database section]({{% param baseURL %}}administrator/administration/database/).

+ 1 - 1
docs/docs-site/content/administrator/configuration/_index.md

@@ -8,4 +8,4 @@ weight: 3
 The source of truth sits in the main [hue.ini](https://github.com/cloudera/hue/blob/master/desktop/conf.dist/hue.ini).
 It consists in several [ini sections](https://en.wikipedia.org/wiki/INI_file#Sections). Lines needs to be uncommented to be active.
 
-Hue is using Hadoop `impersonation` to be able to communicate properly with certain services. This is described in the following [Service Configuration]("#services-pre-configurations).
+Hue is using Hadoop `impersonation` to be able to communicate properly with certain services. This is described in the following [Service Configuration]({{% param baseURL %}}administrator/configuration/).

+ 12 - 2
docs/docs-site/content/administrator/configuration/dashboard/_index.md

@@ -1,5 +1,5 @@
 ---
-title: "Dashboard & Query Builder"
+title: "Dashboard"
 date: 2019-03-13T18:28:09-07:00
 draft: false
 weight: 3
@@ -13,8 +13,18 @@ They consist in 3 types:
 * Query Builder (alpha)
 * Multi-widget reporting (alpha)
 
-This application is getting improved via SQL Dashboards and Query Builder [HUE-3228](https://issues.cloudera.org/browse/HUE-3228).
+## Solr Search
+
+In the `[search]` section of the configuration file, you should
+specify:
+
+    [search]
+      # URL of the Solr Server
+      solr_url=http://solr-server.com:8983/solr/
 
+## SQL
+
+This application is getting improved via SQL Dashboards and Query Builder [HUE-3228](https://issues.cloudera.org/browse/HUE-3228).
 
       [dashboard]
 

+ 6 - 2
docs/docs-site/content/administrator/configuration/editor/_index.md

@@ -1,8 +1,8 @@
 ---
-title: "SQL Editor"
+title: "Editor"
 date: 2019-03-13T18:28:09-07:00
 draft: false
-weight: 1
+weight: 2
 ---
 
 The goal of the Editor is to open-up data to more users by making self service querying easy and productive.
@@ -95,6 +95,10 @@ Display an analysis panel post Impala queries executions with some hints and sug
       [notebook]
       enable_query_analysis=true
 
+### Query Optimization
+
+In the `[metadata]` section, Hue is supporting Cloudera Navigator Optimiser and soon other services. The goal is to provide recommendation on how to write better queries and get risk alerts on dangerous operations directly within the [editor]({{% param baseURL %}}user/editor/).
+
 ### One-click scheduling
 
 Enable the creation of a coordinator for the current SQL query.

+ 14 - 63
docs/docs-site/content/administrator/configuration/external/_index.md

@@ -5,33 +5,11 @@ draft: false
 weight: 5
 ---
 
+
 ## Data Catalog
 
 In the `[metadata]` section, Hue is supporting Cloudera Navigator and soon Apache Atlas ([HUE-8749](https://issues.cloudera.org/browse/HUE-8749)) in order to enrich the [data catalog]({{% param baseURL %}}user/browsers/).
 
-
-## Query Optimization
-
-In the `[metadata]` section, Hue is supporting Cloudera Navigator Optimiser and soon other services. The goal is to provide recommendation on how to write better queries and get risk alerts on dangerous operations directly within the [editor]({{% param baseURL %}}user/editor/).
-
-## YARN Cluster
-
-Hue supports one or two Yarn clusters (two for HA). These clusters should be defined
-under the `[[[default]]]` and `[[[ha]]]` sub-sections.
-
-    # Configuration for YARN (MR2)
-    # ------------------------------------------------------------------------
-    [[yarn_clusters]]
-
-      [[[default]]]
-
-        resourcemanager_host=yarn-rm.com
-        resourcemanager_api_url=http://yarn-rm.com:8088/
-        proxy_api_url=http://yarn-proxy.com:8088/
-        resourcemanager_port=8032
-        history_server_api_url=http://yarn-rhs-com:19888/
-
-
 ## Spark
 
 The `[spark]` section details how to point to [Livy](https://livy.incubator.apache.org/) in order to execute interactive Spark snippets in Scala or Python.
@@ -48,19 +26,9 @@ The `[spark]` section details how to point to [Livy](https://livy.incubator.apac
 The configuration is in `[kafka]` but the service is still experiemental.
 
 
-## Solr Search
-
-In the `[search]` section of the configuration file, you should
-specify:
-
-    [search]
-      # URL of the Solr Server
-      solr_url=http://solr-server.com:8983/solr/
-
 ## Oozie
 
-In the `[liboozie]` section of the configuration file, you should
-specify:
+In oder to schedule workflows, the `[liboozie]` section of the configuration file:
 
     [liboozie]
       oozie_url=http://oozie-server.com:11000/oozie
@@ -79,36 +47,19 @@ To configure Hue as a default proxy user, add the following properties to /etc/o
         <value>*</value>
     </property>
 
-## HBase
-
-Specify the HBase Thrift Server1 in Hue:
-
-    [hbase]
-    hbase_clusters=(Cluster|localhost:9090)
-
-HBase Impersonation:
-
-Enable impersonation for the Thrift server by adding the following properties to hbase-site.xml on each Thrift gateway:
+## YARN Cluster
 
-    <property>
-      <name>hbase.regionserver.thrift.http</name>
-      <value>true</value>
-    </property>
-    <property>
-      <name>hbase.thrift.support.proxyuser</name>
-      <value>true/value>
-    </property>
+Hue supports one or two Yarn clusters (two for HA). These clusters should be defined
+under the `[[[default]]]` and `[[[ha]]]` sub-sections.
 
-Note: If you use framed transport, you cannot use doAs impersonation, because SASL does not work with Thrift framed transport.
+    # Configuration for YARN (MR2)
+    # ------------------------------------------------------------------------
+    [[yarn_clusters]]
 
-doAs Impersonation provides a flexible way to use the same client to impersonate multiple principals. doAs is supported only in Thrift 1.
-Enable doAs support by adding the following properties to hbase-site.xml on each Thrift gateway:
+      [[[default]]]
 
-    <property>
-      <name>hbase.regionserver.thrift.http</name>
-      <value>true</value>
-    </property>
-    <property>
-      <name>hbase.thrift.support.proxyuser</name>
-      <value>true/value>
-    </property>
+        resourcemanager_host=yarn-rm.com
+        resourcemanager_api_url=http://yarn-rm.com:8088/
+        proxy_api_url=http://yarn-proxy.com:8088/
+        resourcemanager_port=8032
+        history_server_api_url=http://yarn-rhs-com:19888/

+ 30 - 4
docs/docs-site/content/administrator/configuration/files/_index.md

@@ -143,8 +143,34 @@ Alternatively (but not recommended for production or secure environments), you c
 
 ## HBase
 
-In the `[hbase]` section of the configuration file, you should
-specify:
+Specify the comma-separated list of HBase Thrift servers for clusters in the format of "(name|host:port)":
 
-    hbase_clusters:
-      Comma-separated list of HBase Thrift servers for clusters in the format of "(name|host:port)".
+    [hbase]
+    hbase_clusters=(Cluster|localhost:9090)
+
+HBase Impersonation:
+
+Enable impersonation for the Thrift server by adding the following properties to hbase-site.xml on each Thrift gateway:
+
+    <property>
+      <name>hbase.regionserver.thrift.http</name>
+      <value>true</value>
+    </property>
+    <property>
+      <name>hbase.thrift.support.proxyuser</name>
+      <value>true/value>
+    </property>
+
+Note: If you use framed transport, you cannot use doAs impersonation, because SASL does not work with Thrift framed transport.
+
+doAs Impersonation provides a flexible way to use the same client to impersonate multiple principals. doAs is supported only in Thrift 1.
+Enable doAs support by adding the following properties to hbase-site.xml on each Thrift gateway:
+
+    <property>
+      <name>hbase.regionserver.thrift.http</name>
+      <value>true</value>
+    </property>
+    <property>
+      <name>hbase.thrift.support.proxyuser</name>
+      <value>true/value>
+    </property>

+ 0 - 305
docs/docs-site/content/administrator/configuration/hue/_index.md

@@ -1,305 +0,0 @@
----
-title: "Hue"
-date: 2019-03-13T18:28:09-07:00
-draft: false
-weight: 1
----
-
-This section is about configuring the Hue server itself.
-These configuration variables are under the `[desktop]` section in
-the `conf/hue.ini` configuration file.
-
-## Specifying the HTTP port
-
-Hue uses CherryPy web server.  You can use the following options to
-change the IP address and port that the web server listens on.
-The default setting is port 8888 on all configured IP addresses.
-
-    # Webserver listens on this address and port
-    http_host=0.0.0.0
-    http_port=8888
-
-[Gunicorn](https://gunicorn.org/) support is planned to come in via [HUE-8739](https://issues.cloudera.org/browse/HUE-8739).
-
-## Specifying the Secret Key
-
-For security, you should also specify the secret key that is used for secure
-hashing in the session store. Enter a long series of random characters
-(30 to 60 characters is recommended).
-
-    secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
-
-NOTE: If you don't specify a secret key, your session cookies will not be
-secure. Hue will run but it will also display error messages telling you to
-set the secret key.
-
-## Disabling some apps
-
-In the Hue ini configuration file, in the [desktop] section, you can enter the names of the app to hide:
-
-<pre>
-[desktop]
-# Comma separated list of apps to not load at server startup.
-app_blacklist=beeswax,impala,security,filebrowser,jobbrowser,rdbms,jobsub,pig,hbase,sqoop,zookeeper,metastore,spark,oozie,indexer
-</pre>
-
-[Read more about it here](http://gethue.com/mini-how-to-disabling-some-apps-from-showing-up/).
-
-## Authentication
-
-By default (`AllowFirstUserDjangoBackend`), the first user who logs in to Hue can choose any
-username and password and becomes an administrator automatically. This
-user can create other user and administrator accounts. User information is
-stored in the Django database in the Django backend.
-
-The authentication system is pluggable. Here is a list of some of the possible authentications:
-
-### Username / Password
-
-This is the default Hue backend. It creates the first user that logs in as the super user. After this, it relies on Django and the user manager to authenticate users.
-
-    desktop.auth.backend.AllowFirstUserDjangoBackend
-
-### Allow All
-
-This backend does not require a password for users to log in. All users are automatically authenticated and the username is set to what is provided.
-
-    desktop.auth.backend.AllowAllBackend
-
-### LDAP
-
-Authenticates users against an LDAP service.
-
-    desktop.auth.backend.LdapBackend
-
-### SAML
-
-Secure Assertion Markup Language (SAML) single sign-on (SSO) backend. Delegates authentication to the configured Identity Provider. See Configuring Hue for SAML for more details.
-
-    libsaml.backend.SAML2Backend
-
-[Read more about it](http://gethue.com/updated-saml-2-0-support/).
-
-### Spnego
-
-SPNEGO is an authentication mechanism negotiation protocol. Authentication can be delegated to an authentication server, such as a Kerberos KDC, depending on the mechanism negotiated.
-
-    desktop.auth.backend.SpnegoDjangoBackend
-
-### PAM
-
-Authenticates users with PAM (pluggable authentication module). The authentication mode depends on the PAM module used.
-
-    desktop.auth.backend.PamBackend
-
-### OAuth Connect
-
-Delegates authentication to a third-party OAuth server.
-
-    desktop.auth.backend.OAuthBackend
-
-### Multiple Authentication Backends
-
-For example, to enable Hue to first attempt LDAP directory lookup before falling back to the database-backed user model, we can update the hue.ini configuration file or Hue safety valve in Cloudera Manager with a list containing first the LdapBackend followed by either the ModelBackend or custom AllowFirstUserDjangoBackend (permits first login and relies on user model for all subsequent authentication):
-
-    [desktop]
-      [[auth]]
-      backend=desktop.auth.backend.LdapBackend,desktop.auth.backend.AllowFirstUserDjangoBackend
-
-This tells Hue to first check against the configured LDAP directory service, and if the username is not found in the directory, then attempt to authenticate the user with the Django user manager.
-
-[Read more about it here](http://gethue.com/configuring-hue-multiple-authentication-backends-and-ldap/).
-
-
-## Change your maps look and feel
-
-The properties we need to tweak are leaflet_tile_layer and leaflet_tile_layer_attribution, that can be configured in the hue.ini file:
-
-    [desktop]
-    leaflet_tile_layer=https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}
-    leaflet_tile_layer_attribution='Tiles &copy; Esri &mdash; Source: Esri, i-cubed, USDA, USGS, AEX, GeoEye, Getmapping, Aerogrid, IGN, IGP, UPR-EGP, and the GIS User Community'
-
-[Read more about it here](http://gethue.com/change-your-maps-look-and-feel/).
-
-## Configure a Proxy
-
-We explained how to run Hue with NGINX serving the static files or under Apache. If you use another proxy, you might need to set these options:
-
-<pre>
-  [desktop]
-  # Enable X-Forwarded-Host header if the load balancer requires it.
-  use_x_forwarded_host=false
-
-  # Support for HTTPS termination at the load-balancer level with SECURE_PROXY_SSL_HEADER.
-  secure_proxy_ssl_header=false
-</pre>
-
-## Configuring SSL
-
-You can configure Hue to serve over HTTPS.
-
-1. Configure Hue to use your private key by adding the following
-options to the `hue.ini` configuration file:
-
-    ssl_certificate=/path/to/certificate
-    ssl_private_key=/path/to/key
-
-2. Ideally, you would have an appropriate key signed by a Certificate Authority.
-If you're just testing, you can create a self-signed key using the `openssl`
-command that may be installed on your system:
-
-Create a key:
-
-    openssl genrsa 1024 > host.key
-
-Create a self-signed certificate:
-
-    openssl req -new -x509 -nodes -sha1 -key host.key > host.cert
-
-
-<div class="note">
-Self-signed Certificates and File Uploads
-
-To upload files using the Hue File Browser over HTTPS requires
-using a proper SSL Certificate.  Self-signed certificates don't
-work.
-</div>
-
-Note: The security vulnerability SWEET32 is also called Birthday attacks against TLS ciphers with 64bit block size and it is assigned CVE-2016-2183. This is due to legacy block ciphers
-having block size of 64 bits are vulnerable to a practical collision attack when used in CBC mode.
-
-DES/3DES are the only ciphers has block size of 64-bit. One way to config Hue not to use them:
-
-    [desktop]
-    ssl_cipher_list=DEFAULT:!DES:!3DES
-
-## SASL
-
-When getting a bigger result set from Hive/Impala or bigger files like images from HBase, the response requires to increase
-the buffer size of SASL lib for thrift sasl communication.
-
-<pre>
-  [desktop]
-  # This property specifies the maximum size of the receive buffer in bytes in thrift sasl communication,
-  # default value is 2097152 (2 MB), which equals to (2 * 1024 * 1024)
-  sasl_max_buffer=2097152
-</pre>
-
-## User Admin Configuration
-In the `[useradmin]` section of the configuration file, you can
-_optionally_ specify the following:
-
-default_user_group::
-  The name of a default group that is suggested when creating a
-  user manually. If the LdapBackend or PamBackend are configured
-  for doing user authentication, new users will automatically be
-  members of the default group.
-
-
-## Banner
-You can add a custom banner to the Hue Web UI by applying HTML directly to the property, banner_top_html. For example:
-
-    banner_top_html=<H4>My company's custom Hue Web UI banner</H4>
-
-## Splash Screen
-You can customize a splash screen on the login page by applying HTML directly to the property, login_splash_html. For example:
-
-    [desktop]
-    [[custom]]
-    login_splash_html=WARNING: You are required to have authorization before you proceed.
-
-
-## Custom Logo
-
-There is also the possibility to change the logo for further personalization.
-
-    [desktop]
-    [[custom]]
-    # SVG code to replace the default Hue logo in the top bar and sign in screen
-    # e.g. <image xlink:href="/static/desktop/art/hue-logo-mini-white.png" x="0" y="0" height="40" width="160" />
-    logo_svg=
-
-You can go crazy and write there any SVG code you want. Please keep in mind your SVG should be designed to fit in a 160×40 pixels space. To have the same ‘hearts logo' you can see above, you can type this code
-
-    [desktop]
-    [[custom]]
-    logo_svg='<g><path stroke="null" id="svg_1" d="m44.41215,11.43463c-4.05017,-10.71473 -17.19753,-5.90773 -18.41353,-0.5567c-1.672,-5.70253 -14.497,-9.95663 -18.411,0.5643c-4.35797,11.71793 16.891,22.23443 18.41163,23.95773c1.5181,-1.36927 22.7696,-12.43803 18.4129,-23.96533z" fill="#ffffff"/> <path stroke="null" id="svg_2" d="m98.41246,10.43463c-4.05016,-10.71473 -17.19753,-5.90773 -18.41353,-0.5567c-1.672,-5.70253 -14.497,-9.95663 -18.411,0.5643c-4.35796,11.71793 16.891,22.23443 18.41164,23.95773c1.5181,-1.36927 22.76959,-12.43803 18.41289,-23.96533z" fill="#FF5A79"/> <path stroke="null" id="svg_3" d="m154.41215,11.43463c-4.05016,-10.71473 -17.19753,-5.90773 -18.41353,-0.5567c-1.672,-5.70253 -14.497,-9.95663 -18.411,0.5643c-4.35796,11.71793 16.891,22.23443 18.41164,23.95773c1.5181,-1.36927 22.76959,-12.43803 18.41289,-23.96533z" fill="#ffffff"/> </g>'
-
-Read more about it in [Hue with a custom logo](http://gethue.com/hue-with-a-custom-logo/) post.
-
-
-## Storing passwords in file script
-
-This [article details how to store passwords in a script](http://gethue.com/storing-passwords-in-script-rather-than-hue-ini-files/) launched from the OS rather than have clear text passwords in the hue*.ini files.
-
-Some passwords go in Hue ini configuration file making them easily visible to Hue admin user or by users of cluster management software. You can use the password_script feature to prevent passwords from being visible.
-
-## Idle session timeout
-
-Hue now offers a new property, idle_session_timeout, that can be configured in the hue.ini file:
-
-    [desktop]
-    [[auth]]
-    idle_session_timeout=600
-
-When idle_session_timeout is set, users will automatically be logged out after N (e.g. – 600) seconds of inactivity and be prompted to login again:
-
-[Read more about it here](http://gethue.com/introducing-the-new-login-modal-and-idle-session-timeout/).
-
-## Auditing
-
-Read more about [Auditing User Administration Operations with Hue and Cloudera Navigator](http://gethue.com/auditing-user-administration-operations-with-hue-and-cloudera-navigator-2/).
-
-## Source Version Control
-
-By default Hue stores the [saved documents]({{% param baseURL %}}user/concept/#documents) in its database. This features aims at pointing to any source versioning systems like GitHub, BitBucket... to open and save queries.
-
-**Note** This feature is experiemental and tracked in [HUE-951](https://issues.cloudera.org/browse/HUE-951).
-
-    [desktop]
-
-    [[vcs]]
-
-    ## [[[git-read-only]]]
-        ## Base URL to Remote Server
-        # remote_url=https://github.com/cloudera/hue/tree/master
-
-        ## Base URL to Version Control API
-        # api_url=https://api.github.com
-    ## [[[github]]]
-
-        ## Base URL to Remote Server
-        # remote_url=https://github.com/cloudera/hue/tree/master
-
-        ## Base URL to Version Control API
-        # api_url=https://api.github.com
-
-        # These will be necessary when you want to write back to the repository.
-        ## Client ID for Authorized Application
-        # client_id=
-
-        ## Client Secret for Authorized Application
-        # client_secret=
-    ## [[[svn]]
-        ## Base URL to Remote Server
-        # remote_url=https://github.com/cloudera/hue/tree/master
-
-        ## Base URL to Version Control API
-        # api_url=https://api.github.com
-
-        # These will be necessary when you want to write back to the repository.
-        ## Client ID for Authorized Application
-        # client_id=
-
-        ## Client Secret for Authorized Application
-        # client_secret=
-
-## Concurrent User Session Limit
-
-If set, limits the number of concurrent user sessions. 1 represents 1 browser session per user. Default: 0 (unlimited sessions per user)
-
-    [desktop]
-    [[session]]
-    concurrent_user_session_limit=0
-
-[Read more about it here](http://gethue.com/restrict-number-of-concurrent-sessions-per-user/).

+ 690 - 0
docs/docs-site/content/administrator/configuration/server/_index.md

@@ -0,0 +1,690 @@
+---
+title: "Server"
+date: 2019-03-13T18:28:09-07:00
+draft: false
+weight: 1
+---
+
+This section is about configuring the Hue server itself.
+These configuration variables are under the `[desktop]` section in
+the `conf/hue.ini` configuration file.
+
+## Basics
+
+### Point to MySQL or Postgres
+
+Directly below the `[[database]]` line, add the following options (and modify accordingly for
+your MySQL setup):
+
+    host=localhost
+    port=3306
+    engine=mysql
+    user=hue
+    password=secretpassword
+    name=hue
+
+And run the table creation one time:
+
+    ./build/env/bin/hue migrate
+
+### Specifying the HTTP port
+
+Hue uses CherryPy web server.  You can use the following options to
+change the IP address and port that the web server listens on.
+The default setting is port 8888 on all configured IP addresses.
+
+    # Webserver listens on this address and port
+    http_host=0.0.0.0
+    http_port=8888
+
+[Gunicorn](https://gunicorn.org/) support is planned to come in via [HUE-8739](https://issues.cloudera.org/browse/HUE-8739).
+
+### Specifying the Secret Key
+
+For security, you should also specify the secret key that is used for secure
+hashing in the session store. Enter a long series of random characters
+(30 to 60 characters is recommended).
+
+    secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
+
+NOTE: If you don't specify a secret key, your session cookies will not be
+secure. Hue will run but it will also display error messages telling you to
+set the secret key.
+
+### Disabling some apps
+
+In the Hue ini configuration file, in the [desktop] section, you can enter the names of the app to hide:
+
+    [desktop]
+    # Comma separated list of apps to not load at server startup.
+    app_blacklist=beeswax,impala,security,filebrowser,jobbrowser,rdbms,jobsub,pig,hbase,sqoop,zookeeper,metastore,spark,oozie,indexer
+
+[Read more about it here](http://gethue.com/mini-how-to-disabling-some-apps-from-showing-up/).
+
+## Authentication
+
+By default (`AllowFirstUserDjangoBackend`), the first user who logs in to Hue can choose any
+username and password and becomes an administrator automatically. This
+user can create other user and administrator accounts. User information is
+stored in the Django database in the Django backend.
+
+The authentication system is pluggable. Here is a list of some of the possible authentications:
+
+### Username / Password
+
+This is the default Hue backend. It creates the first user that logs in as the super user. After this, it relies on Django and the user manager to authenticate users.
+
+    desktop.auth.backend.AllowFirstUserDjangoBackend
+
+### Allow All
+
+This backend does not require a password for users to log in. All users are automatically authenticated and the username is set to what is provided.
+
+    desktop.auth.backend.AllowAllBackend
+
+### LDAP
+
+Authenticates users against an LDAP service.
+
+    desktop.auth.backend.LdapBackend
+
+There are two ways to bind Hue with an LDAP directory service:
+
+* Search Bind: Hue searches for user credentials with search base (and attribute and filter).
+* Direct Bind: Hue authenticates (without searching) in one of two ways:
+  * NT Domain: Bind to Microsoft Active Directory with username@domain (the UPN)or
+  * Username Pattern: Bind to open standard LDAP with full path of directory information tree (DIT).
+
+Note: Username pattern does not work with AD because AD inserts spaces into the UID which Hue cannot process.
+
+Encryption: To prevent credentials from transmitting in the clear, encrypt with LDAP over SSL, using the LDAPS protocol on the LDAPS port (636 by default); or encrypt with the StartTLS extension using the standard LDAP protocol and port (389 by default). Cloudera recommends LDAPS. You must have a CA Certificate in either case.
+
+Hue Supported LDAP Authentication and Encryption Methods
+
+    LDAP Auth Action	Encrypted (LDAPS)	Encrypted (LDAP+TLS)	Not Encrypted (LDAP)
+    Search Bind	AD, LDAP	AD, LDAP	AD, LDAP
+    Direct Bind - NT Domain	AD	AD	AD
+    Direct Bind - User Pattern	LDAP	LDAP	LDAP
+
+Example of a Search Bind configuration encrypted with LDAPS:
+
+    [[custom]]
+    [[auth]]
+    backend=desktop.auth.backend.LdapBackend
+
+    [[ldap]]
+    ldap_url=ldaps://w2k8-1.ad.sec.cloudera.com:636
+    search_bind_authentication=true
+    ldap_cert=/<path_to_cacert>/w2k8-1-root.pem
+    use_start_tls=false
+    create_users_on_login=true
+    base_dn="DC=ad,DC=sec,DC=cloudera,DC=com"
+    bind_dn="<username>@ad.sec.cloudera.com"
+    bind_password_script=<path_to_password_script>/<script.sh>
+    test_ldap_user="testuser1"
+    test_ldap_group="testgroup1"
+
+    [[[users]]]
+    user_filter="objectclass=user"
+    user_name_attr="sAMAccountName"
+
+    [[[groups]]]
+    group_filter="objectclass=group"
+    group_name_attr="cn"
+    group_member_attr="member"
+
+Example of a Direct Bind configuration for Active Directory encrypted with LDAPS:
+
+    [[ldap]]
+    ldap_url=ldaps://w2k8-1.ad.sec.cloudera.com:636
+    search_bind_authentication=false
+    nt_domain=ad.sec.cloudera.com
+    ldap_cert=/<path_to_cacert>/w2k8-1-root.pem
+    use_start_tls=false
+    create_users_on_login=true
+    base_dn="DC=ad,DC=sec,DC=cloudera,DC=com"
+    bind_dn="<username>"
+    bind_password_script=<path_to_password_script>/<script.sh>
+    ...
+
+Example of a Direct Bind configuration for Active Directory encrypted with StartTLS:
+
+    [[ldap]]
+    ldap_url=ldap://w2k8-1.ad.sec.cloudera.com:389
+    search_bind_authentication=false
+    nt_domain=ad.sec.cloudera.com
+    ldap_cert=/opt/cloudera/security/cacerts/w2k8-1-root.pem
+    use_start_tls=true
+    create_users_on_login=true
+    base_dn="DC=ad,DC=sec,DC=cloudera,DC=com"
+    bind_dn="cconner"
+    bind_password_script=<path_to_password_script>/<script.sh>
+    ...
+
+
+#### Search Bind
+
+Search bind authentication does an ldapsearch against one or more directory services and binds with the found distinguished name (DN) and password. Hue searches the subtree from the base distinguished name. If LDAP Username Attribute is set, Hue looks for an entry whose attribute has the same value as the short name given at login.
+
+Important: Search binding works with all directory service types. It is also the only method that allows synchronizing groups at login (set with sync_groups_on_login in a safety-valve).
+
+Set the following required properties:
+
+Authentication Backend	desktop.auth.backend.LdapBackend
+
+    LDAP URL	ldaps://<ldap_server>:636 (or ldap://<ldap_server>:389)
+    LDAP Server CA Certificate	/path_to_certificate/cert.pem
+    LDAP Search Base	DC=mycompany,DC=com
+    LDAP Bind User Distinguished Name	username@domain
+    LDAP Bind Password	bind_user_password
+    Use Search Bind Authentication	TRUE
+    Enable LDAP TLS	FALSE if using LDAPS or not encrypting
+    Create LDAP users on login	TRUE
+
+Note: To encrypt with TLS, set LDAP URL to ldap://<ldap_server>:389 and check Enable LDAP TLS. For a proof of concept without encryption, use ldap://<ldap_server>:389, remove the value for LDAP Server CA Certificate, and uncheck Enable LDAP TLS.
+
+You can optionally improve search performance with attributes and filters.
+
+    LDAP User Filter	objectclass=user (default = *)
+    LDAP Username Attribute	sAMAccountName (AD default), uid (LDAP default)
+    LDAP Group Filter	objectclass=group (default = *)
+    LDAP Group Name Attribute	cn (default)
+    LDAP Group Membership Attribute	member (default)
+
+Note: With the user settings in the table above, the LDAP search filter has the form: (&(objectClass=user)(sAMAccountName=<user entered username>)).
+
+Add any valid user and/or valid group to quickly test your LDAP configuration.
+
+    LDAP Username for Test LDAP Configuration	Any valid user
+    LDAP Group Name for Test LDAP Configuration	Any valid group
+
+Note: The syntax of Bind Distinguished Name differs per bind method:
+
+    Search Bind: username@domain
+    Direct Bind with NT Domain: username
+    Direct Bind with Username Pattern: DN string (full DIT path)
+
+Do not use if anonymous binding is supported.
+
+    ## You can test ldapsearch at the command line as follows:
+    LDAPTLS_CACERT=/<path_to_cert>/<ca_certificate> ldapsearch -H ldaps://<ldap_server>:636 \
+    -D "<bind_dn>" -w <bind_password> -b <base_dn> "samaccountname=<user>"
+
+Note: To run ldapsearch with a CA certificate, you may need to install ldap_utils on Debian/Ubuntu and openldap-clients on RHEL/CentOS.
+
+#### Direct Bind
+
+To authenticate with direct binding, Hue needs either the User Principal Name (UPN) for Active Directory, or the full path to the LDAP user in the Directory Information Tree (DIT) for open standard LDAP.
+
+Important: Direct binding only works with one domain. For multiple directories, use Search Bind.
+
+To directly bind to an Active Directory/LDAP server with NT domain:
+
+Click the Configuration tab and filter by scope=Service-wide and category=Security.
+
+Set LDAP properties exactly like Search Bind with these exceptions:
+    Active Directory Domain	<your NT domain>
+    LDAP Bind User Distinguished Name	<username only> (not username@domain)
+    Use Search Bind Authentication	FALSE
+
+Test your LDAP configuration, and when successful, Restart Hue.
+To directly bind to an open standard LDAP server with a username pattern:
+
+Remove the value for Active Directory Domain.
+
+Set both LDAP Username Pattern and LDAP Bind User Distinguished Name to a DN string that represents the full path of the directory information tree, from UID to top level domain.
+
+Note: When using direct bind, set LDAP Search Base, not for authentication (you can log on to Hue without it), but to Synchronize Hue with LDAP Server.
+
+
+### SAML
+
+Secure Assertion Markup Language (SAML) single sign-on (SSO) backend. Delegates authentication to the configured Identity Provider. See Configuring Hue for SAML for more details.
+
+    libsaml.backend.SAML2Backend
+
+
+Authenticate Hue Users with SAML
+
+Hue supports SAML (Security Assertion Markup Language) for Single Sign-on (SSO) authentication.
+
+The SAML 2.0 Web Browser SSO profile has three components:
+
+* User Agent - Browser that represents you, the user, seeking resources.
+* Service Provider (SP) - Service (Hue) that sends authentication requests to SAML.
+* Identity Provider (IdP) - SAML service that authenticates users.
+
+When a user requests access to an application, the Service Provider (Hue) sends an authentication request from the User Agent (browser) to the Identity Provider. The Identity Provider authenticates the user, sends a response, and redirects the browser back to Hue.
+This page explains how to configure Hue, the Service Provider, and gives guidance on how to configure the Identity Provider, which differs per product.
+
+Configure Hue for SAML Authentication
+
+The Service Provider (Hue) and the Identity Provider use a metadata file to confirm each other's identity. Hue stores metadata from the SAML server, and the IdP stores metadata from Hue server.
+
+In Configure Hue at the Command Line, you must copy the metadata from your IdP's SAML server and store it in an XML file on every ost with a Hue server.
+Important: Read the documentation of your Identity Provider for details on how to procure the XML of the SAML server metadata.
+
+Configure Hue at the Command Line
+Important: You may need to disable cipher algorithms. See SAML SSL Error in Troubleshooting below.
+
+Install the following libraries on all hosts in your cluster:
+
+    ## RHEL/CentOS
+    yum install git gcc python-devel swig openssl
+    ## Ubuntu/Debian
+    apt-get install git gcc python-dev swig openssl
+    ## SLES
+    zypper install git gcc python-devel swig openssl make libxslt-devel libltdl-devel
+
+Install xmlsec1 and xmlsec1-openssl on all hosts in the cluster:
+
+Important: Ensure that the xmlsec1 package is executable by the user, hue.
+
+    ## RHEL/CentOS
+    yum install xmlsec1 xmlsec1-openssl
+    Note: If xmlsec libraries are not available, use the appropriate epel repository:
+    ## For RHEL/CentOS 7
+    wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-6.noarch.rpm
+    rpm -ivh epel-release-7-6.noarch.rpm
+    ## Ubuntu/Debian
+    apt-get install xmlsec1 libxmlsec1-openssl
+    ## SLES (get latest version)
+    wget http://www.aleksey.com/xmlsec/download/xmlsec1-1.2.24.tar.gz
+    tar -xvzf xmlsec1-1.2.24.tar.gz
+    cd xmlsec1-1.2.24
+    ./configure && make
+    make install
+
+Copy metadata from your IdP's SAML server and save it as an XML file on every host with a Hue server.
+For example, if your Identity Provider is Shibboleth, visit https://<idp_host>:8443/idp/shibboleth, copy the metadata content, and paste it into an .xml file.
+
+Note: You may have to edit the copied metadata; for example, the IdP's port number (8443) may be missing from its URL.
+
+    mkdir -pm 755 /opt/cloudera/security/saml/
+    cd /opt/cloudera/security/saml/
+    vim idp-<your idp provider>-metadata.xml
+
+Add key_file and cert_file for encrypted assertions–see Table of SAML Parameters.
+
+Warning: Add key and cert files even if not encrypting assertions. Hue checks for the existence and validity of these files even if they are not needed! They cannot be empty files. This is a known issue.
+
+If necessary, create "valid" dummy files:
+
+    openssl genrsa -des3 -out dummy.key 2048
+    openssl rsa -inform PEM -outform PEM -in dummy.key -pubout -out dummy-nopass.pem
+
+Configure Hue
+
+Currently, all hue.ini properties for SAML must be added to Hue Service safety-valve in Cloudera Manager.
+Log on to Cloudera Manager and go to Hue > Configuration.
+
+    ## Example Settings using Open AM:
+    [desktop]
+    redirect_whitelist="^\/.*$,^http:\/\/clr.sec.cloudera.com:8080\/.*$"
+    [[auth]]
+    backend=libsaml.backend.SAML2Backend
+    [libsaml]
+    xmlsec_binary=/usr/bin/xmlsec1
+    metadata_file=/opt/cloudera/security/saml/idp-openam-metadata.xml
+    key_file=/opt/cloudera/security/saml/host.key
+    cert_file=/opt/cloudera/security/saml/host.pem
+    username_source=nameid
+    name_id_format="urn:oasis:names:tc:SAML:2.0:nameid-format:transient"
+    entity_id=<host base name>
+    logout_enabled=false
+
+Note: For SLES distributions, the xmlsec binary may be in /usr/local/bin/. If so:
+
+* Set Hue Service Advanced Configuration Snippet: xmlsec_binary=/usr/local/bin/xmlsec1
+* Set Hue Service Environment Advanced Configuration Snippet: LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/
+
+Integrate IdP SAML Server with Hue
+
+After Hue is configured and restarted, copy the metadata generated by Hue server and send it to your Identity Provider so they can configure the SAML server.
+
+Ensure Hue is configured, restarted, and running.
+Go to http://<hue_fqdn>:8889/saml2/metadata.
+
+Copy the metadata and send it to your Identity Provider.
+
+Ensure that your Identity Provider configures the SAML server with the Hue metadata (just as you configured the Hue server with SAML metadata).
+
+SAML Properties in hue.ini
+
+Table of SAML Parameters
+
+SAML Parameter	Description
+    authn_requests_signed	Boolean, that when True, signs Hue-initiated authentication requests with X.509 certificate.
+    backend	Hard-coded value set to SAML backend library packaged with Hue (libsaml.backend.SAML2Backend).
+    base_url	URL that SAML Identity Provider uses for responses. Typically used in Load balanced Hue environments.
+    cert_file	Path to X.509 certificate sent with encrypted metadata. File format must be .PEM.
+    create_users_on_login	Boolean, that when True, creates users from OpenId, upon successful login.
+    entity_id	Service provider ID. Can also accept pattern where '<base_url>' is replaced with server URL base.
+    key_file	Path to private key used to encrypt metadata. File format must be .PEM.
+    key_file_password	Password used to decrypt the X.509 certificate in memory.
+    logout_enabled	Boolean, that when True, enables single logout.
+    logout_requests_signed	Boolean, that when True, signs Hue-initiated logout requests with an X.509 certificate.
+    metadata_file	Path to readable metadata XML file copied from Identity Provider.
+    name_id_format	Format of NameID that Hue requests from SAML server.
+    optional_attributes	Comma-separated list of optional attributes that Hue requests from Identity Provider.
+    required_attributes	Comma-separated list of required attributes that Hue requests from Identity Provider. For example, uid and email.
+    redirect_whitelist	Fully qualified domain name of SAML server: "^\/.*$,^https:\/\/<SAML_server_FQDN>\/.*$".
+    user_attribute_mapping	Map of Identity Provider attributes to Hue django user attributes. For example, {'uid':'username', 'email':'email'}.
+    username_source	Declares source of username as nameid or attributes.
+    xmlsec_binary	Path to xmlsec_binary that signs, verifies, encrypts/decrypts SAML requests and assertions. Must be executable by user, hue.
+    Description of some properties to be set in hue.ini (via Cloudera Manager):
+    redirect_whitelist [desktop]
+
+Set to the fully qualified domain name of the SAML server so that Hue can redirect to the SAML server for authentication.
+
+    [desktop]
+    redirect_whitelist=^\/.$,^https:\/\/<SAML_server_fully_qualified_domain_name>\/.$
+    Note: Hue uses redirect_whitelist to protect itself from redirecting to unapproved URLs.
+    backend [desktop]>[[auth]]
+    Point to the SAML backend (packaged with Hue):
+    backend=libsaml.backend.SAML2Backend
+    xmlsec_binary [libsaml]
+    Point to the xmlsec1 library path:
+    xmlsec_binary=/usr/bin/xmlsec1
+    Note: To find the path, run: which xmlsec1
+    metadata_file [libsaml]
+    Point to the path of the XML file you created from the IdP's metadata:
+    metadata_file=/path/to/<your_idp_metadata_file>.xml
+    key_file and cert_file [libsaml]
+
+To encrypt communication between Hue and the Identity Provider, you need a private key and certificate. The private key signs requests sent to the Identity Provider and the certificate file encrypts and decrypts messages from the Identity Provider.
+
+Copy these files from the Identity Provider and set key_file and cert_file to their respective paths. Both files are in PEM format and must be named with the .PEM extension.
+
+Note: The key and certificate files specified by the key_file and cert_file parameters in hue.ini must be .PEM files.
+Users with password-protected certificates can set the property, key_file_password in hue.ini. Hue uses the password to decrypt the SAML certificate in memory and passes it to xmlsec1 through a named pipe. The decrypted certificate never touches the disk. This only works for POSIX-compatible platforms.
+
+Troubleshooting
+
+Remember to Enable DEBUG for logging.
+
+SAML SSL Error
+
+OpenSSL might fail in CDH 5.5.x and higher with this message:
+
+    SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_CHECK_CERT_AND_ALGORITHM', 'dh key too small')]
+
+To resolve, append the following code to the file, /usr/java/<your_jdk_version>-cloudera/jre/lib/security/java.security:
+
+    jdk.tls.disabledAlgorithms=MD5, RC4, DH
+
+SAML Decrypt Error
+
+The following error is an indication that you are using a slightly different SAML protocol from what Hue expects:
+
+    Error: ('failed to decrypt', -1)
+
+To resolve:
+
+Download and rename Python script, fix-xmlsec1.txt.
+
+    wget http://www.cloudera.com/documentation/other/shared/fix-xmlsec1.txt -O fix-xmlsec1.py
+
+Change permissions as appropriate, for example:
+
+    chmod 755 fix-xmlsec1.py
+
+In hue.ini, set xmlsec_binary=<path_to_script>/fix-xmlsec1.py.
+Run fix-xmlsec1.py.
+
+This script repairs the known issue whereby xmlsec1 is not compiled with RetrievalMethod and cannot find the location of the encrypted key. SAML2 responses would sometimes place EncryptedKey outside of the EncryptedData tree. This script moves EncryptedKey under EncryptedData.
+
+
+### Spnego
+
+SPNEGO is an authentication mechanism negotiation protocol. Authentication can be delegated to an authentication server, such as a Kerberos KDC, depending on the mechanism negotiated.
+
+    desktop.auth.backend.SpnegoDjangoBackend
+
+### PAM
+
+Authenticates users with PAM (pluggable authentication module). The authentication mode depends on the PAM module used.
+
+    desktop.auth.backend.PamBackend
+
+### OAuth Connect
+
+Delegates authentication to a third-party OAuth server.
+
+    desktop.auth.backend.OAuthBackend
+
+### Multiple Authentication Backends
+
+For example, to enable Hue to first attempt LDAP directory lookup before falling back to the database-backed user model, we can update the hue.ini configuration file or Hue safety valve in Cloudera Manager with a list containing first the LdapBackend followed by either the ModelBackend or custom AllowFirstUserDjangoBackend (permits first login and relies on user model for all subsequent authentication):
+
+    [desktop]
+      [[auth]]
+      backend=desktop.auth.backend.LdapBackend,desktop.auth.backend.AllowFirstUserDjangoBackend
+
+This tells Hue to first check against the configured LDAP directory service, and if the username is not found in the directory, then attempt to authenticate the user with the Django user manager.
+
+[Read more about it here](http://gethue.com/configuring-hue-multiple-authentication-backends-and-ldap/).
+
+## Security
+
+### Configure a Proxy
+
+We explained how to run Hue with NGINX serving the static files or under Apache. If you use another proxy, you might need to set these options:
+
+<pre>
+  [desktop]
+  # Enable X-Forwarded-Host header if the load balancer requires it.
+  use_x_forwarded_host=false
+
+  # Support for HTTPS termination at the load-balancer level with SECURE_PROXY_SSL_HEADER.
+  secure_proxy_ssl_header=false
+</pre>
+
+### Configuring SSL
+
+You can configure Hue to serve over HTTPS.
+
+1. Configure Hue to use your private key by adding the following
+options to the `hue.ini` configuration file:
+
+    ssl_certificate=/path/to/certificate
+    ssl_private_key=/path/to/key
+
+2. Ideally, you would have an appropriate key signed by a Certificate Authority.
+If you're just testing, you can create a self-signed key using the `openssl`
+command that may be installed on your system:
+
+Create a key:
+
+    openssl genrsa 1024 > host.key
+
+Create a self-signed certificate:
+
+    openssl req -new -x509 -nodes -sha1 -key host.key > host.cert
+
+
+<div class="note">
+Self-signed Certificates and File Uploads
+
+To upload files using the Hue File Browser over HTTPS requires
+using a proper SSL Certificate.  Self-signed certificates don't
+work.
+</div>
+
+Note: The security vulnerability SWEET32 is also called Birthday attacks against TLS ciphers with 64bit block size and it is assigned CVE-2016-2183. This is due to legacy block ciphers
+having block size of 64 bits are vulnerable to a practical collision attack when used in CBC mode.
+
+DES/3DES are the only ciphers has block size of 64-bit. One way to config Hue not to use them:
+
+    [desktop]
+    ssl_cipher_list=DEFAULT:!DES:!3DES
+
+### SASL
+
+When getting a bigger result set from Hive/Impala or bigger files like images from HBase, the response requires to increase
+the buffer size of SASL lib for thrift sasl communication.
+
+<pre>
+  [desktop]
+  # This property specifies the maximum size of the receive buffer in bytes in thrift sasl communication,
+  # default value is 2097152 (2 MB), which equals to (2 * 1024 * 1024)
+  sasl_max_buffer=2097152
+</pre>
+
+### Storing passwords in file script
+
+Hue lets you secure passwords in one consolidated script, or multiple individual scripts. Hue runs each password script at startup and extracts passwords from stdout.
+
+Store scripts in a directory that only Hue can read, write, and execute. You can choose password script names but you cannot change hue.ini property names to which you assign those scripts.
+
+At the command line, create one or more password scripts. For example, create a consolidated script named `my_passwords_script.sh`:
+
+    #!/bin/bash
+
+    SERVICE=$1
+
+    if [[ ${SERVICE} == "ldap_password" ]]
+    then
+    echo "your_ldap_password"
+    fi
+
+    if [[ ${SERVICE} == "ssl_password" ]]
+    then
+    echo "your_ssl_password"
+    fi
+
+    if [[ ${SERVICE} == "bind_password" ]]
+    then
+    echo "your_bind_password"
+    fi
+
+    if [[ ${SERVICE} == "db_password" ]]
+    then
+    echo "your_database_password"
+    fi
+
+Add script properties, for example:
+
+    [desktop]
+    ldap_username=hueservice
+    ldap_password_script="/var/lib/hue/password_script.sh ldap_password"
+    ssl_password_script="/var/lib/hue/password_script.sh ssl_password"
+
+    [[ldap]]
+    bind_password_script="/var/lib/hue/password_script.sh bind_password"
+
+    [[database]]
+    db_password_script="/var/lib/hue/password_script.sh db_password"
+
+
+### Idle session timeout
+
+Hue now offers a new property, idle_session_timeout, that can be configured in the hue.ini file:
+
+    [desktop]
+    [[auth]]
+    idle_session_timeout=600
+
+When idle_session_timeout is set, users will automatically be logged out after N (e.g. – 600) seconds of inactivity and be prompted to login again:
+
+[Read more about it here](http://gethue.com/introducing-the-new-login-modal-and-idle-session-timeout/).
+
+### Auditing
+
+Read more about [Auditing User Administration Operations with Hue and Cloudera Navigator](http://gethue.com/auditing-user-administration-operations-with-hue-and-cloudera-navigator-2/).
+
+### Concurrent User Session Limit
+
+If set, limits the number of concurrent user sessions. 1 represents 1 browser session per user. Default: 0 (unlimited sessions per user)
+
+    [desktop]
+    [[session]]
+    concurrent_user_session_limit=0
+
+[Read more about it here](http://gethue.com/restrict-number-of-concurrent-sessions-per-user/).
+
+## Customize the UI
+
+### Maps look and feel
+
+The properties we need to tweak are leaflet_tile_layer and leaflet_tile_layer_attribution, that can be configured in the hue.ini file:
+
+    [desktop]
+    leaflet_tile_layer=https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}
+    leaflet_tile_layer_attribution='Tiles &copy; Esri &mdash; Source: Esri, i-cubed, USDA, USGS, AEX, GeoEye, Getmapping, Aerogrid, IGN, IGP, UPR-EGP, and the GIS User Community'
+
+[Read more about it here](http://gethue.com/change-your-maps-look-and-feel/).
+
+### Banner
+You can add a custom banner to the Hue Web UI by applying HTML directly to the property, banner_top_html. For example:
+
+    banner_top_html=<H4>My company's custom Hue Web UI banner</H4>
+
+### Splash Screen
+You can customize a splash screen on the login page by applying HTML directly to the property, login_splash_html. For example:
+
+    [desktop]
+    [[custom]]
+    login_splash_html=WARNING: You are required to have authorization before you proceed.
+
+### Custom Logo
+
+There is also the possibility to change the logo for further personalization.
+
+    [desktop]
+    [[custom]]
+    # SVG code to replace the default Hue logo in the top bar and sign in screen
+    # e.g. <image xlink:href="/static/desktop/art/hue-logo-mini-white.png" x="0" y="0" height="40" width="160" />
+    logo_svg=
+
+You can go crazy and write there any SVG code you want. Please keep in mind your SVG should be designed to fit in a 160×40 pixels space. To have the same ‘hearts logo' you can see above, you can type this code
+
+    [desktop]
+    [[custom]]
+    logo_svg='<g><path stroke="null" id="svg_1" d="m44.41215,11.43463c-4.05017,-10.71473 -17.19753,-5.90773 -18.41353,-0.5567c-1.672,-5.70253 -14.497,-9.95663 -18.411,0.5643c-4.35797,11.71793 16.891,22.23443 18.41163,23.95773c1.5181,-1.36927 22.7696,-12.43803 18.4129,-23.96533z" fill="#ffffff"/> <path stroke="null" id="svg_2" d="m98.41246,10.43463c-4.05016,-10.71473 -17.19753,-5.90773 -18.41353,-0.5567c-1.672,-5.70253 -14.497,-9.95663 -18.411,0.5643c-4.35796,11.71793 16.891,22.23443 18.41164,23.95773c1.5181,-1.36927 22.76959,-12.43803 18.41289,-23.96533z" fill="#FF5A79"/> <path stroke="null" id="svg_3" d="m154.41215,11.43463c-4.05016,-10.71473 -17.19753,-5.90773 -18.41353,-0.5567c-1.672,-5.70253 -14.497,-9.95663 -18.411,0.5643c-4.35796,11.71793 16.891,22.23443 18.41164,23.95773c1.5181,-1.36927 22.76959,-12.43803 18.41289,-23.96533z" fill="#ffffff"/> </g>'
+
+Read more about it in [Hue with a custom logo](http://gethue.com/hue-with-a-custom-logo/) post.
+
+## Source Version Control
+
+By default Hue stores the [saved documents]({{% param baseURL %}}user/concept/#documents) in its database. This features aims at pointing to any source versioning systems like GitHub, BitBucket... to open and save queries.
+
+**Note** This feature is experiemental and tracked in [HUE-951](https://issues.cloudera.org/browse/HUE-951).
+
+    [desktop]
+
+    [[vcs]]
+
+    ## [[[git-read-only]]]
+        ## Base URL to Remote Server
+        # remote_url=https://github.com/cloudera/hue/tree/master
+
+        ## Base URL to Version Control API
+        # api_url=https://api.github.com
+    ## [[[github]]]
+
+        ## Base URL to Remote Server
+        # remote_url=https://github.com/cloudera/hue/tree/master
+
+        ## Base URL to Version Control API
+        # api_url=https://api.github.com
+
+        # These will be necessary when you want to write back to the repository.
+        ## Client ID for Authorized Application
+        # client_id=
+
+        ## Client Secret for Authorized Application
+        # client_secret=
+    ## [[[svn]]
+        ## Base URL to Remote Server
+        # remote_url=https://github.com/cloudera/hue/tree/master
+
+        ## Base URL to Version Control API
+        # api_url=https://api.github.com
+
+        # These will be necessary when you want to write back to the repository.
+        ## Client ID for Authorized Application
+        # client_id=
+
+        ## Client Secret for Authorized Application
+        # client_secret=

+ 1 - 1
docs/docs-site/content/administrator/installation/_index.md

@@ -7,7 +7,7 @@ weight: 1
 
 ## Quick Start
 
-Follow the [Quick Start](https://github.com/cloudera/hue#getting-started) or [Docker]({{% param baseURL %}}administrator/installation/docker/) guides.
+Follow the [Quick Start](https://github.com/cloudera/hue#getting-started) or [Container]({{% param baseURL %}}administrator/installation/containers/) guides.
 
 
 ## Installation

+ 16 - 0
docs/docs-site/content/administrator/installation/containers.md

@@ -0,0 +1,16 @@
+---
+title: "Containers"
+date: 2019-03-13T18:28:08-07:00
+draft: false
+weight: 3
+---
+
+## Docker
+
+In addition to providing to providing a ready to use image, the [Docker Guide](https://github.com/cloudera/hue/tree/testing/tools/docker) shows how to build it.
+
+It then details how to start the containers and parameterize them.
+
+## Kubernetes
+
+The [Kubernetes Guide](https://github.com/cloudera/hue/tree/testing/tools/kubernetes) shows how to run the services via Helm or native YAML configs.

+ 4 - 0
docs/docs-site/content/administrator/installation/dependencies/_index.md

@@ -102,13 +102,17 @@ for more details, refer to this link: [https://docs.oracle.com/cd/E37670_01/E590
 * [Oracle Instant Client](http://www.oracle.com/technetwork/database/database-technologies/instant-client/downloads/index.html)
 
 1. Install Dependencies via Homebrew
+
     brew install mysql maven gmp openssl libffi && brew cask install java8
 
 2. Install Xcode command line tools
+
     sudo xcode-select --install
 
 3. Fix openssl errors
+
 Required for MacOS 10.11+
+
     export LDFLAGS=-L/usr/local/opt/openssl/lib && export CPPFLAGS=-I/usr/local/opt/openssl/include
 
 

+ 0 - 10
docs/docs-site/content/administrator/installation/docker.md

@@ -1,10 +0,0 @@
----
-title: "Docker"
-date: 2019-03-13T18:28:08-07:00
-draft: false
-weight: 3
----
-
-Alternatively to building Hue into an image, the [Hue Docker Guide](https://github.com/cloudera/hue/tree/testing/tools/docker) is available.
-
-It details how to start a Hue in one command line or build a container image with all the dependencies.

+ 3 - 1
docs/docs-site/content/administrator/installation/install.md

@@ -5,7 +5,9 @@ draft: false
 weight: 2
 ---
 
-Configure `$PREFIX` with the path where you want to install Hue by running:
+Start from a [release]({{% param baseURL %}}releases/) tarball or [building](https://github.com/cloudera/hue#getting-started) the project yourself.
+
+Then configure `$PREFIX` with the path where you want to install Hue by running:
 
     PREFIX=/usr/share make install
     cd /usr/share/hue

+ 4 - 3
docs/docs-site/content/administrator/installation/starting.md

@@ -52,12 +52,13 @@ software components.
 ## Feedback
 
 Your feedback is welcome. The best way to send feedback is to join the
-[mailing list](https://groups.google.com/a/cloudera.org/group/hue-user), and
-send e-mail, to mailto:hue-user@cloudera.org[hue-user@cloudera.org].
+[mailing list](https://groups.google.com/a/cloudera.org/group/hue-user) and
+send e-mail to [hue-user@cloudera.org](mailto:hue-user@cloudera.org).
+[@gethue](https://twitter.com/gethue) is also a good
 
 ## Reporting Bugs
 
 If you find that something doesn't work, it'll often be helpful to include logs
 from your server. Please include the
 logs as a zip (or cut and paste the ones that look relevant) and send those with
-your bug reports.
+your bug reports.

+ 27 - 0
docs/docs-site/content/developer/development/_index.md

@@ -21,6 +21,33 @@ This section goes into greater detail on how to build and reuse the components o
 * [Bootstrap](http://twitter.github.com/bootstrap/)
 * [Knockout js](http://knockoutjs.com/)
 
+### Quick Start
+
+Build once:
+
+    make apps
+
+Then start the dev server (which will auto reload):
+
+    ./build/env/bin/hue runserver
+
+If you are changing Javascript or CSS files, also start:
+
+    npm run dev
+
+Then it is recommended to use MySQL or PostGres as the [database](({{% param baseURL %}}).
+
+Open the `hue.ini` file in a text editor. Directly below the
+`[[database]]` line, add the following options (and modify accordingly for
+your MySQL setup):
+
+    host=localhost
+    port=3306
+    engine=mysql
+    user=hue
+    password=secretpassword
+    name=hue
+
 ### Javascript
 
 The javascript files are currently being migrated to webpack bundles, during this process some files will live under src/desktop/static/ and some will live under src/dekstop/js