title: How to configure Hue for your Hadoop cluster author: admin type: post date: 2014-10-02T22:01:37+00:00 url: /how-to-configure-hue-in-your-hadoop-cluster/ sf_thumbnail_type:
Hue is a lightweight Web server that lets you use Hadoop directly from your browser. Hue is just a ‘view on top of any Hadoop distribution’ and can be installed on any machine.
There are multiples ways (cf. ‘Download’ section of gethue.com) to install Hue. The next step is then to configure Hue to point to your Hadoop cluster. By default Hue assumes a local cluster (i.e. there is only one machine) is present. In order to interact with a real cluster, Hue needs to know on which hosts are distributed the Hadoop services.
Hue main configuration happens in a hue.ini file. It lists a lot of options but essentially what are the addresses and ports of HDFS, YARN, Oozie, Hive… Depending on the distribution you installed the ini file is located:
**
Note:** To override a value in Cloudera Manager, you need to enter verbatim each mini section from below into the Hue Safety Valve: Hue Service → Configuration → Service-Wide → Advanced → Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini
At any time, you can see the path to the hue.ini and what are its values on the /desktop/dump_config page. Then, for each Hadoop Service, Hue contains a section that needs to be updated with the correct hostnames and ports. Here is an example of the Hive section in the ini file:
{{< highlight bash >}}[beeswax]
# Host where HiveServer2 is running.
hive_server_host=localhost
{{< /highlight >}}
To point to another server, just replaced the host value by 'hiveserver.ent.com':
{{< highlight bash >}}[beeswax]
# Host where HiveServer2 is running.
hive_server_host=hiveserver.ent.com
{{< /highlight >}}
Note: Any line starting with a # is considered as a comment so is not used.
Note: The list of mis-configured services are listed on the /about/admin_wizard page.
Note: After each change in the ini file, Hue should be restarted to pick it up.
Note: In some cases, as explained in how to configure Hadoop for Hue documentation, the API of these services needs to be turned on and Hue set as proxy user.
This article shows how to configure Hue to not show certain apps. The list of all the apps is available on the /desktop/dump_config page of Hue.
Here are the main sections that you will need to update in order to have each service accessible in Hue:
This is required for listing or creating files. Replace localhost by the real address of the NameNode (usually http://localhost:50070).
Enter this in hdfs-site.xml to enable WebHDFS in the NameNode and DataNodes:
{{< highlight xml >}}
dfs.webhdfs.enabled
true
{{< /highlight >}}
Configure Hue as a proxy user for all other users and groups, meaning it may submit a request on behalf of any other user. Add to core-site.xml:
{{< highlight xml >}}
hadoop.proxyuser.hue.hosts
*
hadoop.proxyuser.hue.groups
*
{{< /highlight >}}
Then, if the Namenode is on another host than Hue, don't forget to update in the hue.ini:
{{< highlight bash >}}[hadoop]
[[hdfs_clusters]]
[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://localhost:8020
# Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
webhdfs_url=http://localhost:50070/webhdfs/v1
{{< /highlight >}}
The Resource Manager is often on http://localhost:8088 by default. The ProxyServer and Job History servers also needs to be specified. Then Job Browser will let you list and kill running applications and get their logs.
{{< highlight bash >}}[hadoop]
[[yarn_clusters]]
[[[default]]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=localhost
# Whether to submit jobs to this cluster
submit_to=True
# URL of the ResourceManager API
resourcemanager_api_url=http://localhost:8088
# URL of the ProxyServer API
proxy_api_url=http://localhost:8088
# URL of the HistoryServer API
history_server_api_url=http://localhost:19888
{{< /highlight >}}
Here we need a running HiveServer2 in order to send SQL queries.
{{< highlight bash >}}[beeswax]
# Host where HiveServer2 is running.
hive_server_host=localhost
{{< /highlight >}}
Note:
If HiveServer2 is on another machine and you are using security or customized HiveServer2 configuration, you will need to copy the hive-site.xml on the Hue machine too:
{{< highlight bash >}}[beeswax]
# Host where HiveServer2 is running.
hive_server_host=localhost
# Hive configuration directory, where hive-site.xml is located
hive_conf_dir=/etc/hive/conf
{{< /highlight >}}
We need to specify one of the Impalad address for interactive SQL in the Impala app.
{{< highlight bash >}}[impala]
# Host of the Impala Server (one of the Impalad)
server_host=localhost
{{< /highlight >}}
We just need to specify the address of a Solr Cloud (or non Cloud Solr), then interactive dashboards capabilities are unleashed!
{{< highlight bash >}}[search]
# URL of the Solr Server
solr_url=http://localhost:8983/solr/
{{< /highlight >}}
An Oozie server should be up and running before submitting or monitoring workflows.
{{< highlight bash >}}[liboozie]
# The URL where the Oozie service runs on.
oozie_url=http://localhost:11000/oozie
{{< /highlight >}}
The Pig Editor requires Oozie to be setup with its sharelib.
The HBase app works with a HBase Thrift Server version 1. It lets you browse, query and edit HBase tables.
{{< highlight bash >}}[hbase]
# Comma-separated list of HBase Thrift server 1 for clusters in the format of '(name|host:port)'.
hbase_clusters=(Cluster|localhost:9090)
{{< /highlight >}}
Hue just needs to point to the machine with the Sentry server running.
{{< highlight bash >}}[libsentry]
# Hostname or IP of server.
hostname=localhost
{{< /highlight >}}
And that’s it! Now Hue will let you do Big Data directly from your browser without touching the command line! You can then follow-up with some tutorials.
As usual feel free to comment and send feedback on the hue-user list or @gethue!