Browse Source

[doc] Update of README, manual and SDK

Added Proxy app section in SDK
Romain Rigaux 13 năm trước cách đây
mục cha
commit
04730f6
3 tập tin đã thay đổi với 94 bổ sung44 xóa
  1. 19 16
      README.rst
  2. 29 25
      docs/manual.txt
  3. 46 3
      docs/sdk/sdk.md

+ 19 - 16
README.rst

@@ -6,12 +6,15 @@ Welcome to the repository for Hue
     end users, please put them in ``dist/README``.
 
 Hue is both a Web UI for Hadoop and a framework to create interactive Web
-applications.
+applications. It features:
 
-It features a FileBrowser for accessing HDFS, JobSub and
-JobBrowser applications for submitting and viewing MapReduce jobs, a Beeswax
-application for executing Hive queries. On top of that, a SDK is available
-for creating new apps integrated with Hadoop.
+      * FileBrowser for accessing HDFS
+      * Job Designer and Oozie for submitting and scheduling workflows of MapReduce/Java/Streaming/Pig jobs
+      * JobBrowser for viewing MapReduce jobs
+      * Beeswax application for executing Hive queries
+      * A Pig/HBase shell
+
+On top of that, a SDK is available for creating new apps integrated with Hadoop.
 
 More documentation is available at http://cloudera.github.com/hue/.
 
@@ -99,32 +102,32 @@ your system:
 
 File Layout
 ===========
-The "core" stuff is in ``desktop/core/``, whereas installable apps live in
-``apps/``.  Please place third-party dependencies in the app's ext-py/
+The Hue "framework" is in ``desktop``. ``/core/`` contains the Web components and
+``desktop/libs/`` the API for talking to Hadoop.
+The installable apps live in ``apps/``.  Please place third-party dependencies in the app's ext-py/
 directory.
 
 The typical directory structure for inside an application includes:
 
   src/
-    for Python code
+    for Python/Django code
       models.py
       urls.py
       views.py
       forms.py
       settings.py
-        for Django code
- 
+
   conf/
     for configuration (``.ini``) files to be installed
 
   static/
-    for static HTML and js resources
+    for static HTML/js resources and help doc
 
   templates/
     for data to be put through a template engine
 
-  docs/
-    for helpful notes
+  locales/
+    for localizations in multiple languages
 
 For the URLs within your application, you should make your own ``urls.py``
 which will be automatically rooted at ``/yourappname/`` in the global
@@ -136,9 +139,9 @@ Main Stack
 
    * Python 2.4 - 2.7
    * Django 1.2 https://docs.djangoproject.com/en/1.2/
-   * Mako 0.7
-   * jQuery 1.7
-   * Bootstrap 2
+   * Mako
+   * jQuery
+   * Bootstrap
 
 
 Using and Installing Thrift

+ 29 - 25
docs/manual.txt

@@ -17,8 +17,9 @@ There is also a companion SDK guide that describes how to develop
 new Hue applications:
 link:sdk/sdk.html[Hue SDK Documentation]
 
-IMPORTANT: Hue requires the Hadoop contained in Cloudera's Distribution including
-Apache Hadoop (CDH), version 3 update 4 or later.
+IMPORTANT: Hue requires the Hadoop contained in
+https://ccp.cloudera.com/display/CDH4DOC/CDH4+Quick+Start+Guide[Cloudera's
+Distribution including Apache Hadoop (CDH4)]
 
 .Conventions Used in this Guide:
 * Commands that must be run with +root+ permission have a +#+ command prompt.
@@ -59,6 +60,7 @@ tarball, you must have the following installed:
 Redhat,Ubuntu
 ~~~~~~~~~~
 gcc,gcc
+g+,g++
 libxml2-devel,libxml2-dev
 libxslt-devel,libxslt-dev
 cyrus-sasl-devel,libsasl2-dev
@@ -71,6 +73,8 @@ sqlite-devel,libsqlite3-dev
 ant,ant
 ~~~~~~~~~~
 
+The full list is here: https://github.com/cloudera/hue#development-prerequisites
+
 Build
 ^^^^^
 
@@ -114,7 +118,7 @@ software components.
 Install Hadoop from CDH
 ~~~~~~~~~~~~~~~~~~~~~~~
 
-To use Hue, you must install and run Hadoop from CDH3u4 or later.  If you
+To use Hue, you must install and run Hadoop from CDH4 or later.  If you
 are not running this version of CDH or later, upgrade your cluster before
 proceeding.
 
@@ -124,17 +128,15 @@ proceeding.
   Component   Required   Applications                       Notes
 -------------------------------------------------------------------------------------
   HDFS        Yes        Core, Filebrowser                  HDFS access through WebHdfs or HttpFS
-  MR1         No         JobBrowser, JobDesigner*, Beeswax* Job information access through hue-plugins
-  Yarn        No         JobDesigner*, Beeswax*             Transitive dependency via Hive or Oozie
-  Oozie       No         JobDesigner                        Oozie access through REST API
+  MR1         No         JobBrowser, JobDesigner, Beeswax   Job information access through hue-plugins
+  Yarn        No         JobDesigner, Beeswax               Transitive dependency via Hive or Oozie
+  Oozie       No         JobDesigner, Oozie                 Oozie access through REST API
   Hive        No         Beeswax                            Beeswax uses the Hive client libraries
   Flume       No         Shell                              Optionally provides access to the Flume shell
   HBase       No         Shell                              Optionally provides access to the HBase shell
   Pig         No         Shell                              Optionally provides access to the Pig shell
 -------------------------------------------------------------------------------------
 
-[*] Transitive dependency
-
 
 Hadoop Configuration
 ~~~~~~~~~~~~~~~~~~~~
@@ -274,10 +276,10 @@ Unfortunately, when the variable is unset, you'll end up
 with directories named `/tmp/hadoop-user_name-${hue.suffix}` in
 `/tmp`.  Despite that, Hue will still work.
 
-IMPORTANT: The Beeswax server writes into a local directory on the Hue machine 
-that is specified by `hadoop.tmp.dir` to unpack its jars. That directory 
-needs to be writable by the `hue` user, which is the default user who starts 
-Beeswax Server, or else Beeswax server will not start. You may also make that 
+IMPORTANT: The Beeswax server writes into a local directory on the Hue machine
+that is specified by `hadoop.tmp.dir` to unpack its jars. That directory
+needs to be writable by the `hue` user, which is the default user who starts
+Beeswax Server, or else Beeswax server will not start. You may also make that
 directory world-writable.
 
 
@@ -301,7 +303,7 @@ Your Hive data is stored in HDFS, normally under `/user/hive/warehouse`
 (or any path you specify as `hive.metastore.warehouse.dir` in your
 `hive-site.xml`).  Make sure this location exists and is writable by
 the users whom you expect to be creating tables.  `/tmp` (on the local file
-system) must be world-writable, as Hive makes extensive use of it.
+system) must be world-writable (1777), as Hive makes extensive use of it.
 
 [NOTE]
 If you used the embedded Hive MetaStore functionality of Beeswax in Hue from
@@ -339,8 +341,8 @@ Configuring Hue
 
 Hue ships with a default configuration that will work for
 pseudo-distributed clusters.  If you are running on a real cluster, you must
-make a few changes to the `/etc/hue/hue.ini` configuration file. The
-following sections describe the key configuration options you must make to
+make a few changes to the `/etc/hue/hue.ini` configuration file (or ``pseudo-distributed.ini`
+when in development mode). The following sections describe the key configuration options you must make to
 configure Hue.
 
 
@@ -394,13 +396,13 @@ Specifying the Secret Key
 
 For security, you should also specify the secret key that is used for secure
 hashing in the session store. Enter a long series of random characters
-(30 to 60 characters is recommended). 
+(30 to 60 characters is recommended).
 
     secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
 
-NOTE: If you don't specify a secret key, your session cookies will not be 
+NOTE: If you don't specify a secret key, your session cookies will not be
 secure. Hue will run but it will also display error messages telling you to
-set the secret key. 
+set the secret key.
 
 Authentication
 ^^^^^^^^^^^^^^
@@ -411,7 +413,7 @@ user can create other user and administrator accounts.  User information is
 stored in the Django database in the Django backend.
 
 The authentication system is pluggable. For more information, see the
-http://archive.cloudera.com/cdh3/hue/sdk/sdk.html[Hue SDK Documentation].
+link:sdk/sdk.html[Hue SDK Documentation].
 
 Configuring Hue for SSL
 ^^^^^^^^^^^^^^^^^^^^^^^
@@ -576,10 +578,10 @@ beeswax_server_heapsize::
   The heap size (-Xmx) of the Beeswax Server.
 
 
-JobDesigner Configuration
-~~~~~~~~~~~~~~~~~~~~~~~~~
+JobDesigner and Oozie Configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-In the `[jobsub]` section of the configuration file, you should
+In the `[liboozie]` section of the configuration file, you should
 specify:
 
 oozie_url::
@@ -707,7 +709,7 @@ Inspecting the Hue Database
 
 The default SQLite database used by Hue is located in: `/usr/share/hue/desktop/desktop.db`.
 You can inspect this database from the command line using the `sqlite3`
-program. For example:
+program or typing `/usr/share/hue/build/env/bin/hue dbshell'. For example:
 
   # sqlite3 /usr/share/hue/desktop/desktop.db
   SQLite version 3.6.22
@@ -803,14 +805,16 @@ The Help application guides users through the various installed applications.
 Supported Browsers
 ~~~~~~~~~~~~~~~~~~
 
-Hue is primarily tested on Firefox, Google Chrome and Safari on Windows, Mac,
+Hue is primarily tested on Firefox, Chrome and Safari on Windows, Mac,
 and Linux.
 
+Most of Hue should work in IE8+ and Opera is not tested.
+
 Feedback
 ~~~~~~~~
 
 Your feedback is welcome. The best way to send feedback is to join the
-https://groups.google.com/a/cloudera.org/group/hue-user[mailing list], and 
+https://groups.google.com/a/cloudera.org/group/hue-user[mailing list], and
 send e-mail, to mailto:hue-user@cloudera.org[hue-user@cloudera.org].
 
 Reporting Bugs

+ 46 - 3
docs/sdk/sdk.md

@@ -164,8 +164,8 @@ Download, Unpack, Build Distro
 ------------------------------
 
 The Hue SDK is available from [Github](http://github.com/cloudera/hue). Releases
-can be found on the [download page](https://github.com/cloudera/hue/downloads). 
-Releases are missing a few dependencies that could not be included because of 
+can be found on the [download page](https://github.com/cloudera/hue/downloads).
+Releases are missing a few dependencies that could not be included because of
 licencing issues (e.g. the werkzeug module). So if you prefer to have an
 environment ready from scratch, it is preferable to checkout a particular
 release tag instead.
@@ -341,7 +341,7 @@ page. You can click on any stack frame to get a debugging console:
 Great! Now that we've added a single application, we're going to
 delve further into the back-end.
 
-A Look at Two Existing Apps
+A Look at Three Existing Apps
 ===========================
 
 ![Arch](arch_examples.png)
@@ -362,6 +362,49 @@ No JavaScript was written to make this happen!  Instead, the template
 applied certain CSS classes to the relevant `div`'s, and JFrame
 did the rest.
 
+Proxy
+-----
+
+### Setup
+
+You need to have Hue running:
+
+    $ ./build/env/bin/hue runserver
+
+Then if you want to access localhost/50030/jobtracker.jsp you just do:
+
+    http://127.0.0.1:8000/proxy/localhost/50030/jobtracker.jsp
+
+and the page will be displayed within Hue.
+
+You can configure it in ``desktop/conf/pseudo-distributed.ini``
+
+    [proxy]
+    whitelist="(localhost|127\.0\.0\.1)50030|50070|50060|50075)",
+    #Comma-separated list of regular expressions, which match 'host:port' of requested proxy target.
+
+    blacklist=""
+    #Comma-separated list of regular expressions, which match any prefix of 'host:port/path' of requested proxy target.
+    # This does not support matching GET parameters.
+
+### Usage
+
+You can create a new app (or modify a current one for testing).
+
+Then in order to display the proxied page in your app, you could add in the template of a view of the new app a
+snippet of Javacript similar to this for loading the JobTracker page:
+
+    <script>
+        $.get('/proxy/localhost/50030/jobtracker.jsp', function(data) { $('#proxy-body').html(data); alert('Load was performed.'); });
+    </script>
+
+or alternatively get the page in the view (better solution) with the Hue
+[REST API](https://github.com/cloudera/hue/tree/master/desktop/core/src/desktop/lib/rest). Example of use of
+this API can be found in the [HDFS lib](https://github.com/cloudera/hue/blob/master/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py).
+
+If you need to browse through the proxied page, using an iframe might be a better solution.
+
+
 Beeswax
 -------