Browse Source

HUE-8740 [docs] Refactor some of the old dev content

Romain Rigaux 6 years ago
parent
commit
49e029879b

+ 3 - 1
docs/docs-site/config.toml

@@ -1,4 +1,5 @@
 baseURL = 'http://cloudera.github.io/hue/latest/' # Note: also in [params]
+#baseURL = '/' # Flip both to see images in developement mode
 
 languageCode = "en-us"
 title = "Hue Documentation"
@@ -13,6 +14,7 @@ home = [ "HTML", "RSS", "JSON"]
 [params]
   # For logo and favicon partials
   baseURL = 'http://cloudera.github.io/hue/latest/'
+  #baseURL = '/'
 
   # Prefix URL to edit current page. Will display an "Edit this page" button on top right hand corner of every page.
   # Useful to give opportunity to people to create merge request for your doc.
@@ -37,7 +39,7 @@ home = [ "HTML", "RSS", "JSON"]
   # Hide breadcrumbs in the header and only show the current page title
   disableBreadcrumb = false
   # Hide Next and Previous page buttons normally displayed full height beside content
-  disableNextPrev = false
+  disableNextPrev = true
   # Order sections in menu by "weight" or "title". Default to "weight"
   ordersectionsby = "weight"
   # Change default color scheme with a variant one. Can be "red", "blue", "green".

+ 2 - 0
docs/docs-site/content/_index.md

@@ -8,6 +8,8 @@ pre = "<b>1. </b>"
 
 # Documentation
 
+![Hue Logo Back-end]({{% param baseURL %}}images/hue_logo.png)
+
 The manual to install, use and develop Hue.
 
 Hue is a mature open source workbench for self-service operation and development of SQL applications

+ 113 - 159
docs/docs-site/content/developer/development/_index.md

@@ -7,9 +7,12 @@ weight: 1
 
 This section goes into greater detail on how to build and reuse the components of Hue.
 
-## Dependencies
 
-* The OS specific dependencies are listed [here](/administrator/installation/dependencies/)
+## Building
+
+### Dependencies
+
+* The OS specific install instructions are listed in the [install guide](/administrator/installation/dependencies/)
 * Python 2.7+ (Python 3 support tracked in [HUE-8737](https://issues.cloudera.org/browse/HUE-8737))
 * Django (1.11 already included in the distribution)
 * Java (Java 1.8) (should go away after [HUE-8740](https://issues.cloudera.org/browse/HUE-8740))
@@ -18,7 +21,33 @@ This section goes into greater detail on how to build and reuse the components o
 * [Bootstrap](http://twitter.github.com/bootstrap/)
 * [Knockout js](http://knockoutjs.com/)
 
-## Building
+### Javascript
+
+The javascript files are currently being migrated to webpack bundles, during this process some files will live under src/desktop/static/ and some will live under src/dekstop/js
+
+For changes to the files under src/desktop/js the following applies:
+
+First make sure all third-party dependencies defined in package.json are installed into node_modules/
+
+    npm install
+
+Also run this after making changes to package.json, adding new third-party dependencies etc.
+
+To generate the js bundles run:
+
+    npm run webpack
+    npm run webpack-workers
+    npm run webpack-login
+
+During development the bundles can be autogenerated when it detects changes to the .js files, for this run:
+
+    npm run dev
+
+Before sending a review with changes to the bundles run:
+
+    npm run lint-fix
+
+and possibly fix any issues it might report.
 
 ### Documentation
 
@@ -56,34 +85,6 @@ Old documentation:
 
     make docs
 
-### Javascript
-
-The javascript files are currently being migrated to webpack bundles, during this process some files will live under src/desktop/static/ and some will live under src/dekstop/js
-
-For changes to the files under src/desktop/js the following applies:
-
-First make sure all third-party dependencies defined in package.json are installed into node_modules/
-
-    npm install
-
-Also run this after making changes to package.json, adding new third-party dependencies etc.
-
-To generate the js bundles run:
-
-    npm run webpack
-    npm run webpack-workers
-    npm run webpack-login
-
-During development the bundles can be autogenerated when it detects changes to the .js files, for this run:
-
-    npm run dev
-
-Before sending a review with changes to the bundles run:
-
-    npm run lint-fix
-
-and possibly fix any issues it might report.
-
 ### CSS / LESS
 
 After changing the CSS in a .less file, rebuilding with:
@@ -125,7 +126,26 @@ How to create a new locale for an app:
     cd $APP_ROOT/src/$APP_NAME/locale
     $HUE_ROOT/build/env/bin/pybabel init -D django -i en_US.pot -d . -l fr
 
-## Webapp API
+### Dev environment
+
+#### Debugging with PyCharm
+
+First step is to configure pycharm to use the Hue virtual environment at ./build/env/env
+![Pycharm virtualenv]({{% param baseURL %}}images/pycharm_virtualenv.png)
+Second step is to configure the debug configuration
+![Pycharm debug]({{% param baseURL %}}images/pycharm_debug.png)
+
+#### Debugging with Eclipse
+
+First step is to configure Eclipse to use the Hue virtual environment at ./build/env/env
+![Eclipse interpreter]({{% param baseURL %}}images/eclipse_interpreter.png)
+Second step is to configure the debug configuration
+![Eclipse debug]({{% param baseURL %}}images/eclipse_debug.png)
+![Eclipse debug arguments]({{% param baseURL %}}images/eclipse_debug_arguments.png)
+![Eclipse debug interpreter]({{% param baseURL %}}images/eclipse_debug_interpreter.png)
+
+
+## API Server
 
 ### From 30,000 feet
 
@@ -154,9 +174,8 @@ daemon processes "on the side". Some examples are the `Celery Task Server`, `Cel
 
 ![Interacting with Hadoop]({{% param baseURL %}}images/interactingwithhadoop.png)
 
-Hue provides some APIs for interacting with Hadoop.
-Most noticeably, there are python file-object-like APIs for
-interacting with HDFS.  These APIs work by making REST API or Thrift calls
+Hue provides some APIs for interacting with external services like Databases of File storages.
+These APIs work by making REST API or Thrift calls
 the Hadoop daemons.
 
 ### An Architectural View
@@ -213,7 +232,9 @@ For the URLs within your application, you should make your own ``urls.py``
 which will be automatically rooted at ``/yourappname/`` in the global
 namespace. See ``apps/about/src/about/urls.py`` for an example.
 
-### Configuration File
+### Configurations
+
+#### File
 
 Hue uses a typed configuration system that reads configuration files (in an
 ini-style format).  By default, Hue loads all `*.ini` files in the `build/desktop/conf`
@@ -235,62 +256,60 @@ directory.  The configuration files have the following format:
     # namenode_host = 10.0.0.1
 
 
-### Configuration Variables
+#### Variables
 
 Your application's `conf.py` is special. It provides access to the configuration file (and even
 default configurations not specified in the file). Using the above example, your `conf.py` should
 define the following:
 
-* A `desktop.lib.conf.Config` object for `app_property`, such as:
-<pre>
-  MY_PROPERTY = Config(key='app_property', default='Beatles', help='blah')
-</pre>
+A `desktop.lib.conf.Config` object for `app_property`, such as:
+
+    MY_PROPERTY = Config(key='app_property', default='Beatles', help='blah')
+
   You can access its value by `MY_PROPERTY.get()`.
 
-* A `desktop.lib.conf.ConfigSection` object for `section_a`, such as:
-<pre>
-  SECTION_A = ConfigSection(key='section_a',
-        help='blah',
-        members=dict(
-          AWEIGHT=Config(key='a_weight', type=int, default=0),
-          AHEIGHT=Config(key='a_height', type=int, default=0)))
-</pre>
+A `desktop.lib.conf.ConfigSection` object for `section_a`, such as:
+
+    SECTION_A = ConfigSection(key='section_a',
+          help='blah',
+          members=dict(
+            AWEIGHT=Config(key='a_weight', type=int, default=0),
+            AHEIGHT=Config(key='a_height', type=int, default=0)))
+
   You can access the values by `SECTION_A.AWEIGHT.get()`.
 
-* A `desktop.lib.conf.UnspecifiedConfigSection` object for `filesystems`, such as:
-<pre>
-  FS = UnspecifiedConfigSection(
-      key='filesystems',
-      each=ConfigSection(members=dict(
-          nn_host=Config(key='namenode_host', required=True))
-</pre>
-  An `UnspecifiedConfigSection` is useful when the children of the section are not known.
-  When Hue loads your application's configuration, it binds all sub-sections. You can
-  access the values by:
-<pre>
-  cluster1_val = FS['cluster_1'].nn_host.get()
-  all_clusters = FS.keys()
-  for cluster in all_clusters:
-      val = FS[cluster].nn_host.get()
-</pre>
-
-Your Hue application can automatically detect configuration problems and alert
+A `desktop.lib.conf.UnspecifiedConfigSection` object for `filesystems`, such as:
+
+    FS = UnspecifiedConfigSection(
+        key='filesystems',
+        each=ConfigSection(members=dict(
+            nn_host=Config(key='namenode_host', required=True))
+
+An `UnspecifiedConfigSection` is useful when the children of the section are not known.
+When Hue loads your application's configuration, it binds all sub-sections. You can
+access the values by:
+
+    cluster1_val = FS['cluster_1'].nn_host.get()
+    all_clusters = FS.keys()
+    for cluster in all_clusters:
+        val = FS[cluster].nn_host.get()
+
+
+Application can automatically detect configuration problems and alert
 the admin. To take advantage of this feature, create a `config_validator`
 function in your `conf.py`:
 
-<pre>
-  def config_validator(user):
-    """
-    config_validator(user) -> [(config_variable, error_msg)] or None
-    Called by core check_config() view.
-    """
-    res = [ ]
-    if not REQUIRED_PROPERTY.get():
-      res.append((REQUIRED_PROPERTY, "This variable must be set"))
-    if MY_INT_PROPERTY.get() < 0:
-      res.append((MY_INT_PROPERTY, "This must be a non-negative number"))
-    return res
-</pre>
+    def config_validator(user):
+      """
+      config_validator(user) -> [(config_variable, error_msg)] or None
+      Called by core check_config() view.
+      """
+      res = [ ]
+      if not REQUIRED_PROPERTY.get():
+        res.append((REQUIRED_PROPERTY, "This variable must be set"))
+      if MY_INT_PROPERTY.get() < 0:
+        res.append((MY_INT_PROPERTY, "This must be a non-negative number"))
+      return res
 
 
 <div class="note">
@@ -304,39 +323,6 @@ function in your `conf.py`:
 </div>
 
 
-### Running "Helper Processes"
-
-Some Hue applications need to run separate daemon processes on the side.
-
-Suppose your application needs a helper `my_daemon.py`. You need to register it by:
-
-* In `setup.py`, add to `entry_points`:
-<pre>
-    entry_points = {
-      'desktop.sdk.application': 'my_app = my_app',
-      'desktop.supervisor.specs': [ 'my_daemon = my_app:SUPERVISOR_SPEC' ] }
-</pre>
-
-* In `src/my_app/__init__.py`, tell Hue what to run by adding:
-<pre>
-    SUPERVISOR_SPEC = dict(django_command="my_daemon")
-</pre>
-
-* Then in `src/my_app/management/commands`, create `__init__.py` and `my_daemon.py`. Your
-  daemon program has only one requirement: it must define a class called `Command` that
-  extends `django.core.management.base.BaseCommand`. Please see `kt_renewer.py` for an example.
-
-The next time Hue restarts, your `my_daemon` will start automatically.
-If your daemon program dies (exits with a non-zero exit code), Hue will
-restart it.
-
-"Under the covers:" Threading.  Hue, by default, runs CherryPy web server.
-If Hue is configured (and it may be, in the future)
-to use mod_wsgi under Apache httpd, then there would be multiple python
-processes serving the backend.  This means that your Django application
-code should avoid depending on shared process state.  Instead, place
-the stored state in a database or run a separate server.
-
 ### Saving documents
 
 Each app used to have its own model to store its data (e.g. a saving a SQL query, query history...). All the models have been unified into a single Document2 model in the desktop app:
@@ -349,27 +335,10 @@ The `Document2` model provides automatically creation, sharing and saving. It pe
 are Django's Object-Relational Mapping framework.
 
 
-### Managing users
-
-Except for static content, `request.user` is always populated.  It is a
-standard Django `models.User` object.  If you were to set a breakpoint at the
-`index()` function in our calculator app, you will find:
-
-    >>> request.user
-    <User: test>
-
-<div class="note">
-  "Under the covers:" Django uses a notion called
-  <a href="https://docs.djangoproject.com/en/1.2/topics/http/middleware/">middleware</a>
-  that's called in between the request coming in and the view being executed.
-  That's how <code>request.user</code> gets populated.  There's also a
-  middleware for Hue that makes sure that no pages are displayed unless the
-  user is authenticated.
-</div>
-
 ### Walk-through of a Django View
 
-![Django Flow](django_request.png)
+![Django Request]({{% param baseURL %}}images/django_request.png)
+
 
 Django is an MVC framework, except that the controller is called a
 "[view](https://docs.djangoproject.com/en/1.11/#the-view-layer)" and
@@ -435,10 +404,10 @@ Please download from http://thrift.apache.org/.
 The modules using ``Thrift`` have some helper scripts like ``regenerate_thrift.sh``
 for regenerating the code from the interfaces.
 
-### Profiling Hue Apps
+### Profiling
 
 Hue has a profiling system built in, which can be used to analyze server-side
-performance of applications.  To enable profiling::
+performance of applications.  To enable profiling:
 
     build/env/bin/hue runprofileserver
 
@@ -449,12 +418,12 @@ the file names is /tmp/<app_module>.<page_url>.<time_taken>.<timestamp>.prof.
 Hue uses the hotshot profiling library for instrumentation.  The documentation
 for this library is located at: http://docs.python.org/library/hotshot.html.
 
-You can use kcachegrind to view the profiled data graphically::
+You can use kcachegrind to view the profiled data graphically:
 
-    $ hotshot2calltree /tmp/xyz.prof > /tmp/xyz.trace
-    $ kcachegrind /tmp/xyz.trace
+    hotshot2calltree /tmp/xyz.prof > /tmp/xyz.trace
+    kcachegrind /tmp/xyz.trace
 
-More generally, you can programmatically inspect a trace::
+More generally, you can programmatically inspect a trace:
 
     #!/usr/bin/python
     import hotshot.stats
@@ -503,6 +472,11 @@ In a nutshell, front-end development in Hue is using
 [Knockout js](http://knockoutjs.com/) to layout your app and script the custom
 interactions.
 
+### Node.js
+
+During development the bundles can be autogenerated when it detects changes to the .js files, for this run:
+
+    npm run dev
 
 ### CSS Styles
 
@@ -514,7 +488,7 @@ app look at home in Hue.
 On top of the standard Bootstrap styles, Hue defines a small set of custom
 styles in *desktop/core/static/css/jhue.css*.
 
-### Defining Styles for Your Application
+### Defining Styles
 
 When you create your application it will provision a CSS file for you in the
 *static/css* directory. For organization purposes, your styles should go here
@@ -699,23 +673,3 @@ Because building Hadoop (for the tests that require it) is slow, we've
 separated the Jenkins builds into "fast" and "slow".  Both are run
 via scripts/jenkins.sh, which should be kept updated with the latest
 and greatest in build technologies.
-
-#### Debugging with PyCharm
-
-First step is to configure pycharm to use the Hue virtual environment at ./build/env/env
-![Pycharm virtualenv]({{% param baseURL %}}images/pycharm_virtualenv.png)
-Second step is to configure the debug configuration
-![Pycharm debug]({{% param baseURL %}}images/pycharm_debug.png)
-
-#### Debugging with Eclipse
-
-First step is to configure Eclipse to use the Hue virtual environment at ./build/env/env
-![Eclipse interpreter]({{% param baseURL %}}images/eclipse_interpreter.png)
-Second step is to configure the debug configuration
-![Eclipse debug]({{% param baseURL %}}images/eclipse_debug.png)
-![Eclipse debug arguments]({{% param baseURL %}}images/eclipse_debug_arguments.png)
-![Eclipse debug interpreter]({{% param baseURL %}}images/eclipse_debug_interpreter.png)
-
-
-
-

+ 3 - 1
docs/docs-site/layouts/partials/logo.html

@@ -1 +1,3 @@
-<img src="{{ .Site.Params.baseURL }}images/hue_logo.png" alt="alt text" title="Hue Logo">
+<a href="http://gethue.com">
+  <img src="{{ .Site.Params.baseURL }}images/hue_logo.png" alt="alt text" title="Hue Logo">
+</a>

BIN
docs/docs-site/static/images/django_request.png