README.rst 5.1 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180
  1. Welcome to the repository for Hue
  2. =================================
  3. .. note::
  4. This is the development-oriented readme. If you want to write notes for
  5. end users, please put them in ``dist/README``.
  6. Hue is both a web UI for Hadoop and a framework to create interactive web
  7. applications. It features a FileBrowser for accessing HDFS, JobSub and
  8. JobBrowser applications for submitting and viewing MapReduce jobs, a Beeswax
  9. application for interacting with Hive. On top of that, the web frontend
  10. is mostly built from declarative widgets that require no JavaScript and are
  11. easy to learn.
  12. File Layout
  13. ===========
  14. The "core" stuff is in ``desktop/core/``, whereas installable apps live in
  15. ``apps/``. Please place third-party dependencies in the app's ext-py/
  16. directory.
  17. The typical directory structure for inside an application includes:
  18. src/
  19. for Python code
  20. conf/
  21. for configuration (``.ini``) files to be installed
  22. static/
  23. for static HTML and js resources
  24. templates/
  25. for data to be put through a template engine
  26. docs/
  27. for helpful notes
  28. The python code is structured simply as
  29. ``module/package.py``,
  30. where module may be "filebrowser" or "jobsub". Because it is unlikely that
  31. there are going to be huge conflicts, we're going without a deep nested
  32. hierarchy.
  33. URL Layout
  34. ==========
  35. ``core/src/desktop/urls.py`` contains the current layout for top-level URLs.
  36. For the URLs within your application, you should make your own ``urls.py``
  37. which will be automatically rooted at ``/yourappname/`` in the global
  38. namespace. See ``apps/hello/src/hello/urls.py`` for an example.
  39. Development Prerequisites
  40. ===========================
  41. 1. On your host system, you need to have the python "virtualenv" package
  42. installed.
  43. 2. Also, you'll need these library development packages installed on your
  44. system:
  45. Debian:
  46. * gcc
  47. * libmysqlclient-dev
  48. * libsasl2-dev
  49. * libsqlite3-dev
  50. * libxml2-dev
  51. * libxslt-dev
  52. * python-dev
  53. * python-setuptools
  54. CentOS:
  55. * cyrus-sasl-devel
  56. * gcc
  57. * libxml2-devel
  58. * libxslt-devel
  59. * mysql
  60. * mysql-devel
  61. * python-devel
  62. * python-setuptools
  63. * python-simplejson (for the crepo tool)
  64. * sqlite-devel
  65. MacOS (mac port):
  66. * liblxml
  67. * libxml2
  68. * libxslt
  69. * mysql5-devel
  70. * simplejson (easy_install)
  71. * sqlite3
  72. 3. You need to have crepo installed, and preferably on your path. If it is not
  73. on your path, set the environment variable ``CREPO`` to point to ``crepo.py``
  74. from that distribution. You can clone crepo from
  75. http://github.com/cloudera/crepo.git somewhere else on your system.
  76. Getting Started
  77. ===============
  78. To build and get the core server running::
  79. $ export HADOOP_HOME=<path-to-hadoop-home>
  80. $ git clone http://github.com/cloudera/hue.git
  81. $ cd hue
  82. $ make apps
  83. $ build/env/bin/hue runserver_plus
  84. To start the helper daemons::
  85. $ build/env/bin/hue beeswax_server
  86. $ build/env/bin/hue jobsubd
  87. Now Hue should be running on http://localhost:8000.
  88. Setting up Hadoop
  89. =================
  90. In order to start up a pseudodistributed cluster with the plugins enabled,
  91. run::
  92. $ ./tools/scripts/configure-hadoop.sh all
  93. After doing so, running ``jps`` should show all the daemons running (NN, JT,
  94. TT, DN) and you should be able to see the web UI on http://localhost:50030/ and
  95. http://localhost:50070/.
  96. FAQ
  97. ===
  98. 1: What does "Exception: no app!" mean?
  99. Your template has an error in it. Check for messages from the server that
  100. look like::
  101. INFO:root:Processing exception: Unclosed tag 'if'. Looking for one of: else, endif
  102. 2: What do I do if I get "There was an error launching ..."?
  103. Turn on debugging by issuing ``dbug.cookie()`` in a Firebug console.
  104. Django Conventions
  105. ==================
  106. If you need to name your urls
  107. (http://docs.djangoproject.com/en/dev/topics/http/urls/#naming-url-patterns)
  108. because there's ambiguity in the view, be sure to prefix the name
  109. with the application name. The url name namespace is global. So
  110. ``jobsub.list`` is fine, but ``list`` is not.
  111. Hue is using Django 1.1, which supports the notion of URL namespaces:
  112. http://docs.djangoproject.com/en/dev/topics/http/urls/#url-namespaces.
  113. We have yet to move over our URLs to this construct. Brownie points for the
  114. developer who takes this on.
  115. Using and Installing Thrift
  116. ===========================
  117. Right now, we check in the generated thrift code.
  118. To generate the code, you'll need the thrift binary.
  119. Compile it like so::
  120. $ git clone http://github.com/dreiss/thrift.git
  121. $ cd thrift
  122. $ ./bootstrap.sh
  123. $ ./configure --with-py=no --with-java=no --with-perl=no --prefix=$HOME/pub
  124. We exclude python, java, and perl because they don't like
  125. to install in prefix. If you look around at configure's --help,
  126. there are environment variables that determine where those
  127. runtime bindings are installed.
  128. ::
  129. $ make && make install
  130. When preparing ``.thrift`` files, you can use she-bangs to generate
  131. the python bindings like so::
  132. #!/usr/bin/env thrift -r --gen py:new_style -o ../../../
  133. .. note::
  134. This file is in reStructuredText. You may run
  135. ``rst2html README.rst > README.html`` to produce a HTML.