README.rst 5.7 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216
  1. Welcome to the repository for Hue
  2. =================================
  3. .. note::
  4. This is the development-oriented readme. If you want to write notes for
  5. end users, please put them in ``dist/README``.
  6. Hue is both a Web UI for Hadoop and a framework to create interactive Web
  7. applications. It features:
  8. * FileBrowser for accessing HDFS
  9. * Job Designer for creating MapReduce/Streaming/Java jobs
  10. * Oozie App for submitting and scheduling workflows
  11. * JobBrowser for viewing MapReduce jobs
  12. * Beeswax application for executing Hive queries
  13. * A Pig/HBase shell
  14. On top of that, a SDK is available for creating new apps integrated with Hadoop.
  15. More documentation is available at http://cloudera.github.com/hue/.
  16. Getting Started
  17. ===============
  18. To build and get the core server running::
  19. $ git clone http://github.com/cloudera/hue.git
  20. $ cd hue
  21. $ make apps
  22. $ build/env/bin/hue runserver
  23. If using the Beeswax application, start the daemon::
  24. $ build/env/bin/hue beeswax_server
  25. Now Hue should be running on http://localhost:8000.
  26. The configuration in development mode is ``desktop/conf/pseudo-distributed.ini``.
  27. Note: to start all the servers in one command (but lose the automatic reloading after source modification)::
  28. $ build/env/bin/supervisor
  29. To run the tests::
  30. $ build/env/bin/hue test all
  31. $ build/env/bin/hue test specific filebrowser
  32. $ build/env/bin/hue test specific jobbrowser.tests:test_get_path
  33. Development Prerequisites
  34. ===========================
  35. You'll need these library development packages and tools installed on
  36. your system:
  37. Ubuntu:
  38. * ant
  39. * gcc
  40. * g++
  41. * libkrb5-dev
  42. * libmysqlclient-dev
  43. * libssl-dev
  44. * libsasl2-dev
  45. * libsasl2-modules-gssapi-mit
  46. * libsqlite3-dev
  47. * libtidy-0.99-0 (for unit tests only)
  48. * libxml2-dev
  49. * libxslt-dev
  50. * mvn (from ``maven2`` package or tarball)
  51. * openldap-dev / libldap2-dev
  52. * python-dev
  53. * python-simplejson
  54. * python-setuptools
  55. CentOS:
  56. * ant
  57. * asciidoc
  58. * cyrus-sasl-devel
  59. * cyrus-sasl-gssapi
  60. * gcc
  61. * gcc-c++
  62. * krb5-devel
  63. * libtidy (for unit tests only)
  64. * libxml2-devel
  65. * libxslt-devel
  66. * mvn (from ``maven2`` package or tarball)
  67. * mysql
  68. * mysql-devel
  69. * openldap-devel
  70. * python-devel
  71. * python-simplejson
  72. * sqlite-devel
  73. MacOS (mac port):
  74. * liblxml
  75. * libxml2
  76. * libxslt
  77. * mysql5-devel
  78. * simplejson (easy_install)
  79. * sqlite3
  80. File Layout
  81. ===========
  82. The Hue "framework" is in ``desktop``. ``/core/`` contains the Web components and
  83. ``desktop/libs/`` the API for talking to Hadoop.
  84. The installable apps live in ``apps/``. Please place third-party dependencies in the app's ext-py/
  85. directory.
  86. The typical directory structure for inside an application includes:
  87. src/
  88. for Python/Django code
  89. models.py
  90. urls.py
  91. views.py
  92. forms.py
  93. settings.py
  94. conf/
  95. for configuration (``.ini``) files to be installed
  96. static/
  97. for static HTML/js resources and help doc
  98. templates/
  99. for data to be put through a template engine
  100. locales/
  101. for localizations in multiple languages
  102. For the URLs within your application, you should make your own ``urls.py``
  103. which will be automatically rooted at ``/yourappname/`` in the global
  104. namespace. See ``apps/about/src/about/urls.py`` for an example.
  105. Main Stack
  106. ==========
  107. * Python 2.4 - 2.7
  108. * Django 1.2 https://docs.djangoproject.com/en/1.2/
  109. * Mako
  110. * jQuery
  111. * Bootstrap
  112. Using and Installing Thrift
  113. ===========================
  114. Right now, we check in the generated thrift code.
  115. To generate the code, you'll need the thrift binary version 0.7.0.
  116. Please download from http://thrift.apache.org/.
  117. The modules using ``Thrift`` have some helper scripts like ``regenerate_thrift.sh``
  118. for regenerating the code from the interfaces.
  119. Profiling Hue Apps
  120. ==================
  121. Hue has a profiling system built in, which can be used to analyze server-side
  122. performance of applications. To enable profiling::
  123. $ build/env/bin/hue runprofileserver
  124. Then, access the page that you want to profile. This will create files like
  125. /tmp/useradmin.users.000072ms.2011-02-21T13:03:39.745851.prof. The format for
  126. the file names is /tmp/<app_module>.<page_url>.<time_taken>.<timestamp>.prof.
  127. Hue uses the hotshot profiling library for instrumentation. The documentation
  128. for this library is located at: http://docs.python.org/library/hotshot.html.
  129. You can use kcachegrind to view the profiled data graphically::
  130. $ hotshot2calltree /tmp/xyz.prof > /tmp/xyz.trace
  131. $ kcachegrind /tmp/xyz.trace
  132. More generally, you can programmatically inspect a trace::
  133. #!/usr/bin/python
  134. import hotshot.stats
  135. import sys
  136. stats = hotshot.stats.load(sys.argv[1])
  137. stats.sort_stats('cumulative', 'calls')
  138. stats.print_stats(100)
  139. This script takes in a .prof file, and orders function calls by the cumulative
  140. time spent in that function, followed by the number of times the function was
  141. called, and then prints out the top 100 time-wasters. For information on the
  142. other stats available, take a look at this website:
  143. http://docs.python.org/library/profile.html#pstats.Stats
  144. Internationalization
  145. ====================
  146. How to update all the messages and compile them::
  147. $ make locales
  148. How to update and compile the messages of one app::
  149. $ cd apps/beeswax
  150. $ make compile-locale
  151. How to create a new locale for an app::
  152. $ cd $APP_ROOT/src/$APP_NAME/locale
  153. $ $HUE_ROOT/build/env/bin/pybabel init -D django -i en_US.pot -d . -l fr
  154. License
  155. =======
  156. Apache License, Version 2.0
  157. http://www.apache.org/licenses/LICENSE-2.0