README.rst 5.6 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215
  1. Welcome to the repository for Hue
  2. =================================
  3. .. note::
  4. This is the development-oriented readme. If you want to write notes for
  5. end users, please put them in ``dist/README``.
  6. Hue is both a Web UI for Hadoop and a framework to create interactive Web
  7. applications. It features:
  8. * FileBrowser for accessing HDFS
  9. * Job Designer and Oozie for submitting and scheduling workflows of MapReduce/Java/Streaming/Pig jobs
  10. * JobBrowser for viewing MapReduce jobs
  11. * Beeswax application for executing Hive queries
  12. * A Pig/HBase shell
  13. On top of that, a SDK is available for creating new apps integrated with Hadoop.
  14. More documentation is available at http://cloudera.github.com/hue/.
  15. Getting Started
  16. ===============
  17. To build and get the core server running::
  18. $ git clone http://github.com/cloudera/hue.git
  19. $ cd hue
  20. $ make apps
  21. $ build/env/bin/hue runserver
  22. If using the Beeswax application, start the daemon::
  23. $ build/env/bin/hue beeswax_server
  24. Now Hue should be running on http://localhost:8000.
  25. The configuration in development mode is ``desktop/conf/pseudo-distributed.ini``.
  26. Note: to start all the servers in one command (but lose the automatic reloading after source modification)::
  27. $ build/env/bin/supervisor
  28. To run the tests::
  29. $ build/env/bin/hue test all
  30. $ build/env/bin/hue test specific filebrowser
  31. $ build/env/bin/hue test specific jobbrowser.tests:test_get_path
  32. Development Prerequisites
  33. ===========================
  34. You'll need these library development packages and tools installed on
  35. your system:
  36. Ubuntu:
  37. * ant
  38. * gcc
  39. * g++
  40. * libkrb5-dev
  41. * libmysqlclient-dev
  42. * libssl-dev
  43. * libsasl2-dev
  44. * libsasl2-modules-gssapi-mit
  45. * libsqlite3-dev
  46. * libtidy-0.99-0 (for unit tests only)
  47. * libxml2-dev
  48. * libxslt-dev
  49. * mvn (from ``maven2`` package or tarball)
  50. * openldap-dev / libldap2-dev
  51. * python-dev
  52. * python-simplejson
  53. * python-setuptools
  54. CentOS:
  55. * ant
  56. * asciidoc
  57. * cyrus-sasl-devel
  58. * cyrus-sasl-gssapi
  59. * gcc
  60. * gcc-c++
  61. * krb5-devel
  62. * libtidy (for unit tests only)
  63. * libxml2-devel
  64. * libxslt-devel
  65. * mvn (from ``maven2`` package or tarball)
  66. * mysql
  67. * mysql-devel
  68. * openldap-devel
  69. * python-devel
  70. * python-simplejson
  71. * sqlite-devel
  72. MacOS (mac port):
  73. * liblxml
  74. * libxml2
  75. * libxslt
  76. * mysql5-devel
  77. * simplejson (easy_install)
  78. * sqlite3
  79. File Layout
  80. ===========
  81. The Hue "framework" is in ``desktop``. ``/core/`` contains the Web components and
  82. ``desktop/libs/`` the API for talking to Hadoop.
  83. The installable apps live in ``apps/``. Please place third-party dependencies in the app's ext-py/
  84. directory.
  85. The typical directory structure for inside an application includes:
  86. src/
  87. for Python/Django code
  88. models.py
  89. urls.py
  90. views.py
  91. forms.py
  92. settings.py
  93. conf/
  94. for configuration (``.ini``) files to be installed
  95. static/
  96. for static HTML/js resources and help doc
  97. templates/
  98. for data to be put through a template engine
  99. locales/
  100. for localizations in multiple languages
  101. For the URLs within your application, you should make your own ``urls.py``
  102. which will be automatically rooted at ``/yourappname/`` in the global
  103. namespace. See ``apps/about/src/about/urls.py`` for an example.
  104. Main Stack
  105. ==========
  106. * Python 2.4 - 2.7
  107. * Django 1.2 https://docs.djangoproject.com/en/1.2/
  108. * Mako
  109. * jQuery
  110. * Bootstrap
  111. Using and Installing Thrift
  112. ===========================
  113. Right now, we check in the generated thrift code.
  114. To generate the code, you'll need the thrift binary version 0.7.0.
  115. Please download from http://thrift.apache.org/.
  116. The modules using ``Thrift`` have some helper scripts like ``regenerate_thrift.sh``
  117. for regenerating the code from the interfaces.
  118. Profiling Hue Apps
  119. ==================
  120. Hue has a profiling system built in, which can be used to analyze server-side
  121. performance of applications. To enable profiling::
  122. $ build/env/bin/hue runprofileserver
  123. Then, access the page that you want to profile. This will create files like
  124. /tmp/useradmin.users.000072ms.2011-02-21T13:03:39.745851.prof. The format for
  125. the file names is /tmp/<app_module>.<page_url>.<time_taken>.<timestamp>.prof.
  126. Hue uses the hotshot profiling library for instrumentation. The documentation
  127. for this library is located at: http://docs.python.org/library/hotshot.html.
  128. You can use kcachegrind to view the profiled data graphically::
  129. $ hotshot2calltree /tmp/xyz.prof > /tmp/xyz.trace
  130. $ kcachegrind /tmp/xyz.trace
  131. More generally, you can programmatically inspect a trace::
  132. #!/usr/bin/python
  133. import hotshot.stats
  134. import sys
  135. stats = hotshot.stats.load(sys.argv[1])
  136. stats.sort_stats('cumulative', 'calls')
  137. stats.print_stats(100)
  138. This script takes in a .prof file, and orders function calls by the cumulative
  139. time spent in that function, followed by the number of times the function was
  140. called, and then prints out the top 100 time-wasters. For information on the
  141. other stats available, take a look at this website:
  142. http://docs.python.org/library/profile.html#pstats.Stats
  143. Internationalization
  144. ====================
  145. How to update all the messages and compile them::
  146. $ make locales
  147. How to update and compile the messages of one app::
  148. $ cd apps/beeswax
  149. $ make compile-locale
  150. How to create a new locale for an app::
  151. $ cd $APP_ROOT/src/$APP_NAME/locale
  152. $ $HUE_ROOT/build/env/bin/pybabel init -D django -i en_US.pot -d . -l fr
  153. License
  154. =======
  155. Apache License, Version 2.0
  156. http://www.apache.org/licenses/LICENSE-2.0