|
|
@@ -93,10 +93,8 @@ ones that you may write yourself.
|
|
|
Hue is a web application built on the Django python web framework.
|
|
|
Django, running on the WSGI container/web server (typically CherryPy), manages
|
|
|
the url dispatch, executes application logic code, and puts together the views
|
|
|
-from their templates. Django uses a database (typically sqlite)
|
|
|
-to manage session data, and Hue applications can use it as well
|
|
|
-for their "models". (For example, the saved Editor stores
|
|
|
-saved queries in the database.)
|
|
|
+from their templates. Django uses a database (typically MySql or PostGres) to manage session data, and Hue applications can use it as well
|
|
|
+for their "models". (For example, the saved Editor stores saved queries in the database.)
|
|
|
|
|
|
In addition to the web server, some Hue applications run
|
|
|
daemon processes "on the side". Some examples are the `Celery Task Server`, `Celery Beat`.
|
|
|
@@ -114,20 +112,12 @@ These APIs work by making REST or Thrift calls.
|
|
|
|
|
|

|
|
|
|
|
|
-A Hue application may span three tiers: (1) the UI
|
|
|
-and user interaction in the client's browser, (2) the
|
|
|
-core application logic in the Hue web
|
|
|
-server, and (3) external services with which applications
|
|
|
-may interact.
|
|
|
+A Hue application may span three tiers: (1) the UI and user interaction in the client's browser, (2) the
|
|
|
+core application logic in the Hue web server, and (3) external services with which applications may interact.
|
|
|
|
|
|
-The absolute minimum that you must implement (besides
|
|
|
-boilerplate), is a
|
|
|
-"Django [view](https://docs.djangoproject.com/en/1.11/#the-view-layer/)"
|
|
|
-function that processes the request and the associated template
|
|
|
-to render the response into HTML.
|
|
|
+The absolute minimum that you must implement (besides boilerplate), is a "Django [view](https://docs.djangoproject.com/en/1.11/#the-view-layer/)" function that processes the request and the associated template to render the response into HTML.
|
|
|
|
|
|
-Many apps will evolve to have a bit of custom JavaScript and
|
|
|
-CSS styles. Apps that need to talk to an external service
|
|
|
+Many apps will evolve to have a bit of custom JavaScript and CSS styles. Apps that need to talk to an external service
|
|
|
will pull in the code necessary to talk to that service.
|
|
|
|
|
|
### File Layout
|
|
|
@@ -299,14 +289,11 @@ would say finicky); Django templates are simpler, but are less expressive.
|
|
|
|
|
|
### Authentication Backends
|
|
|
|
|
|
-Hue exposes a configuration flag ("auth") to configure
|
|
|
-a custom authentication backend.
|
|
|
-See http://docs.djangoproject.com/en/dev/topics/auth/#writing-an-authentication-backend
|
|
|
-for writing such a backend.
|
|
|
+Hue exposes a configuration flag ("auth") to configure a custom authentication [backend](https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/auth/backend.py).
|
|
|
+See [writing an authentication backend](http://docs.djangoproject.com/en/dev/topics/auth/#writing-an-authentication-backend)
|
|
|
+for more details.
|
|
|
|
|
|
-In addition to that, backends may support a `manages_passwords_externally()` method, returning
|
|
|
-True or False, to tell the user manager application whether or not changing
|
|
|
-passwords within Hue is possible.
|
|
|
+In addition to that, backends may support a `manages_passwords_externally()` method, returning True or False, to tell the user manager application whether or not changing passwords within Hue is possible.
|
|
|
|
|
|
### Authorization
|
|
|
|
|
|
@@ -382,7 +369,7 @@ After upgrading the version of Hue, running these two commands will make sure th
|
|
|
TEST_IMPALAD_HOST=impalad-01.gethue.com
|
|
|
Point to an Impalad and trigger the Impala tests.
|
|
|
|
|
|
-## User interface
|
|
|
+## User Interface (UI)
|
|
|
|
|
|
Developing applications for Hue requires a minimal amount of CSS
|
|
|
(and potentially JavaScript) to use existing functionality.
|
|
|
@@ -472,7 +459,7 @@ development mode.
|
|
|
|
|
|
### The short story
|
|
|
|
|
|
-Run the unit tests
|
|
|
+Run the API unit tests
|
|
|
|
|
|
./build/env/bin/hue test unit
|
|
|
|
|
|
@@ -484,26 +471,16 @@ How to run just some parts of the tests, e.g.:
|
|
|
build/env/bin/hue test specific impala.tests:TestMockedImpala
|
|
|
build/env/bin/hue test specific impala.tests:TestMockedImpala.test_basic_flow
|
|
|
|
|
|
-Jest tests:
|
|
|
+Run the user interface tests:
|
|
|
|
|
|
npm run test
|
|
|
|
|
|
+### Running the API tests
|
|
|
|
|
|
-### Longer story
|
|
|
+The ``test`` management command prepares the arguments (test app names) and passes them to nose (django_nose.nose_runner). Nose will then magically find all the tests to run.
|
|
|
|
|
|
-The ``test`` management command prepares the arguments (test app names)
|
|
|
-and passes them to nose (django_nose.nose_runner). Nose will then magically
|
|
|
-find all the tests to run.
|
|
|
-
|
|
|
-Tests themselves should be named *_test.py. These will be found
|
|
|
-as long as they're in packages covered by django. You can use the
|
|
|
-unittest frameworks, or you can just name your method with
|
|
|
-the word "test" at a word boundary, and nose will find it.
|
|
|
-
|
|
|
-See apps/hello/src/hello/hello_test.py for an example.
|
|
|
-
|
|
|
-
|
|
|
-#### Running only specific tests
|
|
|
+Tests themselves should be named `*_test.py`. These will be found as long as they're in packages covered by django. You can use the
|
|
|
+unittest frameworks, or you can just name your method with the word "test" at a word boundary, and nose will find it. See `apps/hello/src/hello/hello_test.py` for an example.
|
|
|
|
|
|
To run the unit tests (should take 5-10 minutes):
|
|
|
|
|
|
@@ -543,19 +520,9 @@ Start up pdb on test failures:
|
|
|
|
|
|
build/env/bin/hue test <args> --pdb --pdb-failure -s
|
|
|
|
|
|
-#### Coverage
|
|
|
-
|
|
|
-Add the following options:
|
|
|
-
|
|
|
- ./build/env/bin/hue test unit --with-xunit --with-cover
|
|
|
-
|
|
|
-For js run:
|
|
|
-
|
|
|
- npm run test-coverage
|
|
|
-
|
|
|
-#### Create and run the Jest tests
|
|
|
+### Running the UI tests
|
|
|
|
|
|
-Add them next to the file under test, the filename of the test has to end with ".test.js".
|
|
|
+The tests are next to the file under test, the filename of the test has to end with `.test.js`.
|
|
|
|
|
|
someFile.js <- File under test
|
|
|
someFile.test.js <- File containing tests
|
|
|
@@ -575,69 +542,84 @@ To run the tests in watch mode:
|
|
|
While in watch mode Jest will detect changes to all files and re-run related tests. There are
|
|
|
also options to target specific files or tests. Press 'w' in the console to see the options.
|
|
|
|
|
|
-#### Testing KO components and bindings
|
|
|
+Note: on certain OS like Ubuntu, running the tests via a global jest seems to not hang your system
|
|
|
+
|
|
|
+ npm install jest --global
|
|
|
+
|
|
|
+How to update snapshot tests:
|
|
|
+
|
|
|
+ jest --updateSnapshot
|
|
|
|
|
|
-koSetup provides utilities to test knockout components and bindings using jsdom from jest.
|
|
|
+#### Testing KO Js components
|
|
|
+
|
|
|
+koSetup provides utilities to test Knockout components and bindings using `jsdom` from `jest`.
|
|
|
|
|
|
An example of component test:
|
|
|
|
|
|
-```
|
|
|
-import { koSetup } from 'jest/koTestUtils';
|
|
|
+ import { koSetup } from 'jest/koTestUtils';
|
|
|
|
|
|
-import 'ko/someComponent';
|
|
|
+ import 'ko/someComponent';
|
|
|
|
|
|
-describe('ko.someComponent.js', () => {
|
|
|
- const setup = koSetup(); // Adds the necessary setup and teardown
|
|
|
+ describe('ko.someComponent.js', () => {
|
|
|
+ const setup = koSetup(); // Adds the necessary setup and teardown
|
|
|
|
|
|
- it('should render component', async () => {
|
|
|
- const someParams = {}
|
|
|
+ it('should render component', async () => {
|
|
|
+ const someParams = {}
|
|
|
|
|
|
- const element = await setup.renderComponent('someComponent', someParams);
|
|
|
+ const element = await setup.renderComponent('someComponent', someParams);
|
|
|
|
|
|
- expect(element.innerHTML).toMatchSnapshot();
|
|
|
- });
|
|
|
+ expect(element.innerHTML).toMatchSnapshot();
|
|
|
+ });
|
|
|
|
|
|
- it('should change after observable update', async () => {
|
|
|
- const someParams = { visible: ko.observable(false) };
|
|
|
- const wrapper = await setup.renderComponent('someComponent', someParams);
|
|
|
- expect(wrapper.querySelector('[data-test="some-test-id"]').style['display']).toEqual('none');
|
|
|
+ it('should change after observable update', async () => {
|
|
|
+ const someParams = { visible: ko.observable(false) };
|
|
|
+ const wrapper = await setup.renderComponent('someComponent', someParams);
|
|
|
+ expect(wrapper.querySelector('[data-test="some-test-id"]').style['display']).toEqual('none');
|
|
|
|
|
|
- someParams.visible(true); // Or trigger some event on an elmement etc.
|
|
|
- await setup.waitForKoUpdate(); // Waits for re-render
|
|
|
+ someParams.visible(true); // Or trigger some event on an elmement etc.
|
|
|
+ await setup.waitForKoUpdate(); // Waits for re-render
|
|
|
|
|
|
- expect(wrapper.querySelector('[data-test="some-test-id"]').style['display']).toEqual('inline-block');
|
|
|
- });
|
|
|
-});
|
|
|
-```
|
|
|
+ expect(wrapper.querySelector('[data-test="some-test-id"]').style['display']).toEqual('inline-block');
|
|
|
+ });
|
|
|
+ });
|
|
|
|
|
|
An example of a binding test:
|
|
|
|
|
|
-```
|
|
|
-import ko from 'knockout';
|
|
|
-import { koSetup } from 'jest/koTestUtils';
|
|
|
-import './ko.myBinding';
|
|
|
+ import ko from 'knockout';
|
|
|
+ import { koSetup } from 'jest/koTestUtils';
|
|
|
+ import './ko.myBinding';
|
|
|
|
|
|
-describe('ko.myBinding.js', () => {
|
|
|
- const setup = koSetup();
|
|
|
+ describe('ko.myBinding.js', () => {
|
|
|
+ const setup = koSetup();
|
|
|
|
|
|
- it('should toggle observable', async () => {
|
|
|
- const viewModel = { testObservable: ko.observable(false) };
|
|
|
- const wrapper = await setup.renderKo(
|
|
|
- '<div class="click-test" data-bind="myBinding: testObservable"></div>',
|
|
|
- viewModel
|
|
|
- );
|
|
|
+ it('should toggle observable', async () => {
|
|
|
+ const viewModel = { testObservable: ko.observable(false) };
|
|
|
+ const wrapper = await setup.renderKo(
|
|
|
+ '<div class="click-test" data-bind="myBinding: testObservable"></div>',
|
|
|
+ viewModel
|
|
|
+ );
|
|
|
|
|
|
- expect(viewModel.testObservable()).toBeFalsy();
|
|
|
+ expect(viewModel.testObservable()).toBeFalsy();
|
|
|
|
|
|
- wrapper.querySelector('.click-test').click();
|
|
|
- await setup.waitForKoUpdate();
|
|
|
+ wrapper.querySelector('.click-test').click();
|
|
|
+ await setup.waitForKoUpdate();
|
|
|
|
|
|
- expect(viewModel.testObservable()).toBeTruthy();
|
|
|
- });
|
|
|
-});
|
|
|
-```
|
|
|
+ expect(viewModel.testObservable()).toBeTruthy();
|
|
|
+ });
|
|
|
+ });
|
|
|
+
|
|
|
+
|
|
|
+### Coverage
|
|
|
+
|
|
|
+Add the following options:
|
|
|
+
|
|
|
+ ./build/env/bin/hue test unit --with-xunit --with-cover
|
|
|
+
|
|
|
+For js run:
|
|
|
+
|
|
|
+ npm run test-coverage
|
|
|
|
|
|
-#### Continuous Integration (CI)
|
|
|
+### Continuous Integration (CI)
|
|
|
|
|
|
[CircleCi](https://circleci.com/gh/cloudera/hue) automatically run the unit tests (Python, Javascript, linting) on branch updates and pull requests. Branches containing `ci-commit-master` will try to be auto pushed to master if the run is green and the Github permissions match.
|
|
|
This logic is described in the [config.yml](https://github.com/cloudera/hue/tree/master/.circleci).
|
|
|
@@ -646,7 +628,7 @@ The runs happen in an image based on [latest Hue's image](https://hub.docker.com
|
|
|
|
|
|
Note: until the `desktop/ext-py` dependencies are moved to a `requirement.txt`, adding new Python modules will require adding them first to the Docker image by building a Hue branch which has them.
|
|
|
|
|
|
-#### Integration tests
|
|
|
+### Integration tests
|
|
|
|
|
|
To not fail on integration tests, Hue would need to be configured to point to a live cluster, or you could install the mini cluster (only once) with:
|
|
|
|