reference.md 2.4 KB


title: "Reference Architecture" date: 2019-03-13T18:28:09-07:00 draft: false

weight: 1

A recommended setup consists in:

Load Balancers

Hue is often run with:

  • Cherrypy with NGINX (recommended)
  • Cherrypy with HTTPD (built-in when using Cloudera Manager)
  • Gunicorn is coming soon with HUE-8739
  • Apache mod Python

Task Server

** Not fully supported yet**

The task server is currently a work in progress to outsource all the blocking or resource intensive operations outside of the API server. Follow HUE-8738 for more information on when first usable task will be released.

Until then, here is how to try the task server service.

Make sure you have Rabbit MQ installed and running.

sudo apt-get install rabbitmq-server -y

In hue.ini, telling the API server that the Task Server is available:

[desktop]
[[task_server]]
enabled=true

Starting the Task server:

./build/env/bin/celery worker -l info -A desktop

Running a test tasks:

./build/env/bin/hue shell

from desktop.celery import debug_task

debug_task.delay()
debug_task.delay().get() # Works if result backend is setup and task_server is true in the hue.ini

Task Scheduler server

For schedules configured statically in Python:

./build/env/bin/celery -A desktop beat -l info

For schedules configured dynamically via a table with Django Celery Beat:

[desktop]
[[task_server]]
beat_enabled=false

Then:

./build/env/bin/celery -A desktop beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler

Note: the first time the tables need to be created with:

./build/env/bin/hue migrate

Monitoring

Performing a GET /desktop/debug/is_alive will return a 200 response if running.

Proxy

A Web proxy lets you centralize all the access to a certain URL and prettify the address (e.g. ec2-54-247-321-151.compute-1.amazonaws.com --> demo.gethue.com).

Here is one way to do it with NGINX or Apache.