Explorar o código

[docs] Refactor connector API

Romain Rigaux %!s(int64=4) %!d(string=hai) anos
pai
achega
9ffb06051d

+ 0 - 40
docs/docs-site/content/developer/components/er-diagram/_index.md

@@ -1,40 +0,0 @@
----
-title: "ER-diagram"
-draft: false
----
-
-The [Entity-relationship model](https://en.wikipedia.org/wiki/Entity%E2%80%93relationship_model) (ERD) provides an illustration of various SQL entities, and the relationship between them. Entity types supported by the ERD diagram are currently `Table` & `Literal`. It has a very generic architecture, and more types of entities can be supported in the future.
-
-{{< webcomp name="er-diagram"
-    src="/js/gethue/components/er-diagram.js"
-    attrSrc="demo/attrs.json"
-    events="entity-clicked:name"
-/>}}
-
-## Import
-
-Please refer [here](/developer/components/#using-ui-components-in-your-project) for importing the component in your own project. Also [er-diagram-demo](https://github.com/cloudera/hue/tree/master/tools/examples/components/er-diagram-demo) app have working examples.
-
-Once imported `er-diagram` can be used like a native HTML tag.
-
-    <er-diagram id="erd-element-id"/>
-
-## Attributes
-
-- entities: Array &lt;[IEntity](https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/js/components/er-diagram/lib/interfaces.ts#L21)&gt; - An array of entity objects. Each entity will be a box in the UI.
-- relations: Array &lt;[IRelation](https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/js/components/er-diagram/lib/interfaces.ts#L26)&gt; - An array of relation objects. Each relation will connect two of the above entities.
-
-Please refer the [interfaces](https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/js/components/er-diagram/lib/interfaces.ts) for an idea on the structure of these objects.
-
-## Events
-
-- entity-clicked: Function([IEntity](https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/js/components/er-diagram/lib/interfaces.ts#L21)) - Will be triggered when an entity is clicked.
-
-## Resource Files
-
-- Hue
-  - **Web Component:** gethue/web/er-diagram.js
-  - **Vue Component:** gethue/components/er-diagram/index.vue
-- GetHue
-  - **Web Component:** desktop/core/src/desktop/js/components/er-diagram/webcomp.ts
-  - **Vue Component:** desktop/core/src/desktop/js/components/er-diagram/index.vue

+ 0 - 113
docs/docs-site/content/developer/connectors/_index.md

@@ -1,113 +0,0 @@
----
-title: "Connectors"
-date: 2019-03-13T18:28:09-07:00
-draft: false
-weight: 5
----
-
-Connectors provide pluggable integration to any external data service so that an admin can easily allow end users to interact with them.
-
-* List of all the [existing connectors](/administrator/configuration/connectors/)
-* Check "Potential connectors" ideas in each section
-* Feel free to contact the [community](https://discourse.gethue.com/c/developer-sdk-api)
-
-## Databases
-
-### SqlAlchemy
-
-[SqlAlchemy](https://www.sqlalchemy.org) is the prefered way if the Hive API is not supported by the database. The core implementation is in [`sql_alchemy.py`](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py) and relies on each respective SqlAlchemy dialect.
-
-### Hive Interface
-
-This [asynchronous API](https://github.com/cloudera/hue/tree/master/apps/beeswax) based on the Thrift API of Hive is very mature and powers an excellent integration of Apache Hive and Apache Impala.
-
-### Custom
-
-If the built-in HiveServer2 (Hive, Impala, Spark SQL), SqlAlchemy (MySQL, PostgreSQL, Oracle, Presto...) don’t meet your needs, you can implement your own connector to the notebook app:
-
-* List of [all connectors](https://github.com/cloudera/hue/tree/master/desktop/libs/notebook/src/notebook/connectors)
-* Each connector API subclasses the [Base API](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/base.py) and must implement the methods defined within. Refer to the [JDBC](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/jdbc.py) or [RdbmsApi](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/rdbms.py) for representative examples
-
-* [Kafka SQL](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/ksql.py)
-* [Solr SQL](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/solr.py)
-* [JDBC](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/jdbc.py)
-
-The JDBC API relies on a small JDBC proxy running next to the Hue API. By default it won't be built without setting the `BUILD_DB_PROXY` flag, e.g.:
-
-    export BUILD_DB_PROXY=true make install
-
-**Note** In the long term, SqlAlchemy is prefered as more "Python native".
-
-### Potential connectors
-
-It is recommended to develop an SqlAlchemy connector if yours is not already [existing](/administrator/configuration/connectors/#databases).
-
-## Catalogs
-
-The backends is pluggable by providing alternative [client interfaces](https://github.com/cloudera/hue/tree/master/desktop/libs/metadata/src/metadata/catalog):
-
-* [Apache Atlas](https://atlas.apache.org/)
-* Cloudera Navigator
-* Dummy (skeleton for integrating new catalogs)
-
-### Apache Atlas
-
-* [Client API](https://github.com/cloudera/hue/tree/master/desktop/libs/metadata/src/metadata/catalog/atlas_client.py)
-
-### Potential connectors
-
-* [Linkedin DataHub](https://github.com/linkedin/datahub)
-* [Lift Amundsen](https://github.com/lyft/amundsen)
-* AWS Glue
-* Google Cloud Data Catalog
-* Alation
-
-## Storages
-
-Various storage systems can be interacted with. The [`fsmanager.py`](https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/lib/fsmanager.py) is the main router to each API.
-
-**Note** Apache Ozone as well as Ceph can be used via the S3 browser.
-
-### Hadoop HDFS
-
-* [WebHdfs API](https://github.com/cloudera/hue/blob/master/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py)
-
-### AWS S3
-
-* [S3 API](https://github.com/cloudera/hue/blob/master/desktop/libs/aws/src/aws/s3)
-
-### Azure ADLS
-
-* [ADLS v2](https://github.com/cloudera/hue/blob/master/desktop/libs/azure/src/azure/abfs)
-* [ADLS v1](https://github.com/cloudera/hue/blob/master/desktop/libs/azure/src/azure/adls)
-
-### HBase / Key Value Stores
-
-With just a few changes in the [Python API](https://github.com/cloudera/hue/blob/master/apps/hbase/src/hbase/api.py), the HBase browser could be compatible with Apache Kudu or Google Big Table.
-
-### Potential connectors
-
-* Google Cloud Storage is currently a work in progress with [HUE-8978](https://issues.cloudera.org/browse/HUE-8978)
-
-## Jobs
-
-### Apache Spark / Livy
-
-Based on the [Livy REST API](/administrator/configuration/connectors/#apache-spark).
-
-* [Notebook connector](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/spark_shell.py)
-  * PySpark
-  * Scala
-  * Spark SQL
-* [Batch connector](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/spark_batch.py)
-
-### Schedulers
-
-Currently only Apache Oozie is supported for your Datawarehouse, but the API is getting generic with [HUE-3797](https://issues.cloudera.org/browse/HUE-3797).
-
-### Potential connectors
-
-* Elastic Search: a connector similar to Solr for Searching [HUE-7828](https://issues.cloudera.org/browse/HUE-7828). SQL querying is already supported.
-* [Livy Browser API](https://github.com/cloudera/hue/blob/master/apps/jobbrowser/src/jobbrowser/apis/livy_api.py)
-* [Celery API](https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/lib/scheduler/lib/beat.py)
-* Apache Hive native support of query scheduling [HIVE-21884](https://issues.apache.org/jira/browse/HIVE-21884)

+ 109 - 0
docs/docs-site/content/developer/development/_index.md

@@ -1049,3 +1049,112 @@ How to create a new locale for an app:
 
     cd $APP_ROOT/src/$APP_NAME/locale
     $HUE_ROOT/build/env/bin/pybabel init -D django -i en_US.pot -d . -l fr
+
+## Connectors
+
+Connectors provide pluggable integration to any external data service so that an admin can easily allow end users to interact with them.
+
+* List of all the [existing connectors](/administrator/configuration/connectors/)
+* Check "Potential connectors" ideas in each section
+* Feel free to contact the [community](https://discourse.gethue.com/c/developer-sdk-api)
+
+### Databases
+
+#### SqlAlchemy
+
+[SqlAlchemy](https://www.sqlalchemy.org) is the prefered way if the Hive API is not supported by the database. The core implementation is in [`sql_alchemy.py`](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py) and relies on each respective SqlAlchemy dialect.
+
+#### Hive Interface
+
+This [asynchronous API](https://github.com/cloudera/hue/tree/master/apps/beeswax) based on the Thrift API of Hive is very mature and powers an excellent integration of Apache Hive and Apache Impala.
+
+#### Custom
+
+If the built-in HiveServer2 (Hive, Impala, Spark SQL), SqlAlchemy (MySQL, PostgreSQL, Oracle, Presto...) don’t meet your needs, you can implement your own connector to the notebook app:
+
+* List of [all connectors](https://github.com/cloudera/hue/tree/master/desktop/libs/notebook/src/notebook/connectors)
+* Each connector API subclasses the [Base API](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/base.py) and must implement the methods defined within. Refer to the [JDBC](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/jdbc.py) or [RdbmsApi](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/rdbms.py) for representative examples
+
+* [Kafka SQL](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/ksql.py)
+* [Solr SQL](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/solr.py)
+* [JDBC](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/jdbc.py)
+
+The JDBC API relies on a small JDBC proxy running next to the Hue API. By default it won't be built without setting the `BUILD_DB_PROXY` flag, e.g.:
+
+    export BUILD_DB_PROXY=true make install
+
+**Note** In the long term, SqlAlchemy is prefered as more "Python native".
+
+#### Potential connectors
+
+It is recommended to develop an SqlAlchemy connector if yours is not already [existing](/administrator/configuration/connectors/#databases).
+
+### Catalogs
+
+The backends is pluggable by providing alternative [client interfaces](https://github.com/cloudera/hue/tree/master/desktop/libs/metadata/src/metadata/catalog):
+
+* [Apache Atlas](https://atlas.apache.org/)
+* Cloudera Navigator
+* Dummy (skeleton for integrating new catalogs)
+
+#### Apache Atlas
+
+* [Client API](https://github.com/cloudera/hue/tree/master/desktop/libs/metadata/src/metadata/catalog/atlas_client.py)
+
+#### Potential connectors
+
+* [Linkedin DataHub](https://github.com/linkedin/datahub)
+* [Lift Amundsen](https://github.com/lyft/amundsen)
+* AWS Glue
+* Google Cloud Data Catalog
+* Alation
+
+### Storages
+
+Various storage systems can be interacted with. The [`fsmanager.py`](https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/lib/fsmanager.py) is the main router to each API.
+
+**Note** Apache Ozone as well as Ceph can be used via the S3 browser.
+
+#### Hadoop HDFS
+
+* [WebHdfs API](https://github.com/cloudera/hue/blob/master/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py)
+
+#### AWS S3
+
+* [S3 API](https://github.com/cloudera/hue/blob/master/desktop/libs/aws/src/aws/s3)
+
+#### Azure ADLS
+
+* [ADLS v2](https://github.com/cloudera/hue/blob/master/desktop/libs/azure/src/azure/abfs)
+* [ADLS v1](https://github.com/cloudera/hue/blob/master/desktop/libs/azure/src/azure/adls)
+
+#### HBase / Key Value Stores
+
+With just a few changes in the [Python API](https://github.com/cloudera/hue/blob/master/apps/hbase/src/hbase/api.py), the HBase browser could be compatible with Apache Kudu or Google Big Table.
+
+#### Potential connectors
+
+* Google Cloud Storage is currently a work in progress with [HUE-8978](https://issues.cloudera.org/browse/HUE-8978)
+
+### Jobs
+
+#### Apache Spark / Livy
+
+Based on the [Livy REST API](/administrator/configuration/connectors/#apache-spark).
+
+* [Notebook connector](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/spark_shell.py)
+  * PySpark
+  * Scala
+  * Spark SQL
+* [Batch connector](https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/spark_batch.py)
+
+#### Schedulers
+
+Currently only Apache Oozie is supported for your Datawarehouse, but the API is getting generic with [HUE-3797](https://issues.cloudera.org/browse/HUE-3797).
+
+#### Potential connectors
+
+* Elastic Search: a connector similar to Solr for Searching [HUE-7828](https://issues.cloudera.org/browse/HUE-7828). SQL querying is already supported.
+* [Livy Browser API](https://github.com/cloudera/hue/blob/master/apps/jobbrowser/src/jobbrowser/apis/livy_api.py)
+* [Celery API](https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/lib/scheduler/lib/beat.py)
+* Apache Hive native support of query scheduling [HIVE-21884](https://issues.apache.org/jira/browse/HIVE-21884)