apache_flink.md 2.0 KB

A modern SQL Editor for Apache Flink SQL

Background

Hue Flink Editor

This document is about the Query UX specific improvements and assumes a running Flink cluster is available.

Demo

https://github.com/romainr/flink-sql-gateway/tree/master/docs/demo

Tasks

The big pieces would be to stabilize a query API, improve the SQL autocomplete, make the full setup work out of the box.

API

  • Currently leveraging the Flink SQL Gateway
    • Early stage
    • From Ververica (Apache 2)
    • Note: initial tables needs to be defined and loaded via a config file (which I could not make it work so far)
    • Could be turned to a SqlAchemy module for more standardization
    • List of supported statements
  • Security support?

UI

  • Full Editor
  • Support and display of "live" query/streams
  • Addition of Task Server and Websockets for smoother support

Note: some ways to iteratively get some progress could be to start with these static modules which don't necessarily require a running Hue.

Flink SQL Scratchpad Editor

Admin

  • CDP auto configuration of properties (e.g. Flink Gateway service config, Hue...)