Ver código fonte

HUE-9367 [phoenix] Bundle SqlAlchemy connector

Romain 5 anos atrás
pai
commit
2ce46f129c
40 arquivos alterados com 4761 adições e 0 exclusões
  1. 9 0
      desktop/core/ext-py/phoenixdb/Dockerfile
  2. 9 0
      desktop/core/ext-py/phoenixdb/Dockerfile-pqs
  3. 60 0
      desktop/core/ext-py/phoenixdb/NEWS.rst
  4. 131 0
      desktop/core/ext-py/phoenixdb/README.rst
  5. 12 0
      desktop/core/ext-py/phoenixdb/RELEASING.rst
  6. 7 0
      desktop/core/ext-py/phoenixdb/ci/build-env/Dockerfile
  7. 33 0
      desktop/core/ext-py/phoenixdb/ci/phoenix/Dockerfile
  8. 24 0
      desktop/core/ext-py/phoenixdb/ci/phoenix/docker-entrypoint.sh
  9. 12 0
      desktop/core/ext-py/phoenixdb/ci/phoenix/hbase-site.xml
  10. 192 0
      desktop/core/ext-py/phoenixdb/doc/Makefile
  11. 30 0
      desktop/core/ext-py/phoenixdb/doc/api.rst
  12. 287 0
      desktop/core/ext-py/phoenixdb/doc/conf.py
  13. 27 0
      desktop/core/ext-py/phoenixdb/doc/index.rst
  14. 3 0
      desktop/core/ext-py/phoenixdb/doc/versions.rst
  15. 27 0
      desktop/core/ext-py/phoenixdb/examples/basic.py
  16. 39 0
      desktop/core/ext-py/phoenixdb/gen-protobuf.sh
  17. 152 0
      desktop/core/ext-py/phoenixdb/phoenixdb/__init__.py
  18. 16 0
      desktop/core/ext-py/phoenixdb/phoenixdb/avatica/__init__.py
  19. 529 0
      desktop/core/ext-py/phoenixdb/phoenixdb/avatica/client.py
  20. 0 0
      desktop/core/ext-py/phoenixdb/phoenixdb/avatica/proto/__init__.py
  21. 22 0
      desktop/core/ext-py/phoenixdb/phoenixdb/avatica/proto/common_pb2.py
  22. 22 0
      desktop/core/ext-py/phoenixdb/phoenixdb/avatica/proto/requests_pb2.py
  23. 22 0
      desktop/core/ext-py/phoenixdb/phoenixdb/avatica/proto/responses_pb2.py
  24. 185 0
      desktop/core/ext-py/phoenixdb/phoenixdb/connection.py
  25. 371 0
      desktop/core/ext-py/phoenixdb/phoenixdb/cursor.py
  26. 93 0
      desktop/core/ext-py/phoenixdb/phoenixdb/errors.py
  27. 257 0
      desktop/core/ext-py/phoenixdb/phoenixdb/sqlalchemy_phoenix.py
  28. 64 0
      desktop/core/ext-py/phoenixdb/phoenixdb/tests/__init__.py
  29. 857 0
      desktop/core/ext-py/phoenixdb/phoenixdb/tests/dbapi20.py
  30. 26 0
      desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_avatica.py
  31. 109 0
      desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_db.py
  32. 125 0
      desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_dbapi20.py
  33. 60 0
      desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_errors.py
  34. 79 0
      desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_sqlalchemy.py
  35. 378 0
      desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_types.py
  36. 304 0
      desktop/core/ext-py/phoenixdb/phoenixdb/types.py
  37. 21 0
      desktop/core/ext-py/phoenixdb/requirements.txt
  38. 34 0
      desktop/core/ext-py/phoenixdb/setup.cfg
  39. 108 0
      desktop/core/ext-py/phoenixdb/setup.py
  40. 25 0
      desktop/core/ext-py/phoenixdb/tox.ini

+ 9 - 0
desktop/core/ext-py/phoenixdb/Dockerfile

@@ -0,0 +1,9 @@
+from themattrix/tox-base
+
+RUN apt-get update && apt-get install -y krb5-user libkrb5-dev
+
+ENV PHOENIXDB_TEST_DB_URL=http://host.docker.internal:8765
+ENV PHOENIXDB_TEST_DB_TRUSTSTORE=
+ENV PHOENIXDB_TEST_DB_AUTHENTICATION=
+ENV PHOENIXDB_TEST_DB_AVATICA_USER=
+ENV PHOENIXDB_TEST_DB_AVATICA_PASSWORD=

+ 9 - 0
desktop/core/ext-py/phoenixdb/Dockerfile-pqs

@@ -0,0 +1,9 @@
+from maven:3-jdk-8
+
+RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -yq krb5-user libkrb5-dev
+
+EXPOSE 8765
+
+# copy all the files to the container
+
+CMD mvn clean verify -am -pl queryserver-it -Dtest=foo -Dit.test=QueryServerBasicsIT#startLocalPQS -Ddo.not.randomize.pqs.port=true -Dstart.unsecure.pqs=true

+ 60 - 0
desktop/core/ext-py/phoenixdb/NEWS.rst

@@ -0,0 +1,60 @@
+Changelog
+=========
+
+Unreleased
+----------
+- Replaced bundled requests_kerberos with request_gssapi library
+- Use default SPNEGO Auth settings from request_gssapi
+- Refactored authentication code
+- Added support for specifying server certificate
+- Added support for BASIC and DIGEST authentication
+- Fixed HTTP error parsing
+- Added transaction support
+- Added list support
+- Rewritten type handling
+- Refactored test suite
+- Removed shell example, as it was python2 only
+- Updated documentation
+- Added SQLAlchemy dialect
+
+Version 0.7
+-----------
+
+- Added DictCursor for easier access to columns by their names.
+- Support for Phoenix versions from 4.8 to 4.11.
+
+Version 0.6
+-----------
+
+- Fixed result fetching when using a query with parameters.
+- Support for Phoenix 4.9.
+
+Version 0.5
+-----------
+
+- Added support for Python 3.
+- Switched from the JSON serialization to Protocol Buffers, improved compatibility with Phoenix 4.8.
+- Phoenix 4.6 and older are no longer supported.
+
+Version 0.4
+-----------
+
+- Fixes for the final version of Phoenix 4.7.
+
+Version 0.3
+-----------
+
+- Compatible with Phoenix 4.7.
+
+Version 0.2
+-----------
+
+- Added (configurable) retry on connection errors.
+- Added Vagrantfile for easier testing.
+- Compatible with Phoenix 4.6.
+
+Version 0.1
+-----------
+
+- Initial release.
+- Compatible with Phoenix 4.4.

+ 131 - 0
desktop/core/ext-py/phoenixdb/README.rst

@@ -0,0 +1,131 @@
+Phoenix database adapter for Python
+===================================
+
+``phoenixdb`` is a Python library for accessing the
+`Phoenix SQL database <http://phoenix.apache.org/>`_
+using the
+`remote query server <http://phoenix.apache.org/server.html>`_.
+The library implements the
+standard `DB API 2.0 <https://www.python.org/dev/peps/pep-0249/>`_ interface,
+which should be familiar to most Python programmers.
+
+Installation
+------------
+
+The source code is part of the phoenix-queryserver source distribution.
+You can download it from <https://phoenix.apache.org/>, or get the latest development version
+from <https://github.com/apache/phoenix-queryserver>
+
+Extract the archive and then install it manually::
+
+    cd /path/to/phoenix-queryserver-x.y.z/python/phoenixdb
+    python setup.py install
+
+Usage
+-----
+
+The library implements the standard DB API 2.0 interface, so it can be
+used the same way you would use any other SQL database from Python, for example::
+
+    import phoenixdb
+    import phoenixdb.cursor
+
+    database_url = 'http://localhost:8765/'
+    conn = phoenixdb.connect(database_url, autocommit=True)
+
+    cursor = conn.cursor()
+    cursor.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, username VARCHAR)")
+    cursor.execute("UPSERT INTO users VALUES (?, ?)", (1, 'admin'))
+    cursor.execute("SELECT * FROM users")
+    print(cursor.fetchall())
+
+    cursor = conn.cursor(cursor_factory=phoenixdb.cursor.DictCursor)
+    cursor.execute("SELECT * FROM users WHERE id=1")
+    print(cursor.fetchone()['USERNAME'])
+
+
+Setting up a development environment
+------------------------------------
+
+If you want to quickly try out the included examples, you can set up a
+local `virtualenv <https://virtualenv.pypa.io/en/latest/>`_ with all the
+necessary requirements::
+
+    virtualenv e
+    source e/bin/activate
+    pip install -r requirements.txt
+    python setup.py develop
+
+You can start a Phoenix QueryServer instance on http://localhost:8765 for testing by running
+the following command in the phoenix-queryserver directory::
+
+    mvn clean verify -am -pl queryserver-it -Dtest=foo \
+    -Dit.test=QueryServerBasicsIT\#startLocalPQS \
+    -Ddo.not.randomize.pqs.port=true -Dstart.unsecure.pqs=true
+
+You can start a secure (https+kerberos) Phoenix QueryServer instance on https://localhost:8765
+for testing by running the following command in the phoenix-queryserver directory::
+
+    mvn clean verify -am -pl queryserver-it -Dtest=foo \
+    -Dit.test=SecureQueryServerPhoenixDBIT\#startLocalPQS \
+    -Ddo.not.randomize.pqs.port=true -Dstart.secure.pqs=true
+
+this will also create a shell script in queryserver-it/target/krb_setup.sh, that you can use to set
+up the environment for the tests.
+
+If you want to use the library without installing the phoenixdb library, you can use
+the `PYTHONPATH` environment variable to point to the library directly::
+
+    cd $PHOENIX_HOME/python
+    python setup.py build
+    cd ~/my_project
+    PYTHONPATH=$PHOENIX_HOME/build/lib python my_app.py
+
+Don't forget to run flake8 on your changes.
+
+Running the test suite
+----------------------
+
+The library comes with a test suite for testing Python DB API 2.0 compliance and
+various Phoenix-specific features. In order to run the test suite, you need a
+working Phoenix database and set the ``PHOENIXDB_TEST_DB_URL`` environment variable::
+
+    export PHOENIXDB_TEST_DB_URL='http://localhost:8765/'
+    nosetests
+
+If you use a secure PQS server, you can set the connection parameters via the following environment
+variables:
+
+- PHOENIXDB_TEST_DB_TRUSTSTORE
+- PHOENIXDB_TEST_DB_AUTHENTICATION
+- PHOENIXDB_TEST_DB_AVATICA_USER
+- PHOENIXDB_TEST_DB_AVATICA_PASSWORD
+
+Similarly, tox can be used to run the test suite against multiple Python versions::
+
+    pyenv install 3.5.5
+    pyenv install 3.6.4
+    pyenv install 2.7.14
+    pyenv global 2.7.14 3.5.5 3.6.4
+    PHOENIXDB_TEST_DB_URL='http://localhost:8765' tox
+
+You can use tox and docker to run the tests on all supported python versions without installing the
+environments locally::
+
+    docker build -t toxtest .
+    docker run --rm  -v `pwd`:/src toxtest
+
+You can also run the test suite from maven as part of the Java build by setting the 
+run.full.python.testsuite property. You DO NOT need to set the PHOENIXDB_* enviroment variables,
+maven will set them up for you. The output of the test run will be saved in
+phoenix-queryserver/queryserver-it/target/python-stdout.log and python-stderr.log::
+
+    mvn clean verify -Drun.full.python.testsuite=true
+
+Known issues
+------------
+
+- TIME and DATE columns in Phoenix are stored as full timestamps with a millisecond accuracy,
+  but the remote protocol only exposes the time (hour/minute/second) or date (year/month/day)
+  parts of the columns. (`CALCITE-797 <https://issues.apache.org/jira/browse/CALCITE-797>`_, `CALCITE-798 <https://issues.apache.org/jira/browse/CALCITE-798>`_)
+- TIMESTAMP columns in Phoenix are stored with a nanosecond accuracy, but the remote protocol truncates them to milliseconds. (`CALCITE-796 <https://issues.apache.org/jira/browse/CALCITE-796>`_)

+ 12 - 0
desktop/core/ext-py/phoenixdb/RELEASING.rst

@@ -0,0 +1,12 @@
+Releasing a new version
+=======================
+
+Change the version number ``setup.py`` and ``NEWS.rst``.
+
+Commit the changes and tag the repository::
+
+    git tag -s vX.Y
+
+Upload the package to PyPI::
+
+    python setup.py clean sdist upload

+ 7 - 0
desktop/core/ext-py/phoenixdb/ci/build-env/Dockerfile

@@ -0,0 +1,7 @@
+FROM ubuntu:xenial
+
+RUN apt-get update && \
+    DEBIAN_FRONTEND=noninteractive apt-get install -y python-dev python3-dev tox
+
+RUN apt-get update && \
+    DEBIAN_FRONTEND=noninteractive apt-get install -y git

+ 33 - 0
desktop/core/ext-py/phoenixdb/ci/phoenix/Dockerfile

@@ -0,0 +1,33 @@
+FROM openjdk:8
+
+ARG HBASE_VERSION
+ARG HBASE_DIR
+ARG PHOENIX_VERSION
+ARG PHOENIX_NAME=apache-phoenix
+
+ENV HBASE_URL https://archive.apache.org/dist/hbase/$HBASE_DIR/hbase-$HBASE_VERSION-bin.tar.gz
+
+RUN wget --no-verbose -O hbase.tar.gz "$HBASE_URL" && \
+    mkdir /opt/hbase && \
+    tar xf hbase.tar.gz --strip-components=1 -C /opt/hbase && \
+    rm hbase.tar.gz
+
+ENV PHOENIX_URL https://archive.apache.org/dist/phoenix/apache-phoenix-$PHOENIX_VERSION/bin/apache-phoenix-$PHOENIX_VERSION-bin.tar.gz
+
+RUN wget --no-verbose -O phoenix.tar.gz "$PHOENIX_URL" && \
+    mkdir /opt/phoenix && \
+    tar xf phoenix.tar.gz --strip-components=1 -C /opt/phoenix && \
+    rm phoenix.tar.gz
+
+RUN ln -sv /opt/phoenix/phoenix-*-server.jar /opt/hbase/lib/
+
+ADD hbase-site.xml /opt/hbase/conf/hbase-site.xml
+
+ENV HBASE_CONF_DIR /opt/hbase/conf
+ENV HBASE_CP /opt/hbase/lib
+ENV HBASE_HOME /opt/hbase
+
+EXPOSE 8765
+
+COPY docker-entrypoint.sh /usr/local/bin/
+ENTRYPOINT ["docker-entrypoint.sh"]

+ 24 - 0
desktop/core/ext-py/phoenixdb/ci/phoenix/docker-entrypoint.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+
+pids=()
+
+/opt/hbase/bin/hbase-daemon.sh foreground_start master &
+pids+=($!)
+
+/opt/phoenix/bin/queryserver.py &
+pids+=($!)
+
+cleanup() {
+    if [ ${#pids[@]} -ne 0 ]
+    then
+        pids=($(ps -o pid= -p "${pids[@]}"))
+        if [ ${#pids[@]} -ne 0 ]
+        then
+            kill "${pids[@]}"
+        fi
+    fi
+}
+
+trap cleanup SIGCHLD SIGINT SIGTERM
+
+wait

+ 12 - 0
desktop/core/ext-py/phoenixdb/ci/phoenix/hbase-site.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0"?>
+<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+<configuration>
+    <property>
+        <name>hbase.regionserver.wal.codec</name>
+        <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value>
+    </property>
+	<property>
+		<name>phoenix.schema.isNamespaceMappingEnabled</name>
+		<value>true</value>
+	</property>
+</configuration>

+ 192 - 0
desktop/core/ext-py/phoenixdb/doc/Makefile

@@ -0,0 +1,192 @@
+# Makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = sphinx-build
+PAPER         =
+BUILDDIR      = _build
+
+# User-friendly check for sphinx-build
+ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
+$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
+endif
+
+# Internal variables.
+PAPEROPT_a4     = -D latex_paper_size=a4
+PAPEROPT_letter = -D latex_paper_size=letter
+ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+# the i18n builder cannot share the environment and doctrees with the others
+I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+
+.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest coverage gettext
+
+help:
+	@echo "Please use \`make <target>' where <target> is one of"
+	@echo "  html       to make standalone HTML files"
+	@echo "  dirhtml    to make HTML files named index.html in directories"
+	@echo "  singlehtml to make a single large HTML file"
+	@echo "  pickle     to make pickle files"
+	@echo "  json       to make JSON files"
+	@echo "  htmlhelp   to make HTML files and a HTML help project"
+	@echo "  qthelp     to make HTML files and a qthelp project"
+	@echo "  applehelp  to make an Apple Help Book"
+	@echo "  devhelp    to make HTML files and a Devhelp project"
+	@echo "  epub       to make an epub"
+	@echo "  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
+	@echo "  latexpdf   to make LaTeX files and run them through pdflatex"
+	@echo "  latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
+	@echo "  text       to make text files"
+	@echo "  man        to make manual pages"
+	@echo "  texinfo    to make Texinfo files"
+	@echo "  info       to make Texinfo files and run them through makeinfo"
+	@echo "  gettext    to make PO message catalogs"
+	@echo "  changes    to make an overview of all changed/added/deprecated items"
+	@echo "  xml        to make Docutils-native XML files"
+	@echo "  pseudoxml  to make pseudoxml-XML files for display purposes"
+	@echo "  linkcheck  to check all external links for integrity"
+	@echo "  doctest    to run all doctests embedded in the documentation (if enabled)"
+	@echo "  coverage   to run coverage check of the documentation (if enabled)"
+
+clean:
+	rm -rf $(BUILDDIR)/*
+
+html:
+	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+	@echo
+	@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
+
+dirhtml:
+	$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
+	@echo
+	@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
+
+singlehtml:
+	$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
+	@echo
+	@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
+
+pickle:
+	$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
+	@echo
+	@echo "Build finished; now you can process the pickle files."
+
+json:
+	$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
+	@echo
+	@echo "Build finished; now you can process the JSON files."
+
+htmlhelp:
+	$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
+	@echo
+	@echo "Build finished; now you can run HTML Help Workshop with the" \
+	      ".hhp project file in $(BUILDDIR)/htmlhelp."
+
+qthelp:
+	$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
+	@echo
+	@echo "Build finished; now you can run "qcollectiongenerator" with the" \
+	      ".qhcp project file in $(BUILDDIR)/qthelp, like this:"
+	@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/phoenixdb.qhcp"
+	@echo "To view the help file:"
+	@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/phoenixdb.qhc"
+
+applehelp:
+	$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
+	@echo
+	@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
+	@echo "N.B. You won't be able to view it unless you put it in" \
+	      "~/Library/Documentation/Help or install it in your application" \
+	      "bundle."
+
+devhelp:
+	$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
+	@echo
+	@echo "Build finished."
+	@echo "To view the help file:"
+	@echo "# mkdir -p $$HOME/.local/share/devhelp/phoenixdb"
+	@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/phoenixdb"
+	@echo "# devhelp"
+
+epub:
+	$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
+	@echo
+	@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
+
+latex:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+	@echo
+	@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
+	@echo "Run \`make' in that directory to run these through (pdf)latex" \
+	      "(use \`make latexpdf' here to do that automatically)."
+
+latexpdf:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+	@echo "Running LaTeX files through pdflatex..."
+	$(MAKE) -C $(BUILDDIR)/latex all-pdf
+	@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
+
+latexpdfja:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+	@echo "Running LaTeX files through platex and dvipdfmx..."
+	$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
+	@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
+
+text:
+	$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
+	@echo
+	@echo "Build finished. The text files are in $(BUILDDIR)/text."
+
+man:
+	$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
+	@echo
+	@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
+
+texinfo:
+	$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
+	@echo
+	@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
+	@echo "Run \`make' in that directory to run these through makeinfo" \
+	      "(use \`make info' here to do that automatically)."
+
+info:
+	$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
+	@echo "Running Texinfo files through makeinfo..."
+	make -C $(BUILDDIR)/texinfo info
+	@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
+
+gettext:
+	$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
+	@echo
+	@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
+
+changes:
+	$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
+	@echo
+	@echo "The overview file is in $(BUILDDIR)/changes."
+
+linkcheck:
+	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
+	@echo
+	@echo "Link check complete; look for any errors in the above output " \
+	      "or in $(BUILDDIR)/linkcheck/output.txt."
+
+doctest:
+	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
+	@echo "Testing of doctests in the sources finished, look at the " \
+	      "results in $(BUILDDIR)/doctest/output.txt."
+
+coverage:
+	$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
+	@echo "Testing of coverage in the sources finished, look at the " \
+	      "results in $(BUILDDIR)/coverage/python.txt."
+
+xml:
+	$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
+	@echo
+	@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
+
+pseudoxml:
+	$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
+	@echo
+	@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."

+ 30 - 0
desktop/core/ext-py/phoenixdb/doc/api.rst

@@ -0,0 +1,30 @@
+API Reference
+=============
+
+phoenixdb module
+----------------
+
+.. automodule:: phoenixdb
+    :members:
+    :undoc-members:
+
+phoenixdb.connection module
+---------------------------
+
+.. automodule:: phoenixdb.connection
+    :members:
+    :undoc-members:
+
+phoenixdb.cursor module
+-----------------------
+
+.. automodule:: phoenixdb.cursor
+    :members:
+    :undoc-members:
+
+phoenixdb.avatica module
+------------------------
+
+.. automodule:: phoenixdb.avatica
+    :members:
+    :undoc-members:

+ 287 - 0
desktop/core/ext-py/phoenixdb/doc/conf.py

@@ -0,0 +1,287 @@
+# -*- coding: utf-8 -*-
+#
+# phoenixdb documentation build configuration file, created by
+# sphinx-quickstart on Sun Jun 28 18:07:35 2015.
+#
+# This file is execfile()d with the current directory set to its
+# containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+import sys
+import os
+import shlex
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+sys.path.insert(0, os.path.abspath('../phoenixdb'))
+
+# -- General configuration ------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.doctest',
+    'sphinx.ext.intersphinx',
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The encoding of source files.
+source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = u'phoenixdb'
+copyright = u'2015, Lukas Lalinsky'
+author = u'Lukas Lalinsky'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+exclude_patterns = ['_build']
+
+# The reST default role (used for this markup: `text`) to use for all
+# documents.
+#default_role = None
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#add_module_names = True
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+#show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+# If true, keep warnings as "system message" paragraphs in the built documents.
+#keep_warnings = False
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = False
+
+
+# -- Options for HTML output ----------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+html_theme = 'classic'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#html_theme_options = {}
+
+# Add any paths that contain custom themes here, relative to this directory.
+#html_theme_path = []
+
+# The name for this set of Sphinx documents.  If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar.  Default is the same as html_title.
+#html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#html_logo = None
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#html_favicon = None
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Add any extra paths that contain custom files (such as robots.txt or
+# .htaccess) here, relative to this directory. These files are copied
+# directly to the root of the documentation.
+#html_extra_path = []
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+#html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_domain_indices = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+html_show_sourcelink = False
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#html_show_sphinx = True
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+#html_show_copyright = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it.  The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# This is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = None
+
+# Language to be used for generating the HTML full-text search index.
+# Sphinx supports the following languages:
+#   'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja'
+#   'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr'
+#html_search_language = 'en'
+
+# A dictionary with options for the search language support, empty by default.
+# Now only 'ja' uses this config value
+#html_search_options = {'type': 'default'}
+
+# The name of a javascript file (relative to the configuration directory) that
+# implements a search results scorer. If empty, the default will be used.
+#html_search_scorer = 'scorer.js'
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'phoenixdbdoc'
+
+# -- Options for LaTeX output ---------------------------------------------
+
+#latex_elements = {
+# The paper size ('letterpaper' or 'a4paper').
+#'papersize': 'letterpaper',
+
+# The font size ('10pt', '11pt' or '12pt').
+#'pointsize': '10pt',
+
+# Additional stuff for the LaTeX preamble.
+#'preamble': '',
+
+# Latex figure (float) alignment
+#'figure_align': 'htbp',
+#}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+#latex_documents = [
+#  (master_doc, 'phoenixdb.tex', u'phoenixdb Documentation',
+#   u'Lukas Lalinsky', 'manual'),
+#]
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+#latex_logo = None
+
+# For "manual" documents, if this is true, then toplevel headings are parts,
+# not chapters.
+#latex_use_parts = False
+
+# If true, show page references after internal links.
+#latex_show_pagerefs = False
+
+# If true, show URL addresses after external links.
+#latex_show_urls = False
+
+# Documents to append as an appendix to all manuals.
+#latex_appendices = []
+
+# If false, no module index is generated.
+#latex_domain_indices = True
+
+
+# -- Options for manual page output ---------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    (master_doc, 'phoenixdb', u'phoenixdb Documentation',
+     [author], 1)
+]
+
+# If true, show URL addresses after external links.
+#man_show_urls = False
+
+
+# -- Options for Texinfo output -------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+  (master_doc, 'phoenixdb', u'phoenixdb Documentation',
+   author, 'phoenixdb', 'One line description of project.',
+   'Miscellaneous'),
+]
+
+# Documents to append as an appendix to all manuals.
+#texinfo_appendices = []
+
+# If false, no module index is generated.
+#texinfo_domain_indices = True
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#texinfo_show_urls = 'footnote'
+
+# If true, do not generate a @detailmenu in the "Top" node's menu.
+#texinfo_no_detailmenu = False
+
+
+# Example configuration for intersphinx: refer to the Python standard library.
+intersphinx_mapping = {'https://docs.python.org/': None}

+ 27 - 0
desktop/core/ext-py/phoenixdb/doc/index.rst

@@ -0,0 +1,27 @@
+.. include:: ../README.rst
+
+API Reference
+-------------
+
+.. toctree::
+   :maxdepth: 2
+
+   api
+
+Changelog
+-------------
+
+.. toctree::
+   :maxdepth: 2
+
+   versions
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+
+
+.. _

+ 3 - 0
desktop/core/ext-py/phoenixdb/doc/versions.rst

@@ -0,0 +1,3 @@
+.. include:: ../NEWS.rst
+
+.. _

+ 27 - 0
desktop/core/ext-py/phoenixdb/examples/basic.py

@@ -0,0 +1,27 @@
+#!/usr/bin/env python
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import phoenixdb
+
+with phoenixdb.connect('http://localhost:8765/', autocommit=True) as connection:
+    with connection.cursor() as cursor:
+        cursor.execute("DROP TABLE IF EXISTS test")
+        cursor.execute("CREATE TABLE test (id INTEGER PRIMARY KEY, text VARCHAR)")
+        cursor.executemany("UPSERT INTO test VALUES (?, ?)", [[1, 'hello'], [2, 'world']])
+        cursor.execute("SELECT * FROM test ORDER BY id")
+        for row in cursor:
+            print(row)

+ 39 - 0
desktop/core/ext-py/phoenixdb/gen-protobuf.sh

@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -x
+AVATICA_VER=rel/avatica-1.10.0
+
+set -e
+
+rm -rf avatica-tmp
+
+mkdir avatica-tmp
+cd avatica-tmp
+wget -O avatica.tar.gz https://github.com/apache/calcite-avatica/archive/$AVATICA_VER.tar.gz
+tar -x --strip-components=1 -f avatica.tar.gz
+
+cd ..
+rm -f phoenixdb/avatica/proto/*_pb2.py
+protoc --proto_path=avatica-tmp/core/src/main/protobuf/ --python_out=phoenixdb/avatica/proto avatica-tmp/core/src/main/protobuf/*.proto
+if [[ "$(uname)" == "Darwin" ]]; then
+  sed -i '' 's/import common_pb2/from . import common_pb2/' phoenixdb/avatica/proto/*_pb2.py
+else
+  sed -i 's/import common_pb2/from . import common_pb2/' phoenixdb/avatica/proto/*_pb2.py
+fi
+
+rm -rf avatica-tmp

+ 152 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/__init__.py

@@ -0,0 +1,152 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+
+from phoenixdb import errors, types
+from phoenixdb.avatica import AvaticaClient
+from phoenixdb.connection import Connection
+from phoenixdb.errors import *  # noqa: F401,F403
+from phoenixdb.types import *  # noqa: F401,F403
+
+from requests.auth import HTTPBasicAuth, HTTPDigestAuth
+
+from requests_gssapi import HTTPSPNEGOAuth
+
+if sys.version_info.major == 3:
+    from urllib.parse import urlencode, urlparse, urlunparse, parse_qs
+else:
+    from urllib import urlencode
+    from urlparse import urlparse, urlunparse, parse_qs
+
+__all__ = ['connect', 'apilevel', 'threadsafety', 'paramstyle'] + types.__all__ + errors.__all__
+
+
+apilevel = "2.0"
+"""
+This module supports the `DB API 2.0 interface <https://www.python.org/dev/peps/pep-0249/>`_.
+"""
+
+threadsafety = 1
+"""
+Multiple threads can share the module, but neither connections nor cursors.
+"""
+
+paramstyle = 'qmark'
+"""
+Parmetrized queries should use the question mark as a parameter placeholder.
+
+For example::
+
+ cursor.execute("SELECT * FROM table WHERE id = ?", [my_id])
+"""
+
+
+def connect(url, max_retries=None, auth=None, authentication=None, avatica_user=None, avatica_password=None,
+            truststore=None, verify=None, **kwargs):
+    """Connects to a Phoenix query server.
+
+    :param url:
+        URL to the Phoenix query server, e.g. ``http://localhost:8765/``
+
+    :param autocommit:
+        Switch the connection to autocommit mode.
+
+    :param readonly:
+        Switch the connection to readonly mode.
+
+    :param max_retries:
+        The maximum number of retries in case there is a connection error.
+
+    :param cursor_factory:
+        If specified, the connection's :attr:`~phoenixdb.connection.Connection.cursor_factory`
+        is set to it.
+
+    :param auth:
+        Authentication configuration object as expected by the underlying python_requests and
+        python_requests_gssapi library
+
+    :param authentication:
+        Alternative way to specify the authentication mechanism that mimics
+        the semantics of the JDBC drirver
+
+    :param avatica_user:
+        Username for BASIC or DIGEST authentication. Use in conjunction with the
+        `~authentication' option.
+
+    :param avatica_password:
+        Password for BASIC or DIGEST authentication. Use in conjunction with the
+        `~authentication' option.
+
+    :param verify:
+        The path to the PEM file for verifying the server's certificate. It is passed directly to
+        the `~verify` parameter of the underlying python_requests library.
+        Setting it to false disables the server certificate verification.
+
+    :param truststore:
+        Alias for verify
+
+    :returns:
+        :class:`~phoenixdb.connection.Connection` object.
+    """
+
+    url_parsed = urlparse(url)
+    url_params = parse_qs(url_parsed.query, keep_blank_values=True)
+
+    # Parse supported JDBC compatible options from URL. args have precendece
+    rebuild = False
+    if auth is None and authentication is None and 'authentication' in url_params:
+        authentication = url_params['authentication'][0]
+        del url_params['authentication']
+        rebuild = True
+
+    if avatica_user is None and 'avatica_user' in url_params:
+        avatica_user = url_params['avatica_user'][0]
+        del url_params['avatica_user']
+        rebuild = True
+
+    if avatica_password is None and 'avatica_password' in url_params:
+        avatica_password = url_params['avatica_password'][0]
+        del url_params['avatica_password']
+        rebuild = True
+
+    if verify is None and truststore is None and 'truststore' in url_params:
+        truststore = url_params['truststore'][0]
+        del url_params['truststore']
+        rebuild = True
+
+    if rebuild:
+        url_parsed._replace(query=urlencode(url_params, True))
+        url = urlunparse(url_parsed)
+
+    if auth == "SPNEGO":
+        # Special case for backwards compatibility
+        auth = HTTPSPNEGOAuth(opportunistic_auth=True)
+    elif auth is None and authentication is not None:
+        if authentication == "SPNEGO":
+            auth = HTTPSPNEGOAuth(opportunistic_auth=True)
+        elif authentication == "BASIC" and avatica_user is not None and avatica_password is not None:
+            auth = HTTPBasicAuth(avatica_user, avatica_password)
+        elif authentication == "DIGEST" and avatica_user is not None and avatica_password is not None:
+            auth = HTTPDigestAuth(avatica_user, avatica_password)
+
+    if verify is None and truststore is not None:
+        verify = truststore
+
+    client = AvaticaClient(url, max_retries=max_retries,
+                           auth=auth,
+                           verify=verify
+                           )
+    client.connect()
+    return Connection(client, **kwargs)

+ 16 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/avatica/__init__.py

@@ -0,0 +1,16 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from .client import AvaticaClient  # noqa: F401

+ 529 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/avatica/client.py

@@ -0,0 +1,529 @@
+# Copyright 2015 Lukas Lalinsky
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Implementation of the JSON-over-HTTP RPC protocol used by Avatica."""
+
+import logging
+import math
+import pprint
+import re
+import time
+
+from phoenixdb import errors
+from phoenixdb.avatica.proto import common_pb2, requests_pb2, responses_pb2
+
+import requests
+
+try:
+    import urlparse
+except ImportError:
+    import urllib.parse as urlparse
+
+try:
+    from HTMLParser import HTMLParser
+except ImportError:
+    from html.parser import HTMLParser
+
+__all__ = ['AvaticaClient']
+
+logger = logging.getLogger(__name__)
+
+
+class JettyErrorPageParser(HTMLParser):
+
+    def __init__(self):
+        HTMLParser.__init__(self)
+        self.path = []
+        self.title = []
+        self.message = []
+
+    def handle_starttag(self, tag, attrs):
+        self.path.append(tag)
+
+    def handle_endtag(self, tag):
+        self.path.pop()
+
+    def handle_data(self, data):
+        if len(self.path) > 2 and self.path[0] == 'html' and self.path[1] == 'body':
+            if len(self.path) == 3 and self.path[2] == 'h2':
+                self.title.append(data.strip())
+            elif len(self.path) == 4 and self.path[2] == 'p' and self.path[3] == 'pre':
+                self.message.append(data.strip())
+
+
+def parse_url(url):
+    url = urlparse.urlparse(url)
+    if not url.scheme and not url.netloc and url.path:
+        netloc = url.path
+        if ':' not in netloc:
+            netloc = '{}:8765'.format(netloc)
+        return urlparse.ParseResult('http', netloc, '/', '', '', '')
+    return url
+
+
+# Defined in phoenix-core/src/main/java/org/apache/phoenix/exception/SQLExceptionCode.java
+SQLSTATE_ERROR_CLASSES = [
+    ('08', errors.OperationalError),  # Connection Exception
+    ('22018', errors.IntegrityError),  # Constraint violatioin.
+    ('22', errors.DataError),  # Data Exception
+    ('23', errors.IntegrityError),  # Constraint Violation
+    ('24', errors.InternalError),  # Invalid Cursor State
+    ('25', errors.InternalError),  # Invalid Transaction State
+    ('42', errors.ProgrammingError),  # Syntax Error or Access Rule Violation
+    ('XLC', errors.OperationalError),  # Execution exceptions
+    ('INT', errors.InternalError),  # Phoenix internal error
+]
+
+# Relevant properties as defined by https://calcite.apache.org/avatica/docs/client_reference.html
+OPEN_CONNECTION_PROPERTIES = (
+    'avatica_user',  # User for the database connection
+    'avatica_password',  # Password for the user
+    'auth',
+    'authentication',
+    'truststore',
+    'verify'
+)
+
+
+def raise_sql_error(code, sqlstate, message):
+    for prefix, error_class in SQLSTATE_ERROR_CLASSES:
+        if sqlstate.startswith(prefix):
+            raise error_class(message, code, sqlstate)
+
+
+def parse_and_raise_sql_error(message):
+    match = re.findall(r'(?:([^ ]+): )?ERROR (\d+) \(([0-9A-Z]{5})\): (.*?) ->', message)
+    if match is not None and len(match):
+        exception, code, sqlstate, message = match[0]
+        raise_sql_error(int(code), sqlstate, message)
+
+
+def parse_error_page(html):
+    parser = JettyErrorPageParser()
+    parser.feed(html)
+    if parser.title == ['HTTP ERROR: 500']:
+        message = ' '.join(parser.message).strip()
+        parse_and_raise_sql_error(message)
+        raise errors.InternalError(message)
+
+
+def parse_error_protobuf(text):
+    message = common_pb2.WireMessage()
+    message.ParseFromString(text)
+
+    err = responses_pb2.ErrorResponse()
+    err.ParseFromString(message.wrapped_message)
+
+    parse_and_raise_sql_error(err.error_message)
+    raise_sql_error(err.error_code, err.sql_state, err.error_message)
+    raise errors.InternalError(err.error_message)
+
+
+class AvaticaClient(object):
+    """Client for Avatica's RPC server.
+
+    This exposes all low-level functionality that the Avatica
+    server provides, using the native terminology. You most likely
+    do not want to use this class directly, but rather get connect
+    to a server using :func:`phoenixdb.connect`.
+    """
+
+    def __init__(self, url, max_retries=None, auth=None, verify=None):
+        """Constructs a new client object.
+
+        :param url:
+            URL of an Avatica RPC server.
+        """
+        self.url = parse_url(url)
+        self.max_retries = max_retries if max_retries is not None else 3
+        self.auth = auth
+        self.verify = verify
+        self.connection = None
+
+    def connect(self):
+        """This method used to open a persistent TCP connection
+        requests does not require this"""
+        pass
+
+    def close(self):
+        """Also does nothing per requests"""
+        pass
+
+    def _post_request(self, body, headers):
+        retry_count = self.max_retries
+        while True:
+            logger.debug("POST %s %r %r", self.url.geturl(), body, headers)
+
+            requestArgs = {'data': body, 'stream': True, 'headers': headers}
+
+            if self.auth is not None:
+                requestArgs.update(auth=self.auth)
+
+            if self.verify is not None:
+                requestArgs.update(verify=self.verify)
+
+            try:
+                response = requests.request('post', self.url.geturl(), **requestArgs)
+
+            except requests.HTTPError as e:
+                if retry_count > 0:
+                    delay = math.exp(-retry_count)
+                    logger.debug("HTTP protocol error, will retry in %s seconds...", delay, exc_info=True)
+                    time.sleep(delay)
+                    retry_count -= 1
+                    continue
+                raise errors.InterfaceError('RPC request failed', cause=e)
+            else:
+                if response.status_code == requests.codes.service_unavailable:
+                    if retry_count > 0:
+                        delay = math.exp(-retry_count)
+                        logger.debug("Service unavailable, will retry in %s seconds...", delay, exc_info=True)
+                        time.sleep(delay)
+                        retry_count -= 1
+                        continue
+                return response
+
+    def _apply(self, request_data, expected_response_type=None):
+        logger.debug("Sending request\n%s", pprint.pformat(request_data))
+
+        request_name = request_data.__class__.__name__
+        message = common_pb2.WireMessage()
+        message.name = 'org.apache.calcite.avatica.proto.Requests${}'.format(request_name)
+        message.wrapped_message = request_data.SerializeToString()
+        body = message.SerializeToString()
+        headers = {'content-type': 'application/x-google-protobuf'}
+
+        response = self._post_request(body, headers)
+        response_body = response.raw.read()
+
+        if response.status_code != requests.codes.ok:
+            logger.debug("Received response\n%s", response_body)
+            if b'<html>' in response_body:
+                parse_error_page(response_body.decode(response.encoding))
+            else:
+                # assume the response is in protobuf format
+                parse_error_protobuf(response_body)
+            raise errors.InterfaceError('RPC request returned invalid status code', response.status_code)
+
+        message = common_pb2.WireMessage()
+        message.ParseFromString(response_body)
+
+        logger.debug("Received response\n%s", message)
+
+        if expected_response_type is None:
+            expected_response_type = request_name.replace('Request', 'Response')
+
+        expected_response_type = 'org.apache.calcite.avatica.proto.Responses$' + expected_response_type
+        if message.name != expected_response_type:
+            raise errors.InterfaceError('unexpected response type "{}" expected "{}"'.format(message.name, expected_response_type))
+
+        return message.wrapped_message
+
+    def get_catalogs(self, connection_id):
+        request = requests_pb2.CatalogsRequest()
+        request.connection_id = connection_id
+        return self._apply(request)
+
+    def get_schemas(self, connection_id, catalog=None, schemaPattern=None):
+        request = requests_pb2.SchemasRequest()
+        request.connection_id = connection_id
+        if catalog is not None:
+            request.catalog = catalog
+        if schemaPattern is not None:
+            request.schema_pattern = schemaPattern
+        return self._apply(request)
+
+    def get_tables(self, connection_id, catalog=None, schemaPattern=None, tableNamePattern=None, typeList=None):
+        request = requests_pb2.TablesRequest()
+        request.connection_id = connection_id
+        if catalog is not None:
+            request.catalog = catalog
+        if schemaPattern is not None:
+            request.schema_pattern = schemaPattern
+        if tableNamePattern is not None:
+            request.table_name_pattern = tableNamePattern
+        if typeList is not None:
+            request.type_list = typeList
+        if typeList is not None:
+            request.type_list.extend(typeList)
+        request.has_type_list = typeList is not None
+        return self._apply(request)
+
+    def get_columns(self, connection_id, catalog=None, schemaPattern=None, tableNamePattern=None, columnNamePattern=None):
+        request = requests_pb2.ColumnsRequest()
+        request.connection_id = connection_id
+        if catalog is not None:
+            request.catalog = catalog
+        if schemaPattern is not None:
+            request.schema_pattern = schemaPattern
+        if tableNamePattern is not None:
+            request.table_name_pattern = tableNamePattern
+        if columnNamePattern is not None:
+            request.column_name_pattern = columnNamePattern
+        return self._apply(request)
+
+    def get_table_types(self, connection_id):
+        request = requests_pb2.TableTypesRequest()
+        request.connection_id = connection_id
+        return self._apply(request)
+
+    def get_type_info(self, connection_id):
+        request = requests_pb2.TypeInfoRequest()
+        request.connection_id = connection_id
+        return self._apply(request)
+
+    def connection_sync(self, connection_id, connProps=None):
+        """Synchronizes connection properties with the server.
+
+        :param connection_id:
+            ID of the current connection.
+
+        :param connProps:
+            Dictionary with the properties that should be changed.
+
+        :returns:
+            A ``common_pb2.ConnectionProperties`` object.
+        """
+        if connProps is None:
+            connProps = {}
+
+        request = requests_pb2.ConnectionSyncRequest()
+        request.connection_id = connection_id
+        request.conn_props.auto_commit = connProps.get('autoCommit', False)
+        request.conn_props.has_auto_commit = True
+        request.conn_props.read_only = connProps.get('readOnly', False)
+        request.conn_props.has_read_only = True
+        request.conn_props.transaction_isolation = connProps.get('transactionIsolation', 0)
+        request.conn_props.catalog = connProps.get('catalog', '')
+        request.conn_props.schema = connProps.get('schema', '')
+
+        response_data = self._apply(request)
+        response = responses_pb2.ConnectionSyncResponse()
+        response.ParseFromString(response_data)
+        return response.conn_props
+
+    def open_connection(self, connection_id, info=None):
+        """Opens a new connection.
+
+        :param connection_id:
+            ID of the connection to open.
+        """
+        request = requests_pb2.OpenConnectionRequest()
+        request.connection_id = connection_id
+        if info is not None:
+            # Info is a list of repeated pairs, setting a dict directly fails
+            for k, v in info.items():
+                request.info[k] = v
+
+        response_data = self._apply(request)
+        response = responses_pb2.OpenConnectionResponse()
+        response.ParseFromString(response_data)
+
+    def close_connection(self, connection_id):
+        """Closes a connection.
+
+        :param connection_id:
+            ID of the connection to close.
+        """
+        request = requests_pb2.CloseConnectionRequest()
+        request.connection_id = connection_id
+        self._apply(request)
+
+    def create_statement(self, connection_id):
+        """Creates a new statement.
+
+        :param connection_id:
+            ID of the current connection.
+
+        :returns:
+            New statement ID.
+        """
+        request = requests_pb2.CreateStatementRequest()
+        request.connection_id = connection_id
+
+        response_data = self._apply(request)
+        response = responses_pb2.CreateStatementResponse()
+        response.ParseFromString(response_data)
+        return response.statement_id
+
+    def close_statement(self, connection_id, statement_id):
+        """Closes a statement.
+
+        :param connection_id:
+            ID of the current connection.
+
+        :param statement_id:
+            ID of the statement to close.
+        """
+        request = requests_pb2.CloseStatementRequest()
+        request.connection_id = connection_id
+        request.statement_id = statement_id
+
+        self._apply(request)
+
+    def prepare_and_execute(self, connection_id, statement_id, sql, max_rows_total=None, first_frame_max_size=None):
+        """Prepares and immediately executes a statement.
+
+        :param connection_id:
+            ID of the current connection.
+
+        :param statement_id:
+            ID of the statement to prepare.
+
+        :param sql:
+            SQL query.
+
+        :param max_rows_total:
+            The maximum number of rows that will be allowed for this query.
+
+        :param first_frame_max_size:
+            The maximum number of rows that will be returned in the first Frame returned for this query.
+
+        :returns:
+            Result set with the signature of the prepared statement and the first frame data.
+        """
+        request = requests_pb2.PrepareAndExecuteRequest()
+        request.connection_id = connection_id
+        request.statement_id = statement_id
+        request.sql = sql
+        if max_rows_total is not None:
+            request.max_rows_total = max_rows_total
+        if first_frame_max_size is not None:
+            request.first_frame_max_size = first_frame_max_size
+
+        response_data = self._apply(request, 'ExecuteResponse')
+        response = responses_pb2.ExecuteResponse()
+        response.ParseFromString(response_data)
+        return response.results
+
+    def prepare(self, connection_id, sql, max_rows_total=None):
+        """Prepares a statement.
+
+        :param connection_id:
+            ID of the current connection.
+
+        :param sql:
+            SQL query.
+
+        :param max_rows_total:
+            The maximum number of rows that will be allowed for this query.
+
+        :returns:
+            Signature of the prepared statement.
+        """
+        request = requests_pb2.PrepareRequest()
+        request.connection_id = connection_id
+        request.sql = sql
+        if max_rows_total is not None:
+            request.max_rows_total = max_rows_total
+
+        response_data = self._apply(request)
+        response = responses_pb2.PrepareResponse()
+        response.ParseFromString(response_data)
+        return response.statement
+
+    def execute(self, connection_id, statement_id, signature, parameter_values=None, first_frame_max_size=None):
+        """Returns a frame of rows.
+
+        The frame describes whether there may be another frame. If there is not
+        another frame, the current iteration is done when we have finished the
+        rows in the this frame.
+
+        :param connection_id:
+            ID of the current connection.
+
+        :param statement_id:
+            ID of the statement to fetch rows from.
+
+        :param signature:
+            common_pb2.Signature object
+
+        :param parameter_values:
+            A list of parameter values, if statement is to be executed; otherwise ``None``.
+
+        :param first_frame_max_size:
+            The maximum number of rows that will be returned in the first Frame returned for this query.
+
+        :returns:
+            Frame data, or ``None`` if there are no more.
+        """
+        request = requests_pb2.ExecuteRequest()
+        request.statementHandle.id = statement_id
+        request.statementHandle.connection_id = connection_id
+        request.statementHandle.signature.CopyFrom(signature)
+        if parameter_values is not None:
+            request.parameter_values.extend(parameter_values)
+            request.has_parameter_values = True
+        if first_frame_max_size is not None:
+            request.deprecated_first_frame_max_size = first_frame_max_size
+            request.first_frame_max_size = first_frame_max_size
+
+        response_data = self._apply(request)
+        response = responses_pb2.ExecuteResponse()
+        response.ParseFromString(response_data)
+        return response.results
+
+    def fetch(self, connection_id, statement_id, offset=0, frame_max_size=None):
+        """Returns a frame of rows.
+
+        The frame describes whether there may be another frame. If there is not
+        another frame, the current iteration is done when we have finished the
+        rows in the this frame.
+
+        :param connection_id:
+            ID of the current connection.
+
+        :param statement_id:
+            ID of the statement to fetch rows from.
+
+        :param offset:
+            Zero-based offset of first row in the requested frame.
+
+        :param frame_max_size:
+            Maximum number of rows to return; negative means no limit.
+
+        :returns:
+            Frame data, or ``None`` if there are no more.
+        """
+        request = requests_pb2.FetchRequest()
+        request.connection_id = connection_id
+        request.statement_id = statement_id
+        request.offset = offset
+        if frame_max_size is not None:
+            request.frame_max_size = frame_max_size
+
+        response_data = self._apply(request)
+        response = responses_pb2.FetchResponse()
+        response.ParseFromString(response_data)
+        return response.frame
+
+    def commit(self, connection_id):
+        """TODO Commits the transaction
+
+        :param connection_id:
+            ID of the current connection.
+        """
+        request = requests_pb2.CommitRequest()
+        request.connection_id = connection_id
+        return self._apply(request)
+
+    def rollback(self, connection_id):
+        """TODO Rolls back the transaction
+
+        :param connection_id:
+            ID of the current connection.
+        """
+        request = requests_pb2.RollbackRequest()
+        request.connection_id = connection_id
+        return self._apply(request)

+ 0 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/avatica/proto/__init__.py


Diferenças do arquivo suprimidas por serem muito extensas
+ 22 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/avatica/proto/common_pb2.py


Diferenças do arquivo suprimidas por serem muito extensas
+ 22 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/avatica/proto/requests_pb2.py


Diferenças do arquivo suprimidas por serem muito extensas
+ 22 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/avatica/proto/responses_pb2.py


+ 185 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/connection.py

@@ -0,0 +1,185 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+import uuid
+import weakref
+
+from phoenixdb import errors
+from phoenixdb.avatica.client import OPEN_CONNECTION_PROPERTIES
+from phoenixdb.cursor import Cursor
+from phoenixdb.errors import ProgrammingError
+
+__all__ = ['Connection']
+
+logger = logging.getLogger(__name__)
+
+
+class Connection(object):
+    """Database connection.
+
+    You should not construct this object manually, use :func:`~phoenixdb.connect` instead.
+    """
+
+    cursor_factory = None
+    """
+    The default cursor factory used by :meth:`cursor` if the parameter is not specified.
+    """
+
+    def __init__(self, client, cursor_factory=None, **kwargs):
+        self._client = client
+        self._closed = False
+        if cursor_factory is not None:
+            self.cursor_factory = cursor_factory
+        else:
+            self.cursor_factory = Cursor
+        self._cursors = []
+        # Extract properties to pass to OpenConnectionRequest
+        self._connection_args = {}
+        # The rest of the kwargs
+        self._filtered_args = {}
+        for k in kwargs:
+            if k in OPEN_CONNECTION_PROPERTIES:
+                self._connection_args[k] = kwargs[k]
+            else:
+                self._filtered_args[k] = kwargs[k]
+        self.open()
+        self.set_session(**self._filtered_args)
+
+    def __del__(self):
+        if not self._closed:
+            self.close()
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        if not self._closed:
+            self.close()
+
+    def open(self):
+        """Opens the connection."""
+        self._id = str(uuid.uuid4())
+        self._client.open_connection(self._id, info=self._connection_args)
+
+    def close(self):
+        """Closes the connection.
+        No further operations are allowed, either on the connection or any
+        of its cursors, once the connection is closed.
+
+        If the connection is used in a ``with`` statement, this method will
+        be automatically called at the end of the ``with`` block.
+        """
+        if self._closed:
+            raise ProgrammingError('the connection is already closed')
+        for cursor_ref in self._cursors:
+            cursor = cursor_ref()
+            if cursor is not None and not cursor._closed:
+                cursor.close()
+        self._client.close_connection(self._id)
+        self._client.close()
+        self._closed = True
+
+    @property
+    def closed(self):
+        """Read-only attribute specifying if the connection is closed or not."""
+        return self._closed
+
+    def commit(self):
+        if self._closed:
+            raise ProgrammingError('the connection is already closed')
+        self._client.commit(self._id)
+
+    def rollback(self):
+        if self._closed:
+            raise ProgrammingError('the connection is already closed')
+        self._client.rollback(self._id)
+
+    def cursor(self, cursor_factory=None):
+        """Creates a new cursor.
+
+        :param cursor_factory:
+            This argument can be used to create non-standard cursors.
+            The class returned must be a subclass of
+            :class:`~phoenixdb.cursor.Cursor` (for example :class:`~phoenixdb.cursor.DictCursor`).
+            A default factory for the connection can also be specified using the
+            :attr:`cursor_factory` attribute.
+
+        :returns:
+            A :class:`~phoenixdb.cursor.Cursor` object.
+        """
+        if self._closed:
+            raise ProgrammingError('the connection is already closed')
+        cursor = (cursor_factory or self.cursor_factory)(self)
+        self._cursors.append(weakref.ref(cursor, self._cursors.remove))
+        return cursor
+
+    def set_session(self, autocommit=None, readonly=None):
+        """Sets one or more parameters in the current connection.
+
+        :param autocommit:
+            Switch the connection to autocommit mode.
+
+        :param readonly:
+            Switch the connection to read-only mode.
+        """
+        props = {}
+        if autocommit is not None:
+            props['autoCommit'] = bool(autocommit)
+        if readonly is not None:
+            props['readOnly'] = bool(readonly)
+        props = self._client.connection_sync(self._id, props)
+        self._autocommit = props.auto_commit
+        self._readonly = props.read_only
+        self._transactionisolation = props.transaction_isolation
+
+    @property
+    def autocommit(self):
+        """Read/write attribute for switching the connection's autocommit mode."""
+        return self._autocommit
+
+    @autocommit.setter
+    def autocommit(self, value):
+        if self._closed:
+            raise ProgrammingError('the connection is already closed')
+        props = self._client.connection_sync(self._id, {'autoCommit': bool(value)})
+        self._autocommit = props.auto_commit
+
+    @property
+    def readonly(self):
+        """Read/write attribute for switching the connection's readonly mode."""
+        return self._readonly
+
+    @readonly.setter
+    def readonly(self, value):
+        if self._closed:
+            raise ProgrammingError('the connection is already closed')
+        props = self._client.connection_sync(self._id, {'readOnly': bool(value)})
+        self._readonly = props.read_only
+
+    @property
+    def transactionisolation(self):
+        return self._transactionisolation
+
+    @transactionisolation.setter
+    def transactionisolation(self, value):
+        if self._closed:
+            raise ProgrammingError('the connection is already closed')
+        props = self._client.connection_sync(self._id, {'transactionIsolation': bool(value)})
+        self._transactionisolation = props.transaction_isolation
+
+
+for name in errors.__all__:
+    setattr(Connection, name, getattr(errors, name))

+ 371 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/cursor.py

@@ -0,0 +1,371 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import collections
+import logging
+
+from phoenixdb.avatica.proto import common_pb2
+from phoenixdb.errors import InternalError, ProgrammingError
+from phoenixdb.types import TypeHelper
+
+__all__ = ['Cursor', 'ColumnDescription', 'DictCursor']
+
+logger = logging.getLogger(__name__)
+
+# TODO see note in Cursor.rowcount()
+MAX_INT = 2 ** 64 - 1
+
+ColumnDescription = collections.namedtuple('ColumnDescription', 'name type_code display_size internal_size precision scale null_ok')
+"""Named tuple for representing results from :attr:`Cursor.description`."""
+
+
+class Cursor(object):
+    """Database cursor for executing queries and iterating over results.
+
+    You should not construct this object manually, use :meth:`Connection.cursor() <phoenixdb.connection.Connection.cursor>` instead.
+    """
+
+    arraysize = 1
+    """
+    Read/write attribute specifying the number of rows to fetch
+    at a time with :meth:`fetchmany`. It defaults to 1 meaning to
+    fetch a single row at a time.
+    """
+
+    itersize = 2000
+    """
+    Read/write attribute specifying the number of rows to fetch
+    from the backend at each network roundtrip during iteration
+    on the cursor. The default is 2000.
+    """
+
+    def __init__(self, connection, id=None):
+        self._connection = connection
+        self._id = id
+        self._signature = None
+        self._column_data_types = []
+        self._frame = None
+        self._pos = None
+        self._closed = False
+        self.arraysize = self.__class__.arraysize
+        self.itersize = self.__class__.itersize
+        self._updatecount = -1
+
+    def __del__(self):
+        if not self._connection._closed and not self._closed:
+            self.close()
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        if not self._closed:
+            self.close()
+
+    def __iter__(self):
+        return self
+
+    def __next__(self):
+        row = self.fetchone()
+        if row is None:
+            raise StopIteration
+        return row
+
+    next = __next__
+
+    def close(self):
+        """Closes the cursor.
+        No further operations are allowed once the cursor is closed.
+
+        If the cursor is used in a ``with`` statement, this method will
+        be automatically called at the end of the ``with`` block.
+        """
+        if self._closed:
+            raise ProgrammingError('the cursor is already closed')
+        if self._id is not None:
+            self._connection._client.close_statement(self._connection._id, self._id)
+            self._id = None
+        self._signature = None
+        self._column_data_types = []
+        self._frame = None
+        self._pos = None
+        self._closed = True
+
+    @property
+    def closed(self):
+        """Read-only attribute specifying if the cursor is closed or not."""
+        return self._closed
+
+    @property
+    def description(self):
+        if self._signature is None:
+            return None
+        description = []
+        for column in self._signature.columns:
+            description.append(ColumnDescription(
+                column.column_name,
+                column.type.name,
+                column.display_size,
+                None,
+                column.precision,
+                column.scale,
+                None if column.nullable == 2 else bool(column.nullable),
+            ))
+        return description
+
+    def _set_id(self, id):
+        if self._id is not None and self._id != id:
+            self._connection._client.close_statement(self._connection._id, self._id)
+        self._id = id
+
+    def _set_signature(self, signature):
+        self._signature = signature
+        self._column_data_types = []
+        self._parameter_data_types = []
+        if signature is None:
+            return
+
+        for column in signature.columns:
+            dtype = TypeHelper.from_column(column)
+            self._column_data_types.append(dtype)
+
+        for parameter in signature.parameters:
+            dtype = TypeHelper.from_param(parameter)
+            self._parameter_data_types.append(dtype)
+
+    def _set_frame(self, frame):
+        self._frame = frame
+        self._pos = None
+
+        if frame is not None:
+            if frame.rows:
+                self._pos = 0
+            elif not frame.done:
+                raise InternalError('got an empty frame, but the statement is not done yet')
+
+    def _fetch_next_frame(self):
+        offset = self._frame.offset + len(self._frame.rows)
+        frame = self._connection._client.fetch(
+            self._connection._id, self._id,
+            offset=offset, frame_max_size=self.itersize)
+        self._set_frame(frame)
+
+    def _process_results(self, results):
+        if results:
+            result = results[0]
+            if result.own_statement:
+                self._set_id(result.statement_id)
+            self._set_signature(result.signature if result.HasField('signature') else None)
+            self._set_frame(result.first_frame if result.HasField('first_frame') else None)
+            self._updatecount = result.update_count
+
+    def _transform_parameters(self, parameters):
+        typed_parameters = []
+        for value, data_type in zip(parameters, self._parameter_data_types):
+            field_name, rep, mutate_to, cast_from, is_array = data_type
+            typed_value = common_pb2.TypedValue()
+
+            if value is None:
+                typed_value.null = True
+                typed_value.type = common_pb2.NULL
+            else:
+                typed_value.null = False
+                if is_array:
+                    if type(value) in [list, tuple]:
+                        for element in value:
+                            if mutate_to is not None:
+                                element = mutate_to(element)
+                            typed_element = common_pb2.TypedValue()
+                            if element is None:
+                                typed_element.null = True
+                            else:
+                                typed_element.type = rep
+                                setattr(typed_element, field_name, element)
+                            typed_value.array_value.append(typed_element)
+                        typed_value.type = common_pb2.ARRAY
+                        typed_value.component_type = rep
+                    else:
+                        raise ProgrammingError('scalar value specified for array parameter')
+                else:
+                    if mutate_to is not None:
+                        value = mutate_to(value)
+                    typed_value.type = rep
+                    setattr(typed_value, field_name, value)
+
+            typed_parameters.append(typed_value)
+        return typed_parameters
+
+    def execute(self, operation, parameters=None):
+        if self._closed:
+            raise ProgrammingError('the cursor is already closed')
+        self._updatecount = -1
+        self._set_frame(None)
+        if parameters is None:
+            if self._id is None:
+                self._set_id(self._connection._client.create_statement(self._connection._id))
+            results = self._connection._client.prepare_and_execute(
+                self._connection._id, self._id,
+                operation, first_frame_max_size=self.itersize)
+            self._process_results(results)
+        else:
+            statement = self._connection._client.prepare(
+                self._connection._id, operation)
+            self._set_id(statement.id)
+            self._set_signature(statement.signature)
+
+            results = self._connection._client.execute(
+                self._connection._id, self._id,
+                statement.signature, self._transform_parameters(parameters),
+                first_frame_max_size=self.itersize)
+            self._process_results(results)
+
+    def executemany(self, operation, seq_of_parameters):
+        if self._closed:
+            raise ProgrammingError('the cursor is already closed')
+        self._updatecount = -1
+        self._set_frame(None)
+        statement = self._connection._client.prepare(
+            self._connection._id, operation, max_rows_total=0)
+        self._set_id(statement.id)
+        self._set_signature(statement.signature)
+        for parameters in seq_of_parameters:
+            self._connection._client.execute(
+                self._connection._id, self._id,
+                statement.signature, self._transform_parameters(parameters),
+                first_frame_max_size=0)
+
+    def _transform_row(self, row):
+        """Transforms a Row into Python values.
+
+        :param row:
+            A ``common_pb2.Row`` object.
+
+        :returns:
+            A list of values casted into the correct Python types.
+
+        :raises:
+            NotImplementedError
+        """
+        tmp_row = []
+
+        for i, column in enumerate(row.value):
+            if column.scalar_value.null:
+                tmp_row.append(None)
+            elif column.has_array_value:
+                field_name, rep, mutate_to, cast_from = self._column_data_types[i]
+
+                list_value = []
+                for j, typed_value in enumerate(column.array_value):
+                    value = getattr(typed_value, field_name)
+                    if cast_from is not None:
+                        value = cast_from(value)
+                    list_value.append(value)
+
+                tmp_row.append(list_value)
+            else:
+                field_name, rep, mutate_to, cast_from = self._column_data_types[i]
+
+                # get the value from the field_name
+                value = getattr(column.scalar_value, field_name)
+
+                # cast the value
+                if cast_from is not None:
+                    value = cast_from(value)
+
+                tmp_row.append(value)
+        return tmp_row
+
+    def fetchone(self):
+        if self._frame is None:
+            raise ProgrammingError('no select statement was executed')
+        if self._pos is None:
+            return None
+        rows = self._frame.rows
+        row = self._transform_row(rows[self._pos])
+        self._pos += 1
+        if self._pos >= len(rows):
+            self._pos = None
+            if not self._frame.done:
+                self._fetch_next_frame()
+        return row
+
+    def fetchmany(self, size=None):
+        if size is None:
+            size = self.arraysize
+        rows = []
+        while size > 0:
+            row = self.fetchone()
+            if row is None:
+                break
+            rows.append(row)
+            size -= 1
+        return rows
+
+    def fetchall(self):
+        rows = []
+        while True:
+            row = self.fetchone()
+            if row is None:
+                break
+            rows.append(row)
+        return rows
+
+    def setinputsizes(self, sizes):
+        pass
+
+    def setoutputsize(self, size, column=None):
+        pass
+
+    @property
+    def connection(self):
+        """Read-only attribute providing access to the :class:`Connection <phoenixdb.connection.Connection>`
+        object this cursor was created from."""
+        return self._connection
+
+    @property
+    def rowcount(self):
+        """Read-only attribute specifying the number of rows affected by
+        the last executed DML statement or -1 if the number cannot be
+        determined. Note that this will always be set to -1 for select
+        queries."""
+        # TODO instead of -1, this ends up being set to Integer.MAX_VALUE
+        if self._updatecount == MAX_INT:
+            return -1
+        return self._updatecount
+
+    @property
+    def rownumber(self):
+        """Read-only attribute providing the current 0-based index of the
+        cursor in the result set or ``None`` if the index cannot be
+        determined.
+
+        The index can be seen as index of the cursor in a sequence
+        (the result set). The next fetch operation will fetch the
+        row indexed by :attr:`rownumber` in that sequence.
+        """
+        if self._frame is not None and self._pos is not None:
+            return self._frame.offset + self._pos
+        return self._pos
+
+
+class DictCursor(Cursor):
+    """A cursor which returns results as a dictionary"""
+
+    def _transform_row(self, row):
+        row = super(DictCursor, self)._transform_row(row)
+        d = {}
+        for ind, val in enumerate(row):
+            d[self._signature.columns[ind].column_name] = val
+        return d

+ 93 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/errors.py

@@ -0,0 +1,93 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+__all__ = [
+    'Warning', 'Error', 'InterfaceError', 'DatabaseError', 'DataError',
+    'OperationalError', 'IntegrityError', 'InternalError',
+    'ProgrammingError', 'NotSupportedError',
+]
+
+try:
+    _StandardError = StandardError
+except NameError:
+    _StandardError = Exception
+
+
+class Warning(_StandardError):
+    """Not used by this package, only defined for compatibility
+    with DB API 2.0."""
+
+
+class Error(_StandardError):
+    """Exception that is the base class of all other error exceptions.
+    You can use this to catch all errors with one single except statement."""
+
+    def __init__(self, message, code=None, sqlstate=None, cause=None):
+        super(_StandardError, self).__init__(message, code, sqlstate, cause)
+
+    @property
+    def message(self):
+        return self.args[0]
+
+    @property
+    def code(self):
+        return self.args[1]
+
+    @property
+    def sqlstate(self):
+        return self.args[2]
+
+    @property
+    def cause(self):
+        return self.args[3]
+
+
+class InterfaceError(Error):
+    """Exception raised for errors that are related to the database
+    interface rather than the database itself."""
+
+
+class DatabaseError(Error):
+    """Exception raised for errors that are related to the database."""
+
+
+class DataError(DatabaseError):
+    """Exception raised for errors that are due to problems with the
+    processed data like division by zero, numeric value out of range,
+    etc."""
+
+
+class OperationalError(DatabaseError):
+    """Raised for errors that are related to the database's operation and not
+    necessarily under the control of the programmer, e.g. an unexpected
+    disconnect occurs, the data source name is not found, a transaction could
+    not be processed, a memory allocation error occurred during
+    processing, etc."""
+
+
+class IntegrityError(DatabaseError):
+    """Raised when the relational integrity of the database is affected, e.g. a foreign key check fails."""
+
+
+class InternalError(DatabaseError):
+    """Raised when the database encounters an internal problem."""
+
+
+class ProgrammingError(DatabaseError):
+    """Raises for programming errors, e.g. table not found, syntax error, etc."""
+
+
+class NotSupportedError(DatabaseError):
+    """Raised when using an API that is not supported by the database."""

+ 257 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/sqlalchemy_phoenix.py

@@ -0,0 +1,257 @@
+# Copyright 2017 Dimitri Capitaine
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from sqlalchemy import types
+from sqlalchemy.engine.default import DefaultDialect, DefaultExecutionContext
+from sqlalchemy.exc import CompileError
+from sqlalchemy.sql.compiler import DDLCompiler
+from sqlalchemy.types import BIGINT, BOOLEAN, CHAR, DATE, DECIMAL, FLOAT, INTEGER, NUMERIC,\
+    SMALLINT, TIME, TIMESTAMP, VARBINARY, VARCHAR
+
+import phoenixdb
+import re
+import sys
+
+if sys.version_info.major == 3:
+    from urllib.parse import urlunsplit, SplitResult, urlencode
+else:
+    from urllib import urlencode
+    from urlparse import urlunsplit, SplitResult
+
+
+class PhoenixDDLCompiler(DDLCompiler):
+
+    def visit_primary_key_constraint(self, constraint):
+        if constraint.name is None:
+            raise CompileError("can't create primary key without a name")
+        return DDLCompiler.visit_primary_key_constraint(self, constraint)
+
+
+AUTOCOMMIT_REGEXP = re.compile(
+    r"\s*(?:UPDATE|UPSERT|CREATE|DELETE|DROP|ALTER)", re.I | re.UNICODE
+)
+
+
+class PhoenixExecutionContext(DefaultExecutionContext):
+
+    def should_autocommit_text(self, statement):
+        return AUTOCOMMIT_REGEXP.match(statement)
+
+
+class PhoenixDialect(DefaultDialect):
+    '''Phoenix dialect
+
+    dialect:: phoenix
+    :name: Phoenix
+
+    note::
+
+    The Phoenix dialect for SQLAlchemy is incomplete. It implements the functions required by Hue
+    for basic operation, but little else.
+
+    Connecting
+    ----------
+
+    The connection URL has the format of phoenix://host:port
+
+    This format does not allow for specifying the http scheme, or the URL path the the server uses.
+    Setting tls=True sets the server URL scheme to https.
+    If the path arg is set , it used as the path of the server URL.
+
+    The phoenix-specific authentication options can be set via the standard connect_args argument.
+
+    Connecting to an unsecure server::
+
+        create_engine('phoenix://localhost:8765')
+
+    Connecting to a secure server via SPNEGO (after kinit)::
+
+        create_engine('phoenix://localhost:8765', tls=True, connect_args={'authentication': 'SPNEGO'})
+
+    Connecting to a secure server via Knox::
+
+        create_engine('phoenix://localhost:8765', tls=True, path='/gateway/avatica/'\
+        connect_args={'authentication':'BASIC', 'avatica_user':'user', 'avatica_password':'password'})
+    '''
+
+    name = "phoenix"
+
+    driver = "phoenixdb"
+
+    ddl_compiler = PhoenixDDLCompiler
+
+    execution_ctx_cls = PhoenixExecutionContext
+
+    def __init__(self, tls=False, path='/', **opts):
+        # There is no way to pass these via the SqlAlchemy url object
+        self.tls = tls
+        self.path = path
+        super(PhoenixDialect, self).__init__(self, **opts)
+
+    @classmethod
+    def dbapi(cls):
+        return phoenixdb
+
+    def create_connect_args(self, url):
+        phoenix_url = urlunsplit(SplitResult(
+            scheme='https' if self.tls else 'http',
+            netloc='{}:{}'.format(url.host, 8765 if url.port is None else url.port),
+            path=self.path,
+            query=urlencode(url.query),
+            fragment='',
+        ))
+        return [phoenix_url], {}
+
+    def has_table(self, connection, table_name, schema=None):
+        if schema is None:
+            query = "SELECT 1 FROM system.catalog WHERE table_name = ? LIMIT 1"
+            params = [table_name.upper()]
+        else:
+            query = "SELECT 1 FROM system.catalog WHERE table_name = ? AND TABLE_SCHEM = ? LIMIT 1"
+            params = [table_name.upper(), schema.upper()]
+        return connection.execute(query, params).first() is not None
+
+    def get_schema_names(self, connection, **kw):
+        query = "SELECT DISTINCT TABLE_SCHEM FROM SYSTEM.CATALOG"
+        return [row[0] for row in connection.execute(query)]
+
+    def get_table_names(self, connection, schema=None, **kw):
+        if schema is None:
+            query = "SELECT DISTINCT table_name FROM SYSTEM.CATALOG"
+            params = []
+        else:
+            query = "SELECT DISTINCT table_name FROM SYSTEM.CATALOG WHERE TABLE_SCHEM = ? "
+            params = [schema.upper()]
+        return [row[0] for row in connection.execute(query, params)]
+
+    def get_columns(self, connection, table_name, schema=None, **kw):
+        if schema is None:
+            query = "SELECT COLUMN_NAME,  DATA_TYPE, NULLABLE " \
+                    "FROM system.catalog " \
+                    "WHERE table_name = ? " \
+                    "ORDER BY ORDINAL_POSITION"
+            params = [table_name.upper()]
+        else:
+            query = "SELECT COLUMN_NAME, DATA_TYPE, NULLABLE " \
+                    "FROM system.catalog " \
+                    "WHERE TABLE_SCHEM = ? " \
+                    "AND table_name = ? " \
+                    "ORDER BY ORDINAL_POSITION"
+            params = [schema.upper(), table_name.upper()]
+
+        # get all of the fields for this table
+        c = connection.execute(query, params)
+        cols = []
+        # first always none
+        c.fetchone()
+        while True:
+            row = c.fetchone()
+            if row is None:
+                break
+            name = row[0]
+            col_type = COLUMN_DATA_TYPE[row[1]]
+            nullable = row[2] == 1 if True else False
+
+            col_d = {
+                'name': name,
+                'type': col_type,
+                'nullable': nullable,
+                'default': None
+            }
+
+            cols.append(col_d)
+        return cols
+
+    # TODO This should be possible to implement
+    def get_pk_constraint(self, conn, table_name, schema=None, **kw):
+        return []
+
+    def get_foreign_keys(self, conn, table_name, schema=None, **kw):
+        return []
+
+    # TODO This should be possible to implement
+    def get_indexes(self, conn, table_name, schema=None, **kw):
+        return []
+
+
+class TINYINT(types.Integer):
+    __visit_name__ = "SMALLINT"
+
+
+class UNSIGNED_TINYINT(types.Integer):
+    __visit_name__ = "SMALLINT"
+
+
+class UNSIGNED_INTEGER(types.Integer):
+    __visit_name__ = "INTEGER"
+
+
+class DOUBLE(types.FLOAT):
+    __visit_name__ = "FLOAT"
+
+
+class UNSIGNED_DOUBLE(types.FLOAT):
+    __visit_name__ = "FLOAT"
+
+
+class UNSIGNED_FLOAT(types.FLOAT):
+    __visit_name__ = "FLOAT"
+
+
+class UNSIGNED_LONG(types.BIGINT):
+    __visit_name__ = "BIGINT"
+
+
+class UNSIGNED_TIME(types.TIME):
+    __visit_name__ = "TIME"
+
+
+class UNSIGNED_DATE(types.DATE):
+    __visit_name__ = "DATE"
+
+
+class UNSIGNED_TIMESTAMP(types.TIMESTAMP):
+    __visit_name__ = "TIMESTAMP"
+
+
+class ROWID (types.String):
+    __visit_name__ = "VARCHAR"
+
+
+COLUMN_DATA_TYPE = {
+    -6: TINYINT,
+    -5: BIGINT,
+    -3: VARBINARY,
+    1: CHAR,
+    2: NUMERIC,
+    3: DECIMAL,
+    4: INTEGER,
+    5: SMALLINT,
+    6: FLOAT,
+    8: DOUBLE,
+    9: UNSIGNED_INTEGER,
+    10: UNSIGNED_LONG,
+    11: UNSIGNED_TINYINT,
+    12: VARCHAR,
+    13: ROWID,
+    14: UNSIGNED_FLOAT,
+    15: UNSIGNED_DOUBLE,
+    16: BOOLEAN,
+    18: UNSIGNED_TIME,
+    19: UNSIGNED_DATE,
+    20: UNSIGNED_TIMESTAMP,
+    91: DATE,
+    92: TIME,
+    93: TIMESTAMP
+}

+ 64 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/tests/__init__.py

@@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import unittest
+
+import phoenixdb
+
+TEST_DB_URL = os.environ.get('PHOENIXDB_TEST_DB_URL', "http://localhost:8765")
+TEST_DB_TRUSTSTORE = os.environ.get('PHOENIXDB_TEST_DB_TRUSTSTORE')
+TEST_DB_AUTHENTICATION = os.environ.get('PHOENIXDB_TEST_DB_AUTHENTICATION')
+TEST_DB_AVATICA_USER = os.environ.get('PHOENIXDB_TEST_DB_AVATICA_USER')
+TEST_DB_AVATICA_PASSWORD = os.environ.get('PHOENIXDB_TEST_DB_AVATICA_PASSWORD')
+
+httpArgs = {}
+if TEST_DB_TRUSTSTORE is not None:
+    httpArgs.update(verify=TEST_DB_TRUSTSTORE)
+if TEST_DB_AUTHENTICATION is not None:
+    httpArgs.update(authentication=TEST_DB_AUTHENTICATION)
+if TEST_DB_AVATICA_USER is not None:
+    httpArgs.update(avatica_user=TEST_DB_AVATICA_USER)
+if TEST_DB_AVATICA_PASSWORD is not None:
+    httpArgs.update(avatica_password=TEST_DB_AVATICA_PASSWORD)
+
+
+@unittest.skipIf(TEST_DB_URL is None, "these tests require the PHOENIXDB_TEST_DB_URL environment variable set to a clean database")
+class DatabaseTestCase(unittest.TestCase):
+
+    def setUp(self):
+        self.conn = phoenixdb.connect(TEST_DB_URL, autocommit=True, **httpArgs)
+
+        def closeDb():
+            self.conn.close()
+        self.addCleanup(closeDb)
+
+    def reopen(self, **avaticaArgs):
+        self.conn.close()
+        kwargs = avaticaArgs.copy()
+        kwargs.update(httpArgs)
+        self.conn = phoenixdb.connect(TEST_DB_URL, **kwargs)
+
+    def addTableCleanup(self, name):
+        def dropTable():
+            with self.conn.cursor() as cursor:
+                cursor.execute("DROP TABLE IF EXISTS {table}".format(table=name))
+        self.addCleanup(dropTable)
+
+    def createTable(self, name, statement):
+        with self.conn.cursor() as cursor:
+            cursor.execute("DROP TABLE IF EXISTS {table}".format(table=name))
+            cursor.execute(statement.format(table=name))
+            self.addTableCleanup(name)

+ 857 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/tests/dbapi20.py

@@ -0,0 +1,857 @@
+#!/usr/bin/env python
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+''' Python DB API 2.0 driver compliance unit test suite. 
+    
+    This software is Public Domain and may be used without restrictions.
+
+ "Now we have booze and barflies entering the discussion, plus rumours of
+  DBAs on drugs... and I won't tell you what flashes through my mind each
+  time I read the subject line with 'Anal Compliance' in it.  All around
+  this is turning out to be a thoroughly unwholesome unit test."
+
+    -- Ian Bicking
+'''
+
+__version__ = '1.14.3'
+
+import unittest
+import time
+import sys
+
+if sys.version[0] >= '3': #python 3.x
+    _BaseException = Exception
+    def _failUnless(self, expr, msg=None):
+        self.assertTrue(expr, msg)
+else:                   #python 2.x
+    from exceptions import StandardError as _BaseException
+    def _failUnless(self, expr, msg=None):
+        self.failUnless(expr, msg)  ## deprecated since Python 2.6
+
+def str2bytes(sval):
+    if sys.version_info < (3,0) and isinstance(sval, str):
+        sval = sval.decode("latin1")
+    return sval.encode("latin1") #python 3 make unicode into bytes
+
+class DatabaseAPI20Test(unittest.TestCase):
+    ''' Test a database self.driver for DB API 2.0 compatibility.
+        This implementation tests Gadfly, but the TestCase
+        is structured so that other self.drivers can subclass this 
+        test case to ensure compiliance with the DB-API. It is 
+        expected that this TestCase may be expanded in the future
+        if ambiguities or edge conditions are discovered.
+
+        The 'Optional Extensions' are not yet being tested.
+
+        self.drivers should subclass this test, overriding setUp, tearDown,
+        self.driver, connect_args and connect_kw_args. Class specification
+        should be as follows:
+
+        import dbapi20 
+        class mytest(dbapi20.DatabaseAPI20Test):
+           [...] 
+
+        Don't 'import DatabaseAPI20Test from dbapi20', or you will
+        confuse the unit tester - just 'import dbapi20'.
+    '''
+
+    # The self.driver module. This should be the module where the 'connect'
+    # method is to be found
+    driver = None
+    connect_args = () # List of arguments to pass to connect
+    connect_kw_args = {} # Keyword arguments for connect
+    table_prefix = 'dbapi20test_' # If you need to specify a prefix for tables
+
+    ddl1 = 'create table %sbooze (name varchar(20))' % table_prefix
+    ddl2 = 'create table %sbarflys (name varchar(20), drink varchar(30))' % table_prefix
+    xddl1 = 'drop table %sbooze' % table_prefix
+    xddl2 = 'drop table %sbarflys' % table_prefix
+    insert = 'insert'
+
+    lowerfunc = 'lower' # Name of stored procedure to convert string->lowercase
+        
+    # Some drivers may need to override these helpers, for example adding
+    # a 'commit' after the execute.
+    def executeDDL1(self,cursor):
+        cursor.execute(self.ddl1)
+
+    def executeDDL2(self,cursor):
+        cursor.execute(self.ddl2)
+
+    def setUp(self):
+        ''' self.drivers should override this method to perform required setup
+            if any is necessary, such as creating the database.
+        '''
+        pass
+
+    def tearDown(self):
+        ''' self.drivers should override this method to perform required cleanup
+            if any is necessary, such as deleting the test database.
+            The default drops the tables that may be created.
+        '''
+        try:
+            con = self._connect()
+            try:
+                cur = con.cursor()
+                for ddl in (self.xddl1,self.xddl2):
+                    try:
+                        cur.execute(ddl)
+                        con.commit()
+                    except self.driver.Error:
+                        # Assume table didn't exist. Other tests will check if
+                        # execute is busted.
+                        pass
+            finally:
+                con.close()
+        except _BaseException:
+            pass
+
+    def _connect(self):
+        try:
+             r = self.driver.connect(
+                *self.connect_args,**self.connect_kw_args
+                )
+        except AttributeError:
+            self.fail("No connect method found in self.driver module")
+        return r
+
+    def test_connect(self):
+        con = self._connect()
+        con.close()
+
+    def test_apilevel(self):
+        try:
+            # Must exist
+            apilevel = self.driver.apilevel
+            # Must equal 2.0
+            self.assertEqual(apilevel,'2.0')
+        except AttributeError:
+            self.fail("Driver doesn't define apilevel")
+
+    def test_threadsafety(self):
+        try:
+            # Must exist
+            threadsafety = self.driver.threadsafety
+            # Must be a valid value
+            _failUnless(self, threadsafety in (0,1,2,3))
+        except AttributeError:
+            self.fail("Driver doesn't define threadsafety")
+
+    def test_paramstyle(self):
+        try:
+            # Must exist
+            paramstyle = self.driver.paramstyle
+            # Must be a valid value
+            _failUnless(self, paramstyle in (
+                'qmark','numeric','named','format','pyformat'
+                ))
+        except AttributeError:
+            self.fail("Driver doesn't define paramstyle")
+
+    def test_Exceptions(self):
+        # Make sure required exceptions exist, and are in the
+        # defined heirarchy.
+        if sys.version[0] == '3': #under Python 3 StardardError no longer exists
+            self.assertTrue(issubclass(self.driver.Warning,Exception))
+            self.assertTrue(issubclass(self.driver.Error,Exception))
+        else:
+            self.failUnless(issubclass(self.driver.Warning,StandardError))
+            self.failUnless(issubclass(self.driver.Error,StandardError))
+
+        _failUnless(self,
+            issubclass(self.driver.InterfaceError,self.driver.Error)
+            )
+        _failUnless(self,
+            issubclass(self.driver.DatabaseError,self.driver.Error)
+            )
+        _failUnless(self,
+            issubclass(self.driver.OperationalError,self.driver.Error)
+            )
+        _failUnless(self,
+            issubclass(self.driver.IntegrityError,self.driver.Error)
+            )
+        _failUnless(self,
+            issubclass(self.driver.InternalError,self.driver.Error)
+            )
+        _failUnless(self,
+            issubclass(self.driver.ProgrammingError,self.driver.Error)
+            )
+        _failUnless(self,
+            issubclass(self.driver.NotSupportedError,self.driver.Error)
+            )
+
+    def test_ExceptionsAsConnectionAttributes(self):
+        # OPTIONAL EXTENSION
+        # Test for the optional DB API 2.0 extension, where the exceptions
+        # are exposed as attributes on the Connection object
+        # I figure this optional extension will be implemented by any
+        # driver author who is using this test suite, so it is enabled
+        # by default.
+        con = self._connect()
+        drv = self.driver
+        _failUnless(self,con.Warning is drv.Warning)
+        _failUnless(self,con.Error is drv.Error)
+        _failUnless(self,con.InterfaceError is drv.InterfaceError)
+        _failUnless(self,con.DatabaseError is drv.DatabaseError)
+        _failUnless(self,con.OperationalError is drv.OperationalError)
+        _failUnless(self,con.IntegrityError is drv.IntegrityError)
+        _failUnless(self,con.InternalError is drv.InternalError)
+        _failUnless(self,con.ProgrammingError is drv.ProgrammingError)
+        _failUnless(self,con.NotSupportedError is drv.NotSupportedError)
+
+
+    def test_commit(self):
+        con = self._connect()
+        try:
+            # Commit must work, even if it doesn't do anything
+            con.commit()
+        finally:
+            con.close()
+
+    def test_rollback(self):
+        con = self._connect()
+        # If rollback is defined, it should either work or throw
+        # the documented exception
+        if hasattr(con,'rollback'):
+            try:
+                con.rollback()
+            except self.driver.NotSupportedError:
+                pass
+    
+    def test_cursor(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+        finally:
+            con.close()
+
+    def test_cursor_isolation(self):
+        con = self._connect()
+        try:
+            # Make sure cursors created from the same connection have
+            # the documented transaction isolation level
+            cur1 = con.cursor()
+            cur2 = con.cursor()
+            self.executeDDL1(cur1)
+            cur1.execute("%s into %sbooze values ('Victoria Bitter')" % (
+                self.insert, self.table_prefix
+                ))
+            cur2.execute("select name from %sbooze" % self.table_prefix)
+            booze = cur2.fetchall()
+            self.assertEqual(len(booze),1)
+            self.assertEqual(len(booze[0]),1)
+            self.assertEqual(booze[0][0],'Victoria Bitter')
+        finally:
+            con.close()
+
+    def test_description(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            self.executeDDL1(cur)
+            self.assertEqual(cur.description,None,
+                'cursor.description should be none after executing a '
+                'statement that can return no rows (such as DDL)'
+                )
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            self.assertEqual(len(cur.description),1,
+                'cursor.description describes too many columns'
+                )
+            self.assertEqual(len(cur.description[0]),7,
+                'cursor.description[x] tuples must have 7 elements'
+                )
+            self.assertEqual(cur.description[0][0].lower(),'name',
+                'cursor.description[x][0] must return column name'
+                )
+            self.assertEqual(cur.description[0][1],self.driver.STRING,
+                'cursor.description[x][1] must return column type. Got %r'
+                    % cur.description[0][1]
+                )
+
+            # Make sure self.description gets reset
+            self.executeDDL2(cur)
+            self.assertEqual(cur.description,None,
+                'cursor.description not being set to None when executing '
+                'no-result statements (eg. DDL)'
+                )
+        finally:
+            con.close()
+
+    def test_rowcount(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            self.executeDDL1(cur)
+            _failUnless(self,cur.rowcount in (-1,0),   # Bug #543885
+                'cursor.rowcount should be -1 or 0 after executing no-result '
+                'statements'
+                )
+            cur.execute("%s into %sbooze values ('Victoria Bitter')" % (
+                self.insert, self.table_prefix
+                ))
+            _failUnless(self,cur.rowcount in (-1,1),
+                'cursor.rowcount should == number or rows inserted, or '
+                'set to -1 after executing an insert statement'
+                )
+            cur.execute("select name from %sbooze" % self.table_prefix)
+            _failUnless(self,cur.rowcount in (-1,1),
+                'cursor.rowcount should == number of rows returned, or '
+                'set to -1 after executing a select statement'
+                )
+            self.executeDDL2(cur)
+            _failUnless(self,cur.rowcount in (-1,0),   # Bug #543885
+                'cursor.rowcount should be -1 or 0 after executing no-result '
+                'statements'
+                )
+        finally:
+            con.close()
+
+    lower_func = 'lower'
+    def test_callproc(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            if self.lower_func and hasattr(cur,'callproc'):
+                r = cur.callproc(self.lower_func,('FOO',))
+                self.assertEqual(len(r),1)
+                self.assertEqual(r[0],'FOO')
+                r = cur.fetchall()
+                self.assertEqual(len(r),1,'callproc produced no result set')
+                self.assertEqual(len(r[0]),1,
+                    'callproc produced invalid result set'
+                    )
+                self.assertEqual(r[0][0],'foo',
+                    'callproc produced invalid results'
+                    )
+        finally:
+            con.close()
+
+    def test_close(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+        finally:
+            con.close()
+
+        # cursor.execute should raise an Error if called after connection
+        # closed
+        self.assertRaises(self.driver.Error,self.executeDDL1,cur)
+
+        # connection.commit should raise an Error if called after connection'
+        # closed.'
+        self.assertRaises(self.driver.Error,con.commit)
+
+    def test_non_idempotent_close(self):
+        con = self._connect()
+        con.close()
+        # connection.close should raise an Error if called more than once
+        #!!! reasonable persons differ about the usefulness of this test and this feature !!!
+        self.assertRaises(self.driver.Error,con.close)
+
+    def test_execute(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            self._paraminsert(cur)
+        finally:
+            con.close()
+
+    def _paraminsert(self,cur):
+        self.executeDDL2(cur)
+        cur.execute("%s into %sbarflys values ('Victoria Bitter', 'thi%%s :may ca%%(u)se? troub:1e')" % (
+            self.insert, self.table_prefix
+            ))
+        _failUnless(self,cur.rowcount in (-1,1))
+
+        if self.driver.paramstyle == 'qmark':
+            cur.execute(
+                "%s into %sbarflys values (?, 'thi%%s :may ca%%(u)se? troub:1e')" % (self.insert, self.table_prefix),
+                ("Cooper's",)
+                )
+        elif self.driver.paramstyle == 'numeric':
+            cur.execute(
+                "%s into %sbarflys values (:1, 'thi%%s :may ca%%(u)se? troub:1e')" % (self.insert, self.table_prefix),
+                ("Cooper's",)
+                )
+        elif self.driver.paramstyle == 'named':
+            cur.execute(
+                "%s into %sbarflys values (:beer, 'thi%%s :may ca%%(u)se? troub:1e')" % (self.insert, self.table_prefix),
+                {'beer':"Cooper's"}
+                )
+        elif self.driver.paramstyle == 'format':
+            cur.execute(
+                "%s into %sbarflys values (%%s, 'thi%%%%s :may ca%%%%(u)se? troub:1e')" % (self.insert, self.table_prefix),
+                ("Cooper's",)
+                )
+        elif self.driver.paramstyle == 'pyformat':
+            cur.execute(
+                "%s into %sbarflys values (%%(beer)s, 'thi%%%%s :may ca%%%%(u)se? troub:1e')" % (self.insert, self.table_prefix),
+                {'beer':"Cooper's"}
+                )
+        else:
+            self.fail('Invalid paramstyle')
+        _failUnless(self,cur.rowcount in (-1,1))
+
+        cur.execute('select name, drink from %sbarflys' % self.table_prefix)
+        res = cur.fetchall()
+        self.assertEqual(len(res),2,'cursor.fetchall returned too few rows')
+        beers = [res[0][0],res[1][0]]
+        beers.sort()
+        self.assertEqual(beers[0],"Cooper's",
+            'cursor.fetchall retrieved incorrect data, or data inserted '
+            'incorrectly'
+            )
+        self.assertEqual(beers[1],"Victoria Bitter",
+            'cursor.fetchall retrieved incorrect data, or data inserted '
+            'incorrectly'
+            )
+        trouble = "thi%s :may ca%(u)se? troub:1e"
+        self.assertEqual(res[0][1], trouble,
+            'cursor.fetchall retrieved incorrect data, or data inserted '
+            'incorrectly. Got=%s, Expected=%s' % (repr(res[0][1]), repr(trouble)))      
+        self.assertEqual(res[1][1], trouble,
+            'cursor.fetchall retrieved incorrect data, or data inserted '
+            'incorrectly. Got=%s, Expected=%s' % (repr(res[1][1]), repr(trouble)
+            ))
+        
+    def test_executemany(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            self.executeDDL1(cur)
+            largs = [ ("Cooper's",) , ("Boag's",) ]
+            margs = [ {'beer': "Cooper's"}, {'beer': "Boag's"} ]
+            if self.driver.paramstyle == 'qmark':
+                cur.executemany(
+                    '%s into %sbooze values (?)' % (self.insert, self.table_prefix),
+                    largs
+                    )
+            elif self.driver.paramstyle == 'numeric':
+                cur.executemany(
+                    '%s into %sbooze values (:1)' % (self.insert, self.table_prefix),
+                    largs
+                    )
+            elif self.driver.paramstyle == 'named':
+                cur.executemany(
+                    '%s into %sbooze values (:beer)' % (self.insert, self.table_prefix),
+                    margs
+                    )
+            elif self.driver.paramstyle == 'format':
+                cur.executemany(
+                    '%s into %sbooze values (%%s)' % (self.insert, self.table_prefix),
+                    largs
+                    )
+            elif self.driver.paramstyle == 'pyformat':
+                cur.executemany(
+                    '%s into %sbooze values (%%(beer)s)' % (
+                        self.insert, self.table_prefix
+                        ),
+                    margs
+                    )
+            else:
+                self.fail('Unknown paramstyle')
+            _failUnless(self,cur.rowcount in (-1,2),
+                'insert using cursor.executemany set cursor.rowcount to '
+                'incorrect value %r' % cur.rowcount
+                )
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            res = cur.fetchall()
+            self.assertEqual(len(res),2,
+                'cursor.fetchall retrieved incorrect number of rows'
+                )
+            beers = [res[0][0],res[1][0]]
+            beers.sort()
+            self.assertEqual(beers[0],"Boag's",'incorrect data retrieved')
+            self.assertEqual(beers[1],"Cooper's",'incorrect data retrieved')
+        finally:
+            con.close()
+
+    def test_fetchone(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+
+            # cursor.fetchone should raise an Error if called before
+            # executing a select-type query
+            self.assertRaises(self.driver.Error,cur.fetchone)
+
+            # cursor.fetchone should raise an Error if called after
+            # executing a query that cannnot return rows
+            self.executeDDL1(cur)
+            self.assertRaises(self.driver.Error,cur.fetchone)
+
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            self.assertEqual(cur.fetchone(),None,
+                'cursor.fetchone should return None if a query retrieves '
+                'no rows'
+                )
+            _failUnless(self,cur.rowcount in (-1,0))
+
+            # cursor.fetchone should raise an Error if called after
+            # executing a query that cannnot return rows
+            cur.execute("%s into %sbooze values ('Victoria Bitter')" % (
+                self.insert, self.table_prefix
+                ))
+            self.assertRaises(self.driver.Error,cur.fetchone)
+
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            r = cur.fetchone()
+            self.assertEqual(len(r),1,
+                'cursor.fetchone should have retrieved a single row'
+                )
+            self.assertEqual(r[0],'Victoria Bitter',
+                'cursor.fetchone retrieved incorrect data'
+                )
+            self.assertEqual(cur.fetchone(),None,
+                'cursor.fetchone should return None if no more rows available'
+                )
+            _failUnless(self,cur.rowcount in (-1,1))
+        finally:
+            con.close()
+
+    samples = [
+        'Carlton Cold',
+        'Carlton Draft',
+        'Mountain Goat',
+        'Redback',
+        'Victoria Bitter',
+        'XXXX'
+        ]
+
+    def _populate(self):
+        ''' Return a list of sql commands to setup the DB for the fetch
+            tests.
+        '''
+        populate = [
+            "%s into %sbooze values ('%s')" % (self.insert, self.table_prefix, s)
+                for s in self.samples
+            ]
+        return populate
+
+    def test_fetchmany(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+
+            # cursor.fetchmany should raise an Error if called without
+            #issuing a query
+            self.assertRaises(self.driver.Error,cur.fetchmany,4)
+
+            self.executeDDL1(cur)
+            for sql in self._populate():
+                cur.execute(sql)
+
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            r = cur.fetchmany()
+            self.assertEqual(len(r),1,
+                'cursor.fetchmany retrieved incorrect number of rows, '
+                'default of arraysize is one.'
+                )
+            cur.arraysize=10
+            r = cur.fetchmany(3) # Should get 3 rows
+            self.assertEqual(len(r),3,
+                'cursor.fetchmany retrieved incorrect number of rows'
+                )
+            r = cur.fetchmany(4) # Should get 2 more
+            self.assertEqual(len(r),2,
+                'cursor.fetchmany retrieved incorrect number of rows'
+                )
+            r = cur.fetchmany(4) # Should be an empty sequence
+            self.assertEqual(len(r),0,
+                'cursor.fetchmany should return an empty sequence after '
+                'results are exhausted'
+            )
+            _failUnless(self,cur.rowcount in (-1,6))
+
+            # Same as above, using cursor.arraysize
+            cur.arraysize=4
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            r = cur.fetchmany() # Should get 4 rows
+            self.assertEqual(len(r),4,
+                'cursor.arraysize not being honoured by fetchmany'
+                )
+            r = cur.fetchmany() # Should get 2 more
+            self.assertEqual(len(r),2)
+            r = cur.fetchmany() # Should be an empty sequence
+            self.assertEqual(len(r),0)
+            _failUnless(self,cur.rowcount in (-1,6))
+
+            cur.arraysize=6
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            rows = cur.fetchmany() # Should get all rows
+            _failUnless(self,cur.rowcount in (-1,6))
+            self.assertEqual(len(rows),6)
+            self.assertEqual(len(rows),6)
+            rows = [r[0] for r in rows]
+            rows.sort()
+          
+            # Make sure we get the right data back out
+            for i in range(0,6):
+                self.assertEqual(rows[i],self.samples[i],
+                    'incorrect data retrieved by cursor.fetchmany'
+                    )
+
+            rows = cur.fetchmany() # Should return an empty list
+            self.assertEqual(len(rows),0,
+                'cursor.fetchmany should return an empty sequence if '
+                'called after the whole result set has been fetched'
+                )
+            _failUnless(self,cur.rowcount in (-1,6))
+
+            self.executeDDL2(cur)
+            cur.execute('select name from %sbarflys' % self.table_prefix)
+            r = cur.fetchmany() # Should get empty sequence
+            self.assertEqual(len(r),0,
+                'cursor.fetchmany should return an empty sequence if '
+                'query retrieved no rows'
+                )
+            _failUnless(self,cur.rowcount in (-1,0))
+
+        finally:
+            con.close()
+
+    def test_fetchall(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            # cursor.fetchall should raise an Error if called
+            # without executing a query that may return rows (such
+            # as a select)
+            self.assertRaises(self.driver.Error, cur.fetchall)
+
+            self.executeDDL1(cur)
+            for sql in self._populate():
+                cur.execute(sql)
+
+            # cursor.fetchall should raise an Error if called
+            # after executing a a statement that cannot return rows
+            self.assertRaises(self.driver.Error,cur.fetchall)
+
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            rows = cur.fetchall()
+            _failUnless(self,cur.rowcount in (-1,len(self.samples)))
+            self.assertEqual(len(rows),len(self.samples),
+                'cursor.fetchall did not retrieve all rows'
+                )
+            rows = [r[0] for r in rows]
+            rows.sort()
+            for i in range(0,len(self.samples)):
+                self.assertEqual(rows[i],self.samples[i],
+                'cursor.fetchall retrieved incorrect rows'
+                )
+            rows = cur.fetchall()
+            self.assertEqual(
+                len(rows),0,
+                'cursor.fetchall should return an empty list if called '
+                'after the whole result set has been fetched'
+                )
+            _failUnless(self,cur.rowcount in (-1,len(self.samples)))
+
+            self.executeDDL2(cur)
+            cur.execute('select name from %sbarflys' % self.table_prefix)
+            rows = cur.fetchall()
+            _failUnless(self,cur.rowcount in (-1,0))
+            self.assertEqual(len(rows),0,
+                'cursor.fetchall should return an empty list if '
+                'a select query returns no rows'
+                )
+            
+        finally:
+            con.close()
+    
+    def test_mixedfetch(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            self.executeDDL1(cur)
+            for sql in self._populate():
+                cur.execute(sql)
+
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            rows1  = cur.fetchone()
+            rows23 = cur.fetchmany(2)
+            rows4  = cur.fetchone()
+            rows56 = cur.fetchall()
+            _failUnless(self,cur.rowcount in (-1,6))
+            self.assertEqual(len(rows23),2,
+                'fetchmany returned incorrect number of rows'
+                )
+            self.assertEqual(len(rows56),2,
+                'fetchall returned incorrect number of rows'
+                )
+
+            rows = [rows1[0]]
+            rows.extend([rows23[0][0],rows23[1][0]])
+            rows.append(rows4[0])
+            rows.extend([rows56[0][0],rows56[1][0]])
+            rows.sort()
+            for i in range(0,len(self.samples)):
+                self.assertEqual(rows[i],self.samples[i],
+                    'incorrect data retrieved or inserted'
+                    )
+        finally:
+            con.close()
+
+    def help_nextset_setUp(self,cur):
+        ''' Should create a procedure called deleteme
+            that returns two result sets, first the 
+	    number of rows in booze then "name from booze"
+        '''
+        raise NotImplementedError('Helper not implemented')
+        #sql="""
+        #    create procedure deleteme as
+        #    begin
+        #        select count(*) from booze
+        #        select name from booze
+        #    end
+        #"""
+        #cur.execute(sql)
+
+    def help_nextset_tearDown(self,cur):
+        'If cleaning up is needed after nextSetTest'
+        raise NotImplementedError('Helper not implemented')
+        #cur.execute("drop procedure deleteme")
+
+    def test_nextset(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            if not hasattr(cur,'nextset'):
+                return
+
+            try:
+                self.executeDDL1(cur)
+                sql=self._populate()
+                for sql in self._populate():
+                    cur.execute(sql)
+
+                self.help_nextset_setUp(cur)
+
+                cur.callproc('deleteme')
+                numberofrows=cur.fetchone()
+                assert numberofrows[0]== len(self.samples)
+                assert cur.nextset()
+                names=cur.fetchall()
+                assert len(names) == len(self.samples)
+                s=cur.nextset()
+                assert s == None,'No more return sets, should return None'
+            finally:
+                self.help_nextset_tearDown(cur)
+
+        finally:
+            con.close()
+
+    def test_nextset(self):
+        raise NotImplementedError('Drivers need to override this test')
+
+    def test_arraysize(self):
+        # Not much here - rest of the tests for this are in test_fetchmany
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            _failUnless(self,hasattr(cur,'arraysize'),
+                'cursor.arraysize must be defined'
+                )
+        finally:
+            con.close()
+
+    def test_setinputsizes(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            cur.setinputsizes( (25,) )
+            self._paraminsert(cur) # Make sure cursor still works
+        finally:
+            con.close()
+
+    def test_setoutputsize_basic(self):
+        # Basic test is to make sure setoutputsize doesn't blow up
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            cur.setoutputsize(1000)
+            cur.setoutputsize(2000,0)
+            self._paraminsert(cur) # Make sure the cursor still works
+        finally:
+            con.close()
+
+    def test_setoutputsize(self):
+        # Real test for setoutputsize is driver dependant
+        raise NotImplementedError('Driver needed to override this test')
+
+    def test_None(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            self.executeDDL1(cur)
+            cur.execute("%s into %sbarflys values ('a', NULL)" % (self.insert, self.table_prefix))
+            cur.execute('select drink from %sbarflys' % self.table_prefix)
+            r = cur.fetchall()
+            self.assertEqual(len(r),1)
+            self.assertEqual(len(r[0]),1)
+            self.assertEqual(r[0][0],None,'NULL value not returned as None')
+        finally:
+            con.close()
+
+    def test_Date(self):
+        d1 = self.driver.Date(2002,12,25)
+        d2 = self.driver.DateFromTicks(time.mktime((2002,12,25,0,0,0,0,0,0)))
+        # Can we assume this? API doesn't specify, but it seems implied
+        # self.assertEqual(str(d1),str(d2))
+
+    def test_Time(self):
+        t1 = self.driver.Time(13,45,30)
+        t2 = self.driver.TimeFromTicks(time.mktime((2001,1,1,13,45,30,0,0,0)))
+        # Can we assume this? API doesn't specify, but it seems implied
+        # self.assertEqual(str(t1),str(t2))
+
+    def test_Timestamp(self):
+        t1 = self.driver.Timestamp(2002,12,25,13,45,30)
+        t2 = self.driver.TimestampFromTicks(
+            time.mktime((2002,12,25,13,45,30,0,0,0))
+            )
+        # Can we assume this? API doesn't specify, but it seems implied
+        # self.assertEqual(str(t1),str(t2))
+
+    def test_Binary(self):
+        b = self.driver.Binary(str2bytes('Something'))
+        b = self.driver.Binary(str2bytes(''))
+
+    def test_STRING(self):
+        _failUnless(self, hasattr(self.driver,'STRING'),
+            'module.STRING must be defined'
+            )
+
+    def test_BINARY(self):
+        _failUnless(self, hasattr(self.driver,'BINARY'),
+            'module.BINARY must be defined.'
+            )
+
+    def test_NUMBER(self):
+        _failUnless(self, hasattr(self.driver,'NUMBER'),
+            'module.NUMBER must be defined.'
+            )
+
+    def test_DATETIME(self):
+        _failUnless(self, hasattr(self.driver,'DATETIME'),
+            'module.DATETIME must be defined.'
+            )
+
+    def test_ROWID(self):
+        _failUnless(self, hasattr(self.driver,'ROWID'),
+            'module.ROWID must be defined.'
+            )

+ 26 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_avatica.py

@@ -0,0 +1,26 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+
+from phoenixdb.avatica.client import parse_url, urlparse
+
+
+class ParseUrlTest(unittest.TestCase):
+
+    def test_parse_url(self):
+        self.assertEqual(urlparse.urlparse('http://localhost:8765/'), parse_url('localhost'))
+        self.assertEqual(urlparse.urlparse('http://localhost:2222/'), parse_url('localhost:2222'))
+        self.assertEqual(urlparse.urlparse('http://localhost:2222/'), parse_url('http://localhost:2222/'))

+ 109 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_db.py

@@ -0,0 +1,109 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+
+import phoenixdb.cursor
+from phoenixdb.errors import InternalError
+from phoenixdb.tests import DatabaseTestCase, TEST_DB_URL
+
+
+@unittest.skipIf(TEST_DB_URL is None, "these tests require the PHOENIXDB_TEST_DB_URL environment variable set to a clean database")
+class PhoenixDatabaseTest(DatabaseTestCase):
+
+    def test_select_literal(self):
+        with self.conn.cursor() as cursor:
+            self.createTable("test", "CREATE TABLE {table} (id INTEGER PRIMARY KEY, text VARCHAR)")
+            cursor.executemany("UPSERT INTO test VALUES (?, ?)", [[i, 'text {}'.format(i)] for i in range(10)])
+
+        with self.conn.cursor() as cursor:
+            cursor.itersize = 4
+            cursor.execute("SELECT * FROM test WHERE id>1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[i, 'text {}'.format(i)] for i in range(2, 10)])
+
+    def test_select_parameter(self):
+        with self.conn.cursor() as cursor:
+            self.createTable("test", "CREATE TABLE {table} (id INTEGER PRIMARY KEY, text VARCHAR)")
+            cursor.executemany("UPSERT INTO test VALUES (?, ?)", [[i, 'text {}'.format(i)] for i in range(10)])
+
+        with self.conn.cursor() as cursor:
+            cursor.itersize = 4
+            cursor.execute("SELECT * FROM test WHERE id>? ORDER BY id", [1])
+            self.assertEqual(cursor.fetchall(), [[i, 'text {}'.format(i)] for i in range(2, 10)])
+
+    def _check_dict_cursor(self, cursor):
+        self.createTable("test", "CREATE TABLE {table} (id INTEGER PRIMARY KEY, text VARCHAR)")
+        cursor.execute("UPSERT INTO test VALUES (?, ?)", [1, 'text 1'])
+        cursor.execute("SELECT * FROM test ORDER BY id")
+        self.assertEqual(cursor.fetchall(), [{'ID': 1, 'TEXT': 'text 1'}])
+
+    def test_dict_cursor_default_parameter(self):
+        self.reopen(autocommit=True, cursor_factory=phoenixdb.cursor.DictCursor)
+
+        with self.conn.cursor() as cursor:
+            self._check_dict_cursor(cursor)
+
+    def test_dict_cursor_default_attribute(self):
+        self.conn.cursor_factory = phoenixdb.cursor.DictCursor
+
+        with self.conn.cursor() as cursor:
+            self._check_dict_cursor(cursor)
+
+    def test_dict_cursor(self):
+        self.reopen(autocommit=True, cursor_factory=phoenixdb.cursor.DictCursor)
+
+        with self.conn.cursor(cursor_factory=phoenixdb.cursor.DictCursor) as cursor:
+            self._check_dict_cursor(cursor)
+
+    def test_schema(self):
+
+        with self.conn.cursor() as cursor:
+            try:
+                cursor.execute("CREATE SCHEMA IF NOT EXISTS test_schema")
+            except InternalError as e:
+                if "phoenix.schema.isNamespaceMappingEnabled" in e.message:
+                    self.skipTest(e.message)
+                raise
+
+            self.createTable("test_schema.test", "CREATE TABLE {table} (id INTEGER PRIMARY KEY, text VARCHAR)")
+            cursor.execute("UPSERT INTO test_schema.test VALUES (?, ?)", [1, 'text 1'])
+            cursor.execute("SELECT * FROM test_schema.test ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[1, 'text 1']])
+
+    def test_transaction(self):
+        self.reopen(autocommit=False)
+        with self.conn.cursor() as cursor:
+            self.createTable("test", "CREATE TABLE {table} (id INTEGER PRIMARY KEY, text VARCHAR)")
+
+            cursor.execute("UPSERT INTO test VALUES (?, ?)", [1, 'one'])
+            cursor.execute("SELECT * FROM test ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [])
+
+            self.conn.commit()
+            cursor.execute("SELECT * FROM test ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[1, 'one']])
+            self.assertEqual(self.conn.autocommit, False)
+
+            cursor.execute("UPSERT INTO test VALUES (?, ?)", [2, 'two'])
+            self.conn.rollback()
+            cursor.execute("SELECT * FROM test ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[1, 'one']])
+            self.assertEqual(self.conn.autocommit, False)
+
+            cursor.execute("UPSERT INTO test VALUES (?, ?)", [2, 'two'])
+            # Since we expose the JDBC semantics, this is an implicit commit
+            self.conn.autocommit = True
+            cursor.execute("SELECT * FROM test ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[1, 'one'], [2, 'two']])

+ 125 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_dbapi20.py

@@ -0,0 +1,125 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+
+import phoenixdb
+from phoenixdb.tests import TEST_DB_URL, httpArgs
+
+from . import dbapi20
+
+
+@unittest.skipIf(TEST_DB_URL is None, "these tests require the PHOENIXDB_TEST_DB_URL environment variable set to a clean database")
+class PhoenixDatabaseAPI20Test(dbapi20.DatabaseAPI20Test):
+    driver = phoenixdb
+    connect_args = (TEST_DB_URL,)
+    connect_kw_args = httpArgs
+
+    ddl1 = 'create table %sbooze (name varchar(20) primary key)' % dbapi20.DatabaseAPI20Test.table_prefix
+    ddl2 = 'create table %sbarflys (name varchar(20) primary key, drink varchar(30))' % dbapi20.DatabaseAPI20Test.table_prefix
+    insert = 'upsert'
+
+    def test_nextset(self):
+        pass
+
+    def test_setoutputsize(self):
+        pass
+
+    def _connect(self):
+        con = dbapi20.DatabaseAPI20Test._connect(self)
+        con.autocommit = True
+        return con
+
+    def test_None(self):
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            self.executeDDL2(cur)
+            cur.execute("%s into %sbarflys values ('a', NULL)" % (self.insert, self.table_prefix))
+            cur.execute('select drink from %sbarflys' % self.table_prefix)
+            r = cur.fetchall()
+            self.assertEqual(len(r), 1)
+            self.assertEqual(len(r[0]), 1)
+            self.assertEqual(r[0][0], None, 'NULL value not returned as None')
+        finally:
+            con.close()
+
+    def test_autocommit(self):
+        con = dbapi20.DatabaseAPI20Test._connect(self)
+        self.assertFalse(con.autocommit)
+        con.autocommit = True
+        self.assertTrue(con.autocommit)
+        con.autocommit = False
+        self.assertFalse(con.autocommit)
+        con.close()
+
+    def test_readonly(self):
+        con = dbapi20.DatabaseAPI20Test._connect(self)
+        self.assertFalse(con.readonly)
+        con.readonly = True
+        self.assertTrue(con.readonly)
+        con.readonly = False
+        self.assertFalse(con.readonly)
+        con.close()
+
+    def test_iter(self):
+        # https://www.python.org/dev/peps/pep-0249/#iter
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            if hasattr(cur, '__iter__'):
+                self.assertIs(cur, iter(cur))
+        finally:
+            con.close()
+
+    def test_next(self):
+        # https://www.python.org/dev/peps/pep-0249/#next
+        con = self._connect()
+        try:
+            cur = con.cursor()
+            if not hasattr(cur, 'next'):
+                return
+
+            # cursor.next should raise an Error if called before
+            # executing a select-type query
+            self.assertRaises(self.driver.Error, cur.next)
+
+            # cursor.next should raise an Error if called after
+            # executing a query that cannnot return rows
+            self.executeDDL1(cur)
+            self.assertRaises(self.driver.Error, cur.next)
+
+            # cursor.next should return None if a query retrieves '
+            # no rows
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            self.assertRaises(StopIteration, cur.next)
+            self.failUnless(cur.rowcount in (-1, 0))
+
+            # cursor.next should raise an Error if called after
+            # executing a query that cannnot return rows
+            cur.execute("%s into %sbooze values ('Victoria Bitter')" % (
+                self.insert, self.table_prefix
+                ))
+            self.assertRaises(self.driver.Error, cur.next)
+
+            cur.execute('select name from %sbooze' % self.table_prefix)
+            r = cur.next()
+            self.assertEqual(len(r), 1, 'cursor.next should have retrieved a row with one column')
+            self.assertEqual(r[0], 'Victoria Bitter', 'cursor.next retrieved incorrect data')
+            # cursor.next should raise StopIteration if no more rows available
+            self.assertRaises(StopIteration, cur.next)
+            self.failUnless(cur.rowcount in (-1, 1))
+        finally:
+            con.close()

+ 60 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_errors.py

@@ -0,0 +1,60 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from phoenixdb.tests import DatabaseTestCase
+
+
+class ProgrammingErrorTest(DatabaseTestCase):
+
+    def test_invalid_sql(self):
+        with self.conn.cursor() as cursor:
+            with self.assertRaises(self.conn.ProgrammingError) as cm:
+                cursor.execute("UPS")
+            self.assertEqual("Syntax error. Encountered \"UPS\" at line 1, column 1.", cm.exception.message)
+            self.assertEqual(601, cm.exception.code)
+            self.assertEqual("42P00", cm.exception.sqlstate)
+
+
+class IntegrityErrorTest(DatabaseTestCase):
+
+    def test_null_in_pk(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key)")
+        with self.conn.cursor() as cursor:
+            with self.assertRaises(self.conn.IntegrityError) as cm:
+                cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (NULL)")
+            self.assertEqual("Constraint violation. PHOENIXDB_TEST_TBL1.ID may not be null", cm.exception.message)
+            self.assertEqual(218, cm.exception.code)
+            self.assertIn(cm.exception.sqlstate, ("22018", "23018"))
+
+
+class DataErrorTest(DatabaseTestCase):
+
+    def test_number_outside_of_range(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id tinyint primary key)")
+        with self.conn.cursor() as cursor:
+            with self.assertRaises(self.conn.DataError) as cm:
+                cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (10000)")
+            self.assertEqual("Type mismatch. TINYINT and INTEGER for 10000", cm.exception.message)
+            self.assertEqual(203, cm.exception.code)
+            self.assertEqual("22005", cm.exception.sqlstate)
+
+    def test_division_by_zero(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key)")
+        with self.conn.cursor() as cursor:
+            with self.assertRaises(self.conn.DataError) as cm:
+                cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2/0)")
+            self.assertEqual("Divide by zero.", cm.exception.message)
+            self.assertEqual(202, cm.exception.code)
+            self.assertEqual("22012", cm.exception.sqlstate)

+ 79 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_sqlalchemy.py

@@ -0,0 +1,79 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import sys
+
+import sqlalchemy as db
+from sqlalchemy import text
+
+from . import TEST_DB_URL, TEST_DB_AUTHENTICATION, TEST_DB_AVATICA_USER, TEST_DB_AVATICA_PASSWORD,\
+        TEST_DB_TRUSTSTORE
+
+if sys.version_info.major == 3:
+    from urllib.parse import urlparse, urlunparse
+else:
+    from urlparse import urlparse, urlunparse
+
+
+@unittest.skipIf(TEST_DB_URL is None, "these tests require the PHOENIXDB_TEST_DB_URL environment variable set to a clean database")
+class SQLAlchemyTest(unittest.TestCase):
+
+    def test_connection(self):
+        engine = self._create_engine()
+        # connection = engine.connect()
+        metadata = db.MetaData()
+        catalog = db.Table('CATALOG', metadata, autoload=True, autoload_with=engine)
+        self.assertIn('TABLE_NAME', catalog.columns.keys())
+
+    def test_textual(self):
+        engine = self._create_engine()
+        with engine.connect() as connection:
+            try:
+                connection.execute('drop table if exists ALCHEMY_TEST')
+                connection.execute(text('create table ALCHEMY_TEST (id integer primary key)'))
+                connection.execute(text('upsert into ALCHEMY_TEST values (42)'))
+                # SQLAlchemy autocommit should kick in
+                result = connection.execute(text('select * from ALCHEMY_TEST'))
+                row = result.fetchone()
+                self.assertEqual(row[0], 42)
+            finally:
+                connection.execute('drop table if exists ALCHEMY_TEST')
+
+    @unittest.skip("ORM feature not implemented")
+    def test_orm(self):
+        pass
+
+    def _create_engine(self):
+        ''''Massage the properties that we use for the DBAPI tests so that they apply to
+        SQLAlchemy'''
+
+        url_parts = urlparse(TEST_DB_URL)
+
+        tls = url_parts.scheme.lower == 'https'
+
+        url_parts = url_parts._replace(scheme='phoenix')
+
+        connect_args = dict()
+        if TEST_DB_AUTHENTICATION:
+            connect_args.update(authentication=TEST_DB_AUTHENTICATION)
+        if TEST_DB_AVATICA_USER:
+            connect_args.update(avatica_user=TEST_DB_AVATICA_USER)
+        if TEST_DB_AVATICA_PASSWORD:
+            connect_args.update(avatica_password=TEST_DB_AVATICA_PASSWORD)
+        if TEST_DB_TRUSTSTORE:
+            connect_args.update(trustore=TEST_DB_TRUSTSTORE)
+
+        return db.create_engine(urlunparse(url_parts), tls=tls, connect_args=connect_args)

+ 378 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/tests/test_types.py

@@ -0,0 +1,378 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import datetime
+import sys
+import unittest
+from decimal import Decimal
+
+import phoenixdb
+from phoenixdb.tests import DatabaseTestCase
+
+
+class TypesTest(DatabaseTestCase):
+
+    def checkIntType(self, type_name, min_value, max_value):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val {})".format(type_name, table="{table}"))
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, 1)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, ?)", [1])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [None])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, ?)", [min_value])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (6, ?)", [max_value])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.description[1].type_code, phoenixdb.NUMBER)
+            self.assertEqual(cursor.fetchall(), [[1, 1], [2, None], [3, 1], [4, None], [5, min_value], [6, max_value]])
+
+            self.assertRaises(
+                self.conn.DatabaseError, cursor.execute,
+                "UPSERT INTO phoenixdb_test_tbl1 VALUES (100, {})".format(min_value - 1))
+
+            self.assertRaises(
+                self.conn.DatabaseError, cursor.execute,
+                "UPSERT INTO phoenixdb_test_tbl1 VALUES (100, {})".format(max_value + 1))
+
+            # XXX The server silently truncates the values
+#            self.assertRaises(self.conn.DatabaseError, cursor.execute, "UPSERT INTO phoenixdb_test_tbl1 VALUES (100, ?)", [min_value - 1])
+#            self.assertRaises(self.conn.DatabaseError, cursor.execute, "UPSERT INTO phoenixdb_test_tbl1 VALUES (100, ?)", [max_value + 1])
+
+    def test_integer(self):
+        self.checkIntType("integer", -2147483648, 2147483647)
+
+    def test_unsigned_int(self):
+        self.checkIntType("unsigned_int", 0, 2147483647)
+
+    def test_bigint(self):
+        self.checkIntType("bigint", -9223372036854775808, 9223372036854775807)
+
+    def test_unsigned_long(self):
+        self.checkIntType("unsigned_long", 0, 9223372036854775807)
+
+    def test_tinyint(self):
+        self.checkIntType("tinyint", -128, 127)
+
+    def test_unsigned_tinyint(self):
+        self.checkIntType("unsigned_tinyint", 0, 127)
+
+    def test_smallint(self):
+        self.checkIntType("smallint", -32768, 32767)
+
+    def test_unsigned_smallint(self):
+        self.checkIntType("unsigned_smallint", 0, 32767)
+
+    def checkFloatType(self, type_name, min_value, max_value):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val {})".format(type_name, table="{table}"))
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, 1)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, ?)", [1])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [None])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, ?)", [min_value])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (6, ?)", [max_value])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.description[1].type_code, phoenixdb.NUMBER)
+            rows = cursor.fetchall()
+            self.assertEqual([r[0] for r in rows], [1, 2, 3, 4, 5, 6])
+            self.assertEqual(rows[0][1], 1.0)
+            self.assertEqual(rows[1][1], None)
+            self.assertEqual(rows[2][1], 1.0)
+            self.assertEqual(rows[3][1], None)
+            self.assertAlmostEqual(rows[4][1], min_value)
+            self.assertAlmostEqual(rows[5][1], max_value)
+
+    def test_float(self):
+        self.checkFloatType("float", -3.4028234663852886e+38, 3.4028234663852886e+38)
+
+    def test_unsigned_float(self):
+        self.checkFloatType("unsigned_float", 0, 3.4028234663852886e+38)
+
+    def test_double(self):
+        self.checkFloatType("double", -1.7976931348623158E+308, 1.7976931348623158E+308)
+
+    def test_unsigned_double(self):
+        self.checkFloatType("unsigned_double", 0, 1.7976931348623158E+308)
+
+    def test_decimal(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val decimal(8,3))")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, 33333.333)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, ?)", [33333.333])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [Decimal('33333.333')])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, ?)", [None])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.description[1].type_code, phoenixdb.NUMBER)
+            rows = cursor.fetchall()
+            self.assertEqual([r[0] for r in rows], [1, 2, 3, 4, 5])
+            self.assertEqual(rows[0][1], Decimal('33333.333'))
+            self.assertEqual(rows[1][1], None)
+            self.assertEqual(rows[2][1], Decimal('33333.333'))
+            self.assertEqual(rows[3][1], Decimal('33333.333'))
+            self.assertEqual(rows[4][1], None)
+            self.assertRaises(
+                self.conn.DatabaseError, cursor.execute,
+                "UPSERT INTO phoenixdb_test_tbl1 VALUES (100, ?)", [Decimal('1234567890')])
+            self.assertRaises(
+                self.conn.DatabaseError, cursor.execute,
+                "UPSERT INTO phoenixdb_test_tbl1 VALUES (101, ?)", [Decimal('123456.789')])
+
+    def test_boolean(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val boolean)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, TRUE)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, FALSE)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [True])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, ?)", [False])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (6, ?)", [None])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.description[1].type_code, phoenixdb.BOOLEAN)
+            self.assertEqual(cursor.fetchall(), [[1, True], [2, False], [3, None], [4, True], [5, False], [6, None]])
+
+    @unittest.skip("https://issues.apache.org/jira/browse/PHOENIX-4664")
+    def test_time(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val time)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, '1970-01-01 12:01:02')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, ?)", [phoenixdb.Time(12, 1, 2)])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [datetime.time(12, 1, 2)])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, ?)", [None])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [
+                [1, datetime.time(12, 1, 2)],
+                [2, None],
+                [3, datetime.time(12, 1, 2)],
+                [4, datetime.time(12, 1, 2)],
+                [5, None],
+            ])
+
+    @unittest.skip("https://issues.apache.org/jira/browse/CALCITE-797")
+    def test_time_full(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val time)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, '2015-07-12 13:01:02.123')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, ?)", [datetime.datetime(2015, 7, 12, 13, 1, 2, 123000)])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [
+                [1, datetime.datetime(2015, 7, 12, 13, 1, 2, 123000)],
+                [2, datetime.datetime(2015, 7, 12, 13, 1, 2, 123000)],
+            ])
+
+    @unittest.skip("https://issues.apache.org/jira/browse/PHOENIX-4664")
+    def test_date(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val date)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, '2015-07-12 00:00:00')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, ?)", [phoenixdb.Date(2015, 7, 12)])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [datetime.date(2015, 7, 12)])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [
+                [1, datetime.date(2015, 7, 12)],
+                [3, datetime.date(2015, 7, 12)],
+                [4, datetime.date(2015, 7, 12)],
+            ])
+
+    @unittest.skip("https://issues.apache.org/jira/browse/CALCITE-798")
+    def test_date_full(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val date)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, '2015-07-12 13:01:02.123')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, ?)", [datetime.datetime(2015, 7, 12, 13, 1, 2, 123000)])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [
+                [1, datetime.datetime(2015, 7, 12, 13, 1, 2, 123000)],
+                [2, datetime.datetime(2015, 7, 12, 13, 1, 2, 123000)],
+            ])
+
+    def test_date_null(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val date)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, ?)", [None])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")  # raises NullPointerException on the server
+            self.assertEqual(cursor.fetchall(), [
+                [1, None],
+                [2, None],
+            ])
+
+    @unittest.skip("https://issues.apache.org/jira/browse/PHOENIX-4664")
+    def test_timestamp(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val timestamp)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, '2015-07-12 13:01:02.123')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, ?)", [phoenixdb.Timestamp(2015, 7, 12, 13, 1, 2)])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [datetime.datetime(2015, 7, 12, 13, 1, 2, 123000)])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, ?)", [None])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [
+                [1, datetime.datetime(2015, 7, 12, 13, 1, 2, 123000)],
+                [2, None],
+                [3, datetime.datetime(2015, 7, 12, 13, 1, 2)],
+                [4, datetime.datetime(2015, 7, 12, 13, 1, 2, 123000)],
+                [5, None],
+            ])
+
+    # Minimal date/time/timestamp type test that doesn't trigger PHOENIX-4664
+    def test_time_minimal(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val1 date, val2 time, val3 timestamp)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, '2015-07-12', '2015-07-12 13:01:02', '2015-07-12 13:01:02.123')")
+            cursor.execute("SELECT * FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [
+                [1, datetime.date(2015, 7, 12), datetime.time(13, 1, 2), datetime.datetime(2015, 7, 12, 13, 1, 2, 123000)]
+            ])
+
+    @unittest.skip("https://issues.apache.org/jira/browse/CALCITE-796")
+    def test_timestamp_full(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val timestamp)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, '2015-07-12 13:01:02.123456789')")
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [
+                [1, datetime.datetime(2015, 7, 12, 13, 1, 2, 123456789)],
+            ])
+
+    def test_varchar(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val varchar)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, 'abc')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, ?)", ['abc'])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [None])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, '')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (6, ?)", [''])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[1, 'abc'], [2, None], [3, 'abc'], [4, None], [5, None], [6, None]])
+
+    @unittest.skipIf(sys.version_info[0] < 3, "phoenixdb doesn't support unicode strings in Python2")
+    def test_unicode(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val varchar)")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, \
+            '\u00E1rv\u00EDzt\u0171r\u0151 t\u00FCk\u00F6rf\u00FAr\u00F3g\u00E9p')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, '\u265E')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, '\U0001F600')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)",
+                           ['\u00E1rv\u00EDzt\u0171r\u0151 t\u00FCk\u00F6rf\u00FAr\u00F3g\u00E9p'])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, ?)", ['\u265E'])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (6, ?)", ['\U0001F600'])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(),
+                             [[1, '\u00E1rv\u00EDzt\u0171r\u0151 t\u00FCk\u00F6rf\u00FAr\u00F3g\u00E9p'],
+                              [2, '\u265E'], [3, '\U0001F600'],
+                              [4, '\u00E1rv\u00EDzt\u0171r\u0151 t\u00FCk\u00F6rf\u00FAr\u00F3g\u00E9p'],
+                              [5, '\u265E'], [6, '\U0001F600']])
+
+    def test_varchar_very_long(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val varchar)")
+        with self.conn.cursor() as cursor:
+            value = '1234567890' * 1000
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, ?)", [value])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[1, value]])
+
+    def test_varchar_limited(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val varchar(2))")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, 'ab')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, ?)", ['ab'])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [None])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, '')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (6, ?)", [''])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[1, 'ab'], [2, None], [3, 'ab'], [4, None], [5, None], [6, None]])
+            self.assertRaises(self.conn.DataError, cursor.execute, "UPSERT INTO phoenixdb_test_tbl1 VALUES (100, 'abc')")
+
+    def test_char_null(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val char(2))")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [None])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, '')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (6, ?)", [''])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[2, None], [4, None], [5, None], [6, None]])
+            self.assertRaises(self.conn.DataError, cursor.execute, "UPSERT INTO phoenixdb_test_tbl1 VALUES (100, 'abc')")
+
+    def test_char(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val char(2))")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, 'ab')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, ?)", ['ab'])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, 'a')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", ['b'])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[1, 'ab'], [2, 'ab'], [3, 'a'], [4, 'b']])
+            self.assertRaises(self.conn.DataError, cursor.execute, "UPSERT INTO phoenixdb_test_tbl1 VALUES (100, 'abc')")
+
+    def test_binary(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val binary(2))")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, 'ab')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, ?)", [phoenixdb.Binary(b'ab')])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, '\x01\x00')")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [phoenixdb.Binary(b'\x01\x00')])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [
+                [1, b'ab'],
+                [2, b'ab'],
+                [3, b'\x01\x00'],
+                [4, b'\x01\x00'],
+            ])
+
+    def test_binary_all_bytes(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val binary(256))")
+        with self.conn.cursor() as cursor:
+            if sys.version_info[0] < 3:
+                value = ''.join(map(chr, range(256)))
+            else:
+                value = bytes(range(256))
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, ?)", [phoenixdb.Binary(value)])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [[1, value]])
+
+    def test_array(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val integer[])")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, ARRAY[1, 2])")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, ?)", [[2, 3]])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, ?)", [[4]])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [
+                [1, [1, 2]],
+                [2, [2, 3]],
+                [3, [4]],
+            ])
+
+    def test_array_boolean(self):
+        self.createTable("phoenixdb_test_tbl1", "CREATE TABLE {table} (id integer primary key, val boolean[])")
+        with self.conn.cursor() as cursor:
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (1, ARRAY[TRUE, TRUE, FALSE])")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (2, NULL)")
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (3, ?)", ((1, 0, 1),))
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (4, ?)", [[True, True, True]])
+            cursor.execute("UPSERT INTO phoenixdb_test_tbl1 VALUES (5, ?)", [[]])
+            cursor.execute("SELECT id, val FROM phoenixdb_test_tbl1 ORDER BY id")
+            self.assertEqual(cursor.fetchall(), [
+                [1, [True, True, False]],
+                [2, None],
+                [3, [True, False, True]],
+                [4, [True, True, True]],
+                [5, None]
+            ])

+ 304 - 0
desktop/core/ext-py/phoenixdb/phoenixdb/types.py

@@ -0,0 +1,304 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import datetime
+import sys
+import time
+from decimal import Decimal
+
+from phoenixdb.avatica.proto import common_pb2
+
+
+__all__ = [
+    'Date', 'Time', 'Timestamp', 'DateFromTicks', 'TimeFromTicks', 'TimestampFromTicks',
+    'Binary', 'STRING', 'BINARY', 'NUMBER', 'DATETIME', 'ROWID', 'BOOLEAN',
+    'TypeHelper',
+]
+
+
+def Date(year, month, day):
+    """Constructs an object holding a date value."""
+    return datetime.date(year, month, day)
+
+
+def Time(hour, minute, second):
+    """Constructs an object holding a time value."""
+    return datetime.time(hour, minute, second)
+
+
+def Timestamp(year, month, day, hour, minute, second):
+    """Constructs an object holding a datetime/timestamp value."""
+    return datetime.datetime(year, month, day, hour, minute, second)
+
+
+def DateFromTicks(ticks):
+    """Constructs an object holding a date value from the given UNIX timestamp."""
+    return Date(*time.localtime(ticks)[:3])
+
+
+def TimeFromTicks(ticks):
+    """Constructs an object holding a time value from the given UNIX timestamp."""
+    return Time(*time.localtime(ticks)[3:6])
+
+
+def TimestampFromTicks(ticks):
+    """Constructs an object holding a datetime/timestamp value from the given UNIX timestamp."""
+    return Timestamp(*time.localtime(ticks)[:6])
+
+
+def Binary(value):
+    """Constructs an object capable of holding a binary (long) string value."""
+    return bytes(value)
+
+
+def time_from_java_sql_time(n):
+    dt = datetime.datetime(1970, 1, 1) + datetime.timedelta(milliseconds=n)
+    return dt.time()
+
+
+def time_to_java_sql_time(t):
+    return ((t.hour * 60 + t.minute) * 60 + t.second) * 1000 + t.microsecond // 1000
+
+
+def date_from_java_sql_date(n):
+    return datetime.date(1970, 1, 1) + datetime.timedelta(days=n)
+
+
+def date_to_java_sql_date(d):
+    if isinstance(d, datetime.datetime):
+        d = d.date()
+    td = d - datetime.date(1970, 1, 1)
+    return td.days
+
+
+def datetime_from_java_sql_timestamp(n):
+    return datetime.datetime(1970, 1, 1) + datetime.timedelta(milliseconds=n)
+
+
+def datetime_to_java_sql_timestamp(d):
+    td = d - datetime.datetime(1970, 1, 1)
+    return td.microseconds // 1000 + (td.seconds + td.days * 24 * 3600) * 1000
+
+
+# FIXME This doesn't seem to be used anywhere in the code
+class ColumnType(object):
+
+    def __init__(self, eq_types):
+        self.eq_types = tuple(eq_types)
+        self.eq_types_set = set(eq_types)
+
+    def __eq__(self, other):
+        return other in self.eq_types_set
+
+    def __cmp__(self, other):
+        if other in self.eq_types_set:
+            return 0
+        if other < self.eq_types:
+            return 1
+        else:
+            return -1
+
+
+STRING = ColumnType(['VARCHAR', 'CHAR'])
+"""Type object that can be used to describe string-based columns."""
+
+BINARY = ColumnType(['BINARY', 'VARBINARY'])
+"""Type object that can be used to describe (long) binary columns."""
+
+NUMBER = ColumnType([
+    'INTEGER', 'UNSIGNED_INT', 'BIGINT', 'UNSIGNED_LONG', 'TINYINT', 'UNSIGNED_TINYINT',
+    'SMALLINT', 'UNSIGNED_SMALLINT', 'FLOAT', 'UNSIGNED_FLOAT', 'DOUBLE', 'UNSIGNED_DOUBLE', 'DECIMAL'
+])
+"""Type object that can be used to describe numeric columns."""
+
+DATETIME = ColumnType(['TIME', 'DATE', 'TIMESTAMP', 'UNSIGNED_TIME', 'UNSIGNED_DATE', 'UNSIGNED_TIMESTAMP'])
+"""Type object that can be used to describe date/time columns."""
+
+ROWID = ColumnType([])
+"""Only implemented for DB API 2.0 compatibility, not used."""
+
+BOOLEAN = ColumnType(['BOOLEAN'])
+"""Type object that can be used to describe boolean columns. This is a phoenixdb-specific extension."""
+
+if sys.version_info[0] < 3:
+    _long = long  # noqa: F821
+else:
+    _long = int
+
+FIELD_MAP = {
+    'bool_value': [
+        (common_pb2.BOOLEAN, None, None),
+        (common_pb2.PRIMITIVE_BOOLEAN, None, None),
+    ],
+    'string_value': [
+        (common_pb2.CHARACTER, None, None),
+        (common_pb2.PRIMITIVE_CHAR, None, None),
+        (common_pb2.STRING, None, None),
+        (common_pb2.BIG_DECIMAL, str, Decimal),
+    ],
+    'number_value': [
+        (common_pb2.INTEGER, None, int),
+        (common_pb2.PRIMITIVE_INT, None, int),
+        (common_pb2.SHORT, None, int),
+        (common_pb2.PRIMITIVE_SHORT, None, int),
+        (common_pb2.LONG, None, _long),
+        (common_pb2.PRIMITIVE_LONG, None, _long),
+        (common_pb2.BYTE, None, int),
+        (common_pb2.JAVA_SQL_TIME, time_to_java_sql_time, time_from_java_sql_time),
+        (common_pb2.JAVA_SQL_DATE, date_to_java_sql_date, date_from_java_sql_date),
+        (common_pb2.JAVA_SQL_TIMESTAMP, datetime_to_java_sql_timestamp, datetime_from_java_sql_timestamp),
+    ],
+    'bytes_value': [
+        (common_pb2.BYTE_STRING, Binary, None),
+    ],
+    'double_value': [
+        (common_pb2.DOUBLE, float, float),
+        (common_pb2.PRIMITIVE_DOUBLE, float, float)
+    ]
+}
+"""The master map that describes how to handle types, keyed by TypedData field"""
+
+REP_MAP = dict((v[0], (k, v[0], v[1], v[2])) for k in FIELD_MAP for v in FIELD_MAP[k])
+"""Flips the available types to allow for faster lookup by protobuf Rep
+
+This mapping should be structured as:
+    {
+        'common_pb2.BIG_DECIMAL': ('string_value', common_pb2.BIG_DECIMAL, str, Decimal),),
+        ...
+        '<Rep enum>': (<field_name>, <mutate_to function>, <cast_from function>),
+    }
+"""
+
+JDBC_TO_REP = dict([
+    # These are the standard types that are used in Phoenix
+    (-6, common_pb2.BYTE),  # TINYINT
+    (5, common_pb2.SHORT),  # SMALLINT
+    (4, common_pb2.INTEGER),  # INTEGER
+    (-5, common_pb2.LONG),  # BIGINT
+    (6, common_pb2.DOUBLE),  # FLOAT
+    (8, common_pb2.DOUBLE),  # DOUBLE
+    (2, common_pb2.BIG_DECIMAL),  # NUMERIC
+    (1, common_pb2.STRING),  # CHAR
+    (91, common_pb2.JAVA_SQL_DATE),  # DATE
+    (92, common_pb2.JAVA_SQL_TIME),  # TIME
+    (93, common_pb2.JAVA_SQL_TIMESTAMP),  # TIMESTAMP
+    (-2, common_pb2.BYTE_STRING),  # BINARY
+    (-3, common_pb2.BYTE_STRING),  # VARBINARY
+    (16, common_pb2.BOOLEAN),  # BOOLEAN
+    # These are the Non-standard types defined by Phoenix
+    (19, common_pb2.JAVA_SQL_DATE),  # UNSIGNED_DATE
+    (15, common_pb2.DOUBLE),  # UNSIGNED_DOUBLE
+    (14, common_pb2.DOUBLE),  # UNSIGNED_FLOAT
+    (9, common_pb2.INTEGER),  # UNSIGNED_INT
+    (10, common_pb2.LONG),  # UNSIGNED_LONG
+    (13, common_pb2.SHORT),  # UNSIGNED_SMALLINT
+    (20, common_pb2.JAVA_SQL_TIMESTAMP),  # UNSIGNED_TIMESTAMP
+    (11, common_pb2.BYTE),  # UNSIGNED_TINYINT
+    # The following are not used by Phoenix, but some of these are used by Avaticafor
+    # parameter types
+    (-7, common_pb2.BOOLEAN),  # BIT
+    (7, common_pb2.DOUBLE),  # REAL
+    (3, common_pb2.BIG_DECIMAL),  # DECIMAL
+    (12, common_pb2.STRING),  # VARCHAR
+    (-1, common_pb2.STRING),  # LONGVARCHAR
+    (-4, common_pb2.BYTE_STRING),  # LONGVARBINARY
+    (2004, common_pb2.BYTE_STRING),  # BLOB
+    (2005, common_pb2.STRING),  # CLOB
+    (-15, common_pb2.STRING),  # NCHAR
+    (-9, common_pb2.STRING),  # NVARCHAR
+    (-16, common_pb2.STRING),  # LONGNVARCHAR
+    (2011, common_pb2.STRING),  # NCLOB
+    (2009, common_pb2.STRING),  # SQLXML
+    # These are defined by JDBC, but cannot be mapped
+    # NULL
+    # OTHER
+    # JAVA_OBJECT
+    # DISTINCT
+    # STRUCT
+    # ARRAY 2003 - We are handling this as a special case
+    # REF
+    # DATALINK
+    # ROWID
+    # REF_CURSOR
+    # TIME WITH TIMEZONE
+    # TIMESTAMP WITH TIMEZONE
+
+    ])
+"""Maps the JDBC Type IDs to Protobuf Reps """
+
+JDBC_MAP = {}
+for k, v in JDBC_TO_REP.items():
+    JDBC_MAP[k & 0xffffffff] = REP_MAP[v]
+"""Flips the available types to allow for faster lookup by JDBC type ID
+
+It has the same format as REP_MAP, but is keyed by JDBC type ID
+"""
+
+
+class TypeHelper(object):
+
+    @staticmethod
+    def from_param(param):
+        """Retrieves a field name and functions to cast to/from based on an AvaticaParameter object
+
+        :param param:
+            Protobuf AvaticaParameter object
+
+        :returns: tuple ``(field_name, rep, mutate_to, cast_from, is_array)``
+            WHERE
+            ``field_name`` is the attribute in ``common_pb2.TypedValue``
+            ``rep`` is the common_pb2.Rep enum
+            ``mutate_to`` is the function to cast values into Phoenix values, if any
+            ``cast_from`` is the function to cast from the Phoenix value to the Python value, if any
+            ``is_array`` the param expects an array instead of scalar
+
+        :raises:
+            NotImplementedError
+        """
+        jdbc_code = param.parameter_type
+        if jdbc_code > 2900 and jdbc_code < 3100:
+            return TypeHelper._from_jdbc(jdbc_code-3000) + (True,)
+        else:
+            return TypeHelper._from_jdbc(jdbc_code) + (False,)
+
+    @staticmethod
+    def from_column(column):
+        """Retrieves a field name and functions to cast to/from based on a TypedValue object
+
+        :param column:
+            Protobuf TypedValue object
+
+        :returns: tuple ``(field_name, rep, mutate_to, cast_from)``
+            WHERE
+            ``field_name`` is the attribute in ``common_pb2.TypedValue``
+            ``rep`` is the common_pb2.Rep enum
+            ``mutate_to`` is the function to cast values into Phoenix values, if any
+            ``cast_from`` is the function to cast from the Phoenix value to the Python value, if any
+
+        :raises:
+            NotImplementedError
+        """
+        if column.type.id == 2003:
+            return TypeHelper._from_jdbc(column.type.component.id)
+        else:
+            return TypeHelper._from_jdbc(column.type.id)
+
+    @staticmethod
+    def _from_jdbc(jdbc_code):
+        if jdbc_code not in JDBC_MAP:
+            # This should not happen. It's either a bug, or Avatica has added new types
+            raise NotImplementedError('JDBC TYPE CODE {} is not supported'.format(jdbc_code))
+
+        return JDBC_MAP[jdbc_code]

+ 21 - 0
desktop/core/ext-py/phoenixdb/requirements.txt

@@ -0,0 +1,21 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+protobuf>=3.0.0
+requests
+requests-gssapi

+ 34 - 0
desktop/core/ext-py/phoenixdb/setup.cfg

@@ -0,0 +1,34 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+[nosetests]
+verbosity=2
+testmatch=^test_.+
+where=phoenixdb/tests
+
+[build_sphinx]
+source-dir = doc
+build-dir = doc/build
+all_files = 1
+
+[upload_sphinx]
+upload-dir = doc/build/html
+
+[flake8]
+max-line-length = 140
+exclude =
+  e,e3,env,venv,doc,build,dist,.tox,.idea,
+  ./phoenixdb/tests/dbapi20.py,
+  ./phoenixdb/avatica/proto/*_pb2.py

+ 108 - 0
desktop/core/ext-py/phoenixdb/setup.py

@@ -0,0 +1,108 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+from setuptools import setup, find_packages
+import setuptools
+import sys
+
+cmdclass = {}
+
+try:
+    from sphinx.setup_command import BuildDoc
+    cmdclass['build_sphinx'] = BuildDoc
+except ImportError:
+    pass
+
+
+def readme():
+    with open('README.rst') as f:
+        return f.read()
+
+
+if setuptools.__version__ < '20.8.1':
+    # Workaround for source install on old setuptools
+    # This won't be able to create a proper multi-version pacakage
+    install_requires=[
+        'protobuf>=3.0.0',
+        'requests',
+        'requests-gssapi',
+        'SQLAlchemy'
+    ]
+    if sys.version_info < (3,6):
+        install_requires.append('gssapi<1.6.0')
+    #Don't build the docs on an old stack
+    setup_requires=[]
+else:
+    install_requires=[
+        'protobuf>=3.0.0',
+        'requests',
+        'requests-gssapi',
+        'gssapi<1.6.0;python_version<"3.6"',
+        'SQLAlchemy'
+    ]
+    setup_requires=[
+        'Sphinx;python_version>="3.6"',
+    ],
+
+version = "1.0.0.dev"
+
+setup(
+    name="phoenixdb",
+    version=version,
+    description="Phoenix database adapter for Python",
+    long_description=readme(),
+    author="Apache Software Foundation",
+    author_email="dev@phoenix.apache.org",
+    url="http://phoenix.apache.org/python.html",
+    license="Apache 2",
+    packages=find_packages(),
+    include_package_data=True,
+    cmdclass=cmdclass,
+    command_options={
+        'build_sphinx': {
+            'version': ('setup.py', version),
+            'release': ('setup.py', version),
+        },
+    },
+    classifiers=[
+        'Programming Language :: Python',
+        'Programming Language :: Python :: 2',
+        'Programming Language :: Python :: 2.7',
+        'Programming Language :: Python :: 3',
+        'Programming Language :: Python :: 3.4',
+        'Programming Language :: Python :: 3.5',
+        'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
+    ],
+    install_requires=install_requires,
+    extras_require={
+        'SQLAlchemy': ['SQLAlchemy'],
+    },
+    tests_require=[
+        'SQLAlchemy',
+        'nose',
+        'flake8'
+    ],
+    setup_requires=setup_requires,
+    entry_points={
+        "sqlalchemy.dialects": [
+            "phoenix = phoenixdb.sqlalchemy_phoenix:PhoenixDialect"
+        ]
+    },
+)

+ 25 - 0
desktop/core/ext-py/phoenixdb/tox.ini

@@ -0,0 +1,25 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+[tox]
+envlist = py27,py34,py35,py36,py37,py38
+[testenv]
+passenv = PHOENIXDB_TEST_DB_URL
+commands =
+  flake8 phoenixdb  
+  nosetests -v
+deps = -rrequirements.txt
+  nose
+  flake8

Alguns arquivos não foram mostrados porque muitos arquivos mudaram nesse diff