| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210 |
- ========================
- Developing new Dialects
- ========================
- .. note::
- When studying this file, it's probably a good idea to also
- familiarize with the README.unittests.rst file, which discusses
- SQLAlchemy's usage and extension of the Nose test runner.
- While SQLAlchemy includes many dialects within the core distribution, the
- trend for new dialects should be that they are published as external
- projects. SQLAlchemy has since version 0.5 featured a "plugin" system
- which allows external dialects to be integrated into SQLAlchemy using
- standard setuptools entry points. As of version 0.8, this system has
- been enhanced, so that a dialect can also be "plugged in" at runtime.
- On the testing side, SQLAlchemy as of 0.8 also includes a "dialect
- compliance suite" that is usable by third party libraries. There is no
- longer a strong need for a new dialect to run through SQLAlchemy's full
- testing suite, as a large portion of these tests do not have
- dialect-sensitive functionality. The "dialect compliance suite" should
- be viewed as the primary target for new dialects, and as it continues
- to grow and mature it should become a more thorough and efficient system
- of testing new dialects.
- As of SQLAlchemy 0.9.4, both nose and pytest are supported for running tests,
- and pytest is now preferred.
- Dialect Layout
- ===============
- The file structure of a dialect is typically similar to the following::
- sqlalchemy-<dialect>/
- setup.py
- setup.cfg
- sqlalchemy_<dialect>/
- __init__.py
- base.py
- <dbapi>.py
- requirements.py
- test/
- conftest.py
- __init__.py
- test_suite.py
- test_<dialect_specific_test>.py
- ...
- An example of this structure can be seen in the Access dialect at
- https://bitbucket.org/zzzeek/sqlalchemy-access/.
- Key aspects of this file layout include:
- * setup.py - should specify setuptools entrypoints, allowing the
- dialect to be usable from create_engine(), e.g.::
- entry_points={
- 'sqlalchemy.dialects': [
- 'access = sqlalchemy_access.pyodbc:AccessDialect_pyodbc',
- 'access.pyodbc = sqlalchemy_access.pyodbc:AccessDialect_pyodbc',
- ]
- }
- Above, the two entrypoints ``access`` and ``access.pyodbc`` allow URLs to be
- used such as::
- create_engine("access://user:pw@dsn")
- create_engine("access+pyodbc://user:pw@dsn")
- * setup.cfg - this file contains the traditional contents such as [egg_info],
- [pytest] and [nosetests] directives, but also contains new directives that are used
- by SQLAlchemy's testing framework. E.g. for Access::
- [egg_info]
- tag_build = dev
- [tool:pytest]
- addopts= --tb native -v -r fxX --maxfail=25 -p no:warnings
- python_files=test/*test_*.py
- [sqla_testing]
- requirement_cls=sqlalchemy_access.requirements:Requirements
- profile_file=test/profiles.txt
- [db]
- default=access+pyodbc://admin@access_test
- sqlite=sqlite:///:memory:
- Above, the ``[sqla_testing]`` section contains configuration used by
- SQLAlchemy's test plugin. The ``[tool:pytest]`` section
- include directives to help with these runners; in the case of
- Nose, the directive ``with-sql_testing = true``, which indicates to Nose that
- the SQLAlchemy nose plugin should be used. In the case of pytest, the
- test/conftest.py file will bootstrap SQLAlchemy's plugin.
- * test/conftest.py - This script bootstraps SQLAlchemy's pytest plugin
- into the pytest runner. This
- script can also be used to install your third party dialect into
- SQLAlchemy without using the setuptools entrypoint system; this allows
- your dialect to be present without any explicit setup.py step needed.
- The other portion invokes SQLAlchemy's pytest plugin::
- from sqlalchemy.dialects import registry
- registry.register("access", "sqlalchemy_access.pyodbc", "AccessDialect_pyodbc")
- registry.register("access.pyodbc", "sqlalchemy_access.pyodbc", "AccessDialect_pyodbc")
- from sqlalchemy.testing.plugin.pytestplugin import *
- Where above, the ``registry`` module, introduced in SQLAlchemy 0.8, provides
- an in-Python means of installing the dialect entrypoints without the use
- of setuptools, using the ``registry.register()`` function in a way that
- is similar to the ``entry_points`` directive we placed in our ``setup.py``.
- * requirements.py - The ``requirements.py`` file is where directives
- regarding database and dialect capabilities are set up.
- SQLAlchemy's tests are often annotated with decorators that mark
- tests as "skip" or "fail" for particular backends. Over time, this
- system has been refined such that specific database and DBAPI names
- are mentioned less and less, in favor of @requires directives which
- state a particular capability. The requirement directive is linked
- to target dialects using a ``Requirements`` subclass. The custom
- ``Requirements`` subclass is specified in the ``requirements.py`` file
- and is made available to SQLAlchemy's test runner using the
- ``requirement_cls`` directive inside the ``[sqla_testing]`` section.
- For a third-party dialect, the custom ``Requirements`` class can
- usually specify a simple yes/no answer for a particular system. For
- example, a requirements file that specifies a database that supports
- the RETURNING construct but does not support reflection of tables
- might look like this::
- # sqlalchemy_access/requirements.py
- from sqlalchemy.testing.requirements import SuiteRequirements
- from sqlalchemy.testing import exclusions
- class Requirements(SuiteRequirements):
- @property
- def table_reflection(self):
- return exclusions.closed()
- @property
- def returning(self):
- return exclusions.open()
- The ``SuiteRequirements`` class in
- ``sqlalchemy.testing.requirements`` contains a large number of
- requirements rules, which attempt to have reasonable defaults. The
- tests will report on those requirements found as they are run.
- The requirements system can also be used when running SQLAlchemy's
- primary test suite against the external dialect. In this use case,
- a ``--dburi`` as well as a ``--requirements`` flag are passed to SQLAlchemy's
- main test runner ``./sqla_nose.py`` so that exclusions specific to the
- dialect take place::
- cd /path/to/sqlalchemy
- py.test -v \
- --requirements sqlalchemy_access.requirements:Requirements \
- --dburi access+pyodbc://admin@access_test
- * test_suite.py - Finally, the ``test_suite.py`` module represents a
- stub test suite, which pulls in the actual SQLAlchemy test suite.
- To pull in the suite as a whole, it can be imported in one step::
- # test/test_suite.py
- from sqlalchemy.testing.suite import *
- That's all that's needed - the ``sqlalchemy.testing.suite`` package
- contains an ever expanding series of tests, most of which should be
- annotated with specific requirement decorators so that they can be
- fully controlled. To specifically modify some of the tests, they can
- be imported by name and subclassed::
- from sqlalchemy.testing.suite import *
- from sqlalchemy.testing.suite import ComponentReflectionTest as _ComponentReflectionTest
- class ComponentReflectionTest(_ComponentReflectionTest):
- @classmethod
- def define_views(cls, metadata, schema):
- # bypass the "define_views" section of the
- # fixture
- return
- Going Forward
- ==============
- The third-party dialect can be distributed like any other Python
- module on Pypi. Links to prominent dialects can be featured within
- SQLAlchemy's own documentation; contact the developers (see AUTHORS)
- for help with this.
- While SQLAlchemy includes many dialects built in, it remains to be
- seen if the project as a whole might move towards "plugin" model for
- all dialects, including all those currently built in. Now that
- SQLAlchemy's dialect API is mature and the test suite is not far
- behind, it may be that a better maintenance experience can be
- delivered by having all dialects separately maintained and released.
- As new versions of SQLAlchemy are released, the test suite and
- requirements file will receive new tests and changes. The dialect
- maintainer would normally keep track of these changes and make
- adjustments as needed.
|