README.dialects.rst 8.6 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210
  1. ========================
  2. Developing new Dialects
  3. ========================
  4. .. note::
  5. When studying this file, it's probably a good idea to also
  6. familiarize with the README.unittests.rst file, which discusses
  7. SQLAlchemy's usage and extension of the Nose test runner.
  8. While SQLAlchemy includes many dialects within the core distribution, the
  9. trend for new dialects should be that they are published as external
  10. projects. SQLAlchemy has since version 0.5 featured a "plugin" system
  11. which allows external dialects to be integrated into SQLAlchemy using
  12. standard setuptools entry points. As of version 0.8, this system has
  13. been enhanced, so that a dialect can also be "plugged in" at runtime.
  14. On the testing side, SQLAlchemy as of 0.8 also includes a "dialect
  15. compliance suite" that is usable by third party libraries. There is no
  16. longer a strong need for a new dialect to run through SQLAlchemy's full
  17. testing suite, as a large portion of these tests do not have
  18. dialect-sensitive functionality. The "dialect compliance suite" should
  19. be viewed as the primary target for new dialects, and as it continues
  20. to grow and mature it should become a more thorough and efficient system
  21. of testing new dialects.
  22. As of SQLAlchemy 0.9.4, both nose and pytest are supported for running tests,
  23. and pytest is now preferred.
  24. Dialect Layout
  25. ===============
  26. The file structure of a dialect is typically similar to the following::
  27. sqlalchemy-<dialect>/
  28. setup.py
  29. setup.cfg
  30. sqlalchemy_<dialect>/
  31. __init__.py
  32. base.py
  33. <dbapi>.py
  34. requirements.py
  35. test/
  36. conftest.py
  37. __init__.py
  38. test_suite.py
  39. test_<dialect_specific_test>.py
  40. ...
  41. An example of this structure can be seen in the Access dialect at
  42. https://bitbucket.org/zzzeek/sqlalchemy-access/.
  43. Key aspects of this file layout include:
  44. * setup.py - should specify setuptools entrypoints, allowing the
  45. dialect to be usable from create_engine(), e.g.::
  46. entry_points={
  47. 'sqlalchemy.dialects': [
  48. 'access = sqlalchemy_access.pyodbc:AccessDialect_pyodbc',
  49. 'access.pyodbc = sqlalchemy_access.pyodbc:AccessDialect_pyodbc',
  50. ]
  51. }
  52. Above, the two entrypoints ``access`` and ``access.pyodbc`` allow URLs to be
  53. used such as::
  54. create_engine("access://user:pw@dsn")
  55. create_engine("access+pyodbc://user:pw@dsn")
  56. * setup.cfg - this file contains the traditional contents such as [egg_info],
  57. [pytest] and [nosetests] directives, but also contains new directives that are used
  58. by SQLAlchemy's testing framework. E.g. for Access::
  59. [egg_info]
  60. tag_build = dev
  61. [tool:pytest]
  62. addopts= --tb native -v -r fxX --maxfail=25 -p no:warnings
  63. python_files=test/*test_*.py
  64. [sqla_testing]
  65. requirement_cls=sqlalchemy_access.requirements:Requirements
  66. profile_file=test/profiles.txt
  67. [db]
  68. default=access+pyodbc://admin@access_test
  69. sqlite=sqlite:///:memory:
  70. Above, the ``[sqla_testing]`` section contains configuration used by
  71. SQLAlchemy's test plugin. The ``[tool:pytest]`` section
  72. include directives to help with these runners; in the case of
  73. Nose, the directive ``with-sql_testing = true``, which indicates to Nose that
  74. the SQLAlchemy nose plugin should be used. In the case of pytest, the
  75. test/conftest.py file will bootstrap SQLAlchemy's plugin.
  76. * test/conftest.py - This script bootstraps SQLAlchemy's pytest plugin
  77. into the pytest runner. This
  78. script can also be used to install your third party dialect into
  79. SQLAlchemy without using the setuptools entrypoint system; this allows
  80. your dialect to be present without any explicit setup.py step needed.
  81. The other portion invokes SQLAlchemy's pytest plugin::
  82. from sqlalchemy.dialects import registry
  83. registry.register("access", "sqlalchemy_access.pyodbc", "AccessDialect_pyodbc")
  84. registry.register("access.pyodbc", "sqlalchemy_access.pyodbc", "AccessDialect_pyodbc")
  85. from sqlalchemy.testing.plugin.pytestplugin import *
  86. Where above, the ``registry`` module, introduced in SQLAlchemy 0.8, provides
  87. an in-Python means of installing the dialect entrypoints without the use
  88. of setuptools, using the ``registry.register()`` function in a way that
  89. is similar to the ``entry_points`` directive we placed in our ``setup.py``.
  90. * requirements.py - The ``requirements.py`` file is where directives
  91. regarding database and dialect capabilities are set up.
  92. SQLAlchemy's tests are often annotated with decorators that mark
  93. tests as "skip" or "fail" for particular backends. Over time, this
  94. system has been refined such that specific database and DBAPI names
  95. are mentioned less and less, in favor of @requires directives which
  96. state a particular capability. The requirement directive is linked
  97. to target dialects using a ``Requirements`` subclass. The custom
  98. ``Requirements`` subclass is specified in the ``requirements.py`` file
  99. and is made available to SQLAlchemy's test runner using the
  100. ``requirement_cls`` directive inside the ``[sqla_testing]`` section.
  101. For a third-party dialect, the custom ``Requirements`` class can
  102. usually specify a simple yes/no answer for a particular system. For
  103. example, a requirements file that specifies a database that supports
  104. the RETURNING construct but does not support reflection of tables
  105. might look like this::
  106. # sqlalchemy_access/requirements.py
  107. from sqlalchemy.testing.requirements import SuiteRequirements
  108. from sqlalchemy.testing import exclusions
  109. class Requirements(SuiteRequirements):
  110. @property
  111. def table_reflection(self):
  112. return exclusions.closed()
  113. @property
  114. def returning(self):
  115. return exclusions.open()
  116. The ``SuiteRequirements`` class in
  117. ``sqlalchemy.testing.requirements`` contains a large number of
  118. requirements rules, which attempt to have reasonable defaults. The
  119. tests will report on those requirements found as they are run.
  120. The requirements system can also be used when running SQLAlchemy's
  121. primary test suite against the external dialect. In this use case,
  122. a ``--dburi`` as well as a ``--requirements`` flag are passed to SQLAlchemy's
  123. main test runner ``./sqla_nose.py`` so that exclusions specific to the
  124. dialect take place::
  125. cd /path/to/sqlalchemy
  126. py.test -v \
  127. --requirements sqlalchemy_access.requirements:Requirements \
  128. --dburi access+pyodbc://admin@access_test
  129. * test_suite.py - Finally, the ``test_suite.py`` module represents a
  130. stub test suite, which pulls in the actual SQLAlchemy test suite.
  131. To pull in the suite as a whole, it can be imported in one step::
  132. # test/test_suite.py
  133. from sqlalchemy.testing.suite import *
  134. That's all that's needed - the ``sqlalchemy.testing.suite`` package
  135. contains an ever expanding series of tests, most of which should be
  136. annotated with specific requirement decorators so that they can be
  137. fully controlled. To specifically modify some of the tests, they can
  138. be imported by name and subclassed::
  139. from sqlalchemy.testing.suite import *
  140. from sqlalchemy.testing.suite import ComponentReflectionTest as _ComponentReflectionTest
  141. class ComponentReflectionTest(_ComponentReflectionTest):
  142. @classmethod
  143. def define_views(cls, metadata, schema):
  144. # bypass the "define_views" section of the
  145. # fixture
  146. return
  147. Going Forward
  148. ==============
  149. The third-party dialect can be distributed like any other Python
  150. module on Pypi. Links to prominent dialects can be featured within
  151. SQLAlchemy's own documentation; contact the developers (see AUTHORS)
  152. for help with this.
  153. While SQLAlchemy includes many dialects built in, it remains to be
  154. seen if the project as a whole might move towards "plugin" model for
  155. all dialects, including all those currently built in. Now that
  156. SQLAlchemy's dialect API is mature and the test suite is not far
  157. behind, it may be that a better maintenance experience can be
  158. delivered by having all dialects separately maintained and released.
  159. As new versions of SQLAlchemy are released, the test suite and
  160. requirements file will receive new tests and changes. The dialect
  161. maintainer would normally keep track of these changes and make
  162. adjustments as needed.