database.rst 16 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493
  1. Database creation/re-use
  2. ========================
  3. ``pytest-django`` takes a conservative approach to enabling database
  4. access. By default your tests will fail if they try to access the
  5. database. Only if you explicitly request database access will this be
  6. allowed. This encourages you to keep database-needing tests to a
  7. minimum which is a best practice since next-to-no business logic
  8. should be requiring the database. Moreover it makes it very clear
  9. what code uses the database and catches any mistakes.
  10. Enabling database access in tests
  11. ---------------------------------
  12. You can use `pytest marks <https://pytest.org/en/latest/mark.html>`_ to
  13. tell ``pytest-django`` your test needs database access::
  14. import pytest
  15. @pytest.mark.django_db
  16. def test_my_user():
  17. me = User.objects.get(username='me')
  18. assert me.is_superuser
  19. It is also possible to mark all tests in a class or module at once.
  20. This demonstrates all the ways of marking, even though they overlap.
  21. Just one of these marks would have been sufficient. See the `pytest
  22. documentation
  23. <https://pytest.org/en/latest/example/markers.html#marking-whole-classes-or-modules>`_
  24. for detail::
  25. import pytest
  26. pytestmark = pytest.mark.django_db
  27. @pytest.mark.django_db
  28. class TestUsers:
  29. pytestmark = pytest.mark.django_db
  30. def test_my_user(self):
  31. me = User.objects.get(username='me')
  32. assert me.is_superuser
  33. By default ``pytest-django`` will set up the Django databases the
  34. first time a test needs them. Once setup the database is cached for
  35. used for all subsequent tests and rolls back transactions to isolate
  36. tests from each other. This is the same way the standard Django
  37. `TestCase
  38. <https://docs.djangoproject.com/en/1.9/topics/testing/tools/#testcase>`_
  39. uses the database. However ``pytest-django`` also caters for
  40. transaction test cases and allows you to keep the test databases
  41. configured across different test runs.
  42. Testing transactions
  43. --------------------
  44. Django itself has the ``TransactionTestCase`` which allows you to test
  45. transactions and will flush the database between tests to isolate
  46. them. The downside of this is that these tests are much slower to
  47. set up due to the required flushing of the database.
  48. ``pytest-django`` also supports this style of tests, which you can
  49. select using an argument to the ``django_db`` mark::
  50. @pytest.mark.django_db(transaction=True)
  51. def test_spam():
  52. pass # test relying on transactions
  53. Tests requiring multiple databases
  54. ----------------------------------
  55. Currently ``pytest-django`` does not specifically support Django's
  56. multi-database support.
  57. You can however use normal :class:`~django.test.TestCase` instances to use its
  58. :ref:`django:topics-testing-advanced-multidb` support.
  59. In particular, if your database is configured for replication, be sure to read
  60. about :ref:`django:topics-testing-primaryreplica`.
  61. If you have any ideas about the best API to support multiple databases
  62. directly in ``pytest-django`` please get in touch, we are interested
  63. in eventually supporting this but unsure about simply following
  64. Django's approach.
  65. See `https://github.com/pytest-dev/pytest-django/pull/431` for an idea /
  66. discussion to approach this.
  67. ``--reuse-db`` - reuse the testing database between test runs
  68. --------------------------------------------------------------
  69. Using ``--reuse-db`` will create the test database in the same way as
  70. ``manage.py test`` usually does.
  71. However, after the test run, the test database will not be removed.
  72. The next time a test run is started with ``--reuse-db``, the database will
  73. instantly be re used. This will allow much faster startup time for tests.
  74. This can be especially useful when running a few tests, when there are a lot
  75. of database tables to set up.
  76. ``--reuse-db`` will not pick up schema changes between test runs. You must run
  77. the tests with ``--reuse-db --create-db`` to re-create the database according
  78. to the new schema. Running without ``--reuse-db`` is also possible, since the
  79. database will automatically be re-created.
  80. ``--create-db`` - force re creation of the test database
  81. --------------------------------------------------------
  82. When used with ``--reuse-db``, this option will re-create the database,
  83. regardless of whether it exists or not.
  84. Example work flow with ``--reuse-db`` and ``--create-db``.
  85. -----------------------------------------------------------
  86. A good way to use ``--reuse-db`` and ``--create-db`` can be:
  87. * Put ``--reuse-db`` in your default options (in your project's ``pytest.ini`` file)::
  88. [pytest]
  89. addopts = --reuse-db
  90. * Just run tests with ``pytest``, on the first run the test database will be
  91. created. The next test run it will be reused.
  92. * When you alter your database schema, run ``pytest --create-db``, to force
  93. re-creation of the test database.
  94. ``--nomigrations`` - Disable Django migrations
  95. ----------------------------------------------
  96. Using ``--nomigrations`` will disable Django migrations and create the database
  97. by inspecting all models. It may be faster when there are several migrations to
  98. run in the database setup. You can use ``--migrations`` to force running
  99. migrations in case ``--nomigrations`` is used, e.g. in ``setup.cfg``.
  100. .. _advanced-database-configuration:
  101. Advanced database configuration
  102. -------------------------------
  103. pytest-django provides options to customize the way database is configured. The
  104. default database construction mostly follows Django's own test runner. You can
  105. however influence all parts of the database setup process to make it fit in
  106. projects with special requirements.
  107. This section assumes some familiarity with the Django test runner, Django
  108. database creation and pytest fixtures.
  109. Fixtures
  110. ########
  111. There are some fixtures which will let you change the way the database is
  112. configured in your own project. These fixtures can be overridden in your own
  113. project by specifying a fixture with the same name and scope in ``conftest.py``.
  114. .. admonition:: Use the pytest-django source code
  115. The default implementation of these fixtures can be found in
  116. `fixtures.py <https://github.com/pytest-dev/pytest-django/blob/master/pytest_django/fixtures.py>`_.
  117. The code is relatively short and straightforward and can provide a
  118. starting point when you need to customize database setup in your own
  119. project.
  120. django_db_setup
  121. """""""""""""""
  122. .. fixture:: django_db_setup
  123. This is the top-level fixture that ensures that the test databases are created
  124. and available. This fixture is session scoped (it will be run once per test
  125. session) and is responsible for making sure the test database is available for tests
  126. that need it.
  127. The default implementation creates the test database by applying migrations and removes
  128. databases after the test run.
  129. You can override this fixture in your own ``conftest.py`` to customize how test
  130. databases are constructed.
  131. django_db_modify_db_settings
  132. """"""""""""""""""""""""""""
  133. .. fixture:: django_db_modify_db_settings
  134. This fixture allows modifying `django.conf.settings.DATABASES` just before the
  135. databases are configured.
  136. If you need to customize the location of your test database, this is the
  137. fixture you want to override.
  138. The default implementation of this fixture requests the
  139. :fixture:`django_db_modify_db_settings_parallel_suffix` to provide compatibility
  140. with pytest-xdist.
  141. This fixture is by default requested from :fixture:`django_db_setup`.
  142. django_db_modify_db_settings_parallel_suffix
  143. """"""""""""""""""""""""""""""""""""""""""""
  144. .. fixture:: django_db_modify_db_settings_parallel_suffix
  145. Requesting this fixture will add a suffix to the database name when the tests
  146. are run via `pytest-xdist`, or via `tox` in parallel mode.
  147. This fixture is by default requested from
  148. :fixture:`django_db_modify_db_settings`.
  149. django_db_modify_db_settings_tox_suffix
  150. """""""""""""""""""""""""""""""""""""""
  151. .. fixture:: django_db_modify_db_settings_tox_suffix
  152. Requesting this fixture will add a suffix to the database name when the tests
  153. are run via `tox` in parallel mode.
  154. This fixture is by default requested from
  155. :fixture:`django_db_modify_db_settings_parallel_suffix`.
  156. django_db_modify_db_settings_xdist_suffix
  157. """""""""""""""""""""""""""""""""""""""""
  158. .. fixture:: django_db_modify_db_settings_xdist_suffix
  159. Requesting this fixture will add a suffix to the database name when the tests
  160. are run via `pytest-xdist`.
  161. This fixture is by default requested from
  162. :fixture:`django_db_modify_db_settings_parallel_suffix`.
  163. django_db_use_migrations
  164. """"""""""""""""""""""""
  165. .. fixture:: django_db_use_migrations
  166. Returns whether or not to use migrations to create the test
  167. databases.
  168. The default implementation returns the value of the
  169. ``--migrations``/``--nomigrations`` command line options.
  170. This fixture is by default requested from :fixture:`django_db_setup`.
  171. django_db_keepdb
  172. """"""""""""""""
  173. .. fixture:: django_db_keepdb
  174. Returns whether or not to re-use an existing database and to keep it after the
  175. test run.
  176. The default implementation handles the ``--reuse-db`` and ``--create-db``
  177. command line options.
  178. This fixture is by default requested from :fixture:`django_db_setup`.
  179. django_db_createdb
  180. """"""""""""""""""
  181. .. fixture:: django_db_createdb
  182. Returns whether or not the database is to be re-created before running any
  183. tests.
  184. This fixture is by default requested from :fixture:`django_db_setup`.
  185. django_db_blocker
  186. """""""""""""""""
  187. .. fixture:: django_db_blocker
  188. .. warning::
  189. It does not manage transactions and changes made to the database will not
  190. be automatically restored. Using the ``pytest.mark.django_db`` marker
  191. or :fixture:`db` fixture, which wraps database changes in a transaction and
  192. restores the state is generally the thing you want in tests. This marker
  193. can be used when you are trying to influence the way the database is
  194. configured.
  195. Database access is by default not allowed. ``django_db_blocker`` is the object
  196. which can allow specific code paths to have access to the database. This
  197. fixture is used internally to implement the ``db`` fixture.
  198. :fixture:`django_db_blocker` can be used as a context manager to enable database
  199. access for the specified block::
  200. @pytest.fixture
  201. def myfixture(django_db_blocker):
  202. with django_db_blocker.unblock():
  203. ... # modify something in the database
  204. You can also manage the access manually via these methods:
  205. .. py:method:: django_db_blocker.unblock()
  206. Enable database access. Should be followed by a call to
  207. :func:`~django_db_blocker.restore`.
  208. .. py:method:: django_db_blocker.block()
  209. Disable database access. Should be followed by a call to
  210. :func:`~django_db_blocker.restore`.
  211. .. py:function:: django_db_blocker.restore()
  212. Restore the previous state of the database blocking.
  213. Examples
  214. ########
  215. Using a template database for tests
  216. """""""""""""""""""""""""""""""""""
  217. This example shows how a pre-created PostgreSQL source database can be copied
  218. and used for tests.
  219. Put this into ``conftest.py``::
  220. import pytest
  221. from django.db import connections
  222. import psycopg2
  223. from psycopg2.extensions import ISOLATION_LEVEL_AUTOCOMMIT
  224. def run_sql(sql):
  225. conn = psycopg2.connect(database='postgres')
  226. conn.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)
  227. cur = conn.cursor()
  228. cur.execute(sql)
  229. conn.close()
  230. @pytest.yield_fixture(scope='session')
  231. def django_db_setup():
  232. from django.conf import settings
  233. settings.DATABASES['default']['NAME'] = 'the_copied_db'
  234. run_sql('DROP DATABASE IF EXISTS the_copied_db')
  235. run_sql('CREATE DATABASE the_copied_db TEMPLATE the_source_db')
  236. yield
  237. for connection in connections.all():
  238. connection.close()
  239. run_sql('DROP DATABASE the_copied_db')
  240. Using an existing, external database for tests
  241. """"""""""""""""""""""""""""""""""""""""""""""
  242. This example shows how you can connect to an existing database and use it for
  243. your tests. This example is trivial, you just need to disable all of
  244. pytest-django and Django's test database creation and point to the existing
  245. database. This is achieved by simply implementing a no-op
  246. :fixture:`django_db_setup` fixture.
  247. Put this into ``conftest.py``::
  248. import pytest
  249. @pytest.fixture(scope='session')
  250. def django_db_setup():
  251. settings.DATABASES['default'] = {
  252. 'ENGINE': 'django.db.backends.mysql',
  253. 'HOST': 'db.example.com',
  254. 'NAME': 'external_db',
  255. }
  256. Populate the database with initial test data
  257. """"""""""""""""""""""""""""""""""""""""""""
  258. This example shows how you can populate the test database with test data. The
  259. test data will be saved in the database, i.e. it will not just be part of a
  260. transactions. This example uses Django's fixture loading mechanism, but it can
  261. be replaced with any way of loading data into the database.
  262. Notice that :fixture:`django_db_setup` is in the argument list. This may look
  263. odd at first, but it will make sure that the original pytest-django fixture
  264. is used to create the test database. When ``call_command`` is invoked, the
  265. test database is already prepared and configured.
  266. Put this in ``conftest.py``::
  267. import pytest
  268. from django.core.management import call_command
  269. @pytest.fixture(scope='session')
  270. def django_db_setup(django_db_setup, django_db_blocker):
  271. with django_db_blocker.unblock():
  272. call_command('loaddata', 'your_data_fixture.json')
  273. Use the same database for all xdist processes
  274. """""""""""""""""""""""""""""""""""""""""""""
  275. By default, each xdist process gets its own database to run tests on. This is
  276. needed to have transactional tests that do not interfere with each other.
  277. If you instead want your tests to use the same database, override the
  278. :fixture:`django_db_modify_db_settings` to not do anything. Put this in
  279. ``conftest.py``::
  280. import pytest
  281. @pytest.fixture(scope='session')
  282. def django_db_modify_db_settings():
  283. pass
  284. Randomize database sequences
  285. """"""""""""""""""""""""""""
  286. You can customize the test database after it has been created by extending the
  287. :fixture:`django_db_setup` fixture. This example shows how to give a PostgreSQL
  288. sequence a random starting value. This can be used to detect and prevent
  289. primary key id's from being hard-coded in tests.
  290. Put this in ``conftest.py``::
  291. import random
  292. import pytest
  293. from django.db import connection
  294. @pytest.fixture(scope='session')
  295. def django_db_setup(django_db_setup, django_db_blocker):
  296. with django_db_blocker.unblock():
  297. cur = connection.cursor()
  298. cur.execute('ALTER SEQUENCE app_model_id_seq RESTART WITH %s;',
  299. [random.randint(10000, 20000)])
  300. Create the test database from a custom SQL script
  301. """""""""""""""""""""""""""""""""""""""""""""""""
  302. You can replace the :fixture:`django_db_setup` fixture and run any code in its
  303. place. This includes creating your database by hand by running a SQL script
  304. directly. This example shows sqlite3's executescript method. In a more
  305. general use case, you probably want to load the SQL statements from a file or
  306. invoke the ``psql`` or the ``mysql`` command line tool.
  307. Put this in ``conftest.py``::
  308. import pytest
  309. from django.db import connection
  310. @pytest.fixture(scope='session')
  311. def django_db_setup(django_db_blocker):
  312. with django_db_blocker.unblock():
  313. with connection.cursor() as c:
  314. c.executescript('''
  315. DROP TABLE IF EXISTS theapp_item;
  316. CREATE TABLE theapp_item (id, name);
  317. INSERT INTO theapp_item (name) VALUES ('created from a sql script');
  318. ''')
  319. .. warning::
  320. This snippet shows ``cursor().executescript()`` which is `sqlite` specific, for
  321. other database engines this method might differ. For instance, psycopg2 uses
  322. ``cursor().execute()``.
  323. Use a read only database
  324. """"""""""""""""""""""""
  325. You can replace the ordinary `django_db_setup` to completely avoid database
  326. creation/migrations. If you have no need for rollbacks or truncating tables,
  327. you can simply avoid blocking the database and use it directly. When using this
  328. method you must ensure that your tests do not change the database state.
  329. Put this in ``conftest.py``::
  330. import pytest
  331. @pytest.fixture(scope='session')
  332. def django_db_setup():
  333. """Avoid creating/setting up the test database"""
  334. pass
  335. @pytest.fixture
  336. def db_access_without_rollback_and_truncate(request, django_db_setup, django_db_blocker):
  337. django_db_blocker.unblock()
  338. request.addfinalizer(django_db_blocker.restore)