validate.txt 20 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686
  1. ===================================
  2. Validation Schema with validate.py
  3. ===================================
  4. --------------------------
  5. Using the Validator class
  6. --------------------------
  7. :Authors: `Michael Foord`_, `Nicola Larosa`_, `Mark Andrews`_
  8. :Version: Validate 1.0.1
  9. :Date: 2010/01/09
  10. :Homepage: `Validate Homepage`_
  11. :Repository: `Google code homepage <http://code.google.com/p/configobj/>`_
  12. :PyPI Entry: `Validate on Python Packaging Index <http://pypi.python.org/pypi/validate>`_
  13. :License: `BSD License`_
  14. :Support: `Mailing List`_
  15. .. _Mailing List: http://lists.sourceforge.net/lists/listinfo/configobj-develop
  16. .. _Michael Foord: fuzzyman@voidspace.org.uk
  17. .. _Nicola Larosa: nico@teknico.net
  18. .. _This Document:
  19. .. _Validate Homepage: http://www.voidspace.org.uk/python/validate.html
  20. .. _BSD License: http://www.voidspace.org.uk/python/license.shtml
  21. .. contents:: Validate Manual
  22. .. sectnum::
  23. Introduction
  24. ============
  25. Validation is used to check that supplied values conform to a specification.
  26. The value can be supplied as a string, e.g. from a config file. In this case
  27. the check will also *convert* the value to the required type. This allows you
  28. to add validation as a transparent layer to access data stored as strings. The
  29. validation checks that the data is correct *and* converts it to the expected
  30. type.
  31. Checks are also strings, and are easy to write. One generic system can be used
  32. to validate information from different sources via a single consistent
  33. mechanism.
  34. Checks look like function calls, and map to function calls. They can include
  35. parameters and keyword arguments. These arguments are passed to the relevant
  36. function by the ``Validator`` instance, along with the value being checked.
  37. The syntax for checks also allows for specifying a default value. This default
  38. value can be ``None``, no matter what the type of the check. This can be used
  39. to indicate that a value was missing, and so holds no useful value.
  40. Functions either return a new value, or raise an exception. See `Validator
  41. Exceptions`_ for the low down on the exception classes that ``validate.py``
  42. defines.
  43. Some standard functions are provided, for basic data types; these come built
  44. into every validator. Additional checks are easy to write: they can be provided
  45. when the ``Validator`` is instantiated, or added afterwards.
  46. Validate was primarily written to support ConfigObj_, but is designed to be
  47. applicable to many other situations.
  48. For support and bug reports please use the ConfigObj `Mailing List`_.
  49. .. _ConfigObj: http://www.voidspace.org.uk/python/configobj.html
  50. Downloading
  51. ===========
  52. The current version is **1.0.1**, dated 9th January 2010.
  53. You can get obtain validate in the following ways :
  54. Files
  55. -----
  56. * validate.py_ from Voidspace
  57. * configobj.zip from Voidspace - See the homepage of ConfigObj_ for the latest
  58. version and download links.
  59. This contains validate.py and `this document`_. (As well as ConfigObj_ and
  60. the ConfigObj documentation).
  61. * The latest development version can be obtained from the `Subversion Repository`_.
  62. Documentation
  63. -------------
  64. *configobj.zip* contains `this document`_.
  65. * You can view `this document`_ online as the `Validate Homepage`_.
  66. .. _configobj.py: http://www.voidspace.org.uk/cgi-bin/voidspace/download/configobj.py
  67. .. _configobj.zip: http://www.voidspace.org.uk/cgi-bin/voidspace/download/configobj-4.7.0.zip
  68. .. _validate.py: http://www.voidspace.org.uk/cgi-bin/voidspace/download/validate.py
  69. .. _Subversion Repository: http://code.google.com/p/configobj/
  70. .. _Sourceforge: http://sourceforge.net/projects/configobj
  71. .. _validate: http://www.voidspace.org.uk/python/validate.html
  72. The standard functions
  73. ======================
  74. The standard functions come built-in to every ``Validator`` instance. They work
  75. with the following basic data types :
  76. * integer
  77. * float
  78. * boolean
  79. * string
  80. * ip_addr
  81. plus lists of these datatypes.
  82. Adding additional checks is done through coding simple functions.
  83. The full set of standard checks are :
  84. :'integer': matches integer values (including negative). Takes optional 'min'
  85. and 'max' arguments::
  86. integer()
  87. integer(3, 9) # any value from 3 to 9
  88. integer(min=0) # any positive value
  89. integer(max=9)
  90. :'float': matches float values
  91. Has the same parameters as the integer check.
  92. :'boolean': matches boolean values: ``True`` or ``False``.
  93. Acceptable string values for True are::
  94. true, on, yes, 1
  95. Acceptable string values for False are::
  96. false, off, no, 0
  97. Any other value raises an error.
  98. :'string': matches any string. Takes optional keyword args 'min' and 'max' to
  99. specify min and max length of string.
  100. :'ip_addr': matches an Internet Protocol address, v.4, represented by a
  101. dotted-quad string, i.e. '1.2.3.4'.
  102. :'list': matches any list. Takes optional keyword args 'min', and 'max' to
  103. specify min and max sizes of the list. The list checks always
  104. return a list.
  105. :force_list: matches any list, but if a single value is passed in will
  106. coerce it into a list containing that value. Useful for
  107. configobj if the user forgot the trailing comma to turn
  108. a single value into a list.
  109. :'tuple': matches any list. This check returns a tuple rather than a list.
  110. :'int_list': Matches a list of integers. Takes the same arguments as list.
  111. :'float_list': Matches a list of floats. Takes the same arguments as list.
  112. :'bool_list': Matches a list of boolean values. Takes the same arguments as
  113. list.
  114. :'string_list': Matches a list of strings. Takes the same arguments as list.
  115. :'ip_addr_list': Matches a list of IP addresses. Takes the same arguments as
  116. list.
  117. :'mixed_list': Matches a list with different types in specific positions.
  118. List size must match the number of arguments.
  119. Each position can be one of::
  120. int, str, boolean, float, ip_addr
  121. So to specify a list with two strings followed by two integers,
  122. you write the check as::
  123. mixed_list(str, str, int, int)
  124. :'pass': matches everything: it never fails and the value is unchanged. It is
  125. also the default if no check is specified.
  126. :'option': matches any from a list of options.
  127. You specify this test with::
  128. option('option 1', 'option 2', 'option 3')
  129. The following code will work without you having to specifically add the
  130. functions yourself.
  131. .. code-block:: python
  132. from validate import Validator
  133. #
  134. vtor = Validator()
  135. newval1 = vtor.check('integer', value1)
  136. newval2 = vtor.check('boolean', value2)
  137. # etc ...
  138. .. note::
  139. Of course, if these checks fail they raise exceptions. So you should wrap
  140. them in ``try...except`` blocks. Better still, use ConfigObj for a higher
  141. level interface.
  142. Using Validator
  143. ===============
  144. Using ``Validator`` is very easy. It has one public attribute and one public
  145. method.
  146. Shown below are the different steps in using ``Validator``.
  147. The only additional thing you need to know, is about `Writing check
  148. functions`_.
  149. Instantiate
  150. -----------
  151. .. code-block:: python
  152. from validate import Validator
  153. vtor = Validator()
  154. or even :
  155. .. code-block:: python
  156. from validate import Validator
  157. #
  158. fdict = {
  159. 'check_name1': function1,
  160. 'check_name2': function2,
  161. 'check_name3': function3,
  162. }
  163. #
  164. vtor = Validator(fdict)
  165. The second method adds a set of your functions as soon as your validator is
  166. created. They are stored in the ``vtor.functions`` dictionary. The 'key' you
  167. give them in this dictionary is the name you use in your checks (not the
  168. original function name).
  169. Dictionary keys/functions you pass in can override the built-in ones if you
  170. want.
  171. Adding functions
  172. ----------------
  173. The code shown above, for adding functions on instantiation, has exactly the
  174. same effect as the following code :
  175. .. code-block:: python
  176. from validate import Validator
  177. #
  178. vtor = Validator()
  179. vtor.functions['check_name1'] = function1
  180. vtor.functions['check_name2'] = function2
  181. vtor.functions['check_name3'] = function3
  182. ``vtor.functions`` is just a dictionary that maps names to functions, so we
  183. could also have called ``vtor.functions.update(fdict)``.
  184. Writing the check
  185. -----------------
  186. As we've heard, the checks map to the names in the ``functions`` dictionary.
  187. You've got a full list of `The standard functions`_ and the arguments they
  188. take.
  189. If you're using ``Validator`` from ConfigObj, then your checks will look like::
  190. keyword = int_list(max=6)
  191. but the check part will be identical .
  192. The check method
  193. ----------------
  194. If you're not using ``Validator`` from ConfigObj, then you'll need to call the
  195. ``check`` method yourself.
  196. If the check fails then it will raise an exception, so you'll want to trap
  197. that. Here's the basic example :
  198. .. code-block:: python
  199. from validate import Validator, ValidateError
  200. #
  201. vtor = Validator()
  202. check = "integer(0, 9)"
  203. value = 3
  204. try:
  205. newvalue = vtor.check(check, value)
  206. except ValidateError:
  207. print 'Check Failed.'
  208. else:
  209. print 'Check passed.'
  210. .. caution::
  211. Although the value can be a string, if it represents a list it should
  212. already have been turned into a list of strings.
  213. Default Values
  214. ~~~~~~~~~~~~~~
  215. Some values may not be available, and you may want to be able to specify a
  216. default as part of the check.
  217. You do this by passing the keyword ``missing=True`` to the ``check`` method, as
  218. well as a ``default=value`` in the check. (Constructing these checks is done
  219. automatically by ConfigObj: you only need to know about the ``default=value``
  220. part) :
  221. .. code-block:: python
  222. check1 = 'integer(default=50)'
  223. check2 = 'option("val 1", "val 2", "val 3", default="val 1")'
  224. assert vtor.check(check1, '', missing=True) == 50
  225. assert vtor.check(check2, '', missing=True) == "val 1"
  226. If you pass in ``missing=True`` to the check method, then the actual value is
  227. ignored. If no default is specified in the check, a ``ValidateMissingValue``
  228. exception is raised. If a default is specified then that is passed to the
  229. check instead.
  230. If the check has ``default=None`` (case sensitive) then ``vtor.check`` will
  231. *always* return ``None`` (the object). This makes it easy to tell your program
  232. that this check contains no useful value when missing, i.e. the value is
  233. optional, and may be omitted without harm.
  234. .. note::
  235. As of version 0.3.0, if you specify ``default='None'`` (note the quote marks
  236. around ``None``) then it will be interpreted as the string ``'None'``.
  237. List Values
  238. ~~~~~~~~~~~
  239. It's possible that you would like your default value to be a list. It's even
  240. possible that you will write your own check functions - and would like to pass
  241. them keyword arguments as lists from within the check.
  242. To avoid confusing syntax with commas and quotes you use a list constructor to
  243. specify that keyword arguments are lists. This includes the ``default`` value.
  244. This makes checks look something like::
  245. checkname(default=list('val1', 'val2', 'val3'))
  246. get_default_value
  247. -----------------
  248. ``Validator`` instances have a ``get_default_value`` method. It takes a ``check`` string
  249. (the same string you would pass to the ``check`` method) and returns the default value,
  250. converted to the right type. If the check doesn't define a default value then this method
  251. raises a ``KeyError``.
  252. If the ``check`` has been seen before then it will have been parsed and cached already,
  253. so this method is not expensive to call (however the conversion is done each time).
  254. Validator Exceptions
  255. ====================
  256. .. note::
  257. If you only use Validator through ConfigObj, it traps these Exceptions for
  258. you. You will still need to know about them for writing your own check
  259. functions.
  260. ``vtor.check`` indicates that the check has failed by raising an exception.
  261. The appropriate error should be raised in the check function.
  262. The base error class is ``ValidateError``. All errors (except for ``VdtParamError``)
  263. raised are sub-classes of this.
  264. If an unrecognised check is specified then ``VdtUnknownCheckError`` is
  265. raised.
  266. There are also ``VdtTypeError`` and ``VdtValueError``.
  267. If incorrect parameters are passed to a check function then it will (or should)
  268. raise ``VdtParamError``. As this indicates *programmer* error, rather than an error
  269. in the value, it is a subclass of ``SyntaxError`` instead of ``ValidateError``.
  270. .. note::
  271. This means it *won't* be caught by ConfigObj - but propagated instead.
  272. If the value supplied is the wrong type, then the check should raise
  273. ``VdtTypeError``. e.g. the check requires the value to be an integer (or
  274. representation of an integer) and something else was supplied.
  275. If the value supplied is the right type, but an unacceptable value, then the
  276. check should raise ``VdtValueError``. e.g. the check requires the value to
  277. be an integer (or representation of an integer) less than ten and a higher
  278. value was supplied.
  279. Both ``VdtTypeError`` and ``VdtValueError`` are initialised with the
  280. incorrect value. In other words you raise them like this :
  281. .. code-block:: python
  282. raise VdtTypeError(value)
  283. #
  284. raise VdtValueError(value)
  285. ``VdtValueError`` has the following subclasses, which should be raised if
  286. they are more appropriate.
  287. * ``VdtValueTooSmallError``
  288. * ``VdtValueTooBigError``
  289. * ``VdtValueTooShortError``
  290. * ``VdtValueTooLongError``
  291. Writing check functions
  292. =======================
  293. Writing check functions is easy.
  294. The check function will receive the value as its first argument, followed by
  295. any other parameters and keyword arguments.
  296. If the check fails, it should raise a ``VdtTypeError`` or a
  297. ``VdtValueError`` (or an appropriate subclass).
  298. All parameters and keyword arguments are *always* passed as strings. (Parsed
  299. from the check string).
  300. The value might be a string (or list of strings) and need
  301. converting to the right type - alternatively it might already be a list of
  302. integers. Our function needs to be able to handle either.
  303. If the check passes then it should return the value (possibly converted to the
  304. right type).
  305. And that's it !
  306. Example
  307. -------
  308. Here is an example function that requires a list of integers. Each integer
  309. must be between 0 and 99.
  310. It takes a single argument specifying the length of the list. (Which allows us
  311. to use the same check in more than one place). If the length can't be converted
  312. to an integer then we need to raise ``VdtParamError``.
  313. Next we check that the value is a list. Anything else should raise a
  314. ``VdtTypeError``. The list should also have 'length' entries. If the list
  315. has more or less entries then we will need to raise a
  316. ``VdtValueTooShortError`` or a ``VdtValueTooLongError``.
  317. Then we need to check every entry in the list. Each entry should be an integer
  318. between 0 and 99, or a string representation of an integer between 0 and 99.
  319. Any other type is a ``VdtTypeError``, any other value is a
  320. ``VdtValueError`` (either too big, or too small).
  321. .. code-block:: python
  322. def special_list(value, length):
  323. """
  324. Check that the supplied value is a list of integers,
  325. with 'length' entries, and each entry between 0 and 99.
  326. """
  327. # length is supplied as a string
  328. # we need to convert it to an integer
  329. try:
  330. length = int(length)
  331. except ValueError:
  332. raise VdtParamError('length', length)
  333. #
  334. # Check the supplied value is a list
  335. if not isinstance(value, list):
  336. raise VdtTypeError(value)
  337. #
  338. # check the length of the list is correct
  339. if len(value) > length:
  340. raise VdtValueTooLongError(value)
  341. elif len(value) < length:
  342. raise VdtValueTooShortError(value)
  343. #
  344. # Next, check every member in the list
  345. # converting strings as necessary
  346. out = []
  347. for entry in value:
  348. if not isinstance(entry, (str, unicode, int)):
  349. # a value in the list
  350. # is neither an integer nor a string
  351. raise VdtTypeError(value)
  352. elif isinstance(entry, (str, unicode)):
  353. if not entry.isdigit():
  354. raise VdtTypeError(value)
  355. else:
  356. entry = int(entry)
  357. if entry < 0:
  358. raise VdtValueTooSmallError(value)
  359. elif entry > 99:
  360. raise VdtValueTooBigError(value)
  361. out.append(entry)
  362. #
  363. # if we got this far, all is well
  364. # return the new list
  365. return out
  366. If you are only using validate from ConfigObj then the error type (*TooBig*,
  367. *TooSmall*, etc) is lost - so you may only want to raise ``VdtValueError``.
  368. .. caution::
  369. If your function raises an exception that isn't a subclass of
  370. ``ValidateError``, then ConfigObj won't trap it. This means validation will
  371. fail.
  372. This is why our function starts by checking the type of the value. If we
  373. are passed the wrong type (e.g. an integer rather than a list) we get a
  374. ``VdtTypeError`` rather than bombing out when we try to iterate over
  375. the value.
  376. If you are using validate in another circumstance you may want to create your
  377. own subclasses of ``ValidateError`` which convey more specific information.
  378. Known Issues
  379. ============
  380. The following parses and then blows up. The resulting error message
  381. is confusing:
  382. ``checkname(default=list(1, 2, 3, 4)``
  383. This is because it parses as: ``checkname(default="list(1", 2, 3, 4)``.
  384. That isn't actually unreasonable, but the error message won't help you
  385. work out what has happened.
  386. TODO
  387. ====
  388. * A regex check function ?
  389. * A timestamp check function ? (Using the ``parse`` function from ``DateUtil`` perhaps).
  390. ISSUES
  391. ======
  392. .. note::
  393. Please file any bug reports to `Michael Foord`_ or the ConfigObj
  394. `Mailing List`_.
  395. If we could pull tuples out of arguments, it would be easier
  396. to specify arguments for 'mixed_lists'.
  397. CHANGELOG
  398. =========
  399. 2009/10/25 - Version 1.0.1
  400. --------------------------
  401. * BUGFIX: Fixed compatibility with Python 2.3.
  402. 2009/04/13 - Version 1.0.0
  403. --------------------------
  404. * BUGFIX: can now handle multiline strings.
  405. * Addition of 'force_list' validation option.
  406. As the API is stable and there are no known bugs or outstanding feature requests I am marking this 1.0.
  407. 2008/02/24 - Version 0.3.2
  408. --------------------------
  409. BUGFIX: Handling of None as default value fixed.
  410. 2008/02/05 - Version 0.3.1
  411. --------------------------
  412. BUGFIX: Unicode checks no longer broken.
  413. 2008/02/05 - Version 0.3.0
  414. --------------------------
  415. Improved performance with a parse cache.
  416. New ``get_default_value`` method. Given a check it returns the default
  417. value (converted to the correct type) or raises a ``KeyError`` if the
  418. check doesn't specify a default.
  419. Added 'tuple' check and corresponding 'is_tuple' function (which always returns a tuple).
  420. BUGFIX: A quoted 'None' as a default value is no longer treated as None,
  421. but as the string 'None'.
  422. BUGFIX: We weren't unquoting keyword arguments of length two, so an
  423. empty string didn't work as a default.
  424. BUGFIX: Strings no longer pass the 'is_list' check. Additionally, the
  425. list checks always return lists.
  426. A couple of documentation bug fixes.
  427. Removed CHANGELOG from module.
  428. 2007/02/04 Version 0.2.3
  429. -----------------------------
  430. Release of 0.2.3
  431. 2006/12/17 Version 0.2.3-alpha1
  432. ------------------------------------
  433. By Nicola Larosa
  434. Fixed validate doc to talk of ``boolean`` instead of ``bool``; changed the
  435. ``is_bool`` function to ``is_boolean`` (Sourceforge bug #1531525).
  436. 2006/04/29 Version 0.2.2
  437. -----------------------------
  438. Addressed bug where a string would pass the ``is_list`` test. (Thanks to
  439. Konrad Wojas.)
  440. 2005/12/16 Version 0.2.1
  441. -----------------------------
  442. Fixed bug so we can handle keyword argument values with commas.
  443. We now use a list constructor for passing list values to keyword arguments
  444. (including ``default``)::
  445. default=list("val", "val", "val")
  446. Added the ``_test`` test.
  447. Moved a function call outside a try...except block.
  448. 2005/08/18 Version 0.2.0
  449. -----------------------------
  450. Updated by `Michael Foord`_ and `Nicola Larosa`_
  451. Does type conversion as well.
  452. 2005/02/01 Version 0.1.0
  453. -----------------------------
  454. Initial version developed by `Michael Foord`_
  455. and Mark Andrews.