libhttplib2.rst 17 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483
  1. .. % Template for a library manual section.
  2. .. % PLEASE REMOVE THE COMMENTS AFTER USING THE TEMPLATE
  3. .. %
  4. .. % Complete documentation on the extended LaTeX markup used for Python
  5. .. % documentation is available in ``Documenting Python'', which is part
  6. .. % of the standard documentation for Python. It may be found online
  7. .. % at:
  8. .. %
  9. .. % http://www.python.org/doc/current/doc/doc.html
  10. .. % ==== 0. ====
  11. .. % Copy this file to <mydir>/lib<mymodule>.tex, and edit that file
  12. .. % according to the instructions below.
  13. .. % ==== 1. ====
  14. .. % The section prologue. Give the section a title and provide some
  15. .. % meta-information. References to the module should use
  16. .. % \refbimodindex, \refstmodindex, \refexmodindex or \refmodindex, as
  17. .. % appropriate.
  18. :mod:`httplib2` A comprehensive HTTP client library.
  19. =====================================================
  20. .. module:: httplib2
  21. .. moduleauthor:: Joe Gregorio <joe@bitworking.org>
  22. .. sectionauthor:: Joe Gregorio <joe@bitworking.org>
  23. .. % Choose one of these to specify the module module name. If there's
  24. .. % an underscore in the name, use
  25. .. % \declaremodule[modname]{...}{mod_name} instead.
  26. .. %
  27. .. % not standard, in Python
  28. .. % Portability statement: Uncomment and fill in the parameter to specify the
  29. .. % availability of the module. The parameter can be Unix, IRIX, SunOS, Mac,
  30. .. % Windows, or lots of other stuff. When ``Mac'' is specified, the availability
  31. .. % statement will say ``Macintosh'' and the Module Index may say ``Mac''.
  32. .. % Please use a name that has already been used whenever applicable. If this
  33. .. % is omitted, no availability statement is produced or implied.
  34. .. %
  35. .. % \platform{Unix}
  36. .. % These apply to all modules, and may be given more than once:
  37. .. % Author of the module code;
  38. .. % omit if not known.
  39. .. % Author of the documentation,
  40. .. % even if not a module section.
  41. .. % Leave at least one blank line after this, to simplify ad-hoc tools
  42. .. % that are sometimes used to massage these files.
  43. The :mod:`httplib2` module is a comprehensive HTTP client library with the
  44. following features:
  45. .. % ==== 2. ====
  46. .. % Give a short overview of what the module does.
  47. .. % If it is platform specific, mention this.
  48. .. % Mention other important restrictions or general operating principles.
  49. .. % For example:
  50. .. describe:: HTTP and HTTPS
  51. HTTPS support is only available if the socket module was compiled with SSL
  52. support.
  53. .. describe:: Keep-Alive
  54. Supports HTTP 1.1 Keep-Alive, keeping the socket open and performing multiple
  55. requests over the same connection if possible.
  56. .. describe:: Authentication
  57. The following three types of HTTP Authentication are supported. These can be
  58. used over both HTTP and HTTPS.
  59. * Digest
  60. * Basic
  61. * WSSE
  62. .. describe:: Caching
  63. The module can optionally operate with a private cache that understands the
  64. Cache-Control: header and uses both the ETag and Last-Modified cache validators.
  65. .. describe:: All Methods
  66. The module can handle any HTTP request method, not just GET and POST.
  67. .. describe:: Redirects
  68. Automatically follows 3XX redirects on GETs.
  69. .. describe:: Compression
  70. Handles both ``deflate`` and ``gzip`` types of compression.
  71. .. describe:: Lost update support
  72. Automatically adds back ETags into PUT requests to resources we have already
  73. cached. This implements Section 3.2 of Detecting the Lost Update Problem Using
  74. Unreserved Checkout
  75. The :mod:`httplib2` module defines the following variables:
  76. .. % ==== 3. ====
  77. .. % List the public functions defined by the module. Begin with a
  78. .. % standard phrase. You may also list the exceptions and other data
  79. .. % items defined in the module, insofar as they are important for the
  80. .. % user.
  81. .. % ---- 3.2. ----
  82. .. % Data items are described using a ``datadesc'' block. This has only
  83. .. % one parameter: the item's name.
  84. .. data:: debuglevel
  85. The amount of debugging information to print. The default is 0.
  86. .. data:: RETRIES
  87. A request will be tried 'RETRIES' times if it fails at the socket/connection level.
  88. The default is 2.
  89. The :mod:`httplib2` module may raise the following Exceptions. Note that there
  90. is an option that turns exceptions into normal responses with an HTTP status
  91. code indicating an error occured. See
  92. :attr:`Http.force_exception_to_status_code`
  93. .. % --- 3.3. ---
  94. .. % Exceptions are described using a ``excdesc'' block. This has only
  95. .. % one parameter: the exception name. Exceptions defined as classes in
  96. .. % the source code should be documented using this environment, but
  97. .. % constructor parameters must be omitted.
  98. .. exception:: HttpLib2Error
  99. The Base Exception for all exceptions raised by httplib2.
  100. .. exception:: RedirectMissingLocation
  101. A 3xx redirect response code was provided but no Location: header was provided
  102. to point to the new location.
  103. .. exception:: RedirectLimit
  104. The maximum number of redirections was reached without coming to a final URI.
  105. .. exception:: ServerNotFoundError
  106. Unable to resolve the host name given.
  107. .. exception:: RelativeURIError
  108. A relative, as opposed to an absolute URI, was passed into request().
  109. .. exception:: FailedToDecompressContent
  110. The headers claimed that the content of the response was compressed but the
  111. decompression algorithm applied to the content failed.
  112. .. exception:: UnimplementedDigestAuthOptionError
  113. The server requested a type of Digest authentication that we are unfamiliar
  114. with.
  115. .. exception:: UnimplementedHmacDigestAuthOptionError
  116. The server requested a type of HMACDigest authentication that we are unfamiliar
  117. with.
  118. .. % ---- 3.4. ----
  119. .. % Other standard environments:
  120. .. %
  121. .. % classdesc - Python classes; same arguments are funcdesc
  122. .. % methoddesc - methods, like funcdesc but has an optional parameter
  123. .. % to give the type name: \begin{methoddesc}[mytype]{name}{args}
  124. .. % By default, the type name will be the name of the
  125. .. % last class defined using classdesc. The type name
  126. .. % is required if the type is implemented in C (because
  127. .. % there's no classdesc) or if the class isn't directly
  128. .. % documented (if it's private).
  129. .. % memberdesc - data members, like datadesc, but with an optional
  130. .. % type name like methoddesc.
  131. .. class:: Http([cache=None], [timeout=None], [proxy_info==ProxyInfo.from_environment], [ca_certs=None], [disable_ssl_certificate_validation=False])
  132. The class that represents a client HTTP interface. The *cache* parameter is
  133. either the name of a directory to be used as a flat file cache, or it must an
  134. object that implements the required caching interface. The *timeout* parameter
  135. is the socket level timeout. The *ca_certs* parameter is the filename of the
  136. CA certificates to use. If none is given a default set is used. The
  137. *disable_ssl_certificate_validation* boolean flag determines if ssl certificate validation
  138. is done. The *proxy_info* parameter is an object of type :class:ProxyInfo.
  139. .. class:: ProxyInfo(proxy_type, proxy_host, proxy_port, [proxy_rdns=None], [proxy_user=None], [proxy_pass=None])
  140. Collect information required to use a proxy.
  141. The parameter proxy_type must be set to one of socks.PROXY_TYPE_XXX
  142. constants. For example: ::
  143. p = ProxyInfo(proxy_type=socks.PROXY_TYPE_HTTP, proxy_host='localhost', proxy_port=8000)
  144. .. class:: Response(info)
  145. Response is a subclass of :class:`dict` and instances of this class are
  146. returned from calls to Http.request. The *info* parameter is either an
  147. :class:`rfc822.Message` or an :class:`httplib.HTTPResponse` object.
  148. .. class:: FileCache(dir_name, [safe=safename])
  149. FileCache implements a Cache as a directory of files. The *dir_name* parameter
  150. is the name of the directory to use. If the directory does not exist then
  151. FileCache attempts to create the directory. The optional *safe* parameter is a
  152. funtion which generates the cache filename for each URI. A FileCache object is
  153. constructed and used for caching when you pass a directory name into the
  154. constructor of :class:`Http`.
  155. Http objects have the following methods:
  156. .. % If your module defines new object types (for a built-in module) or
  157. .. % classes (for a module written in Python), you should list the
  158. .. % methods and instance variables (if any) of each type or class in a
  159. .. % separate subsection.
  160. .. _http-objects:
  161. Http Objects
  162. ---------------
  163. .. method:: Http.request(uri, [method="GET", body=None, headers=None, redirections=DEFAULT_MAX_REDIRECTS, connection_type=None])
  164. Performs a single HTTP request. The *uri* is the URI of the HTTP resource and
  165. can begin with either ``http`` or ``https``. The value of *uri* must be an
  166. absolute URI.
  167. The *method* is the HTTP method to perform, such as ``GET``, ``POST``,
  168. ``DELETE``, etc. There is no restriction on the methods allowed.
  169. The *body* is the entity body to be sent with the request. It is a string
  170. object.
  171. Any extra headers that are to be sent with the request should be provided in the
  172. *headers* dictionary.
  173. The maximum number of redirect to follow before raising an exception is
  174. *redirections*. The default is 5.
  175. The *connection_type* is the type of connection object to use. The supplied
  176. class should implement the interface of httplib.HTTPConnection.
  177. The return value is a tuple of (response, content), the first being and instance
  178. of the :class:`Response` class, the second being a string that contains the
  179. response entity body.
  180. .. method:: Http.add_credentials(name, password, [domain=None])
  181. Adds a name and password that will be used when a request requires
  182. authentication. Supplying the optional *domain* name will restrict these
  183. credentials to only be sent to the specified domain. If *domain* is not
  184. specified then the given credentials will be used to try to satisfy every HTTP
  185. 401 challenge.
  186. .. method:: Http.add_certificate(key, cert, domain)
  187. Add a *key* and *cert* that will be used for an SSL connection to the specified
  188. domain. *keyfile* is the name of a PEM formatted file that contains your
  189. private key. *certfile* is a PEM formatted certificate chain file.
  190. .. method:: Http.clear_credentials()
  191. Remove all the names and passwords used for authentication.
  192. .. attribute:: Http.follow_redirects
  193. If ``True``, which is the default, safe redirects are followed, where safe means
  194. that the client is only doing a ``GET`` or ``HEAD`` on the URI to which it is
  195. being redirected. If ``False`` then no redirects are followed. Note that a False
  196. 'follow_redirects' takes precedence over a True 'follow_all_redirects'. Another
  197. way of saying that is for 'follow_all_redirects' to have any affect,
  198. 'follow_redirects' must be True.
  199. .. attribute:: Http.follow_all_redirects
  200. If ``False``, which is the default, only safe redirects are followed, where safe
  201. means that the client is only doing a ``GET`` or ``HEAD`` on the URI to which it
  202. is being redirected. If ``True`` then all redirects are followed. Note that a
  203. False 'follow_redirects' takes precedence over a True 'follow_all_redirects'.
  204. Another way of saying that is for 'follow_all_redirects' to have any affect,
  205. 'follow_redirects' must be True.
  206. .. attribute:: Http.forward_authorization_headers
  207. If ``False``, which is the default, then Authorization: headers are
  208. stripped from redirects. If ``True`` then Authorization: headers are left
  209. in place when following redirects. This parameter only applies if following
  210. redirects is turned on. Note that turning this on could cause your credentials
  211. to leak, so carefully consider the consequences.
  212. .. attribute:: Http.force_exception_to_status_code
  213. If ``True`` then no :mod:`httplib2` exceptions will be
  214. thrown. Instead, those error conditions will be turned into :class:`Response`
  215. objects that will be returned normally.
  216. If ``False``, which is the default, then exceptions will be thrown.
  217. .. attribute:: Http.optimistic_concurrency_methods
  218. By default a list that only contains "PUT", this attribute
  219. controls which methods will get 'if-match' headers attached
  220. to them from cached responses with etags. You can append
  221. new items to this list to add new methods that should
  222. get this support, such as "PATCH".
  223. .. attribute:: Http.ignore_etag
  224. Defaults to ``False``. If ``True``, then any etags present in the cached
  225. response are ignored when processing the current request, i.e. httplib2 does
  226. **not** use 'if-match' for PUT or 'if-none-match' when GET or HEAD requests are
  227. made. This is mainly to deal with broken servers which supply an etag, but
  228. change it capriciously.
  229. If you wish to supply your own caching implementation then you will need to pass
  230. in an object that supports the following methods. Note that the :mod:`memcache`
  231. module supports this interface natively.
  232. .. _cache-objects:
  233. Cache Objects
  234. --------------
  235. .. method:: Cache.get(key)
  236. Takes a string *key* and returns the value as a string.
  237. .. method:: Cache.set(key, value)
  238. Takes a string *key* and *value* and stores it in the cache.
  239. .. method:: Cache.delete(key)
  240. Deletes the cached value stored at *key*. The value of *key* is a string.
  241. Response objects are derived from :class:`dict` and map header names (lower case
  242. with the trailing colon removed) to header values. In addition to the dict
  243. methods a Response object also has:
  244. .. _response-objects:
  245. Response Objects
  246. ------------------
  247. .. attribute:: Response.fromcache
  248. If ``true`` the the response was returned from the cache.
  249. .. attribute:: Response.version
  250. The version of HTTP that the server supports. A value of 11 means '1.1'.
  251. .. attribute:: Response.status
  252. The numerical HTTP status code returned in the response.
  253. .. attribute:: Response.reason
  254. The human readable component of the HTTP response status code.
  255. .. attribute:: Response.previous
  256. If redirects are followed then the :class:`Response` object returned is just for
  257. the very last HTTP request and *previous* points to the previous
  258. :class:`Response` object. In this manner they form a chain going back through
  259. the responses to the very first response. Will be ``None`` if there are no
  260. previous respones.
  261. The Response object also populates the header ``content-location``, that
  262. contains the URI that was ultimately requested. This is useful if redirects were
  263. encountered, you can determine the ultimate URI that the request was sent to.
  264. All Response objects contain this key value, including ``previous`` responses so
  265. you can determine the entire chain of redirects. If
  266. :attr:`Http.force_exception_to_status_code` is ``True`` and the number of
  267. redirects has exceeded the number of allowed number of redirects then the
  268. :class:`Response` object will report the error in the status code, but the
  269. complete chain of previous responses will still be in tact.
  270. To do a simple ``GET`` request just supply the absolute URI of the resource:
  271. .. % ==== 4. ====
  272. .. % Now is probably a good time for a complete example. (Alternatively,
  273. .. % an example giving the flavor of the module may be given before the
  274. .. % detailed list of functions.)
  275. .. _httplib2-example:
  276. Examples
  277. ---------
  278. ::
  279. import httplib2
  280. h = httplib2.Http()
  281. resp, content = h.request("http://bitworking.org/")
  282. assert resp.status == 200
  283. assert resp['content-type'] == 'text/html'
  284. Here is more complex example that does a PUT of some text to a resource that
  285. requires authentication. The Http instance also uses a file cache in the
  286. directory ``.cache``. ::
  287. import httplib2
  288. h = httplib2.Http(".cache")
  289. h.add_credentials('name', 'password')
  290. resp, content = h.request("https://example.org/chap/2",
  291. "PUT", body="This is text",
  292. headers={'content-type':'text/plain'} )
  293. Here is an example that connects to a server that supports the Atom Publishing
  294. Protocol. ::
  295. import httplib2
  296. h = httplib2.Http()
  297. h.add_credentials(myname, mypasswd)
  298. h.follow_all_redirects = True
  299. headers = {'Content-Type': 'application/atom+xml'}
  300. body = """<?xml version="1.0" ?>
  301. <entry xmlns="http://www.w3.org/2005/Atom">
  302. <title>Atom-Powered Robots Run Amok</title>
  303. <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
  304. <updated>2003-12-13T18:30:02Z</updated>
  305. <author><name>John Doe</name></author>
  306. <content>Some text.</content>
  307. </entry>
  308. """
  309. uri = "http://www.example.com/collection/"
  310. resp, content = h.request(uri, "POST", body=body, headers=headers)
  311. Here is an example of providing data to an HTML form processor. In this case we
  312. presume this is a POST form. We need to take our data and format it as
  313. "application/x-www-form-urlencoded" data and use that as a body for a POST
  314. request.
  315. ::
  316. >>> import httplib2
  317. >>> import urllib
  318. >>> data = {'name': 'fred', 'address': '123 shady lane'}
  319. >>> body = urllib.urlencode(data)
  320. >>> body
  321. 'name=fred&address=123+shady+lane'
  322. >>> h = httplib2.Http()
  323. >>> resp, content = h.request("http://example.com", method="POST", body=body)