index.gsl 17 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583
  1. .macro:: header
  2. ..header
  3. ...link
  4. ....rel=stylesheet
  5. ....href=css/guppy.css
  6. .document: index
  7. ..output: html
  8. ..document_title: Guppy-PE: A Python Programming Environment
  9. ..use: header
  10. ..div
  11. ...dl
  12. ....dt
  13. .....h1:
  14. ......big: Guppy-PE
  15. ....dd
  16. .....h1: A Python Programming Environment
  17. ...hr
  18. ...table
  19. ....tr
  20. .....td
  21. ......a: Guppy
  22. .......href=#Guppy
  23. .....td: A fish swimming in Python
  24. ....tr
  25. .....td
  26. ......a: Heapy
  27. .......href=#Heapy
  28. .....td: Heap Analysis Toolset
  29. ....tr
  30. .....td
  31. ......a: GSL
  32. .......href=#GSL
  33. .....td: Guppy Specification Language
  34. ....tr
  35. .....td
  36. ......a: Documentation
  37. .......href=#Documentation
  38. ....tr
  39. .....td
  40. ......a: Download
  41. .......href=#Download
  42. ....tr
  43. .....td
  44. ......a: Credits
  45. .......href=#Credits
  46. ....tr
  47. .....td
  48. ......a: Contact
  49. .......href=#Contact
  50. ...hr
  51. ..p: This is the home page for
  52. ...big:
  53. ....strong: Guppy-PE
  54. ...t:, a programming environment providing object and heap memory
  55. sizing, profiling and analysis. It includes a prototypical specification
  56. language that can be used to formally specify aspects of
  57. ...a: Python
  58. ....href= http://www.python.org
  59. ...t: programs and generate tests and documentation from a common
  60. source.
  61. ..a
  62. ...name=Guppy
  63. ...h2: Guppy
  64. ..p: Guppy is an umbrella package combining Heapy and GSL with support
  65. utilities such as the Glue module that keeps things together.
  66. ..p: The name guppy was chosen because I found it in a backward-dictionary
  67. as a word ending with py and I thought it was cute enough and that it
  68. would not so likely conflict with some other package name. It was to
  69. be a general name since all kinds of packages should fit under this
  70. top level name.
  71. ..p: The name guppy-pe is because there was another project named guppy in
  72. Sourceforge when I was about to register guppy. The other guppy was
  73. not in Python, so I added
  74. ...strong: -pe
  75. ...t: which means Programming Environment. The Python package is just guppy.
  76. ..a
  77. ...name=Heapy
  78. ...h2: Heapy
  79. ..p: The aim of Heapy is to support debugging and optimization
  80. regarding memory related issues in Python programs.
  81. ..p: Such issues can make a program use too much memory, making it
  82. slow by itself as well as slowing down an entire server, or it may
  83. fail to run at all in a limited memory device such as a mobile phone.
  84. ..p: The primary motivation for Heapy is that there has been a lack of
  85. support for the programmer to get information about the memory usage
  86. in Python programs. Heapy is an attempt to improve this situation. A
  87. project with a similar intent is
  88. ...a: PySizer.
  89. ....href=http://pysizer.8325.org
  90. ..p: The problem situation has a number of aspects, which I think can be
  91. characterised, for example, as follows.
  92. ..ul
  93. ...li: What data about memory and objects to get from the system.
  94. ...li: How to get that data technically.
  95. ...li: How portable to be across platforms and versions.
  96. ...li: How to process data to compress it and find structure in it.
  97. ...li: How to present the resulting information to the user.
  98. ...li: How to make a coherent, useful & usable system of it all
  99. that doesn't interfer too much with the target system it is analysing.
  100. ..p: As Heapy has evolved, with considerations like this in mind,
  101. it currently provides the following features.
  102. ..div
  103. ...h3: Data gathering
  104. ...div
  105. ....ul
  106. .....li: Finds reachable and/or unreachable objects in the object
  107. heap, and collects them into special C-implemented 'nodesets'. Can get
  108. data about the objects such as their sizes and how they refer to each
  109. other.
  110. .....li: Uses a C library that can get data about non-standard
  111. types from extension modules, given a function table.
  112. .....li: Optionally uses multiple Python interpreters in the same
  113. process, so one can monitor the other transparently.
  114. ...h3: Data processing
  115. ...div
  116. ....ul:
  117. .....li: Algebraic set operations, for example the set difference can
  118. be used to extract the objects allocated after a reference point in
  119. time.
  120. .....li: Various classifications of object sets, and different
  121. classifiers can be combined.
  122. .....li: Shortest paths to a set of objects from other objects,
  123. which can be used to find out why the objects are retained in memory.
  124. .....li: Calculation of the 'dominated' set from a set of root
  125. objects which yields the set of objects that would be deallocated if the
  126. root objects were deallocated.
  127. ...h3: Presentation
  128. ...div
  129. ....ul
  130. .....li: Tables where each row represents a classification of data.
  131. .....li: Lists of shortest paths where the edges show the
  132. relationships found between the underlying C objects.
  133. .....li: Reference pattern, presenting a spanning tree of the
  134. graph with sets of objects treated as a unit.
  135. .....li: Limits the number of rows when presentation objects are
  136. shown, without depending on an external pager.
  137. .....li: An interactive graphical browser program can show a time
  138. sequence of classified heap data sets as a graph together with a table
  139. detailing the data at a specific time or the difference between two
  140. points in time.
  141. ...h3: Portability aspects
  142. ...div
  143. ....ul
  144. .....li: Can be used with an unmodified C Python, back to version 2.3
  145. AFAIK. Does not depend on any external unix-specific or other
  146. utilities.
  147. .....li: Requires Tk if the graphical browser is to be used.
  148. .....li: Can not be used with Jython or other non-C Python versions.
  149. ...h3: System aspects
  150. ...div
  151. ....ul
  152. .....li: A general 'glue' model provides a session context that
  153. imports modules and creates objects automatically when accessed. The
  154. glue model is not Heapy specific but is used throughout Guppy and
  155. could be used by other packages as well.
  156. .....li: The glue model makes it practical to have everything in Guppy
  157. being dynamically allocated in a session context, so there is no need
  158. for any global module-level variables. The modules themself are stored
  159. as usual in sys.modules but they are not modified.
  160. .....li: To be true there is one exception I come to think of but
  161. it is really exceptional.
  162. ..p: Heapy has been used during development of itself and of the other
  163. parts of Guppy. It has been used to tell how much memory the parts of
  164. compound objects use, to see what could be worthwhile to optimize. It
  165. was used to find a memory leak in the Heapy profile browser, and to
  166. find out the cause, which as far as I can tell was due to a bug in a
  167. library routine which I have reported.
  168. ..a
  169. ...name=GSL
  170. ...h2: GSL
  171. ..p: The Guppy Specification Language is an evolving specification
  172. language. I started experimenting with this language because I felt
  173. the need to have a way to specify documentation and tests from the
  174. same source. GSL can describe aspects of a system, especially its API,
  175. in a way that can be automatically converted to tests as well as to
  176. documents. The documents generated have a formal structure for
  177. describing the formal aspects of the specification, complemented with
  178. descriptive text from the same source documents. A language that is
  179. similar in intent is the
  180. ...a: Assertion Definition Language
  181. ....href=http://adl.opengroup.org/
  182. ...t:.
  183. ..p: Specifications written in GSL can be used for:
  184. ..ul
  185. ...li: Generating documentation
  186. ....p: Documents are generated from a combination of formally
  187. described aspects and descriptive text. The formal aspects include
  188. specifications of attributes and methods and their parameters and
  189. return values. The descriptive text is written together with the
  190. formal specifications. It can include HTML tags and attributes,
  191. specified in the GSL dotted tree format. Output modules can convert
  192. from the GSL form to formats such as HTML or Latex.
  193. ...li: Generating tests
  194. ....p: The tests generated can check the formal aspects of the
  195. specification against an implementation. It can test whether objects
  196. have the promised attributes, and then if the kind of attribute is
  197. specified it is tested again and so on. Methods are checked to see if
  198. they can be called with the forms of parameters specified. The return
  199. value can then be checked up to some limit of recursion.
  200. ...li: Resolving compatibility issues
  201. ....p: A specification can be compared to a previous specification, to
  202. tell according to formalized rules whether or not the new one is
  203. backwards compatible with the old one, and if not, in what way they
  204. differ. For example, according to such rules, an attribute or
  205. parameter type can be added to a specification, but it can not be
  206. removed. The idea is that all tests possibly generated from the old
  207. specification should still succeed with objects conforming to the new
  208. specification. -- A program could likely compare specifications for
  209. compatibility automatically, but this is not yet implemented.
  210. ..p: GSL has been used to generate the documentation for this Guppy
  211. distribution. Some part of the specification has been checked against
  212. the implementation using the generated tests, which did reveal some
  213. discrepancies that were subsequently corrected.
  214. ..p: The documents generated by GSL use a formal syntax to describe
  215. parameter modes.
  216. ...a: This
  217. ....href=docexample.html
  218. ...t: document contains examples of such parameter descriptions and
  219. explains what they mean.
  220. ..a
  221. ...name=Documentation
  222. ...h2: Documentation
  223. ..p: Some documentation is included with the source code distribution
  224. and can also be browsed here via the following links.
  225. ...table:
  226. ....class=list
  227. ....tr:
  228. .....class=even
  229. .....td
  230. ......a: Getting started with Heapy
  231. .......href=heapy_tutorial.html
  232. .....td: An example of how to get started with Heapy
  233. ....tr:
  234. .....class=odd
  235. .....td
  236. ......a: Document example
  237. .......href=docexample.html
  238. .....td: Explains the meaning of some aspects of the documents.
  239. ....tr:
  240. .....class=even
  241. .....td
  242. ......a: Guppy
  243. .......href=guppy.html
  244. .....td: Specification of
  245. ......code: guppy
  246. ......t:, the top level module.
  247. ....tr:
  248. .....class=odd
  249. .....td
  250. ......a: Profile Browser
  251. .......href=ProfileBrowser.html
  252. .....td: How to use the graphical heap profile browser.
  253. ....tr:
  254. .....class=even
  255. .....td
  256. ......a: Screenshot
  257. .......href=pbscreen.jpg
  258. .....td: Example showing the graphical heap profile browser in action.
  259. ....tr:
  260. .....class=odd
  261. .....td
  262. ......a: GSL
  263. .......href=gsl.html
  264. .....td: The Guppy Specification Language.
  265. ....tr:
  266. .....class=even
  267. .....td
  268. ......a: heapyc
  269. .......href=heapyc.html
  270. .....td: Specification of the heapyc extension module. Note that this
  271. is an internal interface and may be subject to change.
  272. ....tr:
  273. .....class=odd
  274. .....td
  275. ......a: sets
  276. .......href=sets.html
  277. .....td: Specification of the interface to the
  278. setsc extension module which contains bitsets and nodesets.
  279. ....tr:
  280. .....td
  281. ......id=last
  282. ......colspan=5
  283. ..p: The following documentation is not included with the source code.
  284. ...table
  285. ....class=list
  286. ....tr:
  287. .....class=odd
  288. .....td
  289. ......a: heapy-thesis.pdf
  290. .......href=http://guppy-pe.sourceforge.net/heapy-thesis.pdf
  291. .....td: The master's thesis, "Heapy: A Memory Profiler and Debugger
  292. for Python", which presents background, design, implementation,
  293. rationale and some use cases for Heapy (version 0.1).
  294. ....tr
  295. .....class=even
  296. .....td
  297. ......a: Metadata and Abstract
  298. .......href=http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-7247
  299. .......c:href=http://www.diva-portal.org/liu/abstract.xsql?dbid=7247
  300. .....td: Published at
  301. ......a: Linköping University Electronic Press.
  302. .......href= http://www.ep.liu.se/
  303. ....tr:
  304. .....class=odd
  305. .....td
  306. ......a: heapy-presentation.pdf
  307. .......href=http://guppy-pe.sourceforge.net/heapy-presentation.pdf
  308. .....td: Slides from the presentation.
  309. ....tr
  310. .....td
  311. ......id=last
  312. ......colspan=5
  313. ..h3: External resources
  314. ..p: Thanks to all that have written about Heapy on various blogs. I
  315. think the following links may be especially useful to get
  316. started. More tips are very welcome!
  317. ..ul
  318. ...li
  319. ....a:How to use guppy/heapy for tracking down memory usage
  320. .....href=http://www.pkgcore.org/trac/pkgcore/doc/dev-notes/heapy.rst
  321. ....t: (pkgcore - Trac)
  322. ...li
  323. ....a:Debugging Django memory leak with TrackRefs and Guppy
  324. .....href=http://blog.redinnovation.com/2008/03/07/debugging-django-memory-leak-with-trackrefs-and-guppy
  325. ....t: (Redi for Life)
  326. ..a
  327. ...name=Download
  328. ...h2: Download
  329. ..p
  330. The latest version is in the svn trunk directory.
  331. ..p
  332. In Version 0.1.9 I have introduced interactive help and compatibility
  333. for MS compilers (see ANNOUNCE and ChangeLog).
  334. ..p: As of 2009-06-23, I have tested the latest revision successfully on
  335. an AMD64 in 64 bits mode with Ubuntu 7.10 with Python 2.3, 2.4, 2.5
  336. and 2.6, and in 32 bits mode with Ubuntu 9.04 with Python 2.4, 2.5 and
  337. 2.6 .
  338. ..p: To check out the latest (HEAD) revision, you can do:
  339. ..pre
  340. svn co https://guppy-pe.svn.sourceforge.net/svnroot/guppy-pe/trunk/guppy guppy
  341. ..p: To check out the latest release as of this writing
  342. (2009-06-23, guppy-0.1.9), you can do:
  343. ..pre
  344. svn co -r79 https://guppy-pe.svn.sourceforge.net/svnroot/guppy-pe/trunk/guppy guppy
  345. ..h3: Source code releases in tarball format
  346. ..table
  347. ...class=list
  348. ...style=margin-bottom: 10px;
  349. ...tr
  350. ....class=even
  351. ....td
  352. .....a: guppy-0.1.9
  353. ......href=http://pypi.python.org/pypi/guppy/0.1.9
  354. ....td: Updated 2009-06-23. Added interactive help and improved
  355. MS compatibility.
  356. ...tr
  357. ....class=odd
  358. ....td
  359. .....a: guppy-0.1.8
  360. ......href=http://pypi.python.org/pypi/guppy/0.1.8
  361. ....td: Updated 2008-04-08. Works with 64 bits and Python 2.6 .
  362. ...tr
  363. ....class=even
  364. ....td
  365. .....a: guppy-0.1.6.tar.gz
  366. ......href=http://guppy-pe.sourceforge.net/guppy-0.1.6.tar.gz
  367. ....td: Updated 2006-10-16. Doesn't work with 64 bits. -- Quick bug fix version,
  368. correcting the name of the Root object in the hpy instance. This is a
  369. kind of bug the automatic test generation should have catched, since
  370. it was specified with another name in the documentation, but I yet
  371. need some time to get that to work... so I am just uploading this
  372. quick fix now.
  373. ...tr
  374. ....class=odd
  375. ....td
  376. .....a: guppy-0.1.5.tar.gz
  377. ......href=http://guppy-pe.sourceforge.net/guppy-0.1.5.tar.gz
  378. ....td: Updated 2006-10-12. Fixed bugs wrt remote
  379. monitoring and HTML rendering. New features include the shorthand sp
  380. for shpaths and representing the source of the shortest paths in terms
  381. of a Root object in the hpy instance. See changelog.
  382. ...tr
  383. ....class=even
  384. ....td
  385. .....a: guppy-0.1.4.tar.gz
  386. ......href=http://guppy-pe.sourceforge.net/guppy-0.1.4.tar.gz
  387. ....td: Updated 2006-10-11. Most changes are to make it work with
  388. Python 2.5; other changes include improved error reporting in Glue.py
  389. and some test improvements.
  390. ...tr
  391. ....class=odd
  392. ....td
  393. .....a: guppy-0.1.3.tar.gz
  394. ......href=http://guppy-pe.sourceforge.net/guppy-0.1.3.tar.gz
  395. ....td: Updated 2006-03-02.
  396. Updates to Monitor so multiple lines work. It also got a command to
  397. interrupt the remote process. Cleanups and bugfixes especially todo
  398. with Python2.4 (used to crash with array objects). A bunch of other
  399. fixes, see changelog.
  400. ...tr
  401. ....class=even
  402. ....td
  403. .....a: guppy-0.1.2.tar.gz
  404. ......href=http://guppy-pe.sourceforge.net/guppy-0.1.2.tar.gz
  405. ....td: Pointer comparison bugs and test portability problems were
  406. fixed. See the included changelog.
  407. ...tr
  408. ....class=odd
  409. ....td
  410. .....a: guppy-0.1.1.tar.gz
  411. ......href=http://guppy-pe.sourceforge.net/guppy-0.1.1.tar.gz
  412. ....td: The C source code for the extension modules
  413. was changed to be ANSI compatible and I also changed some help text that
  414. had become outdated.
  415. ...tr
  416. ....class=even
  417. ....td
  418. .....a: guppy-0.1.tar.gz
  419. ......href=http://guppy-pe.sourceforge.net/guppy-0.1.tar.gz
  420. ....td: Original version. Extension modules could not be compiled using
  421. strict ANSI C compilers.
  422. ...tr
  423. ....td
  424. .....id=last
  425. .....colspan=5
  426. ..c:
  427. ...a: http://sourceforge.net/project/showfiles.php?group_id=105577
  428. ....href=http://sourceforge.net/project/showfiles.php?group_id=105577
  429. ..a
  430. ...name=Credits
  431. ...h2: Credits
  432. ..ul
  433. ...li: Claudio Grondi has tested in Microsoft XP and reported the
  434. problems that occurred. The ANSI-compatibility problems I think have
  435. been fixed but there also seemed to be problems with the .NET C compiler
  436. which I haven't had a chance to sort out yet.
  437. ...li: Patrik Andersson has helped me installing Ubuntu Linux on my new
  438. computer and to make it dual-boot with Microsoft XP. (But we don't have
  439. any C compiler for Windows XP (yet).)
  440. ...li: Chad Austin has supplied a patch for compiling on Windows w/
  441. Visual Studio 2003 and Python 2.5. I think this may fix similar
  442. problems with other MS compilers, Cygwin & Mingw as well.
  443. ...li: Yaroslav Halchenko is packaging and uploading Guppy into the
  444. Debian distribution.
  445. ..a
  446. ...name=Contact
  447. ...h2: Contact
  448. ..dl
  449. ...dt: The author, Sverker Nilsson, may be contacted at:
  450. ...dd
  451. ....address
  452. .....a: svenil@users.sourceforge.net
  453. ......href=mailto:svenil@users.sourceforge.net
  454. ...dt: I have registered a mailing list for discussions, questions,
  455. announcements etc. The list information, subscription form and archives
  456. are available at:
  457. ...dd
  458. ....a: http://lists.sourceforge.net/mailman/listinfo/guppy-pe-list
  459. .....href=http://lists.sourceforge.net/mailman/listinfo/guppy-pe-list
  460. ...dt: Please report bugs preferably via the SourceForge bug tracking
  461. system:
  462. ...dd
  463. ....a: http://sourceforge.net/tracker/?group_id=105577&atid=641821
  464. .....href=http://sourceforge.net/tracker/?group_id=105577&atid=641821
  465. ...dt: The Sourceforge project summary page is:
  466. ...dd
  467. ....a: http://sourceforge.net/projects/guppy-pe
  468. .....href= http://sourceforge.net/projects/guppy-pe