فهرست منبع

HUE-141. Adding Shell app to Hue.

The Shell app is a Hue application that allows for access to command-line utilities
via Hue.
Aditya Acharya 14 سال پیش
والد
کامیت
db03e8e817
100فایلهای تغییر یافته به همراه7994 افزوده شده و 2 حذف شده
  1. 1 0
      README.rst
  2. 1 0
      apps/Makefile
  3. 32 0
      apps/shell/Makefile
  4. 19 0
      apps/shell/conf/hue-shell.ini
  5. 47 0
      apps/shell/setup.py
  6. 25 0
      apps/shell/src/shell/Makefile
  7. 15 0
      apps/shell/src/shell/__init__.py
  8. 75 0
      apps/shell/src/shell/conf.py
  9. 62 0
      apps/shell/src/shell/constants.py
  10. 15 0
      apps/shell/src/shell/forms.py
  11. 15 0
      apps/shell/src/shell/models.py
  12. 27 0
      apps/shell/src/shell/settings.py
  13. 228 0
      apps/shell/src/shell/setuid.c
  14. 685 0
      apps/shell/src/shell/shellmanager.py
  15. BIN
      apps/shell/src/shell/static/art/shell.png
  16. 37 0
      apps/shell/src/shell/static/bootstrap.js
  17. 82 0
      apps/shell/src/shell/static/css/shell.css
  18. 1 0
      apps/shell/src/shell/static/help/index.md
  19. 235 0
      apps/shell/src/shell/static/js/Source/Shell/Poller.js
  20. 450 0
      apps/shell/src/shell/static/js/Source/Shell/Shell.js
  21. 8 0
      apps/shell/src/shell/static/js/package.yml
  22. 19 0
      apps/shell/src/shell/templates/failed_to_create.mako
  23. 30 0
      apps/shell/src/shell/templates/index.mako
  24. 16 0
      apps/shell/src/shell/templates/no_such_user.mako
  25. 15 0
      apps/shell/src/shell/templates/not_running_spawning.mako
  26. 49 0
      apps/shell/src/shell/templates/shared_components.mako
  27. 162 0
      apps/shell/src/shell/tests.py
  28. 28 0
      apps/shell/src/shell/urls.py
  29. 98 0
      apps/shell/src/shell/utils.py
  30. 122 0
      apps/shell/src/shell/views.py
  31. 23 0
      apps/shell/src/shell/windmilltests.py
  32. 7 2
      desktop/conf.dist/hue.ini
  33. 9 0
      desktop/core/ext-py/Spawning-0.9.6/AUTHORS.txt
  34. 19 0
      desktop/core/ext-py/Spawning-0.9.6/LICENSE.txt
  35. 4 0
      desktop/core/ext-py/Spawning-0.9.6/MANIFEST.in
  36. 115 0
      desktop/core/ext-py/Spawning-0.9.6/NEWS.txt
  37. 126 0
      desktop/core/ext-py/Spawning-0.9.6/PKG-INFO
  38. 108 0
      desktop/core/ext-py/Spawning-0.9.6/README.rst
  39. 167 0
      desktop/core/ext-py/Spawning-0.9.6/pip-log.txt
  40. 67 0
      desktop/core/ext-py/Spawning-0.9.6/rc-scripts/conf.d/spawning
  41. 91 0
      desktop/core/ext-py/Spawning-0.9.6/rc-scripts/init.d/spawning
  42. 115 0
      desktop/core/ext-py/Spawning-0.9.6/rc-scripts/init.d/spawning.debian
  43. 5 0
      desktop/core/ext-py/Spawning-0.9.6/setup.cfg
  44. 74 0
      desktop/core/ext-py/Spawning-0.9.6/setup.py
  45. 37 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/__init__.py
  46. 58 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/django_factory.py
  47. 94 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/memory_watcher.py
  48. 102 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/paste_factory.py
  49. 119 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/reloader_dev.py
  50. 143 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/reloader_svn.py
  51. 341 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/spawning_child.py
  52. 521 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/spawning_controller.py
  53. 55 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/util/__init__.py
  54. 171 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/util/log_parser.py
  55. 366 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/util/status.py
  56. 96 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/util/system.py
  57. 82 0
      desktop/core/ext-py/Spawning-0.9.6/src/spawning/wsgi_factory.py
  58. 71 0
      desktop/core/ext-py/eventlet-0.9.14/AUTHORS
  59. 23 0
      desktop/core/ext-py/eventlet-0.9.14/LICENSE
  60. 4 0
      desktop/core/ext-py/eventlet-0.9.14/MANIFEST.in
  61. 334 0
      desktop/core/ext-py/eventlet-0.9.14/NEWS
  62. 68 0
      desktop/core/ext-py/eventlet-0.9.14/PKG-INFO
  63. 50 0
      desktop/core/ext-py/eventlet-0.9.14/README
  64. 181 0
      desktop/core/ext-py/eventlet-0.9.14/README.twisted
  65. 94 0
      desktop/core/ext-py/eventlet-0.9.14/doc/Makefile
  66. BIN
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_images/threading_illustration.png
  67. 4 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/authors.txt
  68. 83 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/basic_usage.txt
  69. 113 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/design_patterns.txt
  70. 21 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/environment.txt
  71. 106 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/examples.txt
  72. 10 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/history.txt
  73. 54 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/hubs.txt
  74. 50 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/index.txt
  75. 19 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules.txt
  76. 27 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/backdoor.txt
  77. 6 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/corolocal.txt
  78. 61 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/db_pool.txt
  79. 5 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/debug.txt
  80. 5 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/event.txt
  81. 6 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/greenpool.txt
  82. 5 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/greenthread.txt
  83. 5 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/pools.txt
  84. 5 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/queue.txt
  85. 11 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/semaphore.txt
  86. 93 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/timeout.txt
  87. 31 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/websocket.txt
  88. 74 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/wsgi.txt
  89. 70 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/patching.txt
  90. 58 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/ssl.txt
  91. 96 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/testing.txt
  92. 30 0
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/threading.txt
  93. BIN
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_static/file.png
  94. BIN
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_static/minus.png
  95. BIN
      desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_static/plus.png
  96. 4 0
      desktop/core/ext-py/eventlet-0.9.14/doc/authors.rst
  97. 83 0
      desktop/core/ext-py/eventlet-0.9.14/doc/basic_usage.rst
  98. 4 0
      desktop/core/ext-py/eventlet-0.9.14/doc/common.txt
  99. 203 0
      desktop/core/ext-py/eventlet-0.9.14/doc/conf.py
  100. 113 0
      desktop/core/ext-py/eventlet-0.9.14/doc/design_patterns.rst

+ 1 - 0
README.rst

@@ -62,6 +62,7 @@ Development Prerequisites
     Debian:
       * asciidoc
       * gcc
+      * g++
       * libmysqlclient-dev
       * libsasl2-dev
       * libsqlite3-dev

+ 1 - 0
apps/Makefile

@@ -37,6 +37,7 @@ APPS := about \
 	jobbrowser \
 	jobsub \
 	proxy \
+        shell \
 	useradmin
 
 # <<<< DEV ONLY

+ 32 - 0
apps/shell/Makefile

@@ -0,0 +1,32 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+ifeq ($(ROOT),)
+  $(error "Error: Expect the environment variable $$ROOT to point to the Desktop installation")
+endif
+
+include $(ROOT)/Makefile.sdk
+
+compile: setuid
+
+clean:: remove_setuid
+
+.PHONY: setuid
+setuid:
+	make -C src/shell setuid
+
+.PHONY: remove_setuid
+remove_setuid:
+	make -C src/shell clean

+ 19 - 0
apps/shell/conf/hue-shell.ini

@@ -0,0 +1,19 @@
+## Configuration options for the Hue Shell app.
+
+[shell]
+## These are commented out so you know where to put lines like these if necessary.
+#shell_buffer_amount=100
+#shell_delegation_token_dir=/tmp/hue_delegation_tokens
+[[ shelltypes ]]
+[[[ flume ]]]
+nice_name = "Flume Shell"
+command = "flume shell"
+help = "The command-line Flume client interface."
+[[[ pig ]]]
+nice_name = "Pig Shell (Grunt)"
+command = "pig -l /dev/null"
+help = "The command-line interpreter for Pig"
+[[[ hbase ]]]
+nice_name = "HBase Shell"
+command = "hbase shell"
+help = "The command-line HBase client interface."

+ 47 - 0
apps/shell/setup.py

@@ -0,0 +1,47 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from setuptools import setup, find_packages
+import os
+
+def expand_package_data(src_dirs, strip=""):
+  ret = []
+  for src_dir in src_dirs:
+    for path, dnames, fnames in os.walk(src_dir):
+      for fname in fnames:
+        ret.append(os.path.join(path, fname).replace(strip, ""))
+  return ret
+
+os.chdir(os.path.dirname(os.path.abspath(__file__)))
+setup(
+  name = "shell",
+  version = "0.1",
+  url = 'http://github.com/cloudera/hue',
+  description = 'Shell interface in Hue',
+  author = 'Hue',
+  packages = find_packages('src'),
+  package_dir = {'': 'src'},
+  install_requires = ['setuptools', 'desktop'],
+  entry_points = { 'desktop.sdk.application': 'shell=shell' },
+  zip_safe = False,
+  package_data = {
+    # Include static resources.  Package_data doesn't
+    # deal well with directory globs, so we enumerate
+    # the files manually.
+    'shell': expand_package_data(
+      ["src/shell/templates", "src/shell/static"],
+      "src/shell/")
+  }
+)

+ 25 - 0
apps/shell/src/shell/Makefile

@@ -0,0 +1,25 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+CC := gcc
+CFLAGS := -Werror -Wall
+
+setuid: setuid.c
+	$(CC) $(CFLAGS) $< -o $@
+	chmod 4711 $@
+
+clean:
+	rm -f setuid

+ 15 - 0
apps/shell/src/shell/__init__.py

@@ -0,0 +1,15 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

+ 75 - 0
apps/shell/src/shell/conf.py

@@ -0,0 +1,75 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+Configuration options for the Shell UI.
+This file specifies the structure that hue-shell.ini should follow.
+See conf/hue-shell.ini to configure which shells are available.
+"""
+from desktop.lib.conf import Config, ConfigSection, UnspecifiedConfigSection
+import shell.utils as utils
+
+SHELL_TYPES = UnspecifiedConfigSection(
+                           key='shelltypes',
+                           each=ConfigSection(members=dict(
+                               nice_name=Config(key='nice_name', required=True),
+                               command=Config(key='command', required=True),
+                               help_doc=Config(key='help', required=False))))
+
+SHELL_BUFFER_AMOUNT = Config(
+  key="shell_buffer_amount",
+  help="Configure the number of output characters buffered for each shell",
+  default=524288,
+  type=int)
+
+SHELL_TIMEOUT = Config(
+  key="shell_timeout",
+  help="Number of seconds to keep shells open for users",
+  default=600,
+  type=int)
+
+SHELL_WRITE_BUFFER_LIMIT = Config(
+  key="shell_write_buffer_limit",
+  help="Number of bytes of commands to buffer for users",
+  default=10000,
+  type=int)
+
+SHELL_OS_READ_AMOUNT = Config(
+  key="shell_os_read_amount",
+  help="Number of bytes to read from child subprocess at a time",
+  default=40960,
+  type=int)
+
+SHELL_DELEGATION_TOKEN_DIR = Config(
+  key="shell_delegation_token_dir",
+  help="The directory to store the temporary delegation tokens used by shell subprocesses",
+  default="/tmp/hue_shell_delegation_tokens",
+  type=str)
+
+def config_validator():
+  """
+  config_validator() -> [ (config_variable, error_message) ]
+
+  Called by core check_config() view.
+  """
+  result = []
+  for item in SHELL_TYPES.keys():
+    command = SHELL_TYPES[item].command.get().strip().split()
+    nice_name = SHELL_TYPES[item].nice_name.get().strip()
+    if not utils.executable_exists(command):
+      result.append((SHELL_TYPES, "Command '%s' for entry '%s' in Shell app configuration cannot \
+                                            be found on the path." % (' '.join(command), item,) ,))
+  return result
+

+ 62 - 0
apps/shell/src/shell/constants.py

@@ -0,0 +1,62 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+A file to store all the constants in one place. Most constants
+are the members of JSON objects, which are stored here for
+easy reference.
+"""
+
+SHELL_ID = "shellId"
+NOT_LOGGED_IN = "notLoggedIn"
+SHELL_KILLED = "shellKilled"
+SUCCESS = "success"
+
+
+# Parameter/JSON object member names
+ALIVE = "alive"
+EXITED = "exited"
+OUTPUT = "output"
+COMMAND = "lineToSend"
+COMMANDS = "commands"
+KEY_NAME = "keyName"
+NICE_NAME = "niceName"
+OFFSET = "offset"
+NEXT_OFFSET = "nextOffset"
+NO_SHELL_EXISTS = "noShellExists"
+BUFFER_EXCEEDED = "bufferExceeded"
+PERIODIC_RESPONSE = "periodicResponse"
+SHELL_CREATE_FAILED = "shellCreateFailed"
+MORE_OUTPUT_AVAILABLE = "moreOutputAvailable"
+NUM_PAIRS = "numPairs"
+CANCELLED = "cancelled"
+NOT_RUNNING_SPAWNING = "notRunningSpawning"
+IS_TAB = "isTab"
+NO_SUCH_USER = "noSuchUser"
+SHELL_NOT_ALLOWED = "shellNotAllowed"
+HOME = "HOME"
+HADOOP_TOKEN_FILE_LOCATION = 'HADOOP_TOKEN_FILE_LOCATION'
+EXISTS = "exists"
+
+# HTTP Headers used
+HUE_INSTANCE_ID = "HTTP_HUE_INSTANCE_ID"
+
+# Required environment variables
+PRESERVED_ENVIRONMENT_VARIABLES = ["JAVA_HOME", "HADOOP_HOME", "PATH", "LC_ALL", "LANG",
+              "LC_COLLATE", "LC_CTYPE", "LC_MESSAGES", "LC_MONETARY", "LC_NUMERIC", "LC_TIME", "TZ",
+              "FLUME_CONF_DIR"]
+
+BROWSER_REQUEST_TIMEOUT = 55    # seconds

+ 15 - 0
apps/shell/src/shell/forms.py

@@ -0,0 +1,15 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

+ 15 - 0
apps/shell/src/shell/models.py

@@ -0,0 +1,15 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

+ 27 - 0
apps/shell/src/shell/settings.py

@@ -0,0 +1,27 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+DJANGO_APPS = [ "shell" ]
+REQUIRES_HADOOP = False
+
+DEPENDER_PACKAGE_YMLS = [
+    "src/shell/static/js/package.yml",
+]
+NICE_NAME = "Hue Shell"
+PERMISSION_ACTIONS = (
+  ("launch_pig", "Launch the Pig Shell"),
+  ("launch_flume", "Launch the Flume Shell"),
+  ("launch_hbase", "Launch the HBase Shell"),
+)

+ 228 - 0
apps/shell/src/shell/setuid.c

@@ -0,0 +1,228 @@
+/*
+ Licensed to Cloudera, Inc. under one
+ or more contributor license agreements.  See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership.  Cloudera, Inc. licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+#include <errno.h>
+#include <grp.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <limits.h>
+#include <sys/stat.h>
+
+static int min_uid = 500;
+
+void log_error(const char *format, ...) {
+  va_list args;
+  va_start(args, format);
+  vfprintf(stderr, format, args);
+  va_end(args);
+  fprintf(stderr, "\n");
+}
+
+/**
+ * Gets the name of the currently executing binary. The caller is responsible for freeing
+ * the returned pointer.
+ */
+char *get_executable_name() {
+  char buffer[PATH_MAX];
+  snprintf(buffer, PATH_MAX, "/proc/%u/exe", getpid());
+
+  char *filename = (char *) calloc(1, PATH_MAX);
+  if (filename == NULL) {
+    log_error("Error: calloc returned null, system out of memory.");
+    return NULL;
+  }
+
+  ssize_t len = readlink(buffer, filename, PATH_MAX);
+  if (len == -1) {
+    log_error("Can't get executable name from \"%s\": %s", buffer, strerror(errno));
+    free(filename);
+    return NULL;
+  }
+
+  if (len >= PATH_MAX) {
+    log_error("Executable name %.*s is longer than %d characters.", PATH_MAX, filename, PATH_MAX);
+    free(filename);
+    return NULL;
+  }
+
+  return filename;
+}
+
+/**
+ * Check the permissions on the setuid binary to make sure that security is
+ * promisable. For this, we need the binary to
+ *    * be user-owned by root
+ *    * others do not have write permissions
+ *    * be setuid
+ */
+int check_binary_permissions() {
+
+  char *executable_file = get_executable_name();
+  if (executable_file == NULL) {
+    return -1;
+  }
+
+  struct stat filestat;
+  if (stat(executable_file, &filestat) != 0) {
+    log_error("Could not stat the executable %s : %s", executable_file, strerror(errno));
+    free(executable_file);
+    return -1;
+  }
+
+  // check others do not have write permissions
+  if ((filestat.st_mode & S_IWOTH) == S_IWOTH) {
+    log_error("The setuid binary should not be writable by others.");
+    free(executable_file);
+    return -1;
+  }
+
+  // Binary should be setuid executable
+  if ((filestat.st_mode & S_ISUID) == 0) {
+    log_error("The setuid binary should be set setuid.");
+    free(executable_file);
+    return -1;
+  }
+
+  free(executable_file);
+  return 0;
+}
+
+int chown_delegation_token_files(char *delegation_token_files, int uid, int gid) {
+  char *modifiable_delegation_token_files = strdup(delegation_token_files);
+  if (modifiable_delegation_token_files == NULL) {
+    log_error("Error: strdup returned NULL, system out of memory.");
+    return -1;
+  }
+
+  char *delegation_token_file = strtok(modifiable_delegation_token_files, ",");
+  while (delegation_token_file != NULL) {
+    int chown_result = chown(delegation_token_file, uid, gid);
+    if (chown_result != 0) {
+      log_error("Could not change ownership of file \"%s\" to UID %d and GID %d : %s", delegation_token_file, uid, gid, strerror(errno));
+      free(modifiable_delegation_token_files);
+      return -1;
+    }
+    delegation_token_file = strtok(NULL, ",");
+  }
+  free(modifiable_delegation_token_files);
+  return 0;
+}
+
+
+/**
+ * Set the real/effective gid and uid.
+ * This is a no-op if the current gid/uid are the same as what's asked for.
+ */
+int set_gid_uid(int gid, int uid) {
+  if (getgid() != gid || getegid() != gid) {
+    gid_t group = gid;
+
+    if (setgroups(1, &group) != 0) {
+      log_error("Error: Could not set groups list to [%d] : %s", gid, strerror(errno));
+      return -1;
+    }
+
+    if (setregid(gid, gid) != 0) {
+      log_error("Error: Could not set real and effective group ID to %d : %s", gid, strerror(errno));
+      return -1;
+    }
+  }
+
+  if (getuid() != uid || geteuid() != uid) {
+    if (setreuid(uid, uid) != 0) {
+      log_error("Error: Could not set real and effective user ID to %d : %s", uid, strerror(errno));
+      return -1;
+    }
+  }
+
+  return 0;
+}
+
+int main(int argc, char **argv) {
+  
+  if (argc < 4){
+    log_error("Usage: setuid <desired user ID> <desired group ID> <executable> <arguments for executable>");
+    return -1;
+  }
+
+  // Because strtol can return the overflow/underflow error codes if it parses those integers correctly,
+  // we have to set errno to 0 before the call. The function will set errno to a non-zero value if an
+  // error occurs.
+  errno = 0;
+  int uid = strtol(argv[1], (char **)NULL, 10);
+  if (errno != 0) {
+    log_error("Error: Invalid value for UID: \"%s\" : %s", argv[1], strerror(errno));
+    return -1;
+  }
+
+  // See comment above for why we have to set errno to 0 before calling strtol.
+  errno = 0;
+  int gid = strtol(argv[2], (char **)NULL, 10);
+  if (errno != 0) {
+    log_error("Error: Invalid value for GID: \"%s\" : %s", argv[2], strerror(errno));
+    return -1;
+  }
+
+  if (uid < min_uid) {
+    log_error("Error: value %d for UID is less than the minimum UID allowed (%d)", uid, min_uid);
+    return -1;
+  }
+
+  if (check_binary_permissions() != 0) {
+    log_error("Error: permissions on setuid binary are not correct. Exiting.");
+    return -1;
+  }
+
+  char *delegation_token_files = getenv("HADOOP_TOKEN_FILE_LOCATION");
+  if (delegation_token_files != NULL) {
+    int chown_result = chown_delegation_token_files(delegation_token_files, uid, gid);
+    if (chown_result != 0) {
+      log_error("Error: Could not change ownership of delegation token files, exiting.");
+      return -1;
+    }
+  }
+
+  int set_gid_uid_result = set_gid_uid(gid, uid);
+  if (set_gid_uid_result != 0) {
+    log_error("Error: Could not correctly change to running as correct user, exiting.");
+    return -1;
+  }
+
+  int executable_index = 3;
+  const char *executable = argv[executable_index];
+  char **param_list = (char **) calloc(argc - executable_index + 1, sizeof(char *));
+
+  if (param_list == NULL) {
+    log_error("Error: calloc returned null, system out of memory.");
+    return -1;
+  }
+
+  int i;
+  for (i = 0; i < argc - executable_index; i++) {
+    param_list[i] = argv[executable_index + i];
+  }
+
+  int result = execvp(executable, param_list);
+
+  log_error("Error: exec returned %d with error: %s", result, strerror(errno));
+  return 0;
+}
+

+ 685 - 0
apps/shell/src/shell/shellmanager.py

@@ -0,0 +1,685 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+This module handles I/O with shells.  Much of the functionality has been pushed down into the
+Shell class itself, but a lot also happens in ShellManager.
+"""
+
+
+import cStringIO
+import desktop.lib.i18n
+import errno
+import eventlet
+import logging
+import pty
+import pwd
+import shell.conf
+import shell.constants as constants
+import shell.utils as utils
+import signal
+import subprocess
+import tempfile
+import tty
+
+from eventlet.green import os
+from eventlet.green import select
+from eventlet.green import time
+from hadoop.cluster import all_mrclusters, get_all_hdfs
+
+LOG = logging.getLogger(__name__)
+
+_SETUID_PROG = os.path.join(os.path.dirname(__file__), 'setuid')
+
+class NewShellInterrupt(Exception):
+  """
+  Eventlet's greenlets only allow for exceptions as the way of communicating between greenlets.
+  We use the NewShellInterrupt for cross-greenlet communication.
+  """
+  def __init__(self, new_shell_pairs):
+    self.new_shell_pairs = new_shell_pairs
+
+class Shell(object):
+  """
+  A class to encapsulate I/O with a shell subprocess.
+  """
+  def __init__(self, shell_command, shell_id, username, delegation_token_dir):
+    subprocess_env = {}
+    env = desktop.lib.i18n.make_utf8_env()
+    for item in constants.PRESERVED_ENVIRONMENT_VARIABLES:
+      value = env.get(item)
+      if value:
+        subprocess_env[item] = value
+
+    try:
+      user_info = pwd.getpwnam(username)
+    except KeyError:
+      LOG.error("Unix user account didn't exist at subprocess creation. Was it deleted?")
+      raise
+
+    parent, child = pty.openpty()
+
+    try:
+      tty.setraw(parent)
+    except tty.error:
+      LOG.debug("Could not set parent fd to raw mode, user will see echoed input.")
+
+    subprocess_env[constants.HOME] = user_info.pw_dir
+    command_to_use = [_SETUID_PROG, str(user_info.pw_uid), str(user_info.pw_gid)]
+    command_to_use.extend(shell_command)
+
+    delegation_token_files = self._get_delegation_tokens(username, delegation_token_dir)
+    if delegation_token_files:
+      delegation_token_files = [token_file.name for token_file in delegation_token_files]
+      subprocess_env[constants.HADOOP_TOKEN_FILE_LOCATION] = ','.join(delegation_token_files)
+
+    try:
+      LOG.debug("Starting subprocess with command '%s' and environment '%s'" %
+                                                             (command_to_use, subprocess_env,))
+      p = subprocess.Popen(command_to_use, stdin=child, stdout=child, stderr=child,
+                                                                 env=subprocess_env, close_fds=True)
+    except (OSError, ValueError):
+      os.close(parent)
+      os.close(child)
+      raise
+
+    # State that shouldn't be touched by any other classes.
+    self._output_buffer_length = 0
+    self._commands = []
+    self._fd = parent
+    self._child_fd = child
+    self.subprocess = p
+    self.pid = p.pid
+    self._write_buffer = cStringIO.StringIO()
+    self._read_buffer = cStringIO.StringIO()
+    self._delegation_token_files = delegation_token_files
+
+    # State that's accessed by other classes.
+    self.shell_id = shell_id
+    # Timestamp that is updated on shell creation and on every output request. Used so that we know
+    # when to kill the shell.
+    self.time_received = time.time()
+    self.last_output_sent = False
+    self.remove_at_next_iteration = False
+    self.destroyed = False
+
+  def _get_delegation_tokens(self, username, delegation_token_dir):
+    """
+    If operating against Kerberized Hadoop, we'll need to have obtained delegation tokens for
+    the user we want to run the subprocess as. We have to do it here rather than in the subprocess
+    because the subprocess does not have Kerberos credentials in that case.
+    """
+    delegation_token_files = []
+    all_clusters = []
+    all_clusters += all_mrclusters().values()
+    all_clusters += get_all_hdfs().values()
+
+    LOG.debug("Clusters to potentially acquire tokens for: %s" % (repr(all_clusters),))
+
+    for cluster in all_clusters:
+      if cluster.security_enabled:
+        current_user = cluster.user
+        try:
+          cluster.setuser(username)
+          token = cluster.get_delegation_token()
+          token_file = tempfile.NamedTemporaryFile(dir=delegation_token_dir)
+          token_file.write(token.delegationTokenBytes)
+          token_file.flush()
+          delegation_token_files.append(token_file)
+        finally:
+          cluster.setuser(current_user)
+
+    return delegation_token_files
+
+  def mark_for_cleanup(self):
+    """
+    Flag this shell to be picked up at the next iteration of handle_periodic.
+    """
+    self.remove_at_next_iteration = True
+
+  def get_previous_output(self):
+    """
+    Called when a Hue session is restored. Returns a tuple of ( all previous output, next offset).
+    """
+    val = self._read_buffer.getvalue()
+    return ( val, len(val))
+
+  def get_previous_commands(self):
+    """
+    Return the list of previously entered commands. This is used for bash_history semantics
+    when restoring Shells.
+    """
+    return self._commands
+
+  def get_cached_output(self, offset):
+    """
+    The offset is not the latest one, so some output has already been generated and is
+    stored in the read buffer. So let's fetch it from there.
+    Returns (output, has_more, new_offset) or None.
+    """
+    self._read_buffer.seek(offset)
+    next_output = self._read_buffer.read()
+    if not next_output:
+      return None
+    more_available = len(next_output) >= shell.conf.SHELL_OS_READ_AMOUNT.get()
+    return (next_output, more_available, self._output_buffer_length)
+
+  def process_command(self, command):
+    """
+    Write the command to the end of the wite buffer, and spawn a greenlet to write it
+    into the subprocess when the subprocess becomes writable.
+
+    Returns a dictionary with {return_code: bool}.
+    """
+    LOG.debug("Command received for pid %d : '%s'" % (self.pid, repr(command),))
+    # TODO(bc): Track the buffer size to avoid calling getvalue() every time
+    if len(self._write_buffer.getvalue()) >= shell.conf.SHELL_WRITE_BUFFER_LIMIT.get():
+      LOG.debug("Write buffer too full, dropping command")
+      return { constants.BUFFER_EXCEEDED : True }
+    else:
+      LOG.debug("Write buffer has room. Adding command to end of write buffer.")
+      self._append_to_write_buffer(command)
+      eventlet.spawn_n(self._write_child_when_able)
+      return { constants.SUCCESS : True }
+
+  def _append_to_write_buffer(self, command):
+    """
+    Append the received command to the write buffer. This buffer is used
+    when the child becomes readable to send commands to the child subprocess.
+    """
+    self._write_buffer.seek(len(self._write_buffer.getvalue()))
+    self._write_buffer.write("%s" % (command,))
+    # We seek back to the beginning so that when the child becomes writable we
+    # feed the commands to the child in the order they were received.
+    self._commands.append(command)
+    while len(self._commands) > 25:
+      self._commands.pop(0)
+
+  def _read_from_write_buffer(self):
+    """
+    Read and return the contents of the write buffer.
+    """
+    self._write_buffer.seek(0)
+    contents = self._write_buffer.read()
+    return contents
+
+  def _write_child_when_able(self):
+    """
+    Select on the child's input file descriptor becoming writable, and then write commands to it.
+    If not successful in writing all the commands, spawn a new greenlet to retry.
+    """
+    LOG.debug("write_child_when_able")
+    buffer_contents = self._read_from_write_buffer()
+    if not buffer_contents:
+      return
+
+    try:
+      r, w, x = select.select([],[self._fd],[])
+    except Exception, e:
+      # The next 9 lines are taken from Facebook's Tornado project, which is open-sourced under
+      # the Apache license.
+      # Depending on python version and poll implementation,
+      # different exception types may be thrown and there are
+      # two ways EINTR might be signaled:
+      # * e.errno == errno.EINTR
+      # * e.args is like (errno.EINTR, 'Interrupted system call')
+      if (getattr(e, 'errno') == errno.EINTR or
+          (isinstance(getattr(e, 'args'), tuple) and
+           len(e.args) == 2 and e.args[0] == errno.EINTR)):
+        LOG.warning("Interrupted system call", exc_info=1)
+        eventlet.spawn_n(self._write_child_when_able)
+      else:
+        LOG.error("Unexpected error on select")
+        self.mark_for_cleanup()
+      return
+
+    if not w:
+      return
+
+    try:
+      bytes_written = os.write(self._fd, buffer_contents)
+      self._advance_write_buffer(bytes_written)
+    except OSError, e:
+      if e.errno == errno.EINTR:
+        eventlet.spawn_n(self._write_child_when_able)
+      elif e.errno != errno.EAGAIN:
+        format_str = "Encountered error while writing to process with PID %d:%s"
+        LOG.error(format_str % (self.pid, e))
+        self.mark_for_cleanup()
+    else: # This else clause is on the try/except above, not the if/elif
+      if bytes_written != len(buffer_contents):
+        eventlet.spawn_n(self._write_child_when_able)
+
+  def _advance_write_buffer(self, num_bytes):
+    """
+    Advance the current position in the write buffer by num_bytes bytes.
+    """
+    # TODO: Replace this system with a list of cStringIO objects so that
+    # it's more efficient. We should do this if this seems to be copying
+    # a lot of memory around.
+    self._write_buffer.seek(num_bytes)
+    new_value = self._write_buffer.read()
+    self._write_buffer.truncate(0)
+    self._write_buffer.write(new_value)
+
+  def read_child_output(self):
+    """
+    Reads up to conf.SHELL_OS_READ_AMOUNT bytes from the child subprocess's stdout.
+    Returns a tuple of (output, more_available, new_offset).
+    The second parameter indicates whether more output might be obtained by
+    another call to read_child_output.
+    """
+    ofd = self._fd
+    result = None
+    try:
+      next_output = os.read(ofd, shell.conf.SHELL_OS_READ_AMOUNT.get())
+      self._read_buffer.seek(self._output_buffer_length)
+      self._read_buffer.write(next_output)
+      length = len(next_output)
+      self._output_buffer_length += length
+      num_excess_chars = self._output_buffer_length - shell.conf.SHELL_BUFFER_AMOUNT.get()
+      if num_excess_chars > 0:
+        self._read_buffer.seek(num_excess_chars)
+        newval = self._read_buffer.read()
+        self._read_buffer.truncate(0)
+        self._read_buffer.write(newval)
+        self._output_buffer_length = len(newval)
+    except OSError, e: # No more output at all
+      if e.errno == errno.EINTR:
+        pass
+      elif e.errno != errno.EAGAIN:
+        format_str = "Encountered error while reading from process with PID %d : %s"
+        LOG.error( format_str % (self.subprocess.pid, e))
+        self.mark_for_cleanup()
+    else:
+      more_available = length >= shell.conf.SHELL_OS_READ_AMOUNT.get()
+      result = (next_output, more_available, self._output_buffer_length)
+
+    return result
+
+  def destroy(self):
+    """
+    Clean up the resources used for this shell.
+    """
+    try:
+      for delegation_token_file in self._delegation_token_files:
+        delegation_token_file.close()
+      self._delegation_token_files = None
+      self._write_buffer.close()
+      self._read_buffer.close()
+
+      os.close(self._fd)
+      os.close(self._child_fd)
+
+      try:
+        LOG.debug("Sending SIGKILL to process with PID %d" % (self.subprocess.pid,))
+        os.kill(self.subprocess.pid, signal.SIGKILL)
+        # We could try figure out which exit statuses are fine and which ones are errors.
+        # But that would be difficult to do correctly since os.wait might block.
+      except OSError:
+        pass # This means the subprocess was already killed, which happens if the command was "quit"
+    finally:
+      self.destroyed = True
+
+class ShellManager(object):
+  """
+  The class that manages state for all shell subprocesses.
+  """
+  def __init__(self):
+    self._shells = {} # Keys are (username, shell_id) tuples. Each user has his/her own set of shell ids.
+    shell_types = [] # List of available shell types. For each shell type, we have a nice name (e.g. "Python Shell") and a short name (e.g. "python")
+    self._command_by_short_name = {} # Map each short name to its command (e.g. ["pig", "-l", "/dev/null"])
+    self._meta = {} # Map usernames to utils.UserMetadata objects
+    self._greenlets_by_hid = {} # Map each Hue Instance ID (HID) to greenlet currently fetching output for that HID.
+    self._hids_by_pid = {} # Map each process ID (PID) to the HID whose greenlet is currently doing a "select" on the process's output fd.
+    self._greenlets_to_notify = {} # For each PID, maintain a set of greenlets who are also interested in the output from that process, but are not doing the select.
+    self._shells_by_fds = {} # Map each file descriptor to the Shell instance whose output it represents.
+    self._greenlet_interruptable = {} # For each greenlet, store if it can be safely interrupted.
+
+    self._delegation_token_dir = shell.conf.SHELL_DELEGATION_TOKEN_DIR.get()
+    if not os.path.exists(self._delegation_token_dir):
+      os.mkdir(self._delegation_token_dir)
+
+    for item in shell.conf.SHELL_TYPES.keys():
+      command = shell.conf.SHELL_TYPES[item].command.get().strip().split()
+      nice_name = shell.conf.SHELL_TYPES[item].nice_name.get().strip()
+      executable_exists = utils.executable_exists(command)
+      if executable_exists:
+        self._command_by_short_name[item] = command
+      shell_types.append({ constants.NICE_NAME: nice_name, constants.KEY_NAME: item, constants.EXISTS:executable_exists })
+    self.shell_types = shell_types
+    eventlet.spawn_after(1, self._handle_periodic)
+
+  @classmethod
+  def global_instance(cls):
+    if not hasattr(cls, "_global_instance"):
+      cls._global_instance = cls()
+    return cls._global_instance
+
+  def available_shell_types(self, user):
+    username = user.username
+    try:
+      user_info = pwd.getpwnam(username)
+    except KeyError:
+      user_info = None
+    if not user_info:
+      return None
+
+    shell_types_for_user = []
+    for item in self.shell_types:
+      if user.has_desktop_permission('launch_%s' % (item[constants.KEY_NAME],), 'shell'):
+        shell_types_for_user.append(item)
+    return shell_types_for_user
+
+  def _interrupt_conditionally(self, green_let, message):
+    """
+    If the greenlet is currently interruptable, (i.e. it's in a try/catch block with a handler
+    for a NewShellInterrupt, then interrupt it with the given message.
+    """
+    if self._greenlet_interruptable.get(green_let):
+      green_let.throw(message)
+
+  def _cleanup_greenlets_for_removed_pids(self, removed_pids):
+    """
+    Clean up any greenlets listening for the removed pids. This includes both selecting
+    greenlets and non-selecting greenlets.
+    """
+    greenlets_to_cleanup = set()
+    for pid in removed_pids:
+      listening_hid = self._hids_by_pid.get(pid)
+      if listening_hid:
+        greenlet_for_hid = self._greenlets_by_hid.get(listening_hid)
+        if greenlet_for_hid:
+          greenlets_to_cleanup.add(greenlet_for_hid)
+      non_selecting_greenlets = self._greenlets_to_notify.get(pid)
+      if non_selecting_greenlets:
+        greenlets_to_cleanup.update(non_selecting_greenlets)
+    nsi = NewShellInterrupt([])
+    for greenlet_to_notify in greenlets_to_cleanup:
+      eventlet.spawn_n(self._interrupt_conditionally, greenlet_to_notify, nsi)
+
+  def _handle_periodic(self):
+    """
+    Called every second. Kills shells which haven't been asked about in conf.SHELL_TIMEOUT
+    seconds (currently 600).
+    """
+    try:
+      keys_to_pop = []
+      current_time = time.time()
+      for key, shell_instance in self._shells.iteritems():
+        if shell_instance.last_output_sent or shell_instance.remove_at_next_iteration:
+          keys_to_pop.append(key)
+        elif shell_instance.subprocess.poll() is not None:
+          keys_to_pop.append(key)
+        else:
+          difftime = current_time - shell_instance.time_received
+          if difftime >= shell.conf.SHELL_TIMEOUT.get():
+            keys_to_pop.append(key)
+      removed_pids = [self._shells.get(key).pid for key in keys_to_pop]
+      for key in keys_to_pop:
+        self._cleanup_shell(key)
+    finally:
+      eventlet.spawn_n(self._cleanup_greenlets_for_removed_pids, removed_pids)
+      eventlet.spawn_after(1, self._handle_periodic)
+
+  def _cleanup_shell(self, key):
+    """
+    Clean up metadata for the shell specified by key.
+    """
+    shell_instance = self._shells[key]
+    shell_instance.destroy()
+    self._shells.pop(key)
+    username = key[0]
+    self._meta[username].decrement_count()
+    self._shells_by_fds.pop(shell_instance._fd)
+
+  def try_create(self, user, shell_name):
+    """
+    Attemps to create a new shell subprocess for the given user. Writes the appropriate failure or
+    success response to the client.
+    """
+    command = self._command_by_short_name.get(shell_name)
+    if not command:
+      return { constants.SHELL_CREATE_FAILED : True }
+
+    username = user.username
+    try:
+      user_info = pwd.getpwnam(username)
+    except KeyError:
+      return { constants.NO_SUCH_USER : True }
+
+    if not user.has_desktop_permission('launch_%s' % (shell_name,), 'shell'):
+      return { constants.SHELL_NOT_ALLOWED : True }
+
+    if not username in self._meta:
+      self._meta[username] = utils.UserMetadata(username)
+
+    user_metadata = self._meta[username]
+    shell_id = user_metadata.get_next_id()
+    try:
+      LOG.debug("Trying to create a %s shell for user %s" % (shell_name, username))
+      shell_instance = Shell(command, shell_id, username, self._delegation_token_dir)
+    except (OSError, ValueError, KeyError):
+      LOG.exception("Could not create %s shell for '%s'" % (shell_name, username))
+      return { constants.SHELL_CREATE_FAILED : True }
+
+    LOG.debug("Shell successfully created")
+    user_metadata.increment_count()
+    self._shells[(username, shell_id)] = shell_instance
+    self._shells_by_fds[shell_instance._fd] = shell_instance
+    return { constants.SUCCESS : True, constants.SHELL_ID : shell_id }
+
+  def kill_shell(self, username, shell_id):
+    """
+    Called when the user closes the Shell app instance in Hue. Kills the subprocess.
+    """
+    shell_instance = self._shells.get((username, shell_id))
+    if not shell_instance:
+      response = "User '%s' has no shell with ID '%s'" % (username, shell_id)
+    else:
+      shell_instance.mark_for_cleanup()
+      response = "Shell successfully killed"
+    LOG.debug(response)
+    return response
+
+  def get_previous_output(self, username, shell_id):
+    """
+    Called when the Hue session is restored. Get the outputs that we have previously written out to
+    the client as one big string.
+    """
+    shell_instance = self._shells.get((username, shell_id))
+    if not shell_instance:
+      return { constants.SHELL_KILLED : True }
+    shell_instance.time_received = time.time()
+    output, next_offset = shell_instance.get_previous_output()
+    commands = shell_instance.get_previous_commands()
+    return { constants.SUCCESS: True, constants.OUTPUT: output, constants.NEXT_OFFSET: next_offset,
+      constants.COMMANDS: commands}
+
+  def process_command(self, username, shell_id, command):
+    """
+    Find the shell specified by the (username, shell_id) tuple, and then write the incoming command
+    to that shell.
+    """
+    shell_instance = self._shells.get((username, shell_id))
+    if not shell_instance:
+      return { constants.NO_SHELL_EXISTS : True }
+    shell_instance.time_received = time.time()
+    command += "\n"
+    return shell_instance.process_command(command)
+
+  def _interrupt_with_output(self, readable):
+    """
+    For each of the readable file descriptors, find all greenlets which were not themselves
+    selecting but were interested in the output, and spawn a greenlet to go wake each
+    of them up.
+    """
+    greenlets_set = set()
+    for fd in readable:
+      shell_instance = self._shells_by_fds.get(fd)
+      if not shell_instance:
+        LOG.error("Shell for readable file descriptor '%d' is missing" % (fd,))
+      else:
+        greenlets_to_notify = self._greenlets_to_notify.get(shell_instance.pid, [])
+        greenlets_set.update(greenlets_to_notify)
+    nsi = NewShellInterrupt([])
+    for greenlet_to_notify in greenlets_set:
+      eventlet.spawn_n(self._interrupt_conditionally, greenlet_to_notify, nsi)
+
+  def _read_helper(self, shell_instance, offset=None):
+    if offset is not None:
+      read_result = shell_instance.get_cached_output(offset)
+    else:
+      read_result = shell_instance.read_child_output()
+    if not read_result:
+      return None
+    total_output, more_available, next_offset = read_result
+    # If this is the last output from the shell, let's tell the JavaScript that.
+    if shell_instance.subprocess.poll() is None:
+      status = constants.ALIVE
+    else:
+      status = constants.EXITED
+      shell_instance.last_output_sent = True
+    return { status : True,
+            constants.OUTPUT : total_output,
+            constants.MORE_OUTPUT_AVAILABLE : more_available,
+            constants.NEXT_OFFSET : next_offset }
+
+  def retrieve_output(self, username, hue_instance_id, shell_pairs):
+    """
+    Called when an output request is received from the client. Sends the request to the appropriate
+    shell instances.
+    """
+    time_received = time.time()
+    current_greenlet = eventlet.getcurrent()
+    self._greenlets_by_hid[hue_instance_id] = current_greenlet
+    shell_pairs = set(shell_pairs)
+
+    # Update the time stamps on all shells
+    self._update_access_time(username,
+                             time_received,
+                             [ p[0] for p in shell_pairs ])
+
+    result = None
+    # The main long-polling loop
+    while (time.time() - time_received) < constants.BROWSER_REQUEST_TIMEOUT:
+      # If we have cached output, find that and return immediately
+      cached_output = self._retrieve_cached_output(username, shell_pairs)
+      if len(cached_output) != 0:
+        return cached_output
+
+      fds_to_listen_for = []
+      shell_instances_for_listened_fds = {}
+
+      #
+      # Figure out which shell we should select on.
+      #
+      # Note that only one greenlet (request handler) may select on a given
+      # shell. So we build a registration mechanism with _hids_by_pid.
+      # If somebody else is already doing a select, then we add ourselves to
+      # _greenlets_to_notify.
+      #
+      # Each hid is generated by the frontend uniquely. It safely maps to a
+      # unique greenlet.
+      #
+      for shell_id, _ in shell_pairs:
+        shell_instance = self._shells.get((username, shell_id))
+        # Here we can assume shell_instance exists because if it didn't, we would have broken out of
+        # the while loop above and we wouldn't be executing this code.
+        listening_hid = self._hids_by_pid.get(shell_instance.pid)
+        if listening_hid is not None and listening_hid != hue_instance_id:
+          self._greenlets_to_notify.setdefault(shell_instance.pid, set()).add(current_greenlet)
+        else:
+          fds_to_listen_for.append(shell_instance._fd)
+          shell_instances_for_listened_fds[shell_instance._fd] = shell_instance
+          self._hids_by_pid[shell_instance.pid] = hue_instance_id
+
+      try:
+        time_remaining = constants.BROWSER_REQUEST_TIMEOUT - (time.time() - time_received)
+        self._greenlet_interruptable[current_greenlet] = True
+        readable, writable, exception_occurred = select.select(fds_to_listen_for, [], [], time_remaining)
+        self._greenlet_interruptable[current_greenlet] = False
+      except NewShellInterrupt, nsi:
+        self._greenlet_interruptable[current_greenlet] = False
+        # Here, I'm assuming that we won't have a situation where one of the (shell_id, offset)
+        # tuples in nsi.new_shell_pairs has the same shell_id as an item in shell_pairs, but
+        # an offset with a different (has to be higher) number.
+        shell_pairs.update(nsi.new_shell_pairs)
+      else:
+        if not readable:
+          result = { constants.PERIODIC_RESPONSE: True }
+        else:
+          result = {}
+          for fd in readable:
+            shell_instance = shell_instances_for_listened_fds[fd]
+            if shell_instance.destroyed:
+              result[shell_instance.shell_id] = { constants.SHELL_KILLED : True }
+            else:
+              result[shell_instance.shell_id] = self._read_helper(shell_instance)
+          eventlet.spawn_n(self._interrupt_with_output, readable)
+          break
+
+    if not result:
+      result = { constants.PERIODIC_RESPONSE: True }
+
+    self._greenlets_by_hid.pop(hue_instance_id)
+    for shell_id, _ in shell_pairs:
+      shell_instance = self._shells.get((username, shell_id))
+      if shell_instance:
+        if self._hids_by_pid.get(shell_instance.pid) == hue_instance_id:
+          self._hids_by_pid.pop(shell_instance.pid)
+        else:
+          try:
+            self._greenlets_to_notify[shell_instance.pid].remove(current_greenlet)
+          except KeyError:
+            LOG.error("Greenlet for pid %d was not found in set of listening greenlets" % (shell_instance.pid,))
+    return result
+
+  def add_to_output(self, username, hue_instance_id, shell_pairs):
+    """
+    Adds the given shell_id, offset pairs to the output connection associated with the given Hue
+    instance ID.
+    """
+    new_shells_interrupt = NewShellInterrupt(shell_pairs)
+    greenlet_for_hid = self._greenlets_by_hid.get(hue_instance_id)
+    if greenlet_for_hid:
+      eventlet.spawn_n(self._interrupt_conditionally, greenlet_for_hid, new_shells_interrupt)
+    return { constants.SUCCESS : True }
+
+  def _update_access_time(self, username, atime, shell_id_list):
+    """Update the time_received field in all specified shells"""
+    for shell_id in shell_id_list:
+      shell_instance = self._shells.get((username, shell_id))
+      if shell_instance:
+        shell_instance.time_received = atime
+
+  def _retrieve_cached_output(self, username, shell_pairs):
+    """
+    Try to get cached output from the shells.
+    Returns a dictionary of { shell_id: output_json }
+    """
+    result = { }
+    for shell_id, offset in shell_pairs:
+      shell_instance = self._shells.get((username, shell_id))
+      if shell_instance:
+        cached_output = self._read_helper(shell_instance, offset)
+        if cached_output:
+          result[shell_id] = cached_output
+      else:
+        LOG.warn("User '%s' has no shell with ID '%s'" % (username, shell_id))
+        result[shell_id] = { constants.NO_SHELL_EXISTS: True }
+
+    return result

BIN
apps/shell/src/shell/static/art/shell.png


+ 37 - 0
apps/shell/src/shell/static/bootstrap.js

@@ -0,0 +1,37 @@
+// Licensed to Cloudera, Inc. under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  Cloudera, Inc. licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+Hue.Desktop.register({
+	Shell : {
+		name : 'Shell',
+		//autolaunch: "/shell/",
+		css : '/shell/static/css/shell.css',
+		require: [ 'shell/Shell' ],
+		launch: function(path, options){
+			// application launch code here 
+			// example code below: 
+			return new Shell(path || '/shell/', options);
+		},
+		menu: {
+			id: 'hue-shell-menu',
+			img: {
+				// Replace this with a real icon!
+				// ideally a 55x55 transparent png
+				src: '/shell/static/art/shell.png'
+			}
+		},
+		help: '/help/shell/'
+	}
+});

+ 82 - 0
apps/shell/src/shell/static/css/shell.css

@@ -0,0 +1,82 @@
+/* 
+	shell styles
+	important: these should be namespaced
+	example:
+	let's say you are creating an app called "calculator"; you should prefix all your styles with your application's name like so
+	
+	.calculator img {
+		border: 1px solid #000;
+	}
+	.calculator p {
+		margin: 10px 0px;
+	}
+	etc...
+
+	other notes:
+	* don't use ids - there may be more than one of your apps open; use classes!
+	* the toolbar element is absolutely positioned and 100% wide (and therefor 0px high);
+	any elements inside it should also be absolutely positioned
+*/
+
+.shell img.shell_icon {
+	width: 55px;
+	height: 55px;
+	position: absolute;
+	top: 27px;
+	left: 3px;
+}
+
+.shell .Button .plus_button{
+	background: url("/static/art/icons/add.png");
+	height: 16px;
+	width: 16px;
+	float:left;
+	margin: 0px 5px 0px 0px;
+}
+
+.shell div.nav_menu{
+	position:absolute;
+	top: 37px;
+	left: 66px;
+}
+
+.shell .Button {
+	padding: 10px;
+	margin: 10px;
+}
+
+.shell .nav_button{
+	margin: -2px;
+}
+
+.shell textarea{
+	border-width:0px;
+	width:100%;
+	overflow:hidden;
+	outline:none;
+	padding:0px;
+	resize:none;
+}
+
+.shell li .Button {
+	display:block;
+	width: 100%;
+	font-size:1.25em;
+	margin-left:0px;
+}
+
+.shell span, .shell textarea{
+	font-family: monospace;
+	font-size: 14px;    
+}
+
+.shell .shell_container div{
+	width: 50%;
+	margin-left:25%;
+	background-color:#ffffff;
+	padding: 2px 10px;
+}
+
+.shell .hidden{
+	display: none;
+}

+ 1 - 0
apps/shell/src/shell/static/help/index.md

@@ -0,0 +1 @@
+Help for your app, written in [MarkDown](http://daringfireball.net/projects/markdown/syntax) syntax.

+ 235 - 0
apps/shell/src/shell/static/js/Source/Shell/Poller.js

@@ -0,0 +1,235 @@
+// Licensed to Cloudera, Inc. under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  Cloudera, Inc. licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+/*
+---
+description: I/O functionality shared between instances of the Shell app.
+provides: [Poller]
+requires: [hue-shared/Hue.Request]
+script: Poller.js
+
+...
+*/
+
+var hueInstanceID = function() {
+	var chars = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXTZabcdefghiklmnopqrstuvwxyz";
+	var lastIndex = chars.length - 1;
+	var stringLength = 128;
+	var randomString = "";
+	for (var i = 0; i < stringLength; i++) {
+		var randomIndex = $random(0, lastIndex);
+		randomString += chars.substring(randomIndex, randomIndex+1);
+	}
+	return randomString;
+}();
+
+Poller = {
+	initialize: function() {
+		this.outputReq = new Request.JSON({
+			method: 'post',
+			url: '/shell/retrieve_output',
+			onSuccess: this.outputReceived.bind(this),
+			onFailure: this.outputRequestFailed.bind(this),
+			headers: { "Hue-Instance-ID" : hueInstanceID }
+		});
+		this.addToOutputReq = new Request.JSON({
+			method: 'post',
+			url: '/shell/add_to_output',
+			onSuccess: this.addToOutputCompleted.bind(this),
+			onFailure: this.addToOutputFailed.bind(this),
+			headers: { "Hue-Instance-ID" : hueInstanceID }
+		});
+		this.numAdditionalReqsSent = 0;
+		this.additionalReqs = [];
+		this.addToOutputReqOpen = false;
+		this.requestOpen = false;
+		this.initialized = true;
+		this.requestsStopped = true;
+		this.dispatchInfo = {};
+		this.backoffTime = 1;
+	},
+
+	listenForShell: function(shellId, offset, callback) {
+		// One-time initialization
+		if (!this.initialized) {
+			this.initialize();
+		}
+
+		// Register the dispatch information for this shell ID.
+		this.dispatchInfo[shellId] = {callback:callback, offset:offset};
+
+		// If an output request is already open, use the secondary channel to add the new shell and
+		// offset to the existing output request.
+		if (this.requestOpen) {
+			this.addToOutputChannel(shellId, offset);
+		}
+		
+		// We might be between openOutputChannel calls, so check to see if we've stopped
+		// the requests or if we're just in between calls. If we've stopped, restart them.
+		if (this.requestsStopped) {
+			this.requestsStopped = false;
+			this.openOutputChannel();
+		}
+	},
+	
+	// Remove the dispatch info for the given shell id. We don't have to do a request.cancel() since
+	// either there's only 1 shell and we won't reissue once the request completes, or there are 
+	// multiple and we might want to reissue.
+	stopShellListener: function(shellId) {
+		this.dispatchInfo[shellId] = null;
+	},
+	
+	// Convert the information stored in this.dispatchInfo into the form that the backend speaks.
+	serializeShellData: function() {
+		var serializedShells = {};
+		var numShells = 0;
+		for (var shellId in this.dispatchInfo) {
+			var shellInfo = this.dispatchInfo[shellId];
+			if (shellInfo) {
+				numShells++;
+				serializedShells['shellId'+numShells] = shellId;
+				serializedShells['offset'+numShells] = shellInfo.offset;
+			}
+		}
+		serializedShells["numPairs"] = numShells;
+		return serializedShells;
+	},
+	
+	openOutputChannel: function() {
+		this.requestOpen = true;
+		var serializedData = this.serializeShellData();
+		this.outputReq.send({ data: serializedData });
+	},
+
+	outputRequestFailed: function() {
+		this.requestOpen = false;
+		setTimeout(this.openOutputChannel.bind(this), this.backoffTime);
+		this.backoffTime *= 2;
+	},
+	
+	outputReceived: function(json, text) {
+		this.requestOpen = false;
+		this.backoffTime = 1;
+
+		var closeOutputChannel = true; // Used to determine if we should issue a new output request.
+		if (json.periodicResponse) {
+			closeOutputChannel = false; // If it's just a "keep-alive", we should reissue.
+		}
+		
+		// The object we got back has a dictionary for every shell ID. We loop through the keys in the object.
+		for (var shellId in json) {
+			var shellInfo = this.dispatchInfo[shellId];
+			// For each key, see if we have any callbacks that are interested in that key. If not, then it
+			// either isn't a shell ID (so it's something like "periodicResponse") or it's a shell ID that 
+			// we no longer care about. That can occur when the user closes the instance of the shell app that
+			// we made this output request for.
+			if (shellInfo) {
+				// Let's pull out the data for this shell.
+				var result = json[shellId];
+				// If it's alive, or it has exited, we might have to reissue an output request. We might reissue
+				// if it's exited because there might be more output available.
+				if (result.alive || result.exited) {
+					//Let's update how far into the output stream we are.
+					shellInfo.offset = result.nextOffset;
+					// If it's not alive and no more output is available, then we stop listening for this shell.
+					if (!(result.alive || result.moreOutputAvailable)) {
+						this.stopShellListener(shellId);
+					}
+				} else {
+					//If it's neither alive nor exited, then we're in some error state, so we definitely stop
+					//listening for this shell
+					this.stopShellListener(shellId);
+				}
+				// Since there was output this one time, let's call the callback with the result for this shell.
+				shellInfo.callback(result);
+			}
+			
+			// Now let's check if we still care about this shell. If not, we'll have called
+			// stopShellListener on it and this.dispatchInfo[shellId] will be null.
+			if (this.dispatchInfo[shellId]) { 
+				closeOutputChannel = false; // We care still, so let's reissue an output req.
+			}
+		}
+
+		if (closeOutputChannel) {
+			//None of the shells in the response are still listening. Check to see if any other is.
+			for (var shellId in this.dispatchInfo) {
+				if (this.dispatchInfo[shellId]) {
+					closeOutputChannel = false; // >=1 shells are listening, so let's reissue
+				}
+			}
+		}
+
+		if (!closeOutputChannel) {
+			//can't use openOutputChannel.delay(0, this) here because it causes buggy behavior in Firefox.
+			setTimeout(this.openOutputChannel.bind(this), 0);
+		} else {
+			// Let's set this flag to true so that we can reopen the channel on the next listener.
+			this.requestsStopped = true;
+		}
+	},
+	
+	addToOutputChannel: function(shellId, offset) {
+		// First let's store the info
+		this.additionalReqs.push({shellId: shellId, offset: offset});
+		this.sendAdditionalReq();
+	},
+	
+	serializeAdditionalReqs: function() {
+		// Convert the additional things we need to register into our output channel into the
+		// same format as used for output requests.
+		var serializedData = {}
+		for (var i = 0; i < this.additionalReqs.length; i++) {
+			serializedData["shellId" + (i+1)] = this.additionalReqs[i].shellId;
+			serializedData["offset" + (i+1)] = this.additionalReqs[i].offset;
+		}
+		serializedData["numPairs"] = this.additionalReqs.length;
+		return serializedData;
+	},
+	
+	sendAdditionalReq: function() {
+		this.addToOutputReqOpen = true;
+		var serializedData = this.serializeAdditionalReqs();
+		this.numAdditionalReqsSent = this.additionalReqs.length;
+		this.addToOutputReq.send({ data: serializedData });
+	},
+	
+	addToOutputCompleted: function(json, text) {
+		this.backoffTime = 1;
+		this.addToOutputReqOpen = false;
+		if (json.success) {
+			this.additionalReqs.splice(0, this.numAdditionalReqsSent);
+			this.numAdditionalReqsSent = 0;
+			if (this.additionalReqs.length) {
+				this.sendAdditionalReq.delay(0, this);
+				setTimeout(this.sendAdditionalReq.bind(this), 0);
+			}
+		} else if (json.restartHue) {
+			alert("Your version of Hue is not up to date. Please restart your browser.");
+		} else if (json.notRunningSpawning) {
+			alert("The server is not running Spawning and cannot support the Shell app.");
+		} else {
+			this.numAdditionalReqsSent = 0;
+			setTimeout(this.sendAdditionalReq.bind(this), 0);
+		}
+	},
+	
+	addToOutputFailed: function() {
+		this.addToOutputReqOpen = false;
+		this.numAdditionalReqsSent = 0;
+		setTimeout(this.sendAdditionalReq.bind(this), this.backoffTime);
+		this.backoffTime *= 2;
+	}
+};

+ 450 - 0
apps/shell/src/shell/static/js/Source/Shell/Shell.js

@@ -0,0 +1,450 @@
+// Licensed to Cloudera, Inc. under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  Cloudera, Inc. licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+/*
+---
+
+script: Shell.js
+
+description: Defines Shell; a Hue application that extends Hue.JBrowser.
+
+authors:
+- Hue
+
+requires: [JFrame/JFrame.Browser, hue-shared/Hue.Request, Core/Element, Core/Native, Poller, Core/Fx, hue-shared/Hue.Desktop]
+provides: [Shell]
+
+...
+*/
+ART.Sheet.define('window.art.browser.shell', {
+	'min-width': 620
+});
+
+(function() {
+	var expressions = [
+		{
+			expr: /&/gm,
+			replacement: '&amp;'
+		},
+		{
+			expr: /</gm,
+			replacement: '&lt;'
+		},
+		{
+			expr: />/gm,
+			replacement: '&gt;'
+		},
+		{
+			expr: /"/gm,
+			replacement: '&quot;'
+		},
+		{
+			expr: /\n/g,
+			replacement: "<br>"
+		}
+	];
+
+	String.implement({
+		escapeHTML: function() {
+			var cleaned = this;
+			expressions.each(function(expression) {
+				cleaned = cleaned.replace(expression.expr, expression.replacement);
+			});
+			return cleaned;
+		}
+	});
+})();
+
+var Shell = new Class({
+
+	Extends: Hue.JBrowser,
+	options: {
+		className: 'art browser logo_header shell',
+		displayHistory: false
+	},
+
+	initialize: function(path, options) {
+		this.parent(path || '/shell/', options);
+		if (this.options && this.options.shellId) {
+			this.shellId = options.shellId;
+		}
+		this.addEvent("load", this.startShell.bind(this));
+	},
+
+	startShell: function(view) {
+		var shellIdContainer = $(this).getElement('.shell_id');
+		if (shellIdContainer) {
+			this.shellId = shellIdContainer.get("text");
+		}
+
+		// Set up some state shared between "fresh" and "restored" shells.
+		this.previousCommands = [];
+		this.currentCommandIndex = -1;
+
+		this.jframe.markForCleanup(this.cleanUp.bind(this));
+		this.shellKilled = false;
+
+		this.background = $(this).getElement('.jframe_contents');
+		this.background.setStyle("background-color", "#ffffff");
+		this.container = $(this).getElement('.shell_container');
+		this.output = new Element('span');
+		this.input = new Element('textarea', {
+			events: {
+				keydown: this.handleKeyDown.bind(this),
+				keyup: this.resizeInput.bind(this)
+			},
+			spellcheck: false
+		});
+
+		this.inputExpander = new Fx.Morph(this.input, { duration:0, transition: Fx.Transitions.linear });
+
+		this.jframe.scroller.setOptions({
+			duration: 200
+		});
+
+		// The command-sending request.
+		this.commandReq = new Request.JSON({
+			method: 'post',
+			url: '/shell/process_command',
+			onSuccess: this.commandProcessed.bind(this)
+		});
+
+		if (this.shellId) {
+			this.startRestore(view);
+		} else {
+			this.setup(view);
+		}
+	},
+
+	startRestore: function(view) {
+		this.view = view;
+		this.restoreReq = new Request.JSON({
+			method: 'post',
+			url: '/shell/restore_shell',
+			onSuccess: this.restoreCompleted.bind(this),
+			onFailure: this.restoreFailed.bind(this)
+		});
+		var shellId = this.shellId;
+		this.restoreReq.send({
+			data: 'shellId='+shellId
+		});
+	},
+
+	restoreCompleted: function(json, text) {
+		this.restoreReq = null;
+		if (json.success) {
+			this.view = null;
+			this.nextOffset = json.nextOffset;
+			this.previousCommands = json.commands;
+			this.currentCommandIndex = this.previousCommands.length - 1;
+			this.jframe.collectElement(this.container);
+			this.container.empty();
+			this.setupTerminal(json.output);
+		} else if (json.notRunningSpawning) {
+			this.errorMessage("Error", "The currently running webserver does not support the Shell app. Please contact your admin.");
+		} else {
+			this.restoreFailed();
+		}
+	},
+
+	restoreFailed: function() {
+		this.restoreReq = null;
+		this.shellId = null;
+		var view = this.view;
+		this.view = null;
+		this.setup(view);
+	},
+
+	setupTerminal: function(initVal) {
+		// Set up the DOM
+		this.container.adopt([this.output, this.input]);
+
+		if (initVal) {
+			this.appendToOutput(initVal);
+
+			// Scroll the jframe and focus the input
+			this.jframe.scroller.toBottom();
+		}
+		this.focusInput();
+
+		// If the user clicks anywhere in the jframe, focus the textarea.
+		this.background.addEvent("click", this.focusInput.bind(this));
+		this.shellCreated = true;
+
+		// Register the shell we have with Poller, so we can be included in the output channel it has.
+		Hue.Desktop.store();
+		Poller.listenForShell(this.shellId, this.nextOffset, this.outputReceived.bind(this));
+	},
+
+	focusInput: function() {
+		if (!this.input.get("disabled")) {
+			this.input.focus();
+		}
+	},
+
+	setup: function(view) {
+		this.shellCreated = false;
+		var mainMenuButtons = $(this).getElements("a.menu_button");
+		mainMenuButtons.each(function(button) {
+			var keyName = button.nextSibling.get("text");
+			button.addEvent('click', this.handleShellSelection.bind(this, keyName));
+		}.bind(this));
+	},
+
+	handleShellSelection: function(keyName) {
+		this.registerReq = new Request.JSON({
+			method: 'post',
+			url: '/shell/create',
+			onSuccess: this.registerCompleted.bind(this),
+			onFailure: this.registerFailed.bind(this)
+		});
+		this.registerReq.send({ data: "keyName="+keyName });
+	},
+
+	resizeInput: function() {
+		var currHeight = this.input.getSize().y;
+		var scrollHeight = this.input.getScrollSize().y;
+		if (scrollHeight < currHeight) {
+			return;
+		}
+		// Credit to Philip Hutchison, http://pipwerks.com/2010/05/07/textareaexpander-class-for-mootools/
+		// for suggesting this way of resizing the input. It works really well. This idea is borrowed and
+		// modified from his MIT-licensed code.
+		this.inputExpander.start({ height: scrollHeight });
+	},
+
+	appendToOutput:function(text) {
+		this.output.set('html', this.output.get('html')+text.escapeHTML());
+	},
+
+	registerFailed: function() {
+		this.registerReq = null;
+		this.choices = null;
+		this.choicesText = null;
+		this.errorMessage('Error',"Error creating shell. Is the shell server running?");
+	},
+
+	registerCompleted: function(json, text) {
+		this.registerReq = null;
+		if (!json.success) {
+			if (json.shellCreateFailed) {
+				this.errorMessage('Error', 'Could not create any more shells. Please try again soon.');
+			} else if (json.notRunningSpawning) {
+				this.errorMessage("Error", "The currently running webserver does not support the Shell app. Please contact your admin.");
+			} else if (json.noSuchUser) {
+				this.errorMessage("Error", "The remote server does not have a Unix user with your username. Please contact your admin.");
+			} else if (json.shellNotAllowed) {
+				this.errorMessage("Error", "You do not have permission to create a Shell of this type. Please contact your admin to get permission.");
+			}
+		} else {
+			this.background.setStyle("background-color","#ffffff");
+			this.shellCreated = true;
+			this.shellId = json.shellId;
+			this.options.shellId = json.shellId;
+			this.nextOffset = 0;
+			this.jframe.collectElement(this.container);
+			this.container.empty();
+			this.setupTerminal();
+		}
+	},
+
+	showPreviousCommand: function() {
+		if (this.currentCommandIndex < 0 || this.currentCommandIndex >= this.previousCommands.length) {
+			this.currentCommandIndex = this.previousCommands.length-1;
+		}
+		var oldCommand = this.previousCommands[this.currentCommandIndex];
+		if (oldCommand) {
+			this.input.set('value', oldCommand);
+			this.currentCommandIndex--;
+			this.focusInput();
+		}
+	},
+
+	showNextCommand: function() {
+		if (this.currentCommandIndex < 0 || this.currentCommandIndex >= this.previousCommands.length) {
+			this.currentCommandIndex = this.previousCommands.length?0:-1;
+		}
+		var oldCommand = this.previousCommands[this.currentCommandIndex];
+		if (oldCommand) {
+			this.input.set('value', oldCommand);
+			this.currentCommandIndex++;
+			this.focusInput();
+		}
+	},
+
+	handleUpKey: function() {
+		var tempInputValue = this.tempInputValue;
+		this.tempInputValue = null;
+		if (tempInputValue === this.input.get("value")) {
+			this.showPreviousCommand();
+		}
+	},
+
+	handleDownKey: function() {
+		var tempInputValue = this.tempInputValue;
+		this.tempInputValue = null;
+		if (tempInputValue === this.input.get("value")) {
+			this.showNextCommand();
+		}
+	},
+
+	handleKeyDown: function(event) {
+		if (event.key=="enter") {
+			this.recordCommand();
+			this.sendCommand();
+		} else if (event.key=="up") {
+			this.tempInputValue = this.input.get("value");
+			// The delay is to deal with a problem differentiating "&" and "up" in Firefox.
+			this.handleUpKey.delay(5, this);
+		} else if (event.key=="down") {
+			this.tempInputValue = this.input.get("value");
+			// The delay is to deal with a problem differentiating "(" (left paren) and "down" in Firefox.
+			this.handleDownKey.delay(5, this);
+		} else if (event.key=="tab") {
+			event.stop();
+		}
+		this.resizeInput.delay(0, this);
+	},
+
+	recordCommand: function() {
+		var enteredCommand = this.input.get("value");
+		if (enteredCommand) {
+			if (this.previousCommands[this.previousCommands.length - 1] != enteredCommand) {
+				this.previousCommands.push(enteredCommand);
+				this.currentCommandIndex = this.previousCommands.length - 1;
+			}
+		}
+	},
+
+	sendCommand: function() {
+		var enteredCommand = this.input.get("value");
+		var shellId = this.shellId;
+		this.disableInput();
+		var dataToSend = {
+			lineToSend : enteredCommand,
+			shellId : shellId
+		};
+		this.commandReq.send({
+			data: dataToSend
+		});
+	},
+
+	commandProcessed: function(json, text) {
+		if (json.success) {
+			this.enableInput();
+			this.input.setStyle("height","auto");
+			this.input.set("value", "");
+		} else {
+			if (json.noShellExists) {
+				this.shellExited();
+			} else if (json.bufferExceeded) {
+				this.errorMessage("Error", "You have entered too many commands. Please try again.");
+			} else if (json.notRunningSpawning) {
+				this.errorMessage("Error", "The currently running webserver does not support the Shell app. Please contact your admin.");
+			}
+		}
+	},
+
+	outputReceived: function(json) {
+		if (json.alive || json.exited) {
+			this.appendToOutput(json.output);
+			this.jframe.scroller.toBottom();
+			if (json.exited) {
+				this.shellExited();
+			}
+		} else {
+			if (json.noShellExists) {
+				this.shellExited();
+			} else if (json.shellKilled) {
+				this.shellExited();
+			} else {
+				this.errorMessage('Error','Received invalid JSON response object from the Shell poller');
+			}
+		}
+	},
+
+	enableInput:function() {
+		this.input.set({
+			disabled: false,
+			styles: {
+				cursor: 'text',
+				display: ''
+			}
+		}).focus();
+	},
+
+	disableInput:function() {
+		this.input.set({
+			disabled: true,
+			styles: {
+				cursor: 'default',
+				display: 'none'
+			}
+		}).blur();
+	},
+
+	errorStatus:function() {
+		this.disableInput();
+		this.background.setStyle("background-color", "#cccccc");
+	},
+
+	errorMessage:function(title, message) {
+		this.errorStatus();
+		this.alert(title, message);
+	},
+
+	shellExited:function() {
+		this.errorStatus();
+		this.appendToOutput("\n[Process completed]");
+		this.shellKilled = true;
+	},
+
+	cleanUp:function() {
+		if (this.registerReq) {
+			this.registerReq.cancel();
+		}
+		if (this.restoreReq) {
+			this.restoreReq.cancel();
+		}
+		if (this.commandReq) {
+			this.commandReq.cancel();
+		}
+
+		//Clear out this.options.shellId and this.shellId, but save the value in a local variable
+		//for the purposes of this function.
+		this.options.shellId = null;
+		var shellId = this.shellId;
+		this.shellId = null;
+
+		//Tell the shell poller to stop listening for shellId. Important to do this before
+		//sending the kill shell request because then the resulting output doesn't cause
+		//a non-existent callback to be called.
+		if (shellId) {
+			Poller.stopShellListener(shellId);
+			if (this.shellCreated && !this.shellKilled) {
+				//A one-time request to tell the server to kill the subprocess if it's still alive.
+				var req = new Request.JSON({
+					method: 'post',
+					url: '/shell/kill_shell'
+				});
+				req.send({
+					data: 'shellId='+shellId
+				});
+			}
+		}
+		Hue.Desktop.store();
+	}
+
+});

+ 8 - 0
apps/shell/src/shell/static/js/package.yml

@@ -0,0 +1,8 @@
+copyright: Apache License v2.0
+version: 0.1
+description: Unknown
+name: shell
+sources: [
+Source/Shell/Shell.js,
+Source/Shell/Poller.js
+]

+ 19 - 0
apps/shell/src/shell/templates/failed_to_create.mako

@@ -0,0 +1,19 @@
+<%namespace name="shared" file="shared_components.mako" />
+
+${shared.header("Hue Shell", False)}
+
+## use double hashes for a mako template comment
+
+## this id in the div below ("index") is stripped by Hue.JFrame
+## and passed along as the "view" argument in its onLoad event
+
+## the class 'jframe_padded' will give the contents of your window a standard padding
+<div id="index" class="view jframe_padded">
+Failed to create a shell of the given type. The possible reasons for this are:
+1. The system is out of PTYs.
+2. The system cannot create more subprocesses.
+3. You do not have permission to create shells of this type.
+4. There is no shell with that name.
+5. There is no Unix user account for you.
+</div>
+${shared.footer()}

+ 30 - 0
apps/shell/src/shell/templates/index.mako

@@ -0,0 +1,30 @@
+<%namespace name="shared" file="shared_components.mako" />
+
+${shared.header("Hue Shell", True, shells)}
+
+## use double hashes for a mako template comment
+
+## this id in the div below ("index") is stripped by Hue.JFrame
+## and passed along as the "view" argument in its onLoad event
+
+## the class 'jframe_padded' will give the contents of your window a standard padding
+<div id="index" class="view jframe_padded shell_container">
+  % if shell_id:
+    <span class="shell_id hidden">${shell_id}</span>
+  % else:
+    <div>
+      <ul>
+        % for item in shells:
+          <li>
+          % if item["exists"]:
+            <a class="round Button menu_button">${item["niceName"]}</a><span class="hidden">${item["keyName"]}</span>
+          % else:
+            <a class="round Button disabled">${item["niceName"]}</a>
+          % endif
+          </li>
+        % endfor
+      </ul>
+    </div>
+  % endif
+</div>
+${shared.footer()}

+ 16 - 0
apps/shell/src/shell/templates/no_such_user.mako

@@ -0,0 +1,16 @@
+<%namespace name="shared" file="shared_components.mako" />
+
+${shared.header("Hue Shell", False)}
+
+## use double hashes for a mako template comment
+
+## this id in the div below ("index") is stripped by Hue.JFrame
+## and passed along as the "view" argument in its onLoad event
+
+## the class 'jframe_padded' will give the contents of your window a standard padding
+<div id="index" class="view jframe_padded">
+The Shell app requires a Unix user account for every user of Hue on the remote webserver.
+Please ask your admin to create a user account for you on the remote webserver as
+described in the Shell documentation.
+</div>
+${shared.footer()}

+ 15 - 0
apps/shell/src/shell/templates/not_running_spawning.mako

@@ -0,0 +1,15 @@
+<%namespace name="shared" file="shared_components.mako" />
+
+${shared.header("Hue Shell", False)}
+
+## use double hashes for a mako template comment
+
+## this id in the div below ("index") is stripped by Hue.JFrame
+## and passed along as the "view" argument in its onLoad event
+
+## the class 'jframe_padded' will give the contents of your window a standard padding
+<div id="index" class="view jframe_padded">
+The webserver currently running Hue does not support the Shell app. Please contact your
+administrator.
+</div>
+${shared.footer()}

+ 49 - 0
apps/shell/src/shell/templates/shared_components.mako

@@ -0,0 +1,49 @@
+<%!
+import datetime
+from django.template.defaultfilters import urlencode, escape
+%>
+<%def name="header(title='Hue Shell', toolbar=True, shells=[])">
+  <!DOCTYPE html>
+  <html>
+    <head>
+      <title>${title}</title>
+    </head>
+    <body>
+      <div class="toolbar">
+        <a href="${url('shell.views.index')}"><img src="/shell/static/art/shell.png" class="shell_icon"/></a>
+        % if toolbar:
+          <div class="nav_menu">
+          % if len(shells) == 1:
+            % if shells[0]["exists"]:
+              <a target="Shell" class="nav_button Button round" href="${url('shell.views.create')}?keyName=${shells[0]["keyName"]}">${shells[0]["niceName"]}</a>
+            % else:
+              <a class="nav_button Button round disabled">${shells[0]["niceName"]}</a>
+            % endif
+          % else:
+            % if shells[0]["exists"]:
+              <a target="Shell" class="nav_button Button roundLeft" href="${url('shell.views.create')}?keyName=${shells[0]["keyName"]}">${shells[0]["niceName"]}</a>
+            % else:
+              <a class="nav_button Button roundLeft disabled">${shells[0]["niceName"]}</a>
+            % endif
+            % for item in shells[1:-1]:
+              % if item["exists"]:
+                <a target="Shell" class="nav_button Button" href="${url('shell.views.create')}?keyName=${item["keyName"]}">${item["niceName"]}</a>
+              % else:
+                <a class="nav_button Button disabled">${item["niceName"]}</a>
+              % endif
+            % endfor
+            % if shells[-1]["exists"]:
+              <a target="Shell" class="nav_button Button roundRight" href="${url('shell.views.create')}?keyName=${shells[-1]["keyName"]}">${shells[-1]["niceName"]}</a>
+            % else:
+              <a class="nav_button Button roundRight disabled">${shells[-1]["niceName"]}</a>
+            % endif
+          % endif
+          </div>
+        % endif
+      </div>
+</%def>
+
+<%def name="footer()">
+    </body>
+  </html>
+</%def>

+ 162 - 0
apps/shell/src/shell/tests.py

@@ -0,0 +1,162 @@
+#!/usr/bin/env python
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Tests for "shell"
+"""
+
+from nose.tools import assert_true, assert_equal
+
+from desktop.lib.django_test_util import make_logged_in_client
+from django.contrib.auth.models import User
+from django.utils.encoding import smart_unicode
+import eventlet
+import eventlet.wsgi
+from eventlet.green import time
+from eventlet.green import os
+from eventlet.green import threading
+import pwd
+import re
+import shell
+import shell.conf as conf
+import shell.constants as constants
+import shell.utils as utils
+import simplejson
+
+class TestServer(threading.Thread):
+  def run(self):
+    def dummy_server(env, start_response):
+      start_response('200 OK', [('Content-Type', 'text/plain')])
+      return ['Dummy response\r\n']
+    eventlet.wsgi.server(eventlet.listen(('',55555)), dummy_server)
+
+class TestRequest():
+  def __init__(self):
+    self.POST = {}
+    self.DICT = {}
+    self.META = {}
+
+def reset_all_users():
+  """Reset to a clean state by deleting all users"""
+  for user in User.objects.all():
+    user.delete()
+
+def test_spawning_check():
+  reset_all_users()
+  client = make_logged_in_client(username="test", is_superuser=True)
+
+  get_urls = ["/shell/", "/shell/create"]
+  post_urls = ["/shell/create", "/shell/process_command", "/shell/restore_shell",
+               "/shell/kill_shell", "/shell/retrieve_output", "/shell/add_to_output"]
+  for url in get_urls:
+    response = client.get(url, follow=True)
+    assert "The webserver currently running Hue does not support the Shell app." in response.content
+
+  for url in post_urls:
+    response = client.post(url, follow=True)
+    assert "notRunningSpawning" in response.content
+
+def test_unix_user_account_check():
+  reset_all_users()
+  nonexistent_username = "user_%s" % (time.strftime("%s"),)
+  client = make_logged_in_client(username=nonexistent_username, is_superuser=True)
+
+  d = { 'eventlet.input' : None }
+  response = client.get('/shell/', follow=True, **d)
+  assert "The Shell app requires a Unix user account for every user of Hue" in response.content
+
+def test_rest():
+  reset_all_users()
+  username = pwd.getpwuid(os.getuid()).pw_name
+  client = make_logged_in_client(username=username, is_superuser=True)
+
+  d = { 'eventlet.input' : None }
+  response = client.get('/shell/', follow=True, **d)
+
+  shell_types_available = []
+  for item in shell.conf.SHELL_TYPES.keys():
+    nice_name = shell.conf.SHELL_TYPES[item].nice_name.get().strip()
+    assert nice_name in response.content
+    shell_types_available.append(item)
+
+  if not shell_types_available:
+    return
+
+  response = client.get('/shell/create?keyName=%s' % (time.strftime("%s"),), follow=True, **d)
+  assert "There is no shell with that name." in response.content
+
+  nonexistent_username = "user_%s" % (time.strftime("%s"),)
+  client2 = make_logged_in_client(username=nonexistent_username, is_superuser=True)
+
+  response = client2.get("/shell/create?keyName=%s" % (shell_types_available[0],), follow=True, **d)
+  assert "There is no Unix user account for you." in response.content
+
+  response = client.get("/shell/create?keyName=%s" % (shell_types_available[0],), follow=True, **d)
+  assert '<span class="shell_id hidden">' in response.content
+
+  shell_id_start = response.content.index('<span class="shell_id hidden">')+len('<span class="shell_id hidden">')
+  shell_id_end = response.content.index('</span>')
+  shell_id = response.content[shell_id_start:shell_id_end]
+  assert re.match(r"^\s*\d+\s*$", shell_id)
+
+  response = client.post("/shell/kill_shell", follow=True, data={constants.SHELL_ID: shell_id}, **d)
+  assert response.content.strip() == "Shell successfully killed"
+
+def test_parse_shell_pairs():
+  request = TestRequest()
+  request.POST[constants.NUM_PAIRS] = 2
+  request.POST["%s2" % (constants.SHELL_ID,)] = '0'
+  request.POST["%s2" % (constants.OFFSET,)] = '0'
+  request.POST["%s1" % (constants.SHELL_ID,)] = '1'
+  request.POST["%s1" % (constants.OFFSET,)] = '1'
+  parsed_pairs = utils.parse_shell_pairs(request)
+  assert parsed_pairs[0][0] == '1'
+  assert parsed_pairs[0][1] == 1
+  assert parsed_pairs[1][0] == '0'
+  assert parsed_pairs[1][1] == 0
+
+  request.POST[constants.NUM_PAIRS] = 1
+  assert len(utils.parse_shell_pairs(request)) == 1
+
+  request.POST[constants.NUM_PAIRS] =  'most definitely not a number'
+  try:
+    utils.parse_shell_pairs(request)
+  except ValueError:
+    pass
+  else:
+    assert False, "parse_shell_pairs did not throw an exception when trying to convert a malformed string to integer"
+

+ 28 - 0
apps/shell/src/shell/urls.py

@@ -0,0 +1,28 @@
+#!/usr/bin/env python
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from django.conf.urls.defaults import patterns, url
+
+urlpatterns = patterns('shell',
+  url(r'^$', 'views.index'),
+  url(r'^process_command/?$', 'views.process_command'),
+  url(r'^restore_shell/?$', 'views.restore_shell'),
+  url(r'^kill_shell/?$', 'views.kill_shell'),
+  url(r'^create/?$', 'views.create'),
+  url(r'^retrieve_output/?$', 'views.retrieve_output'),
+  url(r'^add_to_output/?$', 'views.add_to_output'),
+)

+ 98 - 0
apps/shell/src/shell/utils.py

@@ -0,0 +1,98 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+A mixed bag of utilities that are useful for the Shell app but aren't terribly interesting.
+"""
+
+import desktop.lib.i18n
+import logging
+import shell.constants as constants
+from eventlet.green import os
+
+LOG = logging.getLogger(__name__)
+
+def parse_shell_pairs(request):
+  """
+  Parses out and returns a list of (shell_id, offset) tuples from a descendant of RequestHandler.
+  """
+  shell_pairs = []
+  num_pairs = int(request.POST.get(constants.NUM_PAIRS, ""))
+
+  for i in xrange(1, num_pairs+1):
+    try:
+      shell_id_i = request.POST.get("%s%d" % (constants.SHELL_ID, i), "-1")
+      offset_i = int(request.POST.get("%s%d" % (constants.OFFSET, i), "-1"))
+    except ValueError:
+      LOG.debug('Bad HTTP parameter : "%s%d" has value "%s"' % (constants.OFFSET, i,
+                                      request.POST.get("%s%d" % (constants.SHELL_ID, i), "-1")))
+    else:
+      shell_pairs.append((shell_id_i, offset_i, ))
+  return shell_pairs
+
+
+def executable_exists(executable):
+  if not executable:
+    return False
+  if type(executable) == list:
+    executable = executable[0]
+  env = desktop.lib.i18n.make_utf8_env()
+  path = env.get("PATH", os.defpath)
+  path = [os.path.normpath(item) for item in path.strip().strip(os.pathsep).split(os.pathsep)]
+  for item in path:
+    potential_executable = os.path.join(item, executable)
+    if os.access(potential_executable, os.F_OK | os.X_OK):
+      return True
+  return False
+
+class UserMetadata(object):
+  """
+  A simple class to encapsulate the metadata for a user.
+  """
+  def __init__(self, username):
+    self.num_shells = 0
+    self.current_shell_id = 0
+    self.username = username
+
+  def get_next_id(self):
+    """
+    Return the next available ID. Successive calls to this function will yield two different IDs.
+    Returns a unicode string for compatibility with Tornado.
+    """
+    curr_id = self.current_shell_id
+    self.current_shell_id += 1
+    return unicode(curr_id)
+
+  def decrement_count(self):
+    """
+    Decrement the number of shells currently open for the given user.
+    """
+    if self.num_shells > 0:
+      self.num_shells -= 1
+    else:
+      LOG.error("Num shells is negative for user %s" % (self.username,))
+
+  def increment_count(self):
+    """
+    Increment the number of shells currently open for the given user.
+    """
+    self.num_shells += 1
+
+  def get_shell_count(self):
+    """
+    Return the number of shells currently open for the given user.
+    """
+    return self.num_shells

+ 122 - 0
apps/shell/src/shell/views.py

@@ -0,0 +1,122 @@
+#!/usr/bin/env python
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from desktop.lib.django_util import render
+from django.http import HttpResponse
+import simplejson
+import shell.conf
+import shell.constants as constants
+import shell.utils as utils
+from shell.shellmanager import ShellManager
+import sys
+
+def _running_with_spawning(request):
+  return 'eventlet.input' in request.META
+
+def index(request):
+  if not _running_with_spawning(request):
+    return render('not_running_spawning.mako', request, {})
+  shell_manager = ShellManager.global_instance()
+  result = shell_manager.available_shell_types(request.user)
+  if result is None:
+    return render('no_such_user.mako', request, {})
+  return render('index.mako', request, {'shells':result})
+
+def create(request):
+  if not _running_with_spawning(request):
+    if request.method == "POST":
+      result = simplejson.dumps({ constants.NOT_RUNNING_SPAWNING : True })
+      return HttpResponse(result, mimetype="application/json")
+    else:
+      return render('not_running_spawning.mako', request, {})
+  shell_manager = ShellManager.global_instance()
+  user = request.user
+  if request.method == "POST":
+    key_name = request.POST.get(constants.KEY_NAME, "")
+  else:
+    key_name = request.GET.get(constants.KEY_NAME, "")
+  result = shell_manager.try_create(user, key_name)
+  if request.method == "POST":
+    return HttpResponse(simplejson.dumps(result), mimetype="application/json")
+  else:
+    if constants.SUCCESS in result:
+      shell_types = shell_manager.available_shell_types(user)
+      dict_for_template = { 'shells' : shell_types,
+                            'shell_id' : result.get(constants.SHELL_ID) }
+      return render('index.mako', request, dict_for_template)
+    else:
+      return render('failed_to_create.mako', request, {})
+
+def kill_shell(request):
+  if not _running_with_spawning(request):
+    result = simplejson.dumps({ constants.NOT_RUNNING_SPAWNING : True })
+    return HttpResponse(result, mimetype="application/json")
+  shell_manager = ShellManager.global_instance()
+  username = request.user.username
+  shell_id = request.POST[constants.SHELL_ID]
+  result = shell_manager.kill_shell(username, shell_id)
+  return HttpResponse(result)
+
+def restore_shell(request):
+  if not _running_with_spawning(request):
+    result = simplejson.dumps({ constants.NOT_RUNNING_SPAWNING : True })
+    return HttpResponse(result, mimetype="application/json")
+  shell_manager = ShellManager.global_instance()
+  username = request.user.username
+  shell_id = request.POST[constants.SHELL_ID]
+  result = shell_manager.get_previous_output(username, shell_id)
+  return HttpResponse(simplejson.dumps(result), mimetype="application/json")
+
+def process_command(request):
+  if not _running_with_spawning(request):
+    result = simplejson.dumps({ constants.NOT_RUNNING_SPAWNING : True })
+    return HttpResponse(result, mimetype="application/json")
+  shell_manager = ShellManager.global_instance()
+  username = request.user.username
+  shell_id = request.POST[constants.SHELL_ID]
+  command = request.POST.get(constants.COMMAND, "")
+  result = shell_manager.process_command(username, shell_id, command)
+  return HttpResponse(simplejson.dumps(result), mimetype="application/json")
+
+def retrieve_output(request):
+  if not _running_with_spawning(request):
+    result = simplejson.dumps({ constants.NOT_RUNNING_SPAWNING : True })
+    return HttpResponse(result, mimetype="application/json")
+  shell_manager = ShellManager.global_instance()
+  username = request.user.username
+  hue_instance_id = request.META[constants.HUE_INSTANCE_ID]
+  try:
+    shell_pairs = utils.parse_shell_pairs(request)
+  except ValueError:
+    shell_pairs = []
+  result = shell_manager.retrieve_output(username, hue_instance_id, shell_pairs)
+  return HttpResponse(simplejson.dumps(result), mimetype="application/json")
+
+def add_to_output(request):
+  if not _running_with_spawning(request):
+    result = simplejson.dumps({ constants.NOT_RUNNING_SPAWNING : True })
+    return HttpResponse(result, mimetype="application/json")
+  shell_manager = ShellManager.global_instance()
+  username = request.user.username
+  hue_instance_id = request.META[constants.HUE_INSTANCE_ID]
+  try:
+    shell_pairs = utils.parse_shell_pairs(request)
+  except ValueError:
+    shell_pairs = []
+  result = shell_manager.add_to_output(username, hue_instance_id, shell_pairs)
+  return HttpResponse(simplejson.dumps(result), mimetype="application/json")
+

+ 23 - 0
apps/shell/src/shell/windmilltests.py

@@ -0,0 +1,23 @@
+# Licensed to Cloudera, Inc. under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  Cloudera, Inc. licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from desktop.lib.windmill_util import logged_in_client
+
+def test_shell():
+  """ launches the default view for shell """
+  client = logged_in_client()
+  client.click(id='hue-shell-menu')
+  client.waits.forElement(classname='Hue-SHELL', timeout='2000')

+ 7 - 2
desktop/conf.dist/hue.ini

@@ -33,13 +33,18 @@ django_debug_mode=0
 # Turn off backtrace for server error
 http_500_debug_mode=0
 
+# Set to true to use CherryPy as the webserver, set to false
+# to use Spawning as the webserver. Defaults to Spawning if
+# key is not specified.
+## use_cherrypy_server = false
+
 # Webserver runs as this user
 ## server_user=hue
 ## server_group=hue
 
-# If set to false, runcpserver will not actually start the web server.
+# If set to false, runserver will not actually start the web server.
 # Used if Apache is being used as a WSGI container.
-## enable_cherrypy_server=yes
+## enable_server=yes
 
 # Number of threads used by the CherryPy web server
 ## cherrypy_server_threads=10

+ 9 - 0
desktop/core/ext-py/Spawning-0.9.6/AUTHORS.txt

@@ -0,0 +1,9 @@
+Authors
+
+ - Donovan Preston (creator)
+ - Ben Bangert (contributor)
+ - Ludvig Ericson (contributor)
+ - Elliot Murphy (contributor)
+ - Steve 'Ashcrow' Milner (contributor)
+ - Ryan Williams (contributor)
+ - R. Tyler Croy (current maintainer)

+ 19 - 0
desktop/core/ext-py/Spawning-0.9.6/LICENSE.txt

@@ -0,0 +1,19 @@
+Copyright (c) 2008, Donovan Preston
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

+ 4 - 0
desktop/core/ext-py/Spawning-0.9.6/MANIFEST.in

@@ -0,0 +1,4 @@
+include *.txt *.rst *.py
+recursive-include src/spawning *.py
+recursive-include src/Spawning.egg-info *
+recursive-include rc-scripts *

+ 115 - 0
desktop/core/ext-py/Spawning-0.9.6/NEWS.txt

@@ -0,0 +1,115 @@
+0.9
+====
+    - Remove the unnecessary dependency on eventlet.jsonhttp. This makes spawning work with eventlet 0.9, which removed jsonhttp and put it in another library. (fzzzy)
+    - Preserve the order of sys.path. (verterok)
+    - Work around a python deadlock bug with forked child processes (statik)
+
+0.9.1: Fix news file to move entries from 0.8.13 to 0.9 since I decided to call this release 0.9.
+
+0.9.2: Fix a small typo in the preserve order of sys.path patch from 0.9 that would cause spawning to crash. (schmir)
+
+0.9.3:
+    - Add the ability to disallow HTTP Keepalive via --no-keepalive
+    - Enable "progressive" spawning and reaping of children processes; in
+      effect the parent will receive a signal from children about their
+      impending death and the parent will spin up a new child while the old one
+      expires
+    - SIGHUP to the parent will cause it to cycle children without itself dying
+
+0.9.4:
+    - Script for spawning renamed from "spawn" to "spawning" to avoid confusion
+    - Use O_APPEND when opening files when daemonizing
+    - Debian/Ubuntu init.d scripts using start-stop-daemon
+    - spawning.util module added to house methods that don't belong elsewhere
+    - Update Spawning to use newer Eventlet API calls
+    - Properly handle errors when calling `setproctitle`
+    - Allow optional `eventlet.backdoor` bound to localhost
+    - Operate more cleanly with Eventlet's websockets support with changes to
+      eventlet.tpool integration
+    - Avoid passing certain objects through `eventlet.tpool.Proxy` when using
+      threads for a performance gain
+    - Prevent leaking pipes when handling children processes
+
+
+0.9.5:
+    - Fixed a number of issues with reloading of Spawning children
+    - Added /_sysinfo to provide information about the machine
+    - Added health page that listens on a different port from the controller,
+      providing HTML and JSON formatted information about children
+    - Introduced a basic logfile analyzer for processsing Spawning logs for
+      further information not provided by the health page
+
+
+
+0.8
+====
+
+Fixed a problem where eventlet monkeypatching was inappropriately installed in child processes which use threads. This would result in certain operations (primarily DNS lookup operations) resulting in a greenlet "cannot switch to another thread" exception.
+
+Changed the paste factory to use 10 worker threads by default, to match the paste default. Previously if the paste ini file did not mention how many worker threads to use, spawning would default to 0 and switch into cooperative, non-blocking mode.
+
+Added a deadman timeout to child processes which have been told to exit and are waiting for outstanding requests to finish. If the timeout expires before all requests have completed, we assume the process is hung and kill -9 it. The default timeout is 120 seconds.
+
+If an i/o child process dies with an exit code other than 0, the controller decides something must have gone horribly wrong and restarts all of the children.
+
+0.8.1: Fix a bug where the reloader didn't work with paster serve.
+
+0.8.2: In the svn reloader, watch both spawning's directory and the directory of the wsgi application we are serving. Also, fix the django_factory which was broken by 0.8.
+
+0.8.3: For all svn repositories the svn reloader is watching, also watch any svn:externals repositories contained therein.
+
+0.8.4: Fixed a bug where the controller process dying unexpectedly (such as from a kill -9) would cause the children to have an exception but then keep running forever, preventing any other processes from using the ports again in the future.
+
+0.8.5: Fixed a bug in the svn reloader where files that svn reported as 'not under version control' would cause the reloader to crash and exit immediately. I now use svn's exit code instead of sniffing svn's output, which should also help avoid problems for anyone who is using a localized copy of svn, or if svn ever changes the content of these messages.
+
+0.8.6: Fix a file descriptor leak that occurred when the controller reloaded. If your code changed enough times over the lifetime of the server (thousands of times) it would eventually run out of file descriptors and refuse to start up. Now the number of file descriptors stays constant no matter how many times the server restarts.
+
+0.8.7:
+  - In the svn reloader process, check to see if the controller is still alive, and if not, just exit.
+  - Don't hold on to the web port at all in the reloader_svn process.
+  - Add an exponential backoff to the controller's "panic" restart. Before spawning would restart as fast as possible; now it backs off the time between restarts.
+  - If we can't import the wsgi app, panic.
+  - If we can't fork a child process (out of memory), panic.
+
+0.8.8:
+  - Added --access-log-file command line option to allow writing access logs to someplace other than stdout. Useful for redirecting to /dev/null for speed
+  - Correctly extract the child's exit code and clean up the logging of child exit events.
+  - Add coverage gathering using figleaf if the --coverage command line option is given. When gathering coverage, the figleaf report can be downloaded from the /_coverage url.
+  - Add a --max-memory option to limit the total amount of memory spawning will use. If this is exceeded a SIGHUP will be sent to the controller causing the children to restart.
+  - Add a --max-age option to limit the total amount of time a spawning child is allowed to run. After the time limit is exceeded a SIGHUP will be sent to the controller to cause the children to restart.
+  - Instead of just passing the PYTHONPATH environment variable through to the children, construct the PYTHONPATH from the contents of sys.path.
+  - Instead of just trying to run 'spawn' with /usr/bin/env when restarting, just run sys.executable -m spawning.spawning_controller, making it more likely that the controller will run correctly when restarting.
+  - Add a --verbose option and change the default startup procedure to not log the detailed dictionary of configuration information.
+
+0.8.9: Minor release which provides compatibility with running servers which are using 0.8.7. With 0.8.8, any running servers which are upgraded from 0.8.7->0.8.8 will crash with a KeyError and need to be restarted manually.
+
+0.8.10: When spawning starts up, add the current working directory to sys.path if it is not already there. Also, when calculating the PYTHONPATH to give to the child from sys.path, remove any path which does not exist, preventing setuptools "DistributionNotFound" errors.
+
+0.8.11:
+    - Added Python 2.4 compatibility. (kiorky)
+    - Added license headers to all source files. (statik)
+    - Print exceptions to stderr instead of stdout. (lericson)
+    - Don't assume every OSError the controller process gets is EINTR. (lericson)
+    - Added simple daemonizing support. (lericson)
+    - Added an OpenRC init script. (lericson)
+    - Added an explicit manifest. (lericson)
+
+0.8.12:
+    - Remove the processpool implementation added in 0.7 because an equivalent setup can be achieved using controller processes and 1 thread; now we can just talk about the 'number of processes' and 'number of threads' instead of having two levels of different kind of processes when using the processpool. (fzzzy)
+    - Minor release to fix the explicit manifest and OpenRC init script added in the last release. (lericson)
+    - When running under 2.6 we no longer produce a deprecation warning about the removal of the sets module. (fzzzy)
+
+
+
+
+
+0.7
+====
+
+Added django_factory.
+
+Add an optional worker processpool which can be used as an equivalent to the worker threadpool.
+
+Added command-line script to launch a wsgi application, 'spawn'.
+
+	spawn mymodule.my_wsgi_app

+ 126 - 0
desktop/core/ext-py/Spawning-0.9.6/PKG-INFO

@@ -0,0 +1,126 @@
+Metadata-Version: 1.0
+Name: Spawning
+Version: 0.9.6
+Summary: Spawning is a wsgi server which supports multiple processes, multiple threads, green threads, non-blocking HTTP io, and automatic graceful upgrading of code.
+Home-page: UNKNOWN
+Author: R. Tyler Croy
+Author-email: tyler@monkeypox.org
+License: UNKNOWN
+Description: Spawning is a fast, easy to use, and flexible HTTP server for hosting python web applications which conform to the WSGI interface.
+        
+        Spawning uses eventlet to do non-blocking I/O for http requests and responses. This means the server will scale to a large number of idle keep-alive connections easily. Spawning can be configured to use multiple OS processes and either POSIX threads or eventlet's green threads, which are implemented using greenlet.
+        
+        Spawning is open source software, licensed under the MIT license. If you wish to contribute to development, please check out the source from http://github.com/rtyler/Spawning/ and either submit patches or fork spawning and submit a pull request.
+        
+        Single or Multiple Process
+        ==========================
+        
+        If your wsgi applications store state in memory, Spawning can be configured to run only one Python process. In this configuration your application state will be available to all requests but your application will not be able to take full advantage of multiple processors. Using multiple processes will take advantage of all processors and thus should be used for applications which do not share state.
+        
+        Single or Multiple Worker Thread
+        ================================================================
+        
+        If your wsgi applications perform a certain subset of blocking calls which have been monkeypatched by eventlet to cooperate instead (such as operations in the socket module), you can configure each process to run only a single main thread and cooperate using eventlet's green threads instead. This can be useful if your application needs to scale to a large number of simultaneous open connections, such as a COMET server or an application which uses AJAX polling. However, most existing wsgi applications will probably perform blocking operations (for example, calling database adapter libraries which perform blocking socket operations). Therefore, for most wsgi applications a combination of multiple processes and multiple threads will be ideal.
+        
+        Graceful Code Reloading
+        =======================
+        Spawning can watch all Python files that are imported into sys.modules for changes and performs a graceful reload on change. To enable this behavior, specify --reload=dev on the command line.  Old processes are told to stop accepting requests and finish any outstanding requests they are servicing, and shutdown. Meanwhile, new processes are started and begin accepting requests and servicing them with the new code. At no point will users of your site see "connection refused" errors because the server is continuously listening during reload.
+        
+        Running spawning
+        ================
+        
+        Spawning can be used to launch a wsgi application from the command line using the "spawn" script, or using Python Paste. To use with paste, specify use = egg:Spawning in the [server:main] section of a paste ini file.
+        
+        Spawning can also be used to run a Django application by using --factory=spawning.django_factory.config_factory.
+        
+        Examples of running spawning
+        ============================
+        
+        Run the wsgi application callable called "my_wsgi_application" inside the my_wsgi_module.py file::
+        
+          % spawning my_wsgi_module.my_wsgi_application
+        
+        Run whatever is configured inside of the paste-style configuration file development.ini. Equivalent to using paster serve with an ini file configured to use Spawning as the server::
+        
+          % spawning --factory=spawning.paste_factory.config_factory development.ini
+        
+        Run the Django app mysite::
+        
+          % spawning --factory=spawning.django_factory.config_factory mysite.settings
+        
+        Run the wsgi application wrapped with some middleware. Pass as many middleware strings as desired after the wsgi application name::
+        
+          % spawning my_wsgi_module.my_wsgi_application other_wsgi_module.some_wsgi_middleware
+        
+        Run the wsgi application on port 80, with 4 processes each using a threadpool of size 8::
+        
+          % sudo spawning --port=80 --processes=4 --threads=8 my_wsgi_module.my_wsgi_application
+        
+        Use a threadpool of size 0, which indicates that eventlet monkeypatching should be performed and wsgi applications should all be called in the same thread. Useful for writing a comet-style application where a lot of requests are simply waiting on a server-side event or internal network io to complete::
+        
+          % spawning --processes=4 --threads=0 my_wsgi_module.my_comet_application
+        
+        Additional Useful Arguments
+        ===========================
+        
+        -l ACCESS_LOG_FILE, --access-log-file=ACCESS_LOG_FILE
+        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+        
+            The file to log access log lines to. If not given, log
+            to stdout. Pass /dev/null to discard logs.
+        
+        -c, --coverage
+        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+        
+            If given, gather coverage data from the running
+            program and make the coverage report available from
+            the /_coverage url. See the figleaf docs for more
+            info: http://darcs.idyll.org/~t/projects/figleaf/doc/
+        
+        -m MAX_MEMORY, --max-memory=MAX_MEMORY
+        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+        
+            If given, the maximum amount of memory this instance
+            of Spawning is allowed to use. If all of the processes
+            started by this Spawning controller use more than this
+            amount of memory, send a SIGHUP to the controller to
+            get the children to restart.
+        
+        -a MAX_AGE, --max-age=MAX_AGE
+        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+        
+            If given, the maximum amount of time (in seconds) an
+            instance of spawning_child is allowed to run. Once
+            this time limit has expired a SIGHUP will be sent to
+            spawning_controller, causing it to restart all of the
+            child processes.
+        
+        --status-port=PORT, --status-host=HOST
+        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+        
+            If given, starts up a small web service to give 
+            health status reports on the Spawning server.  The 
+            service listens on two urls, 
+            
+            * http://status_host:status_port/status
+            * http://status_host:status_port/status.json
+            
+            The first is an HTML page that displays the status
+            of the server in a human-pleasing manner.  The .json
+            url is a JSON formatting of the same data.
+            
+            The status web service is only started if the 
+            --status-port option is supplied and different than
+            the service port.  --status-host is useful if
+            monitoring happens on a different ip address than
+            web application requests.
+        
+Platform: UNKNOWN
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python
+Classifier: Operating System :: MacOS :: MacOS X
+Classifier: Operating System :: POSIX
+Classifier: Topic :: Internet
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Intended Audience :: Developers
+Classifier: Development Status :: 4 - Beta

+ 108 - 0
desktop/core/ext-py/Spawning-0.9.6/README.rst

@@ -0,0 +1,108 @@
+Spawning is a fast, easy to use, and flexible HTTP server for hosting python web applications which conform to the WSGI interface.
+
+Spawning uses eventlet to do non-blocking I/O for http requests and responses. This means the server will scale to a large number of idle keep-alive connections easily. Spawning can be configured to use multiple OS processes and either POSIX threads or eventlet's green threads, which are implemented using greenlet.
+
+Spawning is open source software, licensed under the MIT license. If you wish to contribute to development, please check out the source from http://github.com/rtyler/Spawning/ and either submit patches or fork spawning and submit a pull request.
+
+Single or Multiple Process
+==========================
+
+If your wsgi applications store state in memory, Spawning can be configured to run only one Python process. In this configuration your application state will be available to all requests but your application will not be able to take full advantage of multiple processors. Using multiple processes will take advantage of all processors and thus should be used for applications which do not share state.
+
+Single or Multiple Worker Thread
+================================================================
+
+If your wsgi applications perform a certain subset of blocking calls which have been monkeypatched by eventlet to cooperate instead (such as operations in the socket module), you can configure each process to run only a single main thread and cooperate using eventlet's green threads instead. This can be useful if your application needs to scale to a large number of simultaneous open connections, such as a COMET server or an application which uses AJAX polling. However, most existing wsgi applications will probably perform blocking operations (for example, calling database adapter libraries which perform blocking socket operations). Therefore, for most wsgi applications a combination of multiple processes and multiple threads will be ideal.
+
+Graceful Code Reloading
+=======================
+Spawning can watch all Python files that are imported into sys.modules for changes and performs a graceful reload on change. To enable this behavior, specify --reload=dev on the command line.  Old processes are told to stop accepting requests and finish any outstanding requests they are servicing, and shutdown. Meanwhile, new processes are started and begin accepting requests and servicing them with the new code. At no point will users of your site see "connection refused" errors because the server is continuously listening during reload.
+
+Running spawning
+================
+
+Spawning can be used to launch a wsgi application from the command line using the "spawn" script, or using Python Paste. To use with paste, specify use = egg:Spawning in the [server:main] section of a paste ini file.
+
+Spawning can also be used to run a Django application by using --factory=spawning.django_factory.config_factory.
+
+Examples of running spawning
+============================
+
+Run the wsgi application callable called "my_wsgi_application" inside the my_wsgi_module.py file::
+
+  % spawning my_wsgi_module.my_wsgi_application
+
+Run whatever is configured inside of the paste-style configuration file development.ini. Equivalent to using paster serve with an ini file configured to use Spawning as the server::
+
+  % spawning --factory=spawning.paste_factory.config_factory development.ini
+
+Run the Django app mysite::
+
+  % spawning --factory=spawning.django_factory.config_factory mysite.settings
+
+Run the wsgi application wrapped with some middleware. Pass as many middleware strings as desired after the wsgi application name::
+
+  % spawning my_wsgi_module.my_wsgi_application other_wsgi_module.some_wsgi_middleware
+
+Run the wsgi application on port 80, with 4 processes each using a threadpool of size 8::
+
+  % sudo spawning --port=80 --processes=4 --threads=8 my_wsgi_module.my_wsgi_application
+
+Use a threadpool of size 0, which indicates that eventlet monkeypatching should be performed and wsgi applications should all be called in the same thread. Useful for writing a comet-style application where a lot of requests are simply waiting on a server-side event or internal network io to complete::
+
+  % spawning --processes=4 --threads=0 my_wsgi_module.my_comet_application
+
+Additional Useful Arguments
+===========================
+
+-l ACCESS_LOG_FILE, --access-log-file=ACCESS_LOG_FILE
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+    The file to log access log lines to. If not given, log
+    to stdout. Pass /dev/null to discard logs.
+
+-c, --coverage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+    If given, gather coverage data from the running
+    program and make the coverage report available from
+    the /_coverage url. See the figleaf docs for more
+    info: http://darcs.idyll.org/~t/projects/figleaf/doc/
+
+-m MAX_MEMORY, --max-memory=MAX_MEMORY
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+    If given, the maximum amount of memory this instance
+    of Spawning is allowed to use. If all of the processes
+    started by this Spawning controller use more than this
+    amount of memory, send a SIGHUP to the controller to
+    get the children to restart.
+
+-a MAX_AGE, --max-age=MAX_AGE
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+    If given, the maximum amount of time (in seconds) an
+    instance of spawning_child is allowed to run. Once
+    this time limit has expired a SIGHUP will be sent to
+    spawning_controller, causing it to restart all of the
+    child processes.
+
+--status-port=PORT, --status-host=HOST
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+    If given, starts up a small web service to give 
+    health status reports on the Spawning server.  The 
+    service listens on two urls, 
+    
+    * http://status_host:status_port/status
+    * http://status_host:status_port/status.json
+    
+    The first is an HTML page that displays the status
+    of the server in a human-pleasing manner.  The .json
+    url is a JSON formatting of the same data.
+    
+    The status web service is only started if the 
+    --status-port option is supplied and different than
+    the service port.  --status-host is useful if
+    monitoring happens on a different ip address than
+    web application requests.

+ 167 - 0
desktop/core/ext-py/Spawning-0.9.6/pip-log.txt

@@ -0,0 +1,167 @@
+Downloading/unpacking distribute
+  Getting page http://pypi.python.org/simple/distribute
+  URLs to search for versions for distribute:
+  * http://pypi.python.org/simple/distribute/
+  Getting page http://packages.python.org/distribute
+  Analyzing links from page http://pypi.python.org/simple/distribute/
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.6.tar.gz#md5=a0a7541a8169d73842e4bb88d79cf8d3 (from http://pypi.python.org/simple/distribute/), version: 0.6.6
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.7.tar.gz#md5=b79b9422b1ea517bf3b43e277e9c813a (from http://pypi.python.org/simple/distribute/), version: 0.6.7
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.8.tar.gz#md5=c0c6a6b273234cb2cf8c4f82612e375c (from http://pypi.python.org/simple/distribute/), version: 0.6.8
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.10.tar.gz#md5=99fb4b3e4ef0861bba11aa1905e89fed (from http://pypi.python.org/simple/distribute/), version: 0.6.10
+    Skipping link http://pypi.python.org/packages/2.6/d/distribute/distribute-0.6-py2.6.egg#md5=89c46c2ed0c756dd278acc1482aa12f1 (from http://pypi.python.org/simple/distribute/); unknown archive format: .egg
+    Skipping link http://pypi.python.org/packages/2.4/d/distribute/distribute-0.6-py2.4.egg#md5=8fc3eb887ee98c506c38838955f9eee2 (from http://pypi.python.org/simple/distribute/); unknown archive format: .egg
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.4.tar.gz#md5=7a963679fddc64a0e363b2ccf8024952 (from http://pypi.python.org/simple/distribute/), version: 0.6.4
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.1.tar.gz#md5=e6224b1da4636dd8ae53407fc67bb35b (from http://pypi.python.org/simple/distribute/), version: 0.6.1
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.9.tar.gz#md5=5b1a2fde063a361aa241f98e0f9e1931 (from http://pypi.python.org/simple/distribute/), version: 0.6.9
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.2.tar.gz#md5=8e612376b1a0e4dfddcbcaefcc14515e (from http://pypi.python.org/simple/distribute/), version: 0.6.2
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.3.tar.gz#md5=3940fd02a763f001014296cfec5e69f2 (from http://pypi.python.org/simple/distribute/), version: 0.6.3
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.5.tar.gz#md5=ebe726123b35c1efbf5f7fc83116e3a1 (from http://pypi.python.org/simple/distribute/), version: 0.6.5
+    Found link http://pypi.python.org/packages/source/d/distribute/distribute-0.6.tar.gz#md5=698b898596fc35c95dbda8332cfca08b (from http://pypi.python.org/simple/distribute/), version: 0.6
+    Skipping link http://pypi.python.org/packages/2.5/d/distribute/distribute-0.6-py2.5.egg#md5=d87f6492c53d192c62e0334859d18b59 (from http://pypi.python.org/simple/distribute/); unknown archive format: .egg
+    Skipping link http://pypi.python.org/packages/2.3/d/distribute/distribute-0.6-py2.3.egg#md5=66d06db7fc91227585f81b0b27b07bab (from http://pypi.python.org/simple/distribute/); unknown archive format: .egg
+    Skipping link http://packages.python.org/distribute (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/1 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/10 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/12 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/13 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/16 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/18 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/3 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/31 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/40 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/49 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/5 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/50 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/52 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/56 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/6 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/7 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/8 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issue/9 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bitbucket.org/tarek/distribute/issues/ (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bugs.python.org/setuptools/issue2 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bugs.python.org/setuptools/issue20 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bugs.python.org/setuptools/issue33 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bugs.python.org/setuptools/issue39 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bugs.python.org/setuptools/issue40 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bugs.python.org/setuptools/issue41 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://bugs.python.org/setuptools/issue44 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://code.google.com/p/unladen-swallow/source/detail?spec=svn875&amp;r=719 (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://mail.python.org/mailman/listinfo/distutils-sig (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://packages.python.org (from http://pypi.python.org/simple/distribute/); not a file
+    Skipping link http://python-distribute.org/distribute_setup_3k.py (from http://pypi.python.org/simple/distribute/); unknown archive format: .py
+    Skipping link http://python-distribute.org/distribute_setup.py (from http://pypi.python.org/simple/distribute/); unknown archive format: .py
+    Found link http://bitbucket.org/tarek/distribute/get/0.6-maintenance.zip#egg=distribute-dev (from http://pypi.python.org/simple/distribute/), version: dev
+  Analyzing links from page http://packages.python.org/distribute/
+    Skipping link http://packages.python.org/distribute/_static/nature.css (from http://packages.python.org/distribute/); unknown archive format: .css
+    Skipping link http://packages.python.org/distribute/_static/pygments.css (from http://packages.python.org/distribute/); unknown archive format: .css
+    Skipping link http://packages.python.org/distribute/ (from http://packages.python.org/distribute/); not a file
+    Skipping link http://packages.python.org/distribute/roadmap.html (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/genindex.html (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/#welcome-to-distribute-s-documentation (from http://packages.python.org/distribute/); not a file
+    Skipping link http://pixane.com (from http://packages.python.org/distribute/); not a file
+    Skipping link http://packages.python.org/distribute/roadmap.html#x (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/roadmap.html#id1 (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/python3.html (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/python3.html#distribute-as-help-during-porting (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/python3.html#distributing-python-3-modules (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/python3.html#advanced-features (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/python3.html#note-on-compatibility-with-setuptools (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/using.html (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/setuptools.html (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/setuptools.html#developer-s-guide (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/setuptools.html#command-reference (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/setuptools.html#extending-and-reusing-distribute (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/easy_install.html (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/easy_install.html#using-easy-install (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/easy_install.html#reference-manual (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/easy_install.html#history (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/easy_install.html#future-plans (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/pkg_resources.html (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/pkg_resources.html#overview (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/pkg_resources.html#api-reference (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/#indices-and-tables (from http://packages.python.org/distribute/); not a file
+    Skipping link http://packages.python.org/distribute/modindex.html (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/search.html (from http://packages.python.org/distribute/); unknown archive format: .html
+    Skipping link http://packages.python.org/distribute/_sources/index.txt (from http://packages.python.org/distribute/); unknown archive format: .txt
+    Skipping link http://pypi.python.org/pypi/distribute (from http://packages.python.org/distribute/); not a file
+    Skipping link http://sphinx.pocoo.org/ (from http://packages.python.org/distribute/); not a file
+  Using version 0.6.10 (newest of versions: 0.6.10, 0.6.9, 0.6.8, 0.6.7, 0.6.6, 0.6.5, 0.6.4, 0.6.3, 0.6.2, 0.6.1, 0.6, dev)
+  Downloading from URL http://pypi.python.org/packages/source/d/distribute/distribute-0.6.10.tar.gz#md5=99fb4b3e4ef0861bba11aa1905e89fed (from http://pypi.python.org/simple/distribute/)
+  Running setup.py egg_info for package distribute
+    /usr/lib/python2.6/site-packages/Pyrex/Compiler/Errors.py:17: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
+      self.message = message
+    running egg_info
+    creating pip-egg-info/distribute.egg-info
+    writing pip-egg-info/distribute.egg-info/PKG-INFO
+    writing top-level names to pip-egg-info/distribute.egg-info/top_level.txt
+    writing dependency_links to pip-egg-info/distribute.egg-info/dependency_links.txt
+    writing entry points to pip-egg-info/distribute.egg-info/entry_points.txt
+    writing pip-egg-info/distribute.egg-info/PKG-INFO
+    writing top-level names to pip-egg-info/distribute.egg-info/top_level.txt
+    writing dependency_links to pip-egg-info/distribute.egg-info/dependency_links.txt
+    writing entry points to pip-egg-info/distribute.egg-info/entry_points.txt
+    writing manifest file 'pip-egg-info/distribute.egg-info/SOURCES.txt'
+    warning: manifest_maker: standard file '-c' not found
+    reading manifest file 'pip-egg-info/distribute.egg-info/SOURCES.txt'
+    reading manifest template 'MANIFEST.in'
+    writing manifest file 'pip-egg-info/distribute.egg-info/SOURCES.txt'
+Installing collected packages: distribute
+  Running setup.py install for distribute
+    Running command /home/tyler/envs/spawning/bin/python -c "import setuptools; __file__='/home/tyler/envs/spawning/build/distribute/setup.py'; execfile('/home/tyler/envs/spawning/build/distribute/setup.py')" install --single-version-externally-managed --record /tmp/pip-NyB_Gl-record/install-record.txt --install-headers /home/tyler/envs/spawning/lib/include
+    /usr/lib/python2.6/site-packages/Pyrex/Compiler/Errors.py:17: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
+      self.message = message
+    Before install bootstrap.
+    Scanning installed packages
+    Setuptools installation detected at /home/tyler/envs/spawning/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg
+    Egg installation
+    Patching...
+    Renaming /home/tyler/envs/spawning/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg into /home/tyler/envs/spawning/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg.OLD.1272815129.25
+    Patched done.
+    Relaunching...
+    Traceback (most recent call last):
+      File "<string>", line 1, in <module>
+    NameError: name 'install' is not defined
+    Complete output from command /home/tyler/envs/spawning/bin/python -c "import setuptools; __file__='/home/tyler/envs/spawning/build/distribute/setup.py'; execfile('/home/tyler/envs/spawning/build/distribute/setup.py')" install --single-version-externally-managed --record /tmp/pip-NyB_Gl-record/install-record.txt --install-headers /home/tyler/envs/spawning/lib/include:
+    /usr/lib/python2.6/site-packages/Pyrex/Compiler/Errors.py:17: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
+
+  self.message = message
+
+Before install bootstrap.
+
+Scanning installed packages
+
+Setuptools installation detected at /home/tyler/envs/spawning/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg
+
+Egg installation
+
+Patching...
+
+Renaming /home/tyler/envs/spawning/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg into /home/tyler/envs/spawning/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg.OLD.1272815129.25
+
+Patched done.
+
+Relaunching...
+
+Traceback (most recent call last):
+
+  File "<string>", line 1, in <module>
+
+NameError: name 'install' is not defined
+
+----------------------------------------
+Command /home/tyler/envs/spawning/bin/python -c "import setuptools; __file__='/home/tyler/envs/spawning/build/distribute/setup.py'; execfile('/home/tyler/envs/spawning/build/distribute/setup.py')" install --single-version-externally-managed --record /tmp/pip-NyB_Gl-record/install-record.txt --install-headers /home/tyler/envs/spawning/lib/include failed with error code 1
+Exception information:
+Traceback (most recent call last):
+  File "/home/tyler/envs/spawning/lib/python2.6/site-packages/pip-0.6.3-py2.6.egg/pip/basecommand.py", line 115, in main
+    self.run(options, args)
+  File "/home/tyler/envs/spawning/lib/python2.6/site-packages/pip-0.6.3-py2.6.egg/pip/commands/install.py", line 157, in run
+    requirement_set.install(install_options)
+  File "/home/tyler/envs/spawning/lib/python2.6/site-packages/pip-0.6.3-py2.6.egg/pip/req.py", line 1165, in install
+    requirement.install(install_options)
+  File "/home/tyler/envs/spawning/lib/python2.6/site-packages/pip-0.6.3-py2.6.egg/pip/req.py", line 533, in install
+    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
+  File "/home/tyler/envs/spawning/lib/python2.6/site-packages/pip-0.6.3-py2.6.egg/pip/__init__.py", line 207, in call_subprocess
+    % (command_desc, proc.returncode))
+InstallationError: Command /home/tyler/envs/spawning/bin/python -c "import setuptools; __file__='/home/tyler/envs/spawning/build/distribute/setup.py'; execfile('/home/tyler/envs/spawning/build/distribute/setup.py')" install --single-version-externally-managed --record /tmp/pip-NyB_Gl-record/install-record.txt --install-headers /home/tyler/envs/spawning/lib/include failed with error code 1

+ 67 - 0
desktop/core/ext-py/Spawning-0.9.6/rc-scripts/conf.d/spawning

@@ -0,0 +1,67 @@
+# Python Spawning configuration
+
+# DO NOT MODIFY THIS FILE DIRECTLY! CREATE A COPY AND MODIFY THAT INSTEAD!
+
+# Spawning factory name.
+# If you're using Django, you should use:
+#FACTORY=spawning.django_factory.config_factory.
+# If you're using Paste, you should use:
+#FACTORY=spawning.paste_factory.config_factory
+# Or, for regular WSGI, the default is:
+#FACTORY=spawning.wsgi_factory.config_factory
+
+# The WSGI application you want to spawn.
+# This means different things with different factories, for example with
+# Django, the WSGI_APP should be the Django settings module to load.
+WSGI_APP=
+
+# Serve on <HOST>:<PORT>
+HOST=0.0.0.0
+PORT=8080
+
+# Change to user (and group if given). `[user[:group]]`
+#CHUID=
+
+# A colon-separated list of the Python path.
+#PYTHON_LIBS=/usr/local/lib/my-python-packages
+
+# stderr is redirected to ERROR_LOG. The status output from the workers, among
+# other things, will end up there.
+ERROR_LOG=/var/log/spawning/error.log
+
+# stdout is redirected to INFO_LOG.
+INFO_LOG=/var/log/spawning/info.log
+
+# The access log is written separately from stdout and stderr.
+ACCESS_LOG=/var/log/spawning/access.log
+
+# Concurrency model.
+
+# Choose either forks (NUM_WORKERS) or threads (NUM_THREADS).
+# NUM_WORKERS defines the number of forks to make a process-based pool out of.
+#NUM_WORKERS=10
+#NUM_THREADS=0
+
+# NUM_THREADS defines the number of threads to make a thread-based pool out of.
+#NUM_THREADS=10
+#NUM_WORKERS=0
+
+# NUM_PROCS defines the number of forks to make, each fork has its own
+# NUM_WORKERS and NUM_THREADS.
+#NUM_PROCS=1
+
+# Maximum age in seconds of a worker before it's restarted.
+#MAX_AGE=
+
+# Maximum memory usage of _all_ workers before it's restarted.
+# Counts resident set size (RSS) in kilobytes, see ps(1).
+#MAX_MEMORY=
+
+# Watch file. When the file changes, Spawning reloads.
+#WATCH=
+
+# Extra options to pass to Spawning.
+#EXTRA_OPTS=
+
+# Path to Spawning binary.
+#SPAWN_BIN=

+ 91 - 0
desktop/core/ext-py/Spawning-0.9.6/rc-scripts/init.d/spawning

@@ -0,0 +1,91 @@
+#!/sbin/runscript
+# Copyright 2009 Ludvig Ericson
+# Distributed under the terms of the 3-clause BSD license
+
+[[ -z "${SPAWN_BIN}" ]] && SPAWN_BIN="$(which spawning)"
+PIDFILE="/var/run/${SVCNAME}.pid"
+
+opts="${opts} reload"
+
+depend() {
+	need net
+}
+
+check_params() {
+	if [[ "${SVCNAME}" == "spawning" && -z "${I_KNOW}" ]]; then
+		ewarn "It is highly recommended to use a symbolic link for this"
+		ewarn "script and start via that instead. This allows you to run"
+		ewarn "multiple spawn services simultaneously. To do this, simply:"
+		ewarn
+		ewarn "  ln -s /etc/init.d/spawning /etc/init.d/spawning.mysvc"
+		ewarn "  cp /etc/conf.d/spawning /etc/conf.d/spawning.mysvc"
+		ewarn
+		ewarn "If you don't want to be bothered by this message, set I_KNOW=yes"
+		ewarn "in your configuration file."
+		ewarn
+	fi
+
+	if [[ -z "${SPAWN_BIN}" ]]; then
+		eerror "Couldn't find spawning binary and no explicit"
+		eerror "path set in configuration file."
+		return 1
+	fi
+
+	for CONF_VAR in WSGI_APP HOST PORT ERROR_LOG; do
+		if [[ -z "$(eval echo \$${CONF_VAR})" ]]; then
+			eerror "Required configuration variable ${CONF_VAR} not"
+			eerror "set in configuration file."
+			return 1
+		fi
+	done
+
+	if [[ ! -z "${WORKERS}" && -z "${NUM_WORKERS}" ]]; then
+		eerror "WORKERS has changed name to NUM_WORKERS, please update"
+		eerror "your configuration file(s) accordingly."
+		return 1
+	fi
+}
+
+start() {
+	ebegin "Starting ${SVCNAME} on ${HOST}:${PORT}"
+	local OPTS
+
+	check_params || return 1
+
+	OPTS="${OPTS} --host ${HOST} --port ${PORT}"
+	[[ ! -z "${NUM_PROCS}" ]] && OPTS="${OPTS} --processes ${NUM_PROCS}"
+	if [[ ! -z "${NUM_THREADS}" ]]; then
+		OPTS="${OPTS} --threads ${NUM_THREADS}"
+	elif [[ ! -z "${NUM_WORKERS}" ]]; then
+		OPTS="${OPTS} --workers ${NUM_WORKERS}"
+	fi
+	OPTS="${OPTS} --access-log ${ACCESS_LOG:-/dev/null}"
+
+	[[ ! -z "${CHUID}" ]]		&& OPTS="${OPTS} --chuid ${CHUID}"
+	[[ ! -z "${FACTORY}" ]]		&& OPTS="${OPTS} --factory ${FACTORY}"
+	[[ ! -z "${WATCH}" ]]		&& OPTS="${OPTS} --watch ${WATCH}"
+	[[ ! -z "${MAX_MEMORY}" ]]	&& OPTS="${OPTS} --max-memory ${MAX_MEMORY}"
+	[[ ! -z "${MAX_AGE}" ]]		&& OPTS="${OPTS} --max-age ${MAX_AGE}"
+	OPTS="${OPTS} ${EXTRA_OPTS}"
+
+	start-stop-daemon --start --pidfile "${PIDFILE}" \
+		--exec "${SPAWN_BIN}" --env PYTHONPATH="${PYTHON_LIBS}" -- \
+			--daemonize --pidfile "${PIDFILE}" \
+			--stderr "${ERROR_LOG}" --stdout "${INFO_LOG:-/dev/null}" \
+			${OPTS} ${WSGI_APP}
+	eend $?
+}
+
+stop() {
+	check_params || return 1
+	ebegin "Stopping ${SVCNAME}"
+	start-stop-daemon --stop --pidfile "${PIDFILE}" --signal INT
+	eend $?
+}
+
+reload() {
+	check_params || return 1
+	ebegin "Telling ${SVCNAME} to reload itself"
+	kill -HUP "$(cat ${PIDFILE})"
+	eend $?
+}

+ 115 - 0
desktop/core/ext-py/Spawning-0.9.6/rc-scripts/init.d/spawning.debian

@@ -0,0 +1,115 @@
+#!/bin/bash
+
+### BEGIN INIT INFO
+# Provides:          spawning.debian
+# Required-Start:    $all
+# Required-Stop:     $all
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: starts a Spawning daemon
+# Description:       starts a Spawning daemon
+### END INIT INFO
+
+# Usage instructions:
+# - Copy this file to your /etc/init.d directory:
+#   $ sudo cp spawning.debian /etc/init.d/spawning.mysite
+# - Change the options below in the "Configuration" section.
+#   Spawning arguments currently supported are:
+#   - host (defaults to 127.0.0.1)
+#   - port (defaults to 8080)
+#   - access-log (defaults to /dev/null)
+#   - stderr (defaults to /dev/null)
+#   - chuid (optional)
+#   - factory (optional)
+#   - processes (optional)
+#   If you have additional arguments to provide (such as 
+#   max-age, watch, workers etc., add them as follows:
+#   EXTRA_ARGS="max-age=1000 --watch=file" etc.
+# - Initialise the script to survive reboots:
+#   $ sudo update-rc.d spawning.mysite defaults
+# - Start your server:
+#   $ sudo /etc/init.d/spawning.mysite start
+
+NAME="spawning.debian" # change debian to your own site
+DESC="Starts a Spawning daemon to run [domain here]"
+# This is where your app or settings.py lives
+SITE_DIR=/var/www/mysite/
+
+# CONFIGURATION - edit this stuff
+# Configure your own spawning here
+SPAWNING_BIN=/usr/local/bin/spawning #Path to Spawning executable
+PROCESSES=2 # Number of processes to spawn
+# Django by default: delete this var to run a WSGI app
+FACTORY="spawning.django_factory.config_factory"
+PORT="8080"
+HOST="127.0.0.1" # listen on loopback by default
+CHUID=""
+ACCESS_LOG=/dev/null
+ERROR_LOG=/dev/null
+APP="settings" # Django by default, but put your own app here
+# END CONFIGURATION
+
+# Don't edit this
+PIDFILE=/var/run/$NAME.pid
+
+# make sure the access_log file exists
+if [ ! -e $ACCESS_LOG ]; then
+    touch $ACCESS_LOG
+fi
+# make sure the error log file exists
+if [ ! -e $ERROR_LOG ]; then
+    touch $ERROR_LOG
+fi
+
+start() {
+    echo -n "Starting $NAME on $HOST:$PORT...: "
+    DAEMON_ARGS="--host=$HOST --port=$PORT"
+    DAEMON_ARGS="$DAEMON_ARGS --stderr=$ERROR_LOG --access-log-file=$ACCESS_LOG"
+    [ -n "$PROCESSES" ]  && DAEMON_ARGS="$DAEMON_ARGS --processes=$PROCESSES"
+    [ -n "$FACTORY" ]    && DAEMON_ARGS="$DAEMON_ARGS --factory=$FACTORY"
+    [ -n "$CHUID" ]      && DAEMON_ARGS="$DAEMON_ARGS --chuid=$CHUID"
+    [ -n "$EXTRA_ARGS" ] && DAEMON_ARGS="$DAEMON_ARGS $EXTRA_ARGS"
+    DAEMON_ARGS="$DAEMON_ARGS $APP"
+    /sbin/start-stop-daemon --start --background --make-pidfile --pidfile=$PIDFILE --chdir $SITE_DIR --exec $SPAWNING_BIN -- $DAEMON_ARGS || return 2
+    echo $NAME
+    return 0
+}
+
+stop () {
+    echo -n "Stopping $NAME: "
+    /sbin/start-stop-daemon --stop --pidfile $PIDFILE
+    rm -f $PIDFILE
+    echo $NAME
+    return
+}
+
+status() {
+    if [ -f "$PIDFILE" ]; then
+        echo -n "$NAME already running with PIDs: " && cat $PIDFILE && echo
+    else
+        echo "$NAME not running"
+    fi
+    return
+}
+
+case "$1" in
+    start)
+        start
+        ;;
+    stop)
+        stop
+        ;;
+    status)
+        status
+        ;;
+    restart)
+        stop
+        sleep 1
+        start
+        ;;
+    *)
+        echo "Usage: $NAME (start|stop|status|restart)"
+        exit 1
+        ;;
+esac
+exit $?

+ 5 - 0
desktop/core/ext-py/Spawning-0.9.6/setup.cfg

@@ -0,0 +1,5 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+tag_svn_revision = 0
+

+ 74 - 0
desktop/core/ext-py/Spawning-0.9.6/setup.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python
+# Copyright (c) 2008, Donovan Preston
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+import sys
+
+# Insert src/ into our path so we can pull the version and include it
+sys.path.insert(0, 'src')
+from spawning import __version__
+
+from os import path
+
+from setuptools import find_packages, setup
+
+install_requires = ['eventlet >= 0.9.12',]
+
+try:
+    import json
+except ImportError:
+    install_requires.append('simplejson')
+
+setup(
+    name='Spawning',
+    description='Spawning is a wsgi server which supports multiple processes, multiple threads, green threads, non-blocking HTTP io, and automatic graceful upgrading of code.',
+    long_description=file(
+        path.join(
+            path.dirname(__file__),
+            'README.rst'
+        )
+    ).read(),
+    author='Donovan Preston',
+    author_email='dsposx@mac.com',
+    maintainer='R. Tyler Croy',
+    maintainer_email='tyler@monkeypox.org',
+    include_package_data = True,
+    packages = find_packages('src'),
+    package_dir = {'': 'src'},
+    version=__version__,
+    install_requires=install_requires,
+    entry_points={
+        'console_scripts': [],
+        'paste.server_factory': [
+            'main=spawning.paste_factory:server_factory'
+        ]
+    },
+    classifiers=[
+        "License :: OSI Approved :: MIT License",
+        "Programming Language :: Python",
+        "Operating System :: MacOS :: MacOS X",
+        "Operating System :: POSIX",
+        "Topic :: Internet",
+        "Topic :: Software Development :: Libraries :: Python Modules",
+        "Intended Audience :: Developers",
+        "Development Status :: 4 - Beta"
+    ]
+)
+

+ 37 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/__init__.py

@@ -0,0 +1,37 @@
+# Copyright (c) 2008, Donovan Preston
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+"""
+"""
+
+__version__ = '0.9.6'
+
+setproctitle = lambda v: NotImplemented
+
+try:
+    from setproctitle import setproctitle
+except ImportError:
+    try:
+        from procname import setprocname as setproctitle
+    except ImportError:
+        pass
+

+ 58 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/django_factory.py

@@ -0,0 +1,58 @@
+# Copyright (c) 2008, Donovan Preston
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+
+import inspect
+import os
+import django.core.handlers.wsgi
+from django.core.servers.basehttp import AdminMediaHandler
+
+import spawning.util
+
+def config_factory(args):
+    args['django_settings_module'] = args.get('args', [None])[0]
+    args['app_factory'] = 'spawning.django_factory.app_factory'
+
+    ## TODO More directories
+    ## INSTALLED_APPS (list of quals)
+    ## ROOT_URL_CONF (qual)
+    ## MIDDLEWARE_CLASSES (list of quals)
+    ## TEMPLATE_CONTEXT_PROCESSORS (list of quals)
+    settings_module = spawning.util.named(args['django_settings_module'])
+
+    dirs = [os.path.split(
+        inspect.getfile(
+            inspect.getmodule(
+                settings_module)))[0]]
+    args['source_directories'] = dirs
+
+    return args
+
+
+def app_factory(config):
+    os.environ['DJANGO_SETTINGS_MODULE'] = config['django_settings_module']
+
+    app = django.core.handlers.wsgi.WSGIHandler()
+    if config.get('dev'):
+        app = AdminMediaHandler(app)
+    return app
+

+ 94 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/memory_watcher.py

@@ -0,0 +1,94 @@
+# Copyright (c) 2008, Donovan Preston
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+import commands
+import os
+import optparse
+import signal
+import sys
+import time
+
+
+MEMORY_WATCH_INTERVAL = 60
+
+
+def watch_memory(controller_pid, max_memory, max_age):
+    if max_age:
+        end_time = time.time() + max_age
+    else:
+        end_time = None
+
+    process_group = os.getpgrp()
+    while True:
+        if max_age:
+            now = time.time()
+            if now + MEMORY_WATCH_INTERVAL > end_time:
+                time.sleep(end_time - now)
+                print "(%s) *** watcher restarting processes! Time limit exceeded." % (
+                    os.getpid(), )
+                os.kill(controller_pid, signal.SIGHUP)
+                end_time = time.time() + max_age
+                continue
+
+        time.sleep(MEMORY_WATCH_INTERVAL)
+        if max_memory:
+            out = commands.getoutput('ps -o rss -g %s' % (process_group, ))
+            used_mem = sum(int(x) for x in out.split('\n')[1:])
+            if used_mem > max_memory:
+                print "(%s) *** memory watcher restarting processes! Memory usage of %s exceeded %s." % (
+                    os.getpid(), used_mem, max_memory)
+                os.kill(controller_pid, signal.SIGHUP)
+
+
+if __name__ == '__main__':
+    parser = optparse.OptionParser(
+        description="Watch all the processes in the process group"
+        " and if the total memory used goes over a configurable amount, send a SIGHUP"
+        " to a given pid.")
+    parser.add_option('-a', '--max-age', dest='max_age', type='int',
+        help='If given, the maximum amount of time (in seconds) to run before sending a  '
+            'SIGHUP to the given pid.')
+
+    options, positional_args = parser.parse_args()
+
+    if len(positional_args) < 2:
+        parser.error("Usage: %s controller_pid max_memory_in_megabytes")
+
+    controller_pid = int(positional_args[0])
+    max_memory = int(positional_args[1])
+    if max_memory:
+        info = 'memory to %s' % (max_memory, )
+    else:
+        info = ''
+
+    if options.max_age:
+        if info:
+            info += ' and'
+        info = " time to %s" % (options.max_age, )
+
+    print "(%s) watcher starting up, limiting%s." % (
+        os.getpid(), info)
+
+    try:
+        watch_memory(controller_pid, max_memory, options.max_age)
+    except KeyboardInterrupt:
+        pass

+ 102 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/paste_factory.py

@@ -0,0 +1,102 @@
+# Copyright (c) 2008, Donovan Preston
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+import os
+import sys
+
+from paste.deploy import loadwsgi
+
+from spawning import spawning_controller
+
+
+def config_factory(args):
+    if 'config_url' in args:
+        config_url = args['config_url']
+        relative_to = args['relative_to']
+        global_conf = args['global_conf']
+    else:
+        config_file = os.path.abspath(args['args'][0])
+        config_url = 'config:%s' % (os.path.basename(config_file), )
+        relative_to = os.path.dirname(config_file)
+        global_conf = {}
+        for arg in args['args'][1:]:
+            key, value = arg.split('=')
+            global_conf[key] = value
+
+    ctx = loadwsgi.loadcontext(
+        loadwsgi.SERVER,
+        config_url,
+        relative_to=relative_to,
+        global_conf=global_conf)
+
+    watch = args.get('watch', None)
+    if watch is None:
+        watch = []
+    if ctx.global_conf['__file__'] not in watch:
+        watch.append(ctx.global_conf['__file__'])
+    args['watch'] = watch
+
+    args['app_factory'] = 'spawning.paste_factory.app_factory'
+    args['config_url'] = config_url
+    args['relative_to'] = relative_to
+    args['source_directories'] = [relative_to]
+    args['global_conf'] = ctx.global_conf
+
+    debug = ctx.global_conf.get('debug', None)
+    if debug is not None:
+        args['dev'] = (debug == 'true')
+    host = ctx.local_conf.get('host', None)
+    if host is not None:
+        args['host'] = host
+    port = ctx.local_conf.get('port', None)
+    if port is not None:
+        args['port'] = int(port)
+    num_processes = ctx.local_conf.get('num_processes', None)
+    if num_processes is not None:
+        args['num_processes'] = int(num_processes)
+    threadpool_workers = ctx.local_conf.get('threadpool_workers', None)
+    if threadpool_workers is not None:
+        args['threadpool_workers'] = int(threadpool_workers)
+
+    return args
+
+
+def app_factory(config):
+    return loadwsgi.loadapp(
+        config['config_url'],
+        relative_to=config['relative_to'],
+        global_conf=config['global_conf'])
+
+
+def server_factory(global_conf, host, port, *args, **kw):
+    config_url = 'config:' + os.path.split(global_conf['__file__'])[1]
+    relative_to = global_conf['here']
+
+    def run(app):
+        args = spawning_controller.DEFAULTS.copy()
+        args.update(
+            {'config_url': config_url, 'relative_to': relative_to, 'global_conf': global_conf})
+
+        spawning_controller.run_controller(
+            'spawning.paste_factory.config_factory', args)
+
+    return run

+ 119 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/reloader_dev.py

@@ -0,0 +1,119 @@
+# Copyright (c) 2008, Donovan Preston
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+"""Watch files and send a SIGHUP signal to another process
+if any of the files change.
+"""
+
+try:
+	set
+except NameError:
+	import sets
+	set = sets.Set
+
+import optparse, os, signal, sys, tempfile, time
+from os.path import join
+from distutils import sysconfig
+
+import eventlet
+
+try:
+    from procname import setprocname
+except ImportError, e:
+    setprocname = lambda n: None
+
+def watch_forever(pid, interval, files=None):
+    """
+    """
+    limiter = eventlet.GreenPool()
+    module_mtimes = {}
+    last_changed_time = None
+    while True:
+        uniques = set()
+
+        uniques.add(join(sysconfig.get_python_lib(), 'easy-install.pth'))
+        uniques.update(list(get_sys_modules_files()))
+
+        if files:
+            uniques.update(files)
+        ##print uniques
+        changed = False
+        for filename in uniques:
+            try:
+                stat = os.stat(filename)
+                if stat:
+                    mtime = stat.st_mtime
+                else:
+                    mtime = 0
+            except (OSError, IOError):
+                continue
+            if filename.endswith('.pyc') and os.path.exists(filename[:-1]):
+                mtime = max(os.stat(filename[:-1]).st_mtime, mtime)
+            if not module_mtimes.has_key(filename):
+                module_mtimes[filename] = mtime
+            elif module_mtimes[filename] < mtime:
+                changed = True
+                last_changed_time = mtime
+                module_mtimes[filename] = mtime
+                print "(%s) * File %r changed" % (os.getpid(), filename)
+
+        if not changed and last_changed_time is not None:
+            last_changed_time = None
+            if pid:
+                print "(%s) ** Sending SIGHUP to %s at %s" % (
+                    os.getpid(), pid, time.asctime())
+                os.kill(pid, signal.SIGHUP)
+                return ## this process is going to die now, no need to keep watching
+            else:
+                print "EXIT??!!!"
+                os._exit(5)
+
+        eventlet.sleep(interval)
+
+
+def get_sys_modules_files():
+    for module in sys.modules.values():
+        fn = getattr(module, '__file__', None)
+        if fn is not None:
+            yield os.path.abspath(fn)
+
+
+def main():
+    parser = optparse.OptionParser()
+    parser.add_option("-p", "--pid",
+        type="int", dest="pid",
+        help="A pid to SIGHUP when a monitored file changes. "
+        "If not given, just print a message to stdout and kill this process instead.")
+    parser.add_option("-i", "--interval",
+        type="int", dest="interval",
+        help="The time to wait between scans, in seconds.", default=1)
+    options, args = parser.parse_args()
+
+    try:
+        watch_forever(options.pid, options.interval)
+    except KeyboardInterrupt:
+        pass
+
+
+if __name__ == '__main__':
+    main()
+

+ 143 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/reloader_svn.py

@@ -0,0 +1,143 @@
+# Copyright (c) 2008, Donovan Preston
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+"""Watch the svn revision returned from svn info and send a SIGHUP
+to a process when the revision changes.
+"""
+
+
+import commands, optparse, os, signal, sys, tempfile, time
+
+try:
+    from procname import setprocname
+except ImportError, e:
+    setprocname = lambda n: None
+
+
+def get_revision(directory):
+    cmd = 'svn info'
+    if directory is not None:
+        cmd = '%s %s' % (cmd, directory)
+
+    try:
+        out = commands.getoutput(cmd).split('\n')
+    except IOError:
+        return
+
+    for line in out:
+        if line.startswith('Revision: '):
+            return int(line[len('Revision: '):])
+
+
+def watch_forever(directories, pid, interval):
+    setprocname("spawn: svn reloader")
+    if directories is None:
+        directories = ['.']
+    ## Look for externals
+    all_svn_repos = set(directories)
+
+    def visit(parent, subdirname, children):
+        if '.svn' in children:
+            children.remove('.svn')
+        status, out = commands.getstatusoutput('svn propget svn:externals %s' % (subdirname, ))
+        if status:
+            return
+
+        for line in out.split('\n'):
+            line = line.strip()
+            if not line:
+                continue
+            name, _external_url = line.split()
+            fulldir = os.path.join(parent, subdirname, name)
+            ## Don't keep going into the external in the walk()
+            try:
+                children.remove(name)
+            except ValueError:
+                print "*** An entry in svn externals doesn't exist, ignoring:", name
+            else:
+                directories.append(fulldir)
+                all_svn_repos.add(fulldir)
+
+    while directories:
+        dirname = directories.pop(0)
+        os.path.walk(dirname, visit, dirname)
+
+    revisions = {}
+    for dirname in all_svn_repos:
+        revisions[dirname] = get_revision(dirname)
+
+    print "(%s) svn watcher watching directories: %s" % (
+        os.getpid(), list(all_svn_repos))
+
+    while True:
+        if pid:
+            ## Check to see if our controller is still alive; if not, just exit.
+            try:
+                os.getpgid(pid)
+            except OSError:
+                print "(%s) reloader_svn is orphaned; controller %s no longer running. Exiting." % (
+                    os.getpid(), pid)
+                os._exit(0)
+
+        for dirname in all_svn_repos:
+            new_revision = get_revision(dirname)
+
+            if new_revision is not None and new_revision != revisions[dirname]:
+                revisions[dirname] = new_revision
+                if pid:
+                    print "(%s) * SVN revision changed on %s to %s; Sending SIGHUP to %s at %s" % (
+                        os.getpid(), dirname, new_revision, pid, time.asctime())
+                    os.kill(pid, signal.SIGHUP)
+                    os._exit(0)
+                else:
+                    print "(%s) Revision changed, dying at %s" % (
+                        os.getpid(), time.asctime())
+                    os._exit(5)
+
+        time.sleep(interval)
+
+
+def main():
+    parser = optparse.OptionParser()
+    parser.add_option("-d", "--dir", dest='dirs', action="append",
+        help="The directories to do svn info in. If not given, use cwd.")
+    parser.add_option("-p", "--pid",
+        type="int", dest="pid",
+        help="A pid to SIGHUP when the svn revision changes. "
+        "If not given, just print a message to stdout and kill this process instead.")
+    parser.add_option("-i", "--interval",
+        type="int", dest="interval",
+        help="The time to wait between scans, in seconds.", default=10)
+    options, args = parser.parse_args()
+
+    print "(%s) svn watcher running, controller pid %s" % (os.getpid(), options.pid)
+    if options.pid is None:
+        options.pid = os.getpid()
+    try:
+        watch_forever(options.dirs, int(options.pid), options.interval)
+    except KeyboardInterrupt:
+        pass
+
+
+if __name__ == '__main__':
+    main()
+

+ 341 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/spawning_child.py

@@ -0,0 +1,341 @@
+#!/usr/bin/env python
+# Copyright (c) 2008, Donovan Preston
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+"""spawning_child.py
+"""
+
+import eventlet
+import eventlet.event
+import eventlet.greenio
+import eventlet.greenthread
+import eventlet.hubs
+import eventlet.wsgi
+
+import errno
+import optparse
+import os
+import signal
+import socket
+import sys
+import time
+
+import spawning.util
+from spawning import setproctitle, reloader_dev
+
+try:
+    import simplejson as json
+except ImportError:
+    import json
+
+
+class URLInterceptor(object):
+    """
+    Intercepts one or more paths.
+    """
+
+    paths = []
+
+    def __init__(self, app, paths=[]):
+        """
+        Creates an instance.
+
+        :Parameters:
+           - `app`: Application to fall through to
+        """
+        self.app = app
+
+    def _intercept(self, env, start_response):
+        """
+        Executes business logic.
+
+        :Parameters:
+           - `env`: environment information
+           - `start_response`: wsgi response function
+        """
+        raise NotImplementedError('_intercept must be overridden')
+
+    def __call__(self, env, start_response):
+        """
+        Dispatches input to the proper method.
+
+        :Parameters:
+           - `env`: environment information
+           - `start_response`: wsgi response function
+        """
+        if env['PATH_INFO'] in self.paths:
+            return self._intercept(env, start_response)
+        return self.app(env, start_response)
+
+
+class FigleafCoverage(URLInterceptor):
+
+    paths = ['/_coverage']
+
+    def __init__(self, app):
+        URLInterceptor.__init__(self, app)
+        import figleaf
+        figleaf.start()
+
+    def _intercept(self, env, start_response):
+        import figleaf
+        try:
+            import cPickle as pickle
+        except ImportError:
+            import pickle
+
+        coverage = figleaf.get_info()
+        s = pickle.dumps(coverage)
+        start_response("200 OK", [('Content-type', 'application/x-pickle')])
+        return [s]
+
+
+class SystemInfo(URLInterceptor):
+    """
+    Intercepts /_sysinfo path and returns json data.
+    """
+
+    paths = ['/_sysinfo']
+
+    def _intercept(self, env, start_response):
+        """
+        Executes business logic.
+
+        :Parameters:
+           - `env`: environment information
+           - `start_response`: wsgi response function
+        """
+        import spawning.util.system
+        start_response("200 OK", [('Content-type', 'application/json')])
+        return [json.dumps(spawning.util.system.System())]
+
+
+class ExitChild(Exception):
+    pass
+
+class ChildStatus(object):
+    def __init__(self, controller_port):
+        self.controller_url =  "http://127.0.0.1:%s/" % controller_port
+        self.server = None
+        
+    def send_status_to_controller(self):
+        try:
+            child_status = {'pid':os.getpid()}
+            if self.server: 
+                child_status['concurrent_requests'] = \
+                    self.server.outstanding_requests
+            else:
+                child_status['error'] = 'Starting...'
+            body = json.dumps(child_status)
+            import urllib2
+            urllib2.urlopen(self.controller_url, body)
+        except (KeyboardInterrupt, SystemExit,
+             eventlet.greenthread.greenlet.GreenletExit):
+            raise
+        except Exception, e:  
+            # we really don't want exceptions here to stop read_pipe_and_die
+            pass
+
+_g_status = None
+def init_statusobj(status_port):
+    global _g_status
+    if status_port:
+        _g_status = ChildStatus(status_port)
+def get_statusobj():
+    return _g_status
+
+
+def read_pipe_and_die(the_pipe, server_coro):
+    dying = False
+    try:
+        while True:
+            eventlet.hubs.trampoline(the_pipe, read=True)
+            c = os.read(the_pipe, 1)
+            # this is how the controller tells the child to send a status update
+            if c == 's' and get_statusobj():
+                get_statusobj().send_status_to_controller()
+            elif not dying:
+                dying = True  # only send ExitChild once
+                eventlet.greenthread.kill(server_coro, ExitChild)
+                # continue to listen for status pings while dying
+    except socket.error:
+        pass
+    # if here, perhaps the controller's process went down; we should die too if
+    # we aren't already
+    if not dying:
+        eventlet.greenthread.kill(server_coro, KeyboardInterrupt)
+
+def tpool_wsgi(app):
+    from eventlet import tpool
+    def tpooled_application(e, s):
+        result = tpool.execute(app, e, s)
+        # return builtins directly
+        if isinstance(result, (basestring, list, tuple)):
+            return result
+        else:
+            # iterators might execute code when iterating over them,
+            # so we wrap them in a Proxy object so every call to
+            # next() goes through tpool
+            return tpool.Proxy(result)
+    return tpooled_application
+
+
+def warn_controller_of_imminent_death(controller_pid):
+    # The controller responds to a SIGUSR1 by kicking off a new child process.
+    try:
+        os.kill(controller_pid, signal.SIGUSR1)
+    except OSError, e:
+        if not e.errno == errno.ESRCH:
+            raise
+
+
+def serve_from_child(sock, config, controller_pid):
+    threads = config.get('threadpool_workers', 0)
+    wsgi_application = spawning.util.named(config['app_factory'])(config)
+
+    if config.get('coverage'):
+        wsgi_application = FigleafCoverage(wsgi_application)
+    if config.get('sysinfo'):
+        wsgi_application = SystemInfo(wsgi_application)
+
+    if threads >= 1:
+        # proxy calls of the application through tpool
+        wsgi_application = tpool_wsgi(wsgi_application)
+    elif threads != 1:
+        from eventlet.green import socket
+
+    host, port = sock.getsockname()
+
+    access_log_file = config.get('access_log_file')
+    if access_log_file is not None:
+        access_log_file = open(access_log_file, 'a')
+
+    max_age = 0
+    if config.get('max_age'):
+        max_age = int(config.get('max_age'))
+
+    server_event = eventlet.event.Event()
+    # the status object wants to have a reference to the server object
+    if config.get('status_port'):
+        def send_server_to_status(server_event):
+            server = server_event.wait()
+            get_statusobj().server = server
+        eventlet.spawn(send_server_to_status, server_event)
+
+    http_version = config.get('no_keepalive') and 'HTTP/1.0' or 'HTTP/1.1'
+    try:
+        wsgi_args = (sock, wsgi_application)
+        wsgi_kwargs = {'log' : access_log_file, 'server_event' : server_event, 'max_http_version' : http_version}
+        if config.get('no_keepalive'):
+            wsgi_kwargs.update({'keepalive' : False})
+        if max_age:
+            wsgi_kwargs.update({'timeout_value' : True})
+            eventlet.with_timeout(max_age, eventlet.wsgi.server, *wsgi_args,
+                    **wsgi_kwargs)
+            warn_controller_of_imminent_death(controller_pid)
+        else:
+            eventlet.wsgi.server(*wsgi_args, **wsgi_kwargs)
+    except KeyboardInterrupt:
+        # controller probably doesn't know that we got killed by a SIGINT
+        warn_controller_of_imminent_death(controller_pid)
+    except ExitChild:
+        pass  # parent killed us, it already knows we're dying
+
+    ## Once we get here, we should not accept any new sockets, so we should close the server socket.
+    sock.close()
+    
+    server = server_event.wait()
+
+    print "(%s) *** Child exiting" % (os.getpid(),)
+    os.kill(os.getpid(), signal.SIGKILL)
+
+def child_sighup(*args, **kwargs):
+    exit(0)
+
+
+def main():
+    parser = optparse.OptionParser()
+    parser.add_option("-r", "--reload",
+        action='store_true', dest='reload',
+        help='If --reload is passed, reload the server any time '
+        'a loaded module changes.')
+    parser.add_option('--ssl-certificate', dest='ssl_certificate', type='string', default='',
+        help='Absolute path to SSL certificate file.')
+    parser.add_option('--ssl-private-key', dest='ssl_private_key', type='string', default='',
+        help='Absolute path to SSL private key.')
+
+    options, args = parser.parse_args()
+
+    if len(args) != 5:
+        print "Usage: %s controller_pid httpd_fd death_fd factory_qual factory_args" % (
+            sys.argv[0], )
+        sys.exit(1)
+
+    controller_pid, httpd_fd, death_fd, factory_qual, factory_args = args
+    controller_pid = int(controller_pid)
+    config = spawning.util.named(factory_qual)(json.loads(factory_args))
+
+    setproctitle("spawn: child (%s)" % ", ".join(config.get("args")))
+    
+    ## Set up status reporter, if requested
+    init_statusobj(config.get('status_port'))
+
+    ## Set up the reloader
+    if config.get('reload'):
+        watch = config.get('watch', None)
+        if watch:
+            watching = ' and %s' % watch
+        else:
+            watching = ''
+        print "(%s) reloader watching sys.modules%s" % (os.getpid(), watching)
+        eventlet.spawn(
+            reloader_dev.watch_forever, controller_pid, 1, watch)
+
+    ## The parent will catch sigint and tell us to shut down
+    signal.signal(signal.SIGINT, signal.SIG_IGN)
+    ## Expect a SIGHUP when we want the child to die
+    signal.signal(signal.SIGHUP, child_sighup)
+    eventlet.spawn(read_pipe_and_die, int(death_fd), eventlet.getcurrent())
+
+    ## Make the socket object from the fd given to us by the controller
+    sock = eventlet.greenio.GreenSocket(
+        socket.fromfd(int(httpd_fd), socket.AF_INET, socket.SOCK_STREAM))
+
+    if options.ssl_certificate and options.ssl_private_key:
+        # The way spawning works is that there's a parent process which forks off long-lived
+        # children, each of which can be multithreaded. What we're using is a single-threaded,
+        # single-process server that does things asynchronously (using coroutines). Spawning creates
+        # a socket and then calls fork() at least once. In the child process, it exec()'s something
+        # else, so as a result the child loses all context. It puts the file descriptor for the
+        # socket as a command-line argument to the child, which then uses the fromfd function of the
+        # socket module to create a socket object. Unfortunately, the resulting object isn't quite
+        # the same as the socket created by the parent. In particular, when we go to upgrade this
+        # socket to ssl using eventlet's wrap_with_ssl(), it fails because it expects sock.fd to be
+        # of type "socket._socketobject", but it's actually of type "_socket.socket". Patching
+        # up the object in this way solves this problem.
+        sock.fd = socket._socketobject(_sock=sock.fd)
+        sock = eventlet.wrap_ssl(sock, certfile=options.ssl_certificate, keyfile=options.ssl_private_key, server_side=True)
+
+    serve_from_child(sock, config, controller_pid)
+
+if __name__ == '__main__':
+    main()

+ 521 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/spawning_controller.py

@@ -0,0 +1,521 @@
+#!/usr/bin/env python
+# Copyright (c) 2008, Donovan Preston
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+import eventlet
+
+import commands
+import datetime
+import errno
+import logging
+import optparse
+import pprint
+import signal
+from eventlet.green import socket
+import sys
+from eventlet.green import time
+import traceback
+
+try:
+    import simplejson as json
+except ImportError:
+    import json
+
+
+import eventlet.backdoor
+from eventlet.green import os
+
+import spawning
+import spawning.util
+
+KEEP_GOING = True
+RESTART_CONTROLLER = False
+PANIC = False
+
+
+DEFAULTS = {
+    'num_processes': 4,
+    'threadpool_workers': 4,
+    'watch': [],
+    'dev': True,
+    'host': '',
+    'port': 8080,
+    'max_memory': None,
+}
+
+def print_exc(msg="Exception occured!"):
+    print >>sys.stderr, "(%d) %s" % (os.getpid(), msg)
+    traceback.print_exc()
+
+def environ():
+    env = os.environ.copy()
+    # to avoid duplicates in the new sys.path
+    revised_paths = set()
+    new_path = list()
+    for path in sys.path:
+        if os.path.exists(path) and path not in revised_paths:
+            revised_paths.add(path)
+            new_path.append(path)
+    current_directory = os.path.realpath('.')
+    if current_directory not in revised_paths:
+        new_path.append(current_directory)
+
+    env['PYTHONPATH'] = ':'.join(new_path)
+    return env
+
+class Child(object):
+    def __init__(self, pid, kill_pipe):
+        self.pid = pid
+        self.kill_pipe = kill_pipe
+        self.active = True
+        self.forked_at = datetime.datetime.now()
+
+class Controller(object):
+    sock = None
+    factory = None
+    args = None
+    config = None
+    children = None
+    keep_going = True
+    panic = False
+    log = None
+    controller_pid = None
+    num_processes = 0
+
+    def __init__(self, sock, factory, args, **kwargs):
+        self.sock = sock
+        self.factory = factory
+        self.config = spawning.util.named(factory)(args)
+        self.args = args
+        self.children = {}
+        self.log = logging.getLogger('Spawning')
+        if not kwargs.get('log_handler'):
+            self.log.addHandler(logging.StreamHandler())
+        self.log.setLevel(logging.DEBUG)
+        self.controller_pid = os.getpid()
+        self.num_processes = int(self.config.get('num_processes', 0))
+        self.started_at = datetime.datetime.now()
+
+    def spawn_children(self, number=1):
+        parent_pid = os.getpid()
+        self.log.debug('Controller.spawn_children(number=%d)' % number)
+
+        for i in range(number):
+            child_side, parent_side = os.pipe()
+            try:
+                child_pid = os.fork()
+            except:
+                print_exc('Could not fork child! Panic!')
+                ### TODO: restart
+
+            if not child_pid:      # child process
+                os.close(parent_side)
+                command = [sys.executable, '-c',
+                    'import sys; from spawning import spawning_child; spawning_child.main()',
+                    str(parent_pid),
+                    str(self.sock.fileno()),
+                    str(child_side),
+                    self.factory,
+                    json.dumps(self.args)]
+                if self.args['reload'] == 'dev':
+                    command.append('--reload')
+		if self.args.get('ssl_private_key') and self.args.get('ssl_certificate'):
+                    command.append('--ssl-private-key')
+                    command.append(self.args.get('ssl_private_key'))
+                    command.append('--ssl-certificate')
+                    command.append(self.args.get('ssl_certificate'))
+                env = environ()
+                tpool_size = int(self.config.get('threadpool_workers', 0))
+                assert tpool_size >= 0, (tpool_size, 'Cannot have a negative --threads argument')
+                if not tpool_size in (0, 1):
+                    env['EVENTLET_THREADPOOL_SIZE'] = str(tpool_size)
+                os.execve(sys.executable, command, env)
+
+            # controller process
+            os.close(child_side)
+            self.children[child_pid] = Child(child_pid, parent_side)
+
+    def children_count(self):
+        return len(self.children)
+
+    def runloop(self):
+        while self.keep_going:
+            eventlet.sleep(0.1)
+            ## Only start the number of children we need
+            number = self.num_processes - self.children_count()
+            if number > 0:
+                self.log.debug('Should start %d new children', number)
+                self.spawn_children(number=number)
+                continue
+
+            if not self.children:
+                ## If we don't yet have children, let's loop
+                continue
+
+            pid, result = None, None
+            try:
+                pid, result = os.wait()
+            except OSError, e:
+                if e.errno != errno.EINTR:
+                    raise
+
+            if pid and self.children.get(pid):
+                try:
+                    child = self.children.pop(pid)
+                    os.close(child.kill_pipe)
+                except (IOError, OSError):
+                    pass
+
+            if result:
+                signum = os.WTERMSIG(result)
+                exitcode = os.WEXITSTATUS(result)
+                self.log.info('(%s) Child died from signal %s with code %s',
+                              pid, signum, exitcode)
+
+    def handle_sighup(self, *args, **kwargs):
+        ''' Pass `no_restart` to prevent restarting the run loop '''
+        self.kill_children()
+        self.spawn_children(number=self.num_processes)
+        # TODO: nothing seems to use no_restart, can it be removed?
+        if not kwargs.get('no_restart', True):
+            self.runloop()
+
+    def kill_children(self):
+        for pid, child in self.children.items():
+            try:
+                os.write(child.kill_pipe, 'k')
+                child.active = False
+                # all maintenance of children's membership happens in runloop()
+                # as children die and os.wait() gets results
+            except OSError, e:
+                if e.errno != errno.EPIPE:
+                    raise
+
+    def handle_deadlychild(self, *args, **kwargs):
+        if self.keep_going:
+            self.log.debug('A child intends to die, starting replacement before it dies')
+            self.spawn_children(number=1)
+
+    def run(self):
+        self.log.info('(%s) *** Controller starting at %s' % (self.controller_pid,
+                time.asctime()))
+
+        if self.config.get('pidfile'):
+            fd = open(self.config.get('pidfile'), 'w')
+            try:
+                fd.write('%s\n' % self.controller_pid)
+            finally:
+                fd.close()
+
+        spawning.setproctitle("spawn: controller " + self.args.get('argv_str', ''))
+
+        if self.sock is None:
+            self.sock = bind_socket(self.config)
+
+        signal.signal(signal.SIGHUP, self.handle_sighup)
+        signal.signal(signal.SIGUSR1, self.handle_deadlychild)
+
+        if self.config.get('status_port'):
+            from spawning.util import status
+            eventlet.spawn(status.Server, self,
+                self.config['status_host'], self.config['status_port'])
+
+        try:
+            self.runloop()
+        except KeyboardInterrupt:
+            self.keep_going = False
+            self.kill_children()
+        self.log.info('(%s) *** Controller exiting' % (self.controller_pid))
+
+def bind_socket(config):
+    sleeptime = 0.5
+    host = config.get('host', '')
+    port = config.get('port', 8080)
+    for x in range(8):
+        try:
+            sock = eventlet.listen((host, port))
+            break
+        except socket.error, e:
+            if e[0] != errno.EADDRINUSE:
+                raise
+            print "(%s) socket %s:%s already in use, retrying after %s seconds..." % (
+                os.getpid(), host, port, sleeptime)
+            eventlet.sleep(sleeptime)
+            sleeptime *= 2
+    else:
+        print "(%s) could not bind socket %s:%s, dying." % (
+            os.getpid(), host, port)
+        sys.exit(1)
+    return sock
+
+def set_process_owner(spec):
+    import pwd, grp
+    if ":" in spec:
+        user, group = spec.split(":", 1)
+    else:
+        user, group = spec, None
+    if group:
+        os.setgid(grp.getgrnam(group).gr_gid)
+    if user:
+        os.setuid(pwd.getpwnam(user).pw_uid)
+    return user, group
+
+def start_controller(sock, factory, factory_args):
+    c = Controller(sock, factory, factory_args)
+    installGlobal(c)
+    c.run()
+
+def main():
+    current_directory = os.path.realpath('.')
+    if current_directory not in sys.path:
+        sys.path.append(current_directory)
+
+    parser = optparse.OptionParser(description="Spawning is an easy-to-use and flexible wsgi server. It supports graceful restarting so that your site finishes serving any old requests while starting new processes to handle new requests with the new code. For the simplest usage, simply pass the dotted path to your wsgi application: 'spawn my_module.my_wsgi_app'", version=spawning.__version__)
+    parser.add_option('-v', '--verbose', dest='verbose', action='store_true', help='Display verbose configuration '
+        'information when starting up or restarting.')
+    parser.add_option("-f", "--factory", dest='factory', default='spawning.wsgi_factory.config_factory',
+        help="""Dotted path (eg mypackage.mymodule.myfunc) to a callable which takes a dictionary containing the command line arguments and figures out what needs to be done to start the wsgi application. Current valid values are: spawning.wsgi_factory.config_factory, spawning.paste_factory.config_factory, and spawning.django_factory.config_factory. The factory used determines what the required positional command line arguments will be. See the spawning.wsgi_factory module for documentation on how to write a new factory.
+        """)
+    parser.add_option("-i", "--host",
+        dest='host', default=DEFAULTS['host'],
+        help='The local ip address to bind.')
+    parser.add_option("-p", "--port",
+        dest='port', type='int', default=DEFAULTS['port'],
+        help='The local port address to bind.')
+    parser.add_option("-s", "--processes",
+        dest='processes', type='int', default=DEFAULTS['num_processes'],
+        help='The number of unix processes to start to use for handling web i/o.')
+    parser.add_option("-t", "--threads",
+        dest='threads', type='int', default=DEFAULTS['threadpool_workers'],
+        help="The number of posix threads to use for handling web requests. "
+            "If threads is 0, do not use threads but instead use eventlet's cooperative "
+            "greenlet-based microthreads, monkeypatching the socket and pipe operations which normally block "
+            "to cooperate instead. Note that most blocking database api modules will not "
+            "automatically cooperate.")
+    parser.add_option('-d', '--daemonize', dest='daemonize', action='store_true',
+        help="Daemonize after starting children.")
+    parser.add_option('-u', '--chuid', dest='chuid', metavar="ID",
+        help="Change user ID in daemon mode (and group ID if given, "
+             "separate with colon.)")
+    parser.add_option('--pidfile', dest='pidfile', metavar="FILE",
+        help="Write own process ID to FILE in daemon mode.")
+    parser.add_option('--stdout', dest='stdout', metavar="FILE",
+        help="Redirect stdout to FILE in daemon mode.")
+    parser.add_option('--stderr', dest='stderr', metavar="FILE",
+        help="Redirect stderr to FILE in daemon mode.")
+    parser.add_option('-w', '--watch', dest='watch', action='append',
+        help="Watch the given file's modification time. If the file changes, the web server will "
+            'restart gracefully, allowing old requests to complete in the old processes '
+            'while starting new processes with the latest code or configuration.')
+    ## TODO Hook up the svn reloader again
+    parser.add_option("-r", "--reload",
+        type='str', dest='reload',
+        help='If --reload=dev is passed, reload any time '
+        'a loaded module or configuration file changes.')
+    parser.add_option('-l', '--access-log-file', dest='access_log_file', default=None,
+        help='The file to log access log lines to. If not given, log to stdout. Pass /dev/null to discard logs.')
+    parser.add_option('-c', '--coverage', dest='coverage', action='store_true',
+        help='If given, gather coverage data from the running program and make the '
+            'coverage report available from the /_coverage url. See the figleaf docs '
+            'for more info: http://darcs.idyll.org/~t/projects/figleaf/doc/')
+    parser.add_option('--sysinfo', dest='sysinfo', action='store_true',
+        help='If given, gather system information data and make the '
+            'report available from the /_sysinfo url.')
+    parser.add_option('-m', '--max-memory', dest='max_memory', type='int', default=0,
+        help='If given, the maximum amount of memory this instance of Spawning '
+            'is allowed to use. If all of the processes started by this Spawning controller '
+            'use more than this amount of memory, send a SIGHUP to the controller '
+            'to get the children to restart.')
+    parser.add_option('--backdoor', dest='backdoor', action='store_true',
+            help='Start a backdoor bound to localhost:3000')
+    parser.add_option('-a', '--max-age', dest='max_age', type='int',
+        help='If given, the maximum amount of time (in seconds) an instance of spawning_child '
+            'is allowed to run. Once this time limit has expired the child will'
+            'gracefully kill itself while the server starts a replacement.')
+    parser.add_option('--no-keepalive', dest='no_keepalive', action='store_true',
+            help='Disable HTTP/1.1 KeepAlive')
+    parser.add_option('-z', '--z-restart-args', dest='restart_args',
+        help='For internal use only')
+    parser.add_option('--status-port', dest='status_port', type='int', default=0,
+        help='If given, hosts a server status page at that port.  Two pages are served: a human-readable HTML version at http://host:status_port/status, and a machine-readable version at http://host:status_port/status.json')
+    parser.add_option('--status-host', dest='status_host', type='string', default='',
+        help='If given, binds the server status page to the specified local ip address.  Defaults to the same value as --host.  If --status-port is not supplied, the status page will not be activated.')
+    parser.add_option('--ssl-certificate', dest='ssl_certificate', type='string', default='',
+        help='Absolute path to SSL certificate file.')
+    parser.add_option('--ssl-private-key', dest='ssl_private_key', type='string', default='',
+        help='Absolute path to SSL private key.')
+
+    options, positional_args = parser.parse_args()
+
+    if len(positional_args) < 1 and not options.restart_args:
+        parser.error("At least one argument is required. "
+            "For the default factory, it is the dotted path to the wsgi application "
+            "(eg my_package.my_module.my_wsgi_application). For the paste factory, it "
+            "is the ini file to load. Pass --help for detailed information about available options.")
+
+    if options.backdoor:
+        try:
+            eventlet.spawn(eventlet.backdoor.backdoor_server, eventlet.listen(('localhost', 3000)))
+        except Exception, ex:
+            sys.stderr.write('**> Error opening backdoor: %s\n' % ex)
+
+    sock = None
+
+    if options.restart_args:
+        restart_args = json.loads(options.restart_args)
+        factory = restart_args['factory']
+        factory_args = restart_args['factory_args']
+
+        start_delay = restart_args.get('start_delay')
+        if start_delay is not None:
+            factory_args['start_delay'] = start_delay
+            print "(%s) delaying startup by %s" % (os.getpid(), start_delay)
+            time.sleep(start_delay)
+
+        fd = restart_args.get('fd')
+        if fd is not None:
+            sock = socket.fromfd(restart_args['fd'], socket.AF_INET, socket.SOCK_STREAM)
+            ## socket.fromfd doesn't result in a socket object that has the same fd.
+            ## The old fd is still open however, so we close it so we don't leak.
+            os.close(restart_args['fd'])
+        return start_controller(sock, factory, factory_args)
+
+    ## We're starting up for the first time.
+    if options.daemonize:
+        # Do the daemon dance. Note that this isn't what is considered good
+        # daemonization, because frankly it's convenient to keep the file
+        # descriptiors open (especially when there are prints scattered all
+        # over the codebase.)
+        # What we do instead is fork off, create a new session, fork again.
+        # This leaves the process group in a state without a session
+        # leader.
+        pid = os.fork()
+        if not pid:
+            os.setsid()
+            pid = os.fork()
+            if pid:
+                os._exit(0)
+        else:
+            os._exit(0)
+        print "(%s) now daemonized" % (os.getpid(),)
+        # Close _all_ open (and othewise!) files.
+        import resource
+        maxfd = resource.getrlimit(resource.RLIMIT_NOFILE)[1]
+        if maxfd == resource.RLIM_INFINITY:
+            maxfd = 4096
+        for fdnum in xrange(maxfd):
+            try:
+                os.close(fdnum)
+            except OSError, e:
+                if e.errno != errno.EBADF:
+                    raise
+        # Remap std{in,out,err}
+        devnull = os.open(os.path.devnull, os.O_RDWR)
+        oflags = os.O_WRONLY | os.O_CREAT | os.O_APPEND
+        if devnull != 0:  # stdin
+            os.dup2(devnull, 0)
+        if options.stdout:
+            stdout_fd = os.open(options.stdout, oflags)
+            if stdout_fd != 1:
+                os.dup2(stdout_fd, 1)
+                os.close(stdout_fd)
+        else:
+            os.dup2(devnull, 1)
+        if options.stderr:
+            stderr_fd = os.open(options.stderr, oflags)
+            if stderr_fd != 2:
+                os.dup2(stderr_fd, 2)
+                os.close(stderr_fd)
+        else:
+            os.dup2(devnull, 2)
+        # Change user & group ID.
+        if options.chuid:
+            user, group = set_process_owner(options.chuid)
+            print "(%s) set user=%s group=%s" % (os.getpid(), user, group)
+    else:
+        # Become a process group leader only if not daemonizing.
+        os.setpgrp()
+
+    ## Fork off the thing that watches memory for this process group.
+    controller_pid = os.getpid()
+    if options.max_memory and not os.fork():
+        env = environ()
+        from spawning import memory_watcher
+        basedir, cmdname = os.path.split(memory_watcher.__file__)
+        if cmdname.endswith('.pyc'):
+            cmdname = cmdname[:-1]
+
+        os.chdir(basedir)
+        command = [
+            sys.executable,
+            cmdname,
+            '--max-age', str(options.max_age),
+            str(controller_pid),
+            str(options.max_memory)]
+        os.execve(sys.executable, command, env)
+
+    factory = options.factory
+
+    # If you tell me to watch something, I'm going to reload then
+    if options.watch:
+        options.reload = True
+
+    if options.status_port == options.port:
+        options.status_port = None
+        sys.stderr.write('**> Status port cannot be the same as the service port, disabling status.\n')
+
+
+    factory_args = {
+        'verbose': options.verbose,
+        'host': options.host,
+        'port': options.port,
+        'num_processes': options.processes,
+        'threadpool_workers': options.threads,
+        'watch': options.watch,
+        'reload': options.reload,
+        'access_log_file': options.access_log_file,
+        'pidfile': options.pidfile,
+        'coverage': options.coverage,
+        'sysinfo': options.sysinfo,
+        'no_keepalive' : options.no_keepalive,
+        'max_age' : options.max_age,
+        'argv_str': " ".join(sys.argv[1:]),
+        'args': positional_args,
+        'status_port': options.status_port,
+        'status_host': options.status_host or options.host,
+	'ssl_private_key': options.ssl_private_key,
+	'ssl_certificate': options.ssl_certificate
+    }
+    start_controller(sock, factory, factory_args)
+
+_global_attr_name_ = '_spawning_controller_'
+def installGlobal(controller):
+    setattr(sys, _global_attr_name_, controller)
+
+def globalController():
+    return getattr(sys, _global_attr_name_, None)
+
+
+if __name__ == '__main__':
+    main()
+
+
+

+ 55 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/util/__init__.py

@@ -0,0 +1,55 @@
+# Copyright (c) 2010, R. Tyler Ballance
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+def named(name):
+    """Return an object given its name.
+
+    The name uses a module-like syntax, eg::
+
+      os.path.join
+
+    or::
+
+      mulib.mu.Resource
+    """
+    toimport = name
+    obj = None
+    import_err_strings = []
+    while toimport:
+        try:
+            obj = __import__(toimport)
+            break
+        except ImportError, err:
+            # print 'Import error on %s: %s' % (toimport, err)  # debugging spam
+            import_err_strings.append(err.__str__())
+            toimport = '.'.join(toimport.split('.')[:-1])
+    if obj is None:
+        raise ImportError('%s could not be imported.  Import errors: %r' % (name, import_err_strings))
+    for seg in name.split('.')[1:]:
+        try:
+            obj = getattr(obj, seg)
+        except AttributeError:
+            dirobj = dir(obj)
+            dirobj.sort()
+            raise AttributeError('attribute %r missing from %r (%r) %r.  Import errors: %r' % (
+                seg, obj, dirobj, name, import_err_strings))
+    return obj

+ 171 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/util/log_parser.py

@@ -0,0 +1,171 @@
+import time
+from datetime import datetime, timedelta
+import sys
+import optparse
+import re
+
+__all__ = ['parse_line', 'parse_lines', 'parse_casual_time',
+    'group_parsed_lines', 'select_timerange']
+
+month_names = {'Jan': 1, 'Feb': 2, 'Mar':3, 'Apr':4, 'May':5, 'Jun':6, 'Jul':7, 
+    'Aug':8,  'Sep': 9, 'Oct':10, 'Nov': 11, 'Dec': 12}
+
+
+def parse_line(line):
+    """ Parses a Spawning log line into a dictionary of fields.
+    
+    Returns the following fields: 
+    * client_ip : The remote IP address.
+    * date : datetime object representing when the request completed
+    * method : HTTP method
+    * path : url path
+    * version : HTTP version
+    * status_code : HTTP status code
+    * size : length of the body
+    * duration : time in seconds to complete the request
+    """
+    # note that a split-based version of the function is faster than
+    # a regexp-based version
+    segs = line.split()
+    if len(segs) != 11:
+        return None
+    retval = {}
+    try:
+        retval['client_ip'] = segs[0]
+        if segs[1] != '-' or segs[2] != '-':
+            return None
+        if segs[3][0] != '[' or segs[4][-1] != ']':
+            return None
+        # time parsing by explicitly poking at string slices is much faster 
+        # than strptime, but it won't work in non-English locales because of 
+        # the month names
+        d = segs[3]
+        t = segs[4]
+        retval['date'] = datetime(
+            int(d[8:12]),         # year
+            month_names[d[4:7]],  # month
+            int(d[1:3]),          # day
+            int(t[0:2]),          # hour
+            int(t[3:5]),          # minute
+            int(t[6:8]))          # second
+        if segs[5][0] != '"' or segs[7][-1] != '"':
+            return None
+        retval['method'] = segs[5][1:]
+        retval['path'] = segs[6]
+        retval['version'] = segs[7][:-1]
+        retval['status_code'] = int(segs[8])
+        retval['size'] = int(segs[9])
+        retval['duration'] = float(segs[10])
+    except (IndexError, ValueError):
+        return None
+    return retval
+    
+    
+def parse_lines(fd):
+    """Generator function that accepts an iterable file-like object and 
+    yields all the parseable lines found in it.
+    """
+    for line in fd:
+        parsed = parse_line(line)
+        if parsed is not None:
+            yield parsed
+
+
+time_intervals = {"sec":1, "min":60, "hr":3600, "day": 86400,
+                  "second":1, "minute":60, "hour":3600,
+                  "s":1, "m":60, "h":3600, "d":86400}
+for k,v in time_intervals.items():  # pluralize
+    time_intervals[k + "s"] = v
+    
+    
+def parse_casual_time(timestr, relative_to):
+    """Lenient relative time parser.  Returns a datetime object if it can.
+    
+    Accepts such human-friendly times as "-1 hour", "-30s", "15min", "2d", "now".
+    Any such relative time is interpreted as a delta applied to the relative_to
+    argument, which should be a datetime.
+    """
+    timestr = timestr.lower()
+    try:
+        return datetime(*(time.strptime(timestr)[0:6]))
+    except ValueError:
+        pass
+    if timestr == "now":
+        return datetime.now()
+    # match stuff like "-1 hour", "-30s"
+    m = re.match(r'([-0-9.]+)\s*(\w+)?', timestr)
+    if m:
+        intervalsz = 1
+        if len(m.groups()) > 1 and m.group(2) in time_intervals:
+            intervalsz = time_intervals[m.group(2)]
+        relseconds = float(m.group(1)) * intervalsz
+        return relative_to + timedelta(seconds=relseconds)
+
+def group_parsed_lines(lines, field):
+    """Aggregates the parsed log lines by a field.  Counts
+    the log lines in each group and their average duration.  The return
+    value is a dict, where the keys are the unique field values, and the values
+    are dicts of count, avg_duration, and the key.
+    """
+    grouped = {}
+    for parsed in lines:
+        key = parsed[field]
+        summary = grouped.setdefault(key, {'count':0, 'total_duration':0.0})
+        summary[field] = key
+        summary['count'] += 1
+        summary['total_duration'] += parsed['duration']
+    # average dat up
+    for summary in grouped.values():
+        summary['avg_duration'] = summary['total_duration']/summary['count']
+        del summary['total_duration']
+    return grouped
+
+def select_timerange(lines, earliest=None, latest=None):
+    """ Generator that accepts an iterable of parsed log lines and yields
+    the log lines that are between the earliest and latest dates.  If
+    either earliest or latest is None, it is ignored."""
+    for parsed in lines:
+        if earliest and parsed['date'] < earliest:
+            continue
+        if latest and parsed['date'] > latest:
+            continue
+        yield parsed
+
+
+if __name__ == "__main__":
+    parser = optparse.OptionParser()
+    parser.add_option('--earliest', dest='earliest', default=None,
+        help='Earliest date to count, either as a full date or a relative time \
+such as "-1 hour".  Relative to --latest, so you generally want to\
+specify a negative relative.')
+    parser.add_option('--latest', dest='latest', default=None,
+        help='Latest date to count, either as a full date or a relative time\
+such as "-30s".  Relative to now.')
+    parser.add_option('--group-by', dest='group_by', default='path',
+        help='Compute counts and aggregates for log lines grouped by this\
+attribute.  Good values include "status_code", "method", and\
+"path" (the default).')
+    opts, args = parser.parse_args()
+
+    if opts.latest:
+        opts.latest = parse_casual_time(opts.latest, datetime.now())
+    if opts.earliest:
+        opts.earliest = parse_casual_time(opts.earliest, 
+                                            opts.latest or datetime.now())
+    if opts.earliest or opts.latest:
+        print "Including dates between", \
+            opts.earliest or "the beginning of time", "and", opts.latest or "now"
+    
+    parsed_lines = parse_lines(sys.stdin)
+    grouped = group_parsed_lines(
+        select_timerange(parsed_lines, opts.earliest, opts.latest),
+        opts.group_by)
+    
+    flat = grouped.values()
+    flat.sort(key=lambda x: x['count'])
+    flat.reverse()
+    print "Count\tAvg Dur\t%s" % opts.group_by
+    for summary in flat:
+        print "%d\t%.4f\t%s" % (summary['count'], 
+            summary['avg_duration'], summary[opts.group_by])
+

+ 366 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/util/status.py

@@ -0,0 +1,366 @@
+import datetime
+try:
+    import json
+except ImportError:
+    import simplejson as json
+
+import eventlet
+from eventlet import event
+from eventlet import wsgi
+from eventlet.green import os
+
+class Server(object):
+    def __init__(self, controller, host, port):
+        self.controller = controller
+        self.host = host
+        self.port = port
+        self.status_waiter = None
+        self.child_events = {}
+        socket = eventlet.listen((host, port))
+        wsgi.server(socket, self.application)
+
+    def get_status_data(self):
+        # using a waiter because we only want one child collection ping
+        # happening at a time; if there are multiple concurrent status requests,
+        # they all simply share the same set of data results
+        if self.status_waiter is None:
+            self.status_waiter = eventlet.spawn(self._collect_status_data)
+        return self.status_waiter.wait()
+    
+    def _collect_status_data(self):
+        try:
+            now = datetime.datetime.now()
+            children = self.controller.children.values()
+            status_data = {
+                'active_children_count':len([c 
+                    for c in children
+                    if c.active]),
+                'killed_children_count':len([c 
+                    for c in children
+                    if not c.active]),
+                'configured_children_count':self.controller.num_processes,
+                'now':now.ctime(),
+                'pid':os.getpid(),
+                'uptime':format_timedelta(now - self.controller.started_at),
+                'started_at':self.controller.started_at.ctime(),
+                'config':self.controller.config}
+            # fire up a few greenthreads to wait on children's responses
+            p = eventlet.GreenPile()
+            for child in self.controller.children.values():
+                p.spawn(self.collect_child_status, child)
+            status_data['children'] = dict([pid_cd for pid_cd in p])
+            
+            # total concurrent connections
+            status_data['concurrent_requests'] = sum([
+                child.get('concurrent_requests', 0)
+                for child in status_data['children'].values()])
+        finally:
+            # wipe out the waiter so that subsequent requests create new ones
+            self.status_waiter = None
+        return status_data
+
+    def collect_child_status(self, child):
+        self.child_events[child.pid] = event.Event()
+        try:
+            try:
+                # tell the child to POST its status to us, we handle it in the
+                # wsgi application below
+                eventlet.hubs.trampoline(child.kill_pipe, write=True)
+                os.write(child.kill_pipe, 's')
+                t = eventlet.Timeout(1)
+                results = self.child_events[child.pid].wait()
+                t.cancel()
+            except (OSError, IOError), e:
+                results = {'error': "%s %s" % (type(e), e)}
+            except eventlet.Timeout:
+                results = {'error':'Timed out'}
+        finally:
+            self.child_events.pop(child.pid, None)
+            
+        results.update({
+            'pid':child.pid, 
+            'active':child.active,
+            'uptime':format_timedelta(datetime.datetime.now() - child.forked_at),
+            'forked_at':child.forked_at.ctime()})
+        return child.pid, results
+
+    def application(self, environ, start_response):
+        if environ['REQUEST_METHOD'] == 'GET':
+            status_data = self.get_status_data()
+            if environ['PATH_INFO'] == '/status':
+                start_response('200 OK', [('content-type', 'text/html')])
+                return [fill_template(status_data)]
+            elif environ['PATH_INFO'] == '/status.json':
+                start_response('200 OK', [('content-type', 'application/json')])
+                return [json.dumps(status_data, indent=2)]
+                
+        elif environ['REQUEST_METHOD'] == 'POST':
+            # it's a client posting its stats to us
+            body = environ['wsgi.input'].read()
+            child_status = json.loads(body)
+            pid = child_status['pid']
+            if pid in self.child_events:
+                self.child_events[pid].send(child_status)
+                start_response('200 OK', [('content-type', 'application/json')])
+            else:
+                start_response('500 Internal Server Error', 
+                               [('content-type', 'text/plain')])
+                print "Don't know about child pid %s" % pid
+            return [""]
+        
+        # fallthrough case
+        start_response('404 Not Found', [('content-type', 'text/plain')])
+        return [""]
+
+def format_timedelta(t):
+    """Based on how HAProxy's status page shows dates.
+    10d 14h
+    3h 20m
+    1h 0m
+    12m
+    15s
+    """
+    seconds = t.seconds
+    if t.days > 0:
+        return "%sd %sh" % (t.days, int(seconds/3600))
+    else:
+        if seconds > 3600:
+            hours = int(seconds/3600)
+            seconds -= hours*3600
+            return "%sh %sm" % (hours, int(seconds/60))
+        else:
+            if seconds > 60:
+                return "%sm" % int(seconds/60)
+            else:
+                return "%ss" % seconds
+
+class Tag(object):
+    """Yeah, there's a templating DSL in this status module.  Deal with it."""
+    def __init__(self, name, *children, **attrs):
+        self.name = name
+        self.attrs = attrs
+        self.children = list(children)
+
+    def __str__(self):
+        al = []
+        for name, val in self.attrs.iteritems():
+            if name == 'cls':
+                name = "class"
+            if isinstance(val, (list, tuple)):
+                val = " ".join(val)
+            else:
+                val = str(val)
+            al.append('%s="%s"' % (name, val))
+        if al:
+            attrstr = " " + " ".join(al) + " "
+        else:
+            attrstr = ""
+        cl = []
+        for child in self.children:
+            cl.append(str(child))
+        if cl:
+            childstr = "\n" + "\n".join(cl) + "\n"
+        else:
+            childstr = ""
+        return "<%s%s>%s</%s>" % (self.name, attrstr, childstr, self.name)
+
+def make_tag(name):
+    return lambda *c, **a: Tag(name, *c, **a)
+p = make_tag('p')
+div = make_tag('div')
+table = make_tag('table')
+tr = make_tag('tr')
+th = make_tag('th')
+td = make_tag('td')
+h2 = make_tag('h2')
+span = make_tag('span')
+
+def fill_template(status_data):
+    # controller status
+    cont_div = table(id='controller')
+    cont_div.children.append(tr(th("PID:", title="Controller Process ID"), 
+        td(status_data['pid'])))
+    cont_div.children.append(tr(th("Uptime:", title="Time since launch"), 
+        td(status_data['uptime'])))
+    cont_div.children.append(tr(th("Host:", title="Host and port server is listening on, all means all interfaces."), 
+        td("%s:%s" % (status_data['config']['host'] or "all",
+            status_data['config']['port']))))
+    cont_div.children.append(tr(th("Threads:", title="Threads per child"), 
+        td(status_data['config']['threadpool_workers'])))
+    cont_div = div(cont_div)
+    
+    # children headers and summaries
+    child_div = div(h2("Child Processes"))
+    count_td = td(status_data['active_children_count'], "/", 
+                  status_data['configured_children_count'])
+    if status_data['active_children_count'] < \
+       status_data['configured_children_count']:
+        count_td.attrs['cls'] = "error"
+        count_td.children.append(
+            span("(", status_data['killed_children_count'], ")"))
+    children_table = table(
+      tr(
+        th('PID', title="Process ID"), 
+        th('Active', title="Accepting New Requests"), 
+        th('Uptime', title="Uptime"), 
+        th('Concurrent', title="Concurrent Requests")),
+      tr(
+        td("Total"),
+        count_td,
+        td(),  # no way to "total" uptime
+        td(status_data['concurrent_requests'])),
+      id="children")
+    child_div.children.append(children_table)
+    
+    # children themselves
+    odd = True
+    for pid in sorted(status_data['children'].keys()):
+        child = status_data['children'][pid]
+        row = tr(td(pid), cls=['child'])
+        if odd:
+            row.attrs['cls'].append('odd')
+        odd = not odd
+        
+        # active handling
+        row.children.append(td({True:'Y', False:'N'}[child['active']]))
+        if not child['active']:
+            row.attrs['cls'].append('dying')
+            
+        # errors
+        if child.get('error'):
+            row.attrs['cls'].append('error')
+            row.children.append(td(child['error'], colspan=2))
+        else:
+            # no errors
+            row.children.append(td(child['uptime']))
+            row.children.append(td(child['concurrent_requests']))
+            
+        children_table.children.append(row)
+        
+    # config dump
+    config_div = div(
+        h2("Configuration"),
+        table(*[tr(th(key),  td(status_data['config'][key]))
+            for key in sorted(status_data['config'].keys())]), 
+        id='config')
+        
+    to_format = {'cont_div': cont_div, 'child_div':child_div,
+                 'config_div':config_div}
+    to_format.update(status_data)
+    return HTML_SHELL % to_format
+
+HTML_SHELL = """
+<!DOCTYPE html>
+<html><head>
+<title>Spawning Status</title>
+<style type="text/css">
+html, p, div, table, h1, h2, input, form {
+	margin: 0;
+	padding: 0;
+	border: 0;
+	outline: 0;
+	font-size: 12px;
+	font-family: Helvetica, Arial, sans-serif;
+	vertical-align: baseline;
+}
+body {
+	line-height: 1.2;
+	color: black;
+	background: white;
+	margin: 3em;
+}
+table {
+	border-collapse: separate;
+	border-spacing: 0;
+}
+th, td {
+	text-align: center;
+	padding: .1em;
+    padding-right: .4em;
+}
+#controller td, #controller th {
+    text-align: left;
+}
+#config td, #config th {
+    text-align: left;
+}
+#children {
+    clear: both;
+}
+#options {
+    float: right;
+    border: 1px solid #dfdfdf;
+    padding:.5em;
+}
+h1,h2 {
+    margin: .5em;
+    margin-left: 0em;
+    font-size: 130%%;  
+}
+h2 {
+    font-size: 115%%;  
+}
+tr.odd {
+    background: #dfdfdf;
+}
+input {
+    border: 1px solid grey;
+}
+#refresh form {
+    display: inline;
+}
+tr.child.dying {
+    font-style: italic;
+    color: #444444;
+}
+.error {
+    background: #ff4444;
+}
+
+/* Cut out the fat for mobile devices */
+@media screen and (max-width: 400px) {
+    body {
+        margin-left: .2em;
+        margin-right: .2em;
+    }
+    #options {
+        float: none;
+    }
+}
+</style>
+</head><body>
+<h1>Spawning Status</h1>
+<div id="options">
+<p>%(now)s</p>
+<div id="refresh">
+<a href="">Refresh</a> (<form>
+  <input type="checkbox" /> every
+  <input type="text" value="5" size=2 />s
+</form>)
+</div>
+<a href="status.json">JSON</a>
+</div>
+%(cont_div)s
+%(child_div)s
+%(config_div)s
+<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
+<script type="text/javascript">
+$(document).ready(function() {
+    var timer;
+    var arrangeTimeout = function () {
+        clearTimeout(timer);
+        if($('#refresh input[type=checkbox]').attr('checked')) {
+            timer = setTimeout(
+                function() {window.location.reload();},
+                $('#refresh input[type=text]').val() * 1000);
+        }
+        if($(this).is('form')) {
+            return false;
+        }
+    };
+    $('#refresh input[type=checkbox]').click(arrangeTimeout);
+    $('#refresh form').submit(arrangeTimeout).submit();
+});
+</script>
+</body></html>
+"""

+ 96 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/util/system.py

@@ -0,0 +1,96 @@
+# Copyright (c) 2010, Steve 'Ashcrow' MIlner
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+"""
+Platform related items.
+"""
+
+import os
+import platform
+import sys
+import tempfile
+
+
+class System(dict):
+    """
+    Class to make finding out system information all in one place.
+
+    **Note**: You can not add attributes to an instance of this class.
+    """
+
+    def __init__(self):
+        dict.__init__(self, {
+            'architecture': platform.architecture(),
+            'max_int': sys.maxint,
+            'max_size': sys.maxsize,
+            'max_unicode': sys.maxunicode,
+            'name': platform.node(),
+            'path_seperator': os.path.sep,
+            'processor': platform.processor(),
+            'python_version': platform.python_version(),
+            'python_branch': platform.python_branch(),
+            'python_build': platform.python_build(),
+            'python_compiler': platform.python_compiler(),
+            'python_implementation': platform.python_implementation(),
+            'python_revision': platform.python_revision(),
+            'python_version_tuple': platform.python_version_tuple(),
+            'python_path': sys.path,
+            'login': os.getlogin(),
+            'system': platform.system(),
+            'temp_directory': tempfile.gettempdir(),
+            'uname': platform.uname(),
+    })
+
+    def __getattr__(self, name):
+        """
+        Looks in the dictionary for items **only**.
+
+        :Parameters:
+           - 'name': name of the attribute to get.
+        """
+        data = dict(self).get(name)
+        if data == None:
+            raise AttributeError("'%s' has no attribute '%s'" % (
+                self.__class__.__name__, name))
+        return data
+
+    def __setattr__(self, key, value):
+        """
+        Setting attributes is **not** allowed.
+
+        :Parameters:
+           - `key`: attribute name to set.
+           - `value`: value to set attribute to.
+        """
+        raise AttributeError("can't set attribute")
+
+    def __repr__(self):
+        """
+        Nice object representation.
+        """
+        return unicode(
+            "<Platform: system='%s', name='%s', arch=%s, processor='%s'>" % (
+            self.system, self.name, self.architecture, self.processor))
+
+    # Method aliases
+    __str__ = __repr__
+    __unicode__ = __repr__
+    __setitem__ = __setattr__

+ 82 - 0
desktop/core/ext-py/Spawning-0.9.6/src/spawning/wsgi_factory.py

@@ -0,0 +1,82 @@
+# Copyright (c) 2008, Donovan Preston
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to
+# deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+# FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+"""The config_factory takes a dictionary containing the command line arguments
+and should return the same dictionary after modifying any of the settings it wishes.
+At the very least the config_factory must set the 'app_factory' key in the returned
+argument dictionary, which should be the dotted path to the function which will be
+called to actually return the wsgi application which will be served.  Also, the
+config_factory can look at the 'args' key for any additional positional command-line
+arguments that were passed to spawn, and modify the configuration dictionary
+based on it's contents.
+
+Return value of config_factory should be a dict containing:
+    app_factory: The dotted path to the wsgi application factory.
+        Will be called with the result of factory_qual as the argument.
+    host: The local ip to bind to.
+    port: The local port to bind to.
+    num_processes: The number of processes to spawn.
+    num_threads: The number of threads to use in the threadpool in each process.
+        If 0, install the eventlet monkeypatching and do not use the threadpool.
+        Code which blocks instead of cooperating will block the process, possibly
+        causing stalls. (TODO sigalrm?)
+    dev: If True, watch all files in sys.modules, easy-install.pth, and any additional
+        file paths in the 'watch' list for changes and restart child
+        processes on change. If False, only reload if the svn revision of the
+        current directory changes.
+    watch: List of additional files to watch for changes and reload when changed.
+"""
+import inspect
+import os
+import time
+
+import spawning.util
+
+def config_factory(args):
+    args['app_factory'] = 'spawning.wsgi_factory.app_factory'
+    args['app'] = args['args'][0]
+    args['middleware'] = args['args'][1:]
+
+    args['source_directories'] = [os.path.split(
+        inspect.getfile(
+            inspect.getmodule(
+                spawning.util.named(args['app']))))[0]]
+    return args
+
+
+def app_factory(config):
+    app = spawning.util.named(config['app'])
+    for mid in config['middleware']:
+        app = spawning.util.named(mid)(app)
+    return app
+
+
+def hello_world(env, start_response):
+    start_response('200 OK', [('Content-type', 'text/plain')])
+    return ['Hello, World!\r\n']
+
+
+def really_long(env, start_response):
+    start_response('200 OK', [('Content-type', 'text/plain')])
+    time.sleep(180)
+    return ['Goodbye, World!\r\n']
+

+ 71 - 0
desktop/core/ext-py/eventlet-0.9.14/AUTHORS

@@ -0,0 +1,71 @@
+Maintainer (i.e., Who To Hassle If You Find Bugs)
+-------------------------------------------------
+Ryan Williams, rdw on Freenode, breath@alum.mit.edu
+
+Original Authors
+----------------
+* Bob Ippolito
+* Donovan Preston
+
+Contributors
+------------
+* AG Projects
+* Chris AtLee
+* R\. Tyler Ballance
+* Denis Bilenko
+* Mike Barton
+* Patrick Carlisle
+* Ben Ford
+* Andrew Godwin
+* Brantley Harris
+* Gregory Holt
+* Joe Malicki
+* Chet Murthy
+* Eugene Oden
+* radix
+* Scott Robinson
+* Tavis Rudd
+* Sergey Shepelev
+* Chuck Thier
+* Nick V
+* Daniele Varrazzo
+* Ryan Williams
+
+Linden Lab Contributors
+-----------------------
+* John Beisley
+* Tess Chu
+* Nat Goodspeed
+* Dave Kaprielian
+* Kartic Krishnamurthy
+* Bryan O'Sullivan
+* Kent Quirk
+* Ryan Williams
+
+Thanks To
+---------
+* AdamKG, giving the hint that invalid argument errors were introduced post-0.9.0
+* Luke Tucker, bug report regarding wsgi + webob
+* Taso Du Val, reproing an exception squelching bug, saving children's lives  ;-)
+* Luci Stanescu, for reporting twisted hub bug
+* Marcus Cavanaugh, for test case code that has been incredibly useful in tracking down bugs
+* Brian Brunswick, for many helpful questions and suggestions on the mailing list
+* Cesar Alaniz, for uncovering bugs of great import
+* the grugq, for contributing patches, suggestions, and use cases
+* Ralf Schmitt, for wsgi/webob incompatibility bug report and suggested fix
+* Benoit Chesneau, bug report on green.os and patch to fix it
+* Slant, better iterator implementation in tpool
+* Ambroff, nice pygtk hub example
+* Michael Carter, websocket patch to improve location handling
+* Marcin Bachry, nice repro of a bug and good diagnosis leading to the fix
+* David Ziegler, reporting issue #53
+* Favo Yang, twisted hub patch
+* Schmir, patch that fixes readline method with chunked encoding in wsgi.py, advice on patcher
+* Slide, for open-sourcing gogreen
+* Holger Krekel, websocket example small fix
+* mikepk, debugging MySQLdb/tpool issues
+* Malcolm Cleaton, patch for Event exception handling
+* Alexey Borzenkov, for finding and fixing issues with Windows error detection (#66, #69), reducing dependencies in zeromq hub (#71)
+* Anonymous, finding and fixing error in websocket chat example (#70)
+* Edward George, finding and fixing an issue in the [e]poll hubs (#74)
+* Ruijun Luo, figuring out incorrect openssl import for wrap_ssl (#73)

+ 23 - 0
desktop/core/ext-py/eventlet-0.9.14/LICENSE

@@ -0,0 +1,23 @@
+Unless otherwise noted, the files in Eventlet are under the following MIT license:
+
+Copyright (c) 2005-2006, Bob Ippolito
+Copyright (c) 2007-2010, Linden Research, Inc.
+Copyright (c) 2008-2010, Eventlet Contributors (see AUTHORS)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

+ 4 - 0
desktop/core/ext-py/eventlet-0.9.14/MANIFEST.in

@@ -0,0 +1,4 @@
+recursive-include tests *.py *.crt *.key
+recursive-include doc *.rst *.txt *.py Makefile *.png
+recursive-include examples *.py *.html
+include MANIFEST.in README.twisted NEWS AUTHORS LICENSE README

+ 334 - 0
desktop/core/ext-py/eventlet-0.9.14/NEWS

@@ -0,0 +1,334 @@
+0.9.14
+======
+* Many fixes to the ZeroMQ hub, which now requires version 2.0.10 or later.  Thanks to Ben Ford.
+* ZeroMQ hub no longer depends on pollhub, and thus works on Windows (thanks, Alexey Borzenkov)
+* Better handling of connect errors on Windows, thanks again to Alexey Borzenkov.
+* More-robust Event delivery, thanks to Malcolm Cleaton
+* wsgi.py now distinguishes between an empty query string ("") and a non-existent query string (no entry in environ).
+* wsgi.py handles ipv6 correctly (thanks, redbo)
+* Better behavior in tpool when you give it nonsensical numbers, thanks to R. Tyler for the nonsense.  :)
+* Fixed importing on 2.5 (#73, thanks to Ruijun Luo)
+* Hub doesn't hold on to invalid fds (#74, thanks to Edward George)
+* Documentation for eventlet.green.zmq, courtesy of Ben Ford
+
+0.9.13
+======
+* ZeroMQ hub, and eventlet.green.zmq make supersockets green.  Thanks to Ben Ford!
+* eventlet.green.MySQLdb added.  It's an interface to MySQLdb that uses tpool to make it appear nonblocking
+* Greenthread affinity in tpool.  Each greenthread is assigned to the same thread when using tpool, making it easier to work with non-thread-safe libraries.
+* Eventlet now depends on greenlet 0.3 or later.
+* Fixed a hang when using tpool during an import causes another import.  Thanks to mikepk for tracking that down.
+* Improved websocket draft 76 compliance, thanks to Nick V.
+* Rare greenthread.kill() bug fixed, which was probably brought about by a bugfix in greenlet 0.3.
+* Easy_installing eventlet should no longer print an ImportError about greenlet
+* Support for serving up SSL websockets, thanks to chwagssd for reporting #62
+* eventlet.wsgi properly sets 'wsgi.url_scheme' environment variable to 'https', and 'HTTPS' to 'on' if serving over ssl
+* Blocking detector uses setitimer on 2.6 or later, allowing for sub-second block detection, thanks to rtyler.
+* Blocking detector is documented now, too
+* socket.create_connection properly uses dnspython for nonblocking dns.  Thanks to rtyler.
+* Removed EVENTLET_TPOOL_DNS, nobody liked that.  But if you were using it, install dnspython instead.  Thanks to pigmej and gholt.
+* Removed _main_wrapper from greenthread, thanks to Ambroff adding keyword arguments to switch() in 0.3!
+
+0.9.12
+======
+* Eventlet no longer uses the Twisted hub if Twisted is imported -- you must call eventlet.hubs.use_hub('twistedr') if you want to use it.  This prevents strange race conditions for those who want to use both Twisted and Eventlet separately.
+* Removed circular import in twistedr.py
+* Added websocket multi-user chat example
+* Not using exec() in green modules anymore.
+* eventlet.green.socket now contains all attributes of the stdlib socket module, even those that were left out by bugs.
+* Eventlet.wsgi doesn't call print anymore, instead uses the logfiles for everything (it used to print exceptions in one place).
+* Eventlet.wsgi properly closes the connection when an error is raised
+* Better documentation on eventlet.event.Event.send_exception
+* Adding websocket.html to tarball so that you can run the examples without checking out the source
+
+0.9.10
+======
+* Greendns: if dnspython is installed, Eventlet will automatically use it to provide non-blocking DNS queries.  Set the environment variable 'EVENTLET_NO_GREENDNS' if you don't want greendns but have dnspython installed.
+* Full test suite passes on Python 2.7.
+* Tests no longer depend on simplejson for >2.6.
+* Potential-bug fixes in patcher (thanks to Schmir, and thanks to Hudson)
+* Websockets work with query strings (thanks to mcarter)
+* WSGI posthooks that get called after the request completed (thanks to gholt, nice docs, too)
+* Blocking detector merged -- use it to detect places where your code is not yielding to the hub for > 1 second.
+* tpool.Proxy can wrap callables
+* Tweaked Timeout class to do something sensible when True is passed to the constructor
+
+0.9.9
+=====
+* A fix for monkeypatching on systems with psycopg version 2.0.14.
+* Improved support for chunked transfers in wsgi, plus a bunch of tests from schmir (ported from gevent by redbo)
+* A fix for the twisted hub from Favo Yang
+
+0.9.8
+=====
+* Support for psycopg2's asynchronous mode, from Daniele Varrazzo
+* websocket module is now part of core Eventlet with 100% unit test coverage thanks to Ben Ford.  See its documentation at http://eventlet.net/doc/modules/websocket.html
+* Added wrap_ssl convenience method, meaning that we truly no longer need api or util modules.
+* Multiple-reader detection code protects against the common mistake of having multiple greenthreads read from the same socket at the same time, which can be overridden if you know what you're doing.
+* Cleaner monkey_patch API: the "all" keyword is no longer necessary.
+* Pool objects have a more convenient constructor -- no more need to subclass
+* amajorek's reimplementation of GreenPipe
+* Many bug fixes, major and minor.
+
+0.9.7
+=====
+* GreenPipe is now a context manager (thanks, quad)
+* tpool.Proxy supports iterators properly
+* bug fixes in eventlet.green.os (thanks, Benoit)
+* much code cleanup from Tavis
+* a few more example apps
+* multitudinous improvements in Py3k compatibility from amajorek
+
+
+0.9.6
+=====
+* new EVENTLET_HUB environment variable allows you to select a hub without code
+* improved GreenSocket and GreenPipe compatibility with stdlib
+* bugfixes on GreenSocket and GreenPipe objects
+* code coverage increased across the board
+* Queue resizing
+* internal DeprecationWarnings largely eliminated
+* tpool is now reentrant (i.e., can call tpool.execute(tpool.execute(foo)))
+* more reliable access to unpatched modules reduces some race conditions when monkeypatching
+* completely threading-compatible corolocal implementation, plus tests and enthusiastic adoption
+* tests stomp on each others' toes less
+* performance improvements in timers, hubs, greenpool
+* Greenlet-aware profile module courtesy of CCP
+* support for select26 module's epoll
+* better PEP-8 compliance and import cleanup
+* new eventlet.serve convenience function for easy TCP servers
+
+
+0.9.5
+=====
+* support psycopg in db_pool
+* smart patcher that does the right patching when importing without needing to understand plumbing of patched module
+* patcher.monkey_patch() method replacing util.wrap_*
+* monkeypatch threading support
+* removed api.named
+* imported timeout module from gevent, replace exc_after and with_timeout()
+* replace call_after with spawn_after; this is so that users don't see the Timer class
+* added cancel() method to GreenThread to support the semantic of "abort if not already in the middle of something"
+* eventlet.green.os with patched read() and write(), etc
+* moved stuff from wrap_pipes_with_coroutine_pipe into green.os
+* eventlet.green.subprocess instead of eventlet.processes
+* improve patching docs, explaining more about patcher and why you'd use eventlet.green
+* better documentation on greenpiles
+* deprecate api.py completely
+* deprecate util.py completely
+* deprecate saranwrap
+* performance improvements in the hubs
+* much better documentation overall
+* new convenience functions: eventlet.connect and eventlet.listen.  Thanks, Sergey!
+
+
+0.9.4
+=====
+* Deprecated coros.Queue and coros.Channel (use queue.Queue instead)
+* Added putting and getting methods to queue.Queue.
+* Added eventlet.green.Queue which is a greened clone of stdlib Queue, along with stdlib tests.
+* Changed __init__.py so that the version number is readable even if greenlet's not installed.
+* Bugfixes in wsgi, greenpool
+
+0.9.3
+=====
+
+* Moved primary api module to __init__ from api.  It shouldn't be necessary to import eventlet.api anymore; import eventlet should do the same job.
+* Proc module deprecated in favor of greenthread
+* New module greenthread, with new class GreenThread.
+* New GreenPool class that replaces pool.Pool.
+* Deprecated proc module (use greenthread module instead)
+* tpooled gethostbyname is configurable via environment variable EVENTLET_TPOOL_GETHOSTBYNAME
+* Removed greenio.Green_fileobject and refactored the code therein to be more efficient.  Only call makefile() on sockets now; makeGreenFile() is deprecated.  The main loss here is that of the readuntil method.  Also, Green_fileobjects used to be auto-flushing; flush() must be called explicitly now.
+* Added epoll support
+* Improved documentation across the board.
+* New queue module, API-compatible with stdlib Queue
+* New debug module, used for enabling verbosity within Eventlet that can help debug applications or Eventlet itself.
+* Bugfixes in tpool, green.select, patcher
+* Deprecated coros.execute (use eventlet.spawn instead)
+* Deprecated coros.semaphore (use semaphore.Semaphore or semaphore.BoundedSemaphore instead)
+* Moved coros.BoundedSemaphore to semaphore.BoundedSemaphore
+* Moved coros.Semaphore to semaphore.Semaphore
+* Moved coros.event to event.Event
+* Deprecated api.tcp_listener, api.connect_tcp, api.ssl_listener
+* Moved get_hub, use_hub, get_default_hub from eventlet.api to eventlet.hubs
+* Renamed libevent hub to pyevent.
+* Removed previously-deprecated features tcp_server, GreenSSL, erpc, and trap_errors.
+* Removed saranwrap as an option for making db connections nonblocking in db_pool.
+
+0.9.2
+=====
+
+* Bugfix for wsgi.py where it was improperly expecting the environ variable to be a constant when passed to the application.
+* Tpool.py now passes its tests on Windows.
+* Fixed minor performance issue in wsgi.
+
+0.9.1
+=====
+
+* PyOpenSSL is no longer required for Python 2.6: use the eventlet.green.ssl module. 2.5 and 2.4 still require PyOpenSSL.
+* Cleaned up the eventlet.green packages and their associated tests, this should result in fewer version-dependent bugs with these modules.
+* PyOpenSSL is now fully wrapped in eventlet.green.OpenSSL; using it is therefore more consistent with using other green modules.
+* Documentation on using SSL added.
+* New green modules: ayncore, asynchat, SimpleHTTPServer, CGIHTTPServer, ftplib.
+* Fuller thread/threading compatibility: patching threadlocal with corolocal so coroutines behave even more like threads.
+* Improved Windows compatibility for tpool.py
+* With-statement compatibility for pools.Pool objects.
+* Refactored copyrights in the files, added LICENSE and AUTHORS files.
+* Added support for logging x-forwarded-for header in wsgi.
+* api.tcp_server is now deprecated, will be removed in a future release.
+* Added instructions on how to generate coverage reports to the documentation.
+* Renamed GreenFile to Green_fileobject, to better reflect its purpose.
+* Deprecated erpc method in tpool.py
+* Bug fixes in: wsgi.py, twistedr.py, poll.py, greenio.py, util.py, select.py, processes.py, selects.py
+
+0.9.0
+=====
+
+* Full-duplex sockets (simultaneous readers and writers in the same process).
+* Remove modules that distract from the core mission of making it straightforward to write event-driven networking apps: 
+    httpd, httpc, channel, greenlib, httpdate, jsonhttp, logutil
+* Removed test dependency on sqlite, using nose instead.
+* Marked known-broken tests using nose's mechanism (most of these are not broken but are simply run in the incorrect context, such as threading-related tests that are incompatible with the libevent hub).
+* Remove copied code from python standard libs (in tests).
+* Added eventlet.patcher which can be used to import "greened" modules.
+
+0.8.16
+======
+* GreenSSLObject properly masks ZeroReturnErrors with an empty read; with unit test.
+* Fixed 2.6 SSL compatibility issue.
+
+0.8.15
+======
+
+* GreenSSL object no longer converts ZeroReturnErrors into empty reads, because that is more compatible with the underlying SSLConnection object.
+* Fixed issue caused by SIGCHLD handler in processes.py
+* Stopped supporting string exceptions in saranwrap and fixed a few test failures.
+
+0.8.14
+======
+* Fixed some more Windows compatibility problems, resolving EVT-37 : 
+http://jira.secondlife.com/browse/EVT-37
+* waiting() method on Pool class, which was lost when the Pool implementation
+replaced CoroutinePool.
+
+0.8.13
+======
+* 2.6 SSL compatibility patch by Marcus Cavanaugh.
+* Added greenlet and pyopenssl as dependencies in setup.py.
+
+0.8.12
+======
+
+* The ability to resize() pools of coroutines, which was lost when the 
+Pool implementation replaced CoroutinePool.
+* Fixed Cesar's issue with SSL connections, and furthermore did a 
+complete overhaul of SSL handling in eventlet so that it's much closer 
+to the behavior of the built-in libraries.  In particular, users of 
+GreenSSL sockets must now call shutdown() before close(), exactly 
+like SSL.Connection objects.
+* A small patch that makes Eventlet work on Windows.  This is the first 
+release of Eventlet that works on Windows.
+
+0.8.11
+======
+
+Eventlet can now run on top of twisted reactor. Twisted-based hub is enabled automatically if
+twisted.internet.reactor is imported. It is also possible to "embed" eventlet into a twisted
+application via eventlet.twistedutil.join_reactor. See the examples for details.
+
+A new package, eventlet.twistedutil, is added that makes integration of twisted and eventlet
+easier. It has block_on function that allows to wait for a Deferred to fire and it wraps
+twisted's Protocol in a synchronous interface. This is similar to and is inspired by Christopher
+Armstrong's corotwine library. Thanks to Dan Pascu for reviewing the package.
+
+Another new package, eventlet.green, was added to provide some of the standard modules
+that are fixed not to block other greenlets. This is an alternative to monkey-patching
+the socket, which is impossible to do if you are running twisted reactor.
+The package includes socket, httplib, urllib2.
+
+Much of the core functionality has been refactored and cleaned up, including the removal
+of eventlet.greenlib. This means that it is now possible to use plain greenlets without
+modification in eventlet, and the subclasses of greenlet instead of the old
+eventlet.greenlib.GreenletContext. Calling eventlet.api.get_hub().switch() now checks to
+see whether the current greenlet has a "switch_out" method and calls it if so, providing the
+same functionality that the GreenletContext.swap_out used to. The swap_in behavior can be
+duplicated by overriding the switch method, and the finalize functionality can be duplicated
+by having a try: finally: block around the greenlet's main implementation. The eventlet.backdoor
+module has been ported to this new scheme, although it's signature had to change slightly so
+existing code that used the backdoor will have to be modified.
+
+A number of bugs related to improper scheduling of switch calls has been fixed.
+The fixed functions and classes include api.trampoline, api.sleep, coros.event,
+coros.semaphore, coros.queue.
+
+Many methods of greenio.GreenSocket were fixed to make its behavior more like that of a regular
+socket. Thanks to Marcin Bachry for fixing GreenSocket.dup to preserve the timeout.
+
+Added proc module which provides an easy way to subscribe to coroutine's results. This makes
+it easy to wait for a single greenlet or for a set of greenlets to complete.
+
+wsgi.py now supports chunked transfer requests (patch by Mike Barton)
+
+The following modules were deprecated or removed because they were broken:
+hubs.nginx, hubs.libev, support.pycurls, support.twisteds, cancel method of coros.event class
+
+The following classes are still present but will be removed in the future version:
+- channel.channel (use coros.Channel)
+- coros.CoroutinePool (use pool.Pool)
+
+saranwrap.py now correctly closes the child process when the referring object is deleted,
+received some fixes to its detection of child process death, now correctly deals with the in
+keyword, and it is now possible to use coroutines in a non-blocking fashion in the child process.
+
+Time-based expiry added to db_pool.  This adds the ability to expire connections both by idleness
+and also by total time open.  There is also a connection timeout option.
+
+A small bug in httpd's error method was fixed.
+
+Python 2.3 is no longer supported.
+
+A number of tests was added along with a script to run all of them for all the configurations.
+The script generates an html page with the results.
+
+Thanks to Brian Brunswick for investigation of popen4 badness (eventlet.process)
+Thanks to Marcus Cavanaugh for pointing out some coros.queue(0) bugs.
+
+The twisted integration as well as many other improvements were funded by AG Projects (http://ag-projects.com), thanks!
+
+0.8.x
+=====
+
+Fix a CPU leak that would cause the poll hub to consume 100% CPU in certain conditions, for example the echoserver example. (Donovan Preston)
+
+Fix the libev hub to match libev's callback signature. (Patch by grugq)
+
+Add a backlog argument to api.tcp_listener (Patch by grugq)
+
+0.7.x
+=====
+
+Fix a major memory leak when using the libevent or libev hubs. Timers were not being removed from the hub after they fired. (Thanks Agusto Becciu and the grugq). Also, make it possible to call wrap_socket_with_coroutine_socket without using the threadpool to make dns operations non-blocking (Thanks the grugq).
+
+It's now possible to use eventlet's SSL client to talk to eventlet's SSL server. (Thanks to Ryan Williams)
+
+Fixed a major CPU leak when using select hub. When adding a descriptor to the hub, entries were made in all three dictionaries, readers, writers, and exc, even if the callback is None. Thus every fd would be passed into all three lists when calling select regardless of whether there was a callback for that event or not. When reading the next request out of a keepalive socket, the socket would come back as ready for writing, the hub would notice the callback is None and ignore it, and then loop as fast as possible consuming CPU.
+
+0.6.x
+=====
+
+Fixes some long-standing bugs where sometimes failures in accept() or connect() would cause the coroutine that was waiting to be double-resumed, most often resulting in SwitchingToDeadGreenlet exceptions as well as weird tuple-unpacking exceptions in the CoroutinePool main loop.
+
+0.6.1: Added eventlet.tpool.killall. Blocks until all of the threadpool threads have been told to exit and join()ed. Meant to be used to clean up the threadpool on exit or if calling execv. Used by Spawning.
+
+0.5.x
+=====
+
+"The Pycon 2008 Refactor": The first release which incorporates libevent support. Also comes with significant refactoring and code cleanup, especially to the eventlet.wsgi http server. Docstring coverage is much higher and there is new extensive documentation: http://wiki.secondlife.com/wiki/Eventlet/Documentation
+
+The point releases of 0.5.x fixed some bugs in the wsgi server, most notably handling of Transfer-Encoding: chunked; previously, it would happily send chunked encoding to clients which asked for HTTP/1.0, which isn't legal.
+
+0.2
+=====
+
+Initial re-release of forked linden branch.

+ 68 - 0
desktop/core/ext-py/eventlet-0.9.14/PKG-INFO

@@ -0,0 +1,68 @@
+Metadata-Version: 1.0
+Name: eventlet
+Version: 0.9.14
+Summary: Highly concurrent networking library
+Home-page: http://eventlet.net
+Author: Linden Lab
+Author-email: eventletdev@lists.secondlife.com
+License: UNKNOWN
+Description: Eventlet is a concurrent networking library for Python that allows you to change how you run your code, not how you write it.
+        
+        It uses epoll or libevent for highly scalable non-blocking I/O.  Coroutines ensure that the developer uses a blocking style of programming that is similar to threading, but provide the benefits of non-blocking I/O.  The event dispatch is implicit, which means you can easily use Eventlet from the Python interpreter, or as a small part of a larger application.
+        
+        It's easy to get started using Eventlet, and easy to convert existing 
+        applications to use it.  Start off by looking at the `examples`_, 
+        `common design patterns`_, and the list of `basic API primitives`_.
+        
+        .. _examples: http://eventlet.net/doc/examples.html
+        .. _common design patterns: http://eventlet.net/doc/design_patterns.html
+        .. _basic API primitives: http://eventlet.net/doc/basic_usage.html
+        
+        Quick Example
+        ===============
+        
+        Here's something you can try right on the command line::
+        
+            % python
+            >>> import eventlet 
+            >>> from eventlet.green import urllib2
+            >>> gt = eventlet.spawn(urllib2.urlopen, 'http://eventlet.net')
+            >>> gt2 = eventlet.spawn(urllib2.urlopen, 'http://secondlife.com')
+            >>> gt2.wait()
+            >>> gt.wait()
+        
+        
+        Getting Eventlet
+        ==================
+        
+        The easiest way to get Eventlet is to use easy_install or pip::
+        
+          easy_install eventlet
+          pip install eventlet
+        
+        The development `tip`_ is available via easy_install as well::
+        
+          easy_install 'eventlet==dev'
+          pip install 'eventlet==dev'
+        
+        .. _tip: http://bitbucket.org/which_linden/eventlet/get/tip.zip#egg=eventlet-dev
+        
+        Building the Docs Locally
+        =========================
+        
+        To build a complete set of HTML documentation, you must have Sphinx, which can be found at http://sphinx.pocoo.org/ (or installed with `easy_install sphinx`)
+        
+          cd doc
+          make html
+          
+        The built html files can be found in doc/_build/html afterward.
+Platform: UNKNOWN
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python
+Classifier: Operating System :: MacOS :: MacOS X
+Classifier: Operating System :: POSIX
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Topic :: Internet
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Intended Audience :: Developers
+Classifier: Development Status :: 4 - Beta

+ 50 - 0
desktop/core/ext-py/eventlet-0.9.14/README

@@ -0,0 +1,50 @@
+Eventlet is a concurrent networking library for Python that allows you to change how you run your code, not how you write it.
+
+It uses epoll or libevent for highly scalable non-blocking I/O.  Coroutines ensure that the developer uses a blocking style of programming that is similar to threading, but provide the benefits of non-blocking I/O.  The event dispatch is implicit, which means you can easily use Eventlet from the Python interpreter, or as a small part of a larger application.
+
+It's easy to get started using Eventlet, and easy to convert existing 
+applications to use it.  Start off by looking at the `examples`_, 
+`common design patterns`_, and the list of `basic API primitives`_.
+
+.. _examples: http://eventlet.net/doc/examples.html
+.. _common design patterns: http://eventlet.net/doc/design_patterns.html
+.. _basic API primitives: http://eventlet.net/doc/basic_usage.html
+
+Quick Example
+===============
+
+Here's something you can try right on the command line::
+
+    % python
+    >>> import eventlet 
+    >>> from eventlet.green import urllib2
+    >>> gt = eventlet.spawn(urllib2.urlopen, 'http://eventlet.net')
+    >>> gt2 = eventlet.spawn(urllib2.urlopen, 'http://secondlife.com')
+    >>> gt2.wait()
+    >>> gt.wait()
+
+
+Getting Eventlet
+==================
+
+The easiest way to get Eventlet is to use easy_install or pip::
+
+  easy_install eventlet
+  pip install eventlet
+
+The development `tip`_ is available via easy_install as well::
+
+  easy_install 'eventlet==dev'
+  pip install 'eventlet==dev'
+
+.. _tip: http://bitbucket.org/which_linden/eventlet/get/tip.zip#egg=eventlet-dev
+
+Building the Docs Locally
+=========================
+
+To build a complete set of HTML documentation, you must have Sphinx, which can be found at http://sphinx.pocoo.org/ (or installed with `easy_install sphinx`)
+
+  cd doc
+  make html
+  
+The built html files can be found in doc/_build/html afterward.

+ 181 - 0
desktop/core/ext-py/eventlet-0.9.14/README.twisted

@@ -0,0 +1,181 @@
+--work in progress--
+
+Introduction
+------------
+Twisted provides solid foundation for asynchronous programming in Python.
+Eventlet makes asynchronous programming look like synchronous, thus
+achieving higher signal-to-noise ratio than traditional twisted programs have.
+
+Eventlet on top of twisted provides:
+ * stable twisted
+ * usable and readable synchronous style
+ * existing twisted code can be used without any changes
+ * existing blocking code can be used after trivial changes applied
+
+NOTE: the maintainer of Eventlet's Twisted support no longer supports it; it still exists but may have had some breakage along the way.  Please treat it as experimental, and if you'd like to maintain it, please do!
+
+Eventlet features:
+
+ * utilities for spawning and controlling greenlet execution:
+   api.spawn, api.kill, proc module
+ * utilities for communicating between greenlets:
+   event.Event, queue.Queue, semaphore.Semaphore
+ * standard Python modules that won't block the reactor:
+   eventlet.green package
+ * utilities specific to twisted hub:
+   eventlet.twistedutil package
+
+
+Getting started with eventlet on twisted
+----------------------------------------
+
+This section will only mention stuff that may be useful but it
+won't explain in details how to use it. For that, refer to the
+docstrings of the modules and the examples.
+
+There are 2 ways of using twisted with eventlet, one that is
+familiar to twisted developers and another that is familiar
+to eventlet developers:
+
+ 1. explicitly start the main loop in the main greenlet;
+ 2. implicitly start the main loop in a dedicated greenlet.
+
+To enable (1), add this line at the top of your program:
+from eventlet.twistedutil import join_reactor
+then start the reactor as you would do in a regular twisted application.
+
+For (2) just make sure that you have reactor installed before using
+any of eventlet functions. Otherwise a non-twisted hub will be selected
+and twisted code won't work.
+
+Most of examples/twisted_* use twisted style with the exception of
+twisted_client.py and twisted_srvconnector.py. All of the non-twisted
+examples in examples directory use eventlet-style (they work with any
+of eventlet's hubs, not just twisted-based).
+
+Eventlet implements "blocking" operations by switching to the main loop
+greenlet, thus it's impossible to call a blocking function when you are
+already in the main loop. Therefore one must be cautious in a twisted
+callback, calling only a non-blocking subset of eventlet API here. The
+following functions won't unschedule the current greenlet and are safe
+to call from anywhere:
+
+1. Greenlet creation functions: api.spawn, proc.spawn,
+   twistedutil.deferToGreenThread and others based on api.spawn.
+
+2. send(), send_exception(), poll(), ready() methods of event.Event
+   and queue.Queue.
+
+3. wait(timeout=0) is identical to poll(). Currently only Proc.wait
+   supports timeout parameter.
+
+4. Proc.link/link_value/link_exception
+
+Other classes that use these names should follow the convention.
+
+For an example on how to take advantage of eventlet in a twisted
+application using deferToGreenThread see examples/twisted_http_proxy.py
+
+Although eventlet provides eventlet.green.socket module that implements
+interface of the standard Python socket, there's also a way to use twisted's
+network code in a synchronous fashion via GreenTransport class.
+A GreenTransport interface is reminiscent of socket but it's not a drop-in
+replacement. It combines features of TCPTransport and Protocol in a single
+object:
+
+ * all of transport methods (like getPeer()) are available directly on
+   a GreenTransport instance; in addition, underlying transport object
+   is available via 'transport' attribute;
+ * write method is overriden: it may block if transport write buffer is full;
+ * read() and recv() methods are provided to retrieve the data from protocol
+   synchronously.
+
+To make a GreenTransport instance use twistedutil.protocol.GreenClientCreator
+(usage is similar to that of twisted.internet.protocol.ClientCreator)
+
+For an example on how to get a connected GreenTransport instance,
+see twisted_client.py, twisted_srvconnect.py or twisted_portforward.py.
+For an example on how to use GreenTransport for incoming connections,
+see twisted_server.py, twisted_portforward.py.
+
+
+also
+* twistedutil.block_on - wait for a deferred to fire
+  block_on(reactor.callInThread(func, args))
+* twistedutil.protocol.basic.LineOnlyReceiverTransport - a green transport
+  variant built on top of LineOnlyReceiver protocol. Demonstrates how
+  to convert a protocol to a synchronous mode.
+
+
+Coroutines
+----------
+
+To understand how eventlet works, one has to understand how to use greenlet:
+http://codespeak.net/py/dist/greenlet.html
+
+Essential points
+
+* There always exists MAIN greenlet
+* Every greenlet except MAIN has a parent. MAIN therefore could be detected as g.parent is None
+* When greenlet is finished it's return value is propagated to the parent (i.e. switch() call
+  in the parent greenlet returns it)
+* When an exception leaves a greelen, it's propagated to the parent (i.e. switch() in the parent
+  re-raises it) unless it's a subclass of GreenletExit, which is returned as a value.
+* parent can be reassigned (by simply setting 'parent' attribute). A cycle would be detected and
+  rejected with ValueError
+
+
+Note, that there's no scheduler of any sort; if a coroutine wants to be
+scheduled again it must take care of it itself. As an application developer,
+however, you don't need to worry about it as that's what eventlet does behind
+the scenes. The cost of that is that you should not use greenlet's switch() and
+throw() methods, they will likely leave the current greenlet unscheduled
+forever. Eventlet also takes advantage of greenlet's `parent' attribute,
+so you should not meddle with it either.
+
+
+How does eventlet work
+----------------------
+
+Twisted's reactor and eventlet's hub are very similar in what they do.
+Both continuously perform polling on the list of registered descriptors
+and each time a specific event is fired, the associated callback function
+is called. In addition, both maintain a list of scheduled calls.
+
+Polling is performed by the main loop - a function that both reactor and hub have.
+When twisted calls user's callback it's expected to return almost immediately,
+without any blocking I/O calls.
+
+Eventlet runs the main loop in a dedicated greenlet (MAIN_LOOP). It is the same
+greenlet as MAIN if you use join_reactor. Otherwise it's a separate greenlet
+started implicitly. The execution is organized in a such way that the switching
+always involves MAIN_LOOP. All of functions in eventlet that appear "blocking"
+use the following algorithm:
+
+1. register a callback that switches back to the current greenlet when
+   an event of interest happens
+2. switch to the MAIN_LOOP
+
+For example, here's what eventlet's socket recv() does:
+
+= blocking operation RECV on socket d =
+
+user's greenlet (USER)             main loop's greenlet (MAIN_LOOP)
+      |
+(inside d.recv() call)
+      |
+add_descriptor(d, RECV)
+      |
+data=MAIN_LOOP.switch() ---------> poll for events
+  ^---------------------\               |
+                        |              ... ---------------------------> may execute other greenlets here
+                        |               |
+                        |          event RECV on descriptor d?
+                        |               |
+                        |          d.remove_descriptor(d, RECV)
+                        |               |
+                        |          data = d.recv() # calling blocking op that will return immediately
+                        |               |
+                        \--------- USER.switch(data) # argument data here becomes return value in user's switch
+  return data
+

+ 94 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/Makefile

@@ -0,0 +1,94 @@
+# Makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = PYTHONPATH=../:$(PYTHONPATH) sphinx-build
+PAPER         =
+
+# Internal variables.
+PAPEROPT_a4     = -D latex_paper_size=a4
+PAPEROPT_letter = -D latex_paper_size=letter
+ALLSPHINXOPTS   = -d _build/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+
+.PHONY: help clean html dirhtml pickle json htmlhelp qthelp latex changes linkcheck doctest
+
+help:
+	@echo "Please use \`make <target>' where <target> is one of"
+	@echo "  html      to make standalone HTML files"
+	@echo "  dirhtml   to make HTML files named index.html in directories"
+	@echo "  pickle    to make pickle files"
+	@echo "  json      to make JSON files"
+	@echo "  htmlhelp  to make HTML files and a HTML help project"
+	@echo "  qthelp    to make HTML files and a qthelp project"
+	@echo "  latex     to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
+	@echo "  changes   to make an overview of all changed/added/deprecated items"
+	@echo "  linkcheck to check all external links for integrity"
+	@echo "  doctest   to run all doctests embedded in the documentation (if enabled)"
+	@echo "  coverage  to generate a docstring coverage report"
+
+clean:
+	-rm -rf _build/*
+
+html:
+	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) _build/html
+	@echo
+	@echo "Build finished. The HTML pages are in _build/html."
+
+dirhtml:
+	$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) _build/dirhtml
+	@echo
+	@echo "Build finished. The HTML pages are in _build/dirhtml."
+
+pickle:
+	$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) _build/pickle
+	@echo
+	@echo "Build finished; now you can process the pickle files."
+
+json:
+	$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) _build/json
+	@echo
+	@echo "Build finished; now you can process the JSON files."
+
+htmlhelp:
+	$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) _build/htmlhelp
+	@echo
+	@echo "Build finished; now you can run HTML Help Workshop with the" \
+	      ".hhp project file in _build/htmlhelp."
+
+qthelp:
+	$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) _build/qthelp
+	@echo
+	@echo "Build finished; now you can run "qcollectiongenerator" with the" \
+	      ".qhcp project file in _build/qthelp, like this:"
+	@echo "# qcollectiongenerator _build/qthelp/Eventlet.qhcp"
+	@echo "To view the help file:"
+	@echo "# assistant -collectionFile _build/qthelp/Eventlet.qhc"
+
+latex:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) _build/latex
+	@echo
+	@echo "Build finished; the LaTeX files are in _build/latex."
+	@echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \
+	      "run these through (pdf)latex."
+
+changes:
+	$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) _build/changes
+	@echo
+	@echo "The overview file is in _build/changes."
+
+linkcheck:
+	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) _build/linkcheck
+	@echo
+	@echo "Link check complete; look for any errors in the above output " \
+	      "or in _build/linkcheck/output.txt."
+
+doctest:
+	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) _build/doctest
+	@echo "Testing of doctests in the sources finished, look at the " \
+	      "results in _build/doctest/output.txt."
+	      
+coverage:
+	$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) _build/coverage
+	@echo "Coverage report finished, look at the " \
+		  "results in _build/coverage/python.txt."

BIN
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_images/threading_illustration.png


+ 4 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/authors.txt

@@ -0,0 +1,4 @@
+Authors
+=======
+
+.. include:: ../AUTHORS

+ 83 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/basic_usage.txt

@@ -0,0 +1,83 @@
+Basic Usage
+=============
+
+If it's your first time to Eventlet, you may find the illuminated examples in the :ref:`design-patterns` document to be a good starting point.
+
+Eventlet is built around the concept of green threads (i.e. coroutines, we use the terms interchangeably) that are launched to do network-related work.  Green threads differ from normal threads in two main ways:
+
+* Green threads are so cheap they are nearly free.  You do not have to conserve green threads like you would normal threads.  In general, there will be at least one green thread per network connection.
+* Green threads cooperatively yield to each other instead of preemptively being scheduled.  The major advantage from this behavior is that shared data structures don't need locks, because only if a yield is explicitly called can another green thread have access to the data structure.  It is also possible to inspect primitives such as queues to see if they have any pending data.
+
+Primary API
+===========
+
+The design goal for Eventlet's API is simplicity and readability.  You should be able to read its code and understand what it's doing.  Fewer lines of code are preferred over excessively clever implementations.  `Like Python itself <http://www.python.org/dev/peps/pep-0020/>`_, there should be one, and only one obvious way to do it in Eventlet!
+
+Though Eventlet has many modules, much of the most-used stuff is accessible simply by doing ``import eventlet``.  Here's a quick summary of the functionality available in the ``eventlet`` module, with links to more verbose documentation on each.
+
+Greenthread Spawn
+-----------------------
+
+.. function:: eventlet.spawn(func, *args, **kw)
+   
+   This launches a greenthread to call *func*.  Spawning off multiple greenthreads gets work done in parallel.  The return value from ``spawn`` is a :class:`greenthread.GreenThread` object, which can be used to retrieve the return value of *func*.  See :func:`spawn <eventlet.greenthread.spawn>` for more details.
+   
+.. function:: eventlet.spawn_n(func, *args, **kw)
+   
+   The same as :func:`spawn`, but it's not possible to know how the function terminated (i.e. no return value or exceptions).  This makes execution faster.  See :func:`spawn_n <eventlet.greenthread.spawn_n>` for more details.
+
+.. function:: eventlet.spawn_after(seconds, func, *args, **kw)
+   
+    Spawns *func* after *seconds* have elapsed; a delayed version of :func:`spawn`.   To abort the spawn and prevent *func* from being called, call :meth:`GreenThread.cancel` on the return value of :func:`spawn_after`.  See :func:`spawn_after <eventlet.greenthread.spawn_after>` for more details.
+
+Greenthread Control
+-----------------------
+
+.. function:: eventlet.sleep(seconds=0)
+
+    Suspends the current greenthread and allows others a chance to process.  See :func:`sleep <eventlet.greenthread.sleep>` for more details.
+
+.. class:: eventlet.GreenPool
+
+   Pools control concurrency.  It's very common in applications to want to consume only a finite amount of memory, or to restrict the amount of connections that one part of the code holds open so as to leave more for the rest, or to behave consistently in the face of unpredictable input data.  GreenPools provide this control.  See :class:`GreenPool <eventlet.greenpool.GreenPool>` for more on how to use these.
+
+.. class:: eventlet.GreenPile
+
+    GreenPile objects represent chunks of work.  In essence a GreenPile is an iterator that can be stuffed with work, and the results read out later. See :class:`GreenPile <eventlet.greenpool.GreenPile>` for more details.
+    
+.. class:: eventlet.Queue
+
+    Queues are a fundamental construct for communicating data between execution units.  Eventlet's Queue class is used to communicate between greenthreads, and provides a bunch of useful features for doing that.  See :class:`Queue <eventlet.queue.Queue>` for more details.
+    
+.. class:: eventlet.Timeout
+
+    This class is a way to add timeouts to anything.  It raises *exception* in the current greenthread after *timeout* seconds.  When *exception* is omitted or ``None``, the Timeout instance itself is raised.
+    
+    Timeout objects are context managers, and so can be used in with statements.
+    See :class:`Timeout <eventlet.timeout.Timeout>` for more details.
+
+Patching Functions
+---------------------
+    
+.. function:: eventlet.import_patched(modulename, *additional_modules, **kw_additional_modules)
+
+    Imports a module in a way that ensures that the module uses "green" versions of the standard library modules, so that everything works nonblockingly.  The only required argument is the name of the module to be imported.  For more information see :ref:`import-green`.
+
+.. function:: eventlet.monkey_patch(all=True, os=False, select=False, socket=False, thread=False, time=False)
+
+    Globally patches certain system modules to be greenthread-friendly. The keyword arguments afford some control over which modules are patched. If *all* is True, then all modules are patched regardless of the other arguments. If it's False, then the rest of the keyword arguments control patching of specific subsections of the standard library.  Most patch the single module of the same name (os, time, select).  The exceptions are socket, which also patches the ssl module if present; and thread, which patches thread, threading, and Queue.  It's safe to call monkey_patch multiple times.  For more information see :ref:`monkey-patch`.
+
+Network Convenience Functions
+------------------------------
+
+.. autofunction:: eventlet.connect
+
+.. autofunction:: eventlet.listen
+
+.. autofunction:: eventlet.wrap_ssl
+
+.. autofunction:: eventlet.serve
+
+.. autoclass:: eventlet.StopServe
+    
+These are the basic primitives of Eventlet; there are a lot more out there in the other Eventlet modules; check out the :doc:`modules`.

+ 113 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/design_patterns.txt

@@ -0,0 +1,113 @@
+.. _design-patterns:
+
+Design Patterns
+=================
+
+There are a bunch of basic patterns that Eventlet usage falls into.  Here are a few examples that show their basic structure.
+
+Client Pattern
+--------------------
+
+The canonical client-side example is a web crawler.  This use case is given a list of urls and wants to retrieve their bodies for later processing.  Here is a very simple example::
+
+
+  urls = ["http://www.google.com/intl/en_ALL/images/logo.gif",
+         "https://wiki.secondlife.com/w/images/secondlife.jpg",
+         "http://us.i1.yimg.com/us.yimg.com/i/ww/beta/y3.gif"]
+  
+  import eventlet
+  from eventlet.green import urllib2  
+
+  def fetch(url):
+      return urllib2.urlopen(url).read()
+  
+  pool = eventlet.GreenPool()
+  for body in pool.imap(fetch, urls):
+      print "got body", len(body)
+
+There is a slightly more complex version of this in the :ref:`web crawler example <web_crawler_example>`.  Here's a tour of the interesting lines in this crawler. 
+
+``from eventlet.green import urllib2`` is how you import a cooperatively-yielding version of urllib2.  It is the same in all respects to the standard version, except that it uses green sockets for its communication.  This is an example of the :ref:`import-green` pattern.
+
+``pool = eventlet.GreenPool()`` constructs a :class:`GreenPool <eventlet.greenpool.GreenPool>` of a thousand green threads.  Using a pool is good practice because it provides an upper limit on the amount of work that this crawler will be doing simultaneously, which comes in handy when the input data changes dramatically.
+
+``for body in pool.imap(fetch, urls):`` iterates over the results of calling the fetch function in parallel.  :meth:`imap <eventlet.greenpool.GreenPool.imap>` makes the function calls in parallel, and the results are returned in the order that they were executed.
+
+The key aspect of the client pattern is that it involves collecting the results of each function call; the fact that each fetch is done concurrently is essentially an invisible optimization.  Note also that imap is memory-bounded and won't consume gigabytes of memory if the list of urls grows to the tens of thousands (yes, we had that problem in production once!).
+
+
+Server Pattern
+--------------------
+
+Here's a simple server-side example, a simple echo server::
+    
+    import eventlet
+    
+    def handle(client):
+        while True:
+            c = client.recv(1)
+            if not c: break
+            client.sendall(c)
+    
+    server = eventlet.listen(('0.0.0.0', 6000))
+    pool = eventlet.GreenPool(10000)
+    while True:
+        new_sock, address = server.accept()
+        pool.spawn_n(handle, new_sock)
+
+The file :ref:`echo server example <echo_server_example>` contains a somewhat more robust and complex version of this example.
+
+``server = eventlet.listen(('0.0.0.0', 6000))`` uses a convenience function to create a listening socket.
+
+``pool = eventlet.GreenPool(10000)`` creates a pool of green threads that could handle ten thousand clients.  
+
+``pool.spawn_n(handle, new_sock)`` launches a green thread to handle the new client.  The accept loop doesn't care about the return value of the ``handle`` function, so it uses :meth:`spawn_n <eventlet.greenpool.GreenPool.spawn_n>`, instead of :meth:`spawn <eventlet.greenpool.GreenPool.spawn>`.
+
+The difference between the server and the client patterns boils down to the fact that the server has a ``while`` loop calling ``accept()`` repeatedly, and that it hands off the client socket completely to the handle() method, rather than collecting the results.
+
+Dispatch Pattern
+-------------------
+
+One common use case that Linden Lab runs into all the time is a "dispatch" design pattern.  This is a server that is also a client of some other services.  Proxies, aggregators, job workers, and so on are all terms that apply here.  This is the use case that the :class:`GreenPile <eventlet.greenpool.GreenPile>` was designed for.
+
+Here's a somewhat contrived example: a server that receives POSTs from clients that contain a list of urls of RSS feeds.  The server fetches all the feeds concurrently and responds with a list of their titles to the client.  It's easy to imagine it doing something more complex than this, and this could be easily modified to become a Reader-style application::
+
+    import eventlet
+    feedparser = eventlet.import_patched('feedparser')
+    
+    pool = eventlet.GreenPool()
+    
+    def fetch_title(url):
+        d = feedparser.parse(url)
+        return d.feed.get('title', '')
+    
+    def app(environ, start_response):
+        pile = eventlet.GreenPile(pool)
+        for url in environ['wsgi.input'].readlines():
+            pile.spawn(fetch_title, url)
+        titles = '\n'.join(pile)
+        start_response('200 OK', [('Content-type', 'text/plain')])
+        return [titles]
+
+The full version of this example is in the :ref:`feed_scraper_example`, which includes code to start the WSGI server on a particular port.
+
+This example uses a global (gasp) :class:`GreenPool <eventlet.greenpool.GreenPool>` to control concurrency.  If we didn't have a global limit on the number of outgoing requests, then a client could cause the server to open tens of thousands of concurrent connections to external servers, thereby getting feedscraper's IP banned, or various other accidental-or-on-purpose bad behavior.  The pool isn't a complete DoS protection, but it's the bare minimum.
+
+.. highlight:: python
+    :linenothreshold: 1
+
+The interesting lines are in the app function::
+
+    pile = eventlet.GreenPile(pool)
+    for url in environ['wsgi.input'].readlines():
+        pile.spawn(fetch_title, url)
+    titles = '\n'.join(pile)
+
+.. highlight:: python
+    :linenothreshold: 1000
+
+Note that in line 1, the Pile is constructed using the global pool as its argument.  That ties the Pile's concurrency to the global's.  If there are already 1000 concurrent fetches from other clients of feedscraper, this one will block until some of those complete.  Limitations are good!
+
+Line 3 is just a spawn, but note that we don't store any return value from it.  This is because the return value is kept in the Pile itself.  This becomes evident in the next line...
+
+Line 4 is where we use the fact that the Pile is an iterator.  Each element in the iterator is one of the return values from the fetch_title function, which are strings.  We can use a normal Python idiom (:func:`join`) to concatenate these incrementally as they happen.

+ 21 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/environment.txt

@@ -0,0 +1,21 @@
+.. _env_vars:
+
+Environment Variables
+======================
+
+Eventlet's behavior can be controlled by a few environment variables.
+These are only for the advanced user.
+
+EVENTLET_HUB 
+
+   Used to force Eventlet to use the specified hub instead of the
+   optimal one.  See :ref:`understanding_hubs` for the list of
+   acceptable hubs and what they mean (note that picking a hub not on
+   the list will silently fail).  Equivalent to calling
+   :meth:`eventlet.hubs.use_hub` at the beginning of the program.
+
+EVENTLET_THREADPOOL_SIZE
+
+   The size of the threadpool in :mod:`~eventlet.tpool`.  This is an
+   environment variable because tpool constructs its pool on first
+   use, so any control of the pool size needs to happen before then.

+ 106 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/examples.txt

@@ -0,0 +1,106 @@
+Examples
+========
+
+Here are a bunch of small example programs that use Eventlet.  All of these examples can be found in the ``examples`` directory of a source copy of Eventlet.
+
+.. _web_crawler_example:
+
+Web Crawler
+------------
+``examples/webcrawler.py``
+
+.. literalinclude:: ../examples/webcrawler.py
+
+.. _wsgi_server_example:
+
+WSGI Server
+------------
+``examples/wsgi.py``
+
+.. literalinclude:: ../examples/wsgi.py
+
+.. _echo_server_example:
+
+Echo Server
+-----------
+``examples/echoserver.py``
+
+.. literalinclude:: ../examples/echoserver.py
+
+.. _socket_connect_example:
+
+Socket Connect
+--------------
+``examples/connect.py``
+
+.. literalinclude:: ../examples/connect.py
+
+.. _chat_server_example:
+
+Multi-User Chat Server
+-----------------------
+``examples/chat_server.py``
+
+This is a little different from the echo server, in that it broadcasts the 
+messages to all participants, not just the sender.
+        
+.. literalinclude:: ../examples/chat_server.py
+
+.. _feed_scraper_example:
+
+Feed Scraper
+-----------------------
+``examples/feedscraper.py``
+
+This example requires `Feedparser <http://www.feedparser.org/>`_ to be installed or on the PYTHONPATH.
+
+.. literalinclude:: ../examples/feedscraper.py
+
+.. _forwarder_example:
+
+Port Forwarder
+-----------------------
+``examples/forwarder.py``
+
+.. literalinclude:: ../examples/forwarder.py
+
+.. _recursive_crawler_example:
+
+Recursive Web Crawler
+-----------------------------------------
+``examples/recursive_crawler.py``
+
+This is an example recursive web crawler that fetches linked pages from a seed url.
+
+.. literalinclude:: ../examples/recursive_crawler.py
+
+.. _producer_consumer_example:
+
+Producer Consumer Web Crawler
+-----------------------------------------
+``examples/producer_consumer.py``
+
+This is an example implementation of the producer/consumer pattern as well as being identical in functionality to the recursive web crawler.
+
+.. literalinclude:: ../examples/producer_consumer.py
+
+.. _websocket_example:
+
+Websocket Server Example
+--------------------------
+``examples/websocket.py``
+
+This exercises some of the features of the websocket server
+implementation.
+
+.. literalinclude:: ../examples/websocket.py
+
+.. _websocket_chat_example:
+
+Websocket Multi-User Chat Example
+-----------------------------------
+``examples/websocket_chat.py``
+
+This is a mashup of the websocket example and the multi-user chat example, showing how you can do the same sorts of things with websockets that you can do with regular sockets.
+
+.. literalinclude:: ../examples/websocket_chat.py

+ 10 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/history.txt

@@ -0,0 +1,10 @@
+History
+-------
+
+Eventlet began life as Donovan Preston was talking to Bob Ippolito about coroutine-based non-blocking networking frameworks in Python. Most non-blocking frameworks require you to run the "main loop" in order to perform all network operations, but Donovan wondered if a library written using a trampolining style could get away with transparently running the main loop any time i/o was required, stopping the main loop once no more i/o was scheduled. Bob spent a few days during PyCon 2006 writing a proof-of-concept. He named it eventlet, after the coroutine implementation it used, `greenlet <http://cheeseshop.python.org/pypi/greenlet greenlet>`_. Donovan began using eventlet as a light-weight network library for his spare-time project `Pavel <http://soundfarmer.com/Pavel/trunk/ Pavel>`_, and also began writing some unittests.
+
+* http://svn.red-bean.com/bob/eventlet/trunk/
+
+When Donovan started at Linden Lab in May of 2006, he added eventlet as an svn external in the ``indra/lib/python directory``, to be a dependency of the yet-to-be-named backbone project (at the time, it was named restserv). However, including eventlet as an svn external meant that any time the externally hosted project had hosting issues, Linden developers were not able to perform svn updates. Thus, the eventlet source was imported into the linden source tree at the same location, and became a fork.
+
+Bob Ippolito has ceased working on eventlet and has stated his desire for Linden to take it's fork forward to the open source world as "the" eventlet.

+ 54 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/hubs.txt

@@ -0,0 +1,54 @@
+.. _understanding_hubs:
+
+Understanding Eventlet Hubs
+===========================
+
+A hub forms the basis of Eventlet's event loop, which dispatches I/O events and schedules greenthreads.  It is the existence of the hub that promotes coroutines (which can be tricky to program with) into greenthreads (which are easy).
+
+Eventlet has multiple hub implementations, and when you start using it, it tries to select the best hub implementation for your system.  The hubs that it supports are (in order of preference):
+
+**epolls**
+    Requires Python 2.6 or the `python-epoll <http://pypi.python.org/pypi/python-epoll/1.0>`_ package, and Linux.  This is the fastest pure-Python hub.
+**poll**
+    On platforms that support it
+**selects**
+    Lowest-common-denominator, available everywhere.
+**pyevent**
+    This is a libevent-based backend and is thus the fastest.  It's disabled by default, because it does not support native threads, but you can enable it yourself if your use case doesn't require them.  (You have to install pyevent, too.)
+
+If the selected hub is not idea for the application, another can be selected.  You can make the selection either with the environment variable :ref:`EVENTLET_HUB <env_vars>`, or with use_hub.
+
+.. function:: eventlet.hubs.use_hub(hub=None)
+
+    Use this to control which hub Eventlet selects.  Call it with the name of the desired hub module.  Make sure to do this before the application starts doing any I/O!  Calling use_hub completely eliminates the old hub, and any file descriptors or timers that it had been managing will be forgotten.  Put the call as one of the first lines in the main module.::
+    
+        """ This is the main module """
+        from eventlet import hubs
+        hubs.use_hub("pyevent")
+    
+    Hubs are implemented as thread-local class instances.  :func:`eventlet.hubs.use_hub` only operates on the current thread.  When using multiple threads that each need their own hub, call :func:`eventlet.hubs.use_hub` at the beginning of each thread function that needs a specific hub.  In practice, it may not be necessary to specify a hub in each thread; it works to use one special hub for the main thread, and let other threads use the default hub; this hybrid hub configuration will work fine.
+    
+    It is also possible to use a third-party hub module in place of one of the built-in ones.  Simply pass the module itself to :func:`eventlet.hubs.use_hub`.  The task of writing such a hub is a little beyond the scope of this document, it's probably a good idea to simply inspect the code of the existing hubs to see how they work.::
+
+         from eventlet import hubs    
+         from mypackage import myhub
+         hubs.use_hub(myhub)
+    
+    Supplying None as the argument to :func:`eventlet.hubs.use_hub` causes it to select the default hub.
+
+
+How the Hubs Work
+-----------------
+
+The hub has a main greenlet, MAINLOOP.  When one of the running coroutines needs
+to do some I/O, it registers a listener with the hub (so that the hub knows when to wake it up again), and then switches to MAINLOOP (via ``get_hub().switch()``).  If there are other coroutines that are ready to run, MAINLOOP switches to them, and when they complete or need to do more I/O, they switch back to the MAINLOOP.  In this manner, MAINLOOP ensures that every coroutine gets scheduled when it has some work to do.
+
+MAINLOOP is launched only when the first I/O operation happens, and it is not the same greenlet that __main__ is running in.  This lazy launching is why it's not necessary to explicitly call a dispatch() method like other frameworks, which in turn means that code can start using Eventlet without needing to be substantially restructured.
+
+More Hub-Related Functions
+---------------------------
+
+.. autofunction:: eventlet.hubs.get_hub
+.. autofunction:: eventlet.hubs.get_default_hub
+.. autofunction:: eventlet.hubs.trampoline
+

+ 50 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/index.txt

@@ -0,0 +1,50 @@
+Eventlet Documentation
+====================================
+
+Code talks!  This is a simple web crawler that fetches a bunch of urls concurrently::
+
+    urls = ["http://www.google.com/intl/en_ALL/images/logo.gif",
+         "https://wiki.secondlife.com/w/images/secondlife.jpg",
+         "http://us.i1.yimg.com/us.yimg.com/i/ww/beta/y3.gif"]
+    
+    import eventlet
+    from eventlet.green import urllib2  
+    
+    def fetch(url):
+      return urllib2.urlopen(url).read()
+    
+    pool = eventlet.GreenPool()
+    for body in pool.imap(fetch, urls):
+      print "got body", len(body)
+
+Contents
+=========
+
+.. toctree::
+   :maxdepth: 2
+
+   basic_usage
+   design_patterns
+   patching
+   examples
+   ssl
+   threading
+   hubs
+   testing
+   environment
+
+   modules
+   
+   authors
+   history
+
+License
+---------
+Eventlet is made available under the terms of the open source `MIT license <http://www.opensource.org/licenses/mit-license.php>`_
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

+ 19 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules.txt

@@ -0,0 +1,19 @@
+Module Reference
+======================
+
+.. toctree::
+   :maxdepth: 2
+
+   modules/backdoor
+   modules/corolocal
+   modules/debug
+   modules/db_pool
+   modules/event
+   modules/greenpool
+   modules/greenthread
+   modules/pools
+   modules/queue
+   modules/semaphore
+   modules/timeout
+   modules/websocket
+   modules/wsgi

+ 27 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/backdoor.txt

@@ -0,0 +1,27 @@
+:mod:`backdoor` -- Python interactive interpreter within a running process
+===============================================================================
+
+The backdoor module is convenient for inspecting the state of a long-running process.  It supplies the normal Python interactive interpreter in a way that does not block the normal operation of the application.  This can be useful for debugging, performance tuning, or simply learning about how things behave in situ.
+
+In the application, spawn a greenthread running backdoor_server on a listening socket::
+    
+    eventlet.spawn(backdoor.backdoor_server, eventlet.listen(('localhost', 3000)))
+    
+When this is running, the backdoor is accessible via telnet to the specified port.
+
+.. code-block:: sh
+
+  $ telnet localhost 3000
+  Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39) 
+  [GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
+  Type "help", "copyright", "credits" or "license" for more information.
+  >>> import myapp
+  >>> dir(myapp)
+  ['__all__', '__doc__', '__name__', 'myfunc']
+  >>>
+  
+The backdoor cooperatively yields to the rest of the application between commands, so on a running server continuously serving requests, you can observe the internal state changing between interpreter commands.
+
+.. automodule:: eventlet.backdoor
+	:members:
+

+ 6 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/corolocal.txt

@@ -0,0 +1,6 @@
+:mod:`corolocal` -- Coroutine local storage
+=============================================
+
+.. automodule:: eventlet.corolocal
+	:members:
+	:undoc-members:

+ 61 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/db_pool.txt

@@ -0,0 +1,61 @@
+:mod:`db_pool` -- DBAPI 2 database connection pooling
+========================================================
+
+The db_pool module is useful for managing database connections.  It provides three primary benefits: cooperative yielding during database operations, concurrency limiting to a database host, and connection reuse.  db_pool is intended to be database-agnostic, compatible with any DB-API 2.0 database module.
+
+*It has currently been tested and used with both MySQLdb and psycopg2.*
+
+A ConnectionPool object represents a pool of connections open to a particular database.  The arguments to the constructor include the database-software-specific module, the host name, and the credentials required for authentication.  After construction, the ConnectionPool object decides when to create and sever connections with the target database.
+
+>>> import MySQLdb
+>>> cp = ConnectionPool(MySQLdb, host='localhost', user='root', passwd='')
+
+Once you have this pool object, you connect to the database by calling :meth:`~eventlet.db_pool.ConnectionPool.get` on it:
+
+>>> conn = cp.get()
+
+This call may either create a new connection, or reuse an existing open connection, depending on whether it has one open already or not.  You can then use the connection object as normal.  When done, you must return the connection to the pool:
+
+>>> conn = cp.get()
+>>> try:
+...     result = conn.cursor().execute('SELECT NOW()')
+... finally:
+...     cp.put(conn)
+
+After you've returned a connection object to the pool, it becomes useless and will raise exceptions if any of its methods are called.
+
+Constructor Arguments
+----------------------
+
+In addition to the database credentials, there are a bunch of keyword constructor arguments to the ConnectionPool that are useful.
+
+* min_size, max_size : The normal Pool arguments.  max_size is the most important constructor argument -- it determines the number of concurrent connections can be open to the destination database.  min_size is not very useful.
+* max_idle : Connections are only allowed to remain unused in the pool for a limited amount of time.  An asynchronous timer periodically wakes up and closes any connections in the pool that have been idle for longer than they are supposed to be.  Without this parameter, the pool would tend to have a 'high-water mark', where the number of connections open at a given time corresponds to the peak historical demand.  This number only has effect on the connections in the pool itself -- if you take a connection out of the pool, you can hold on to it for as long as you want.  If this is set to 0, every connection is closed upon its return to the pool.
+* max_age : The lifespan of a connection.  This works much like max_idle, but the timer is measured from the connection's creation time, and is tracked throughout the connection's life.  This means that if you take a connection out of the pool and hold on to it for some lengthy operation that exceeds max_age, upon putting the connection back in to the pool, it will be closed.  Like max_idle, max_age will not close connections that are taken out of the pool, and, if set to 0, will cause every connection to be closed when put back in the pool.
+* connect_timeout : How long to wait before raising an exception on connect().  If the database module's connect() method takes too long, it raises a ConnectTimeout exception from the get() method on the pool.
+
+DatabaseConnector
+-----------------
+
+If you want to connect to multiple databases easily (and who doesn't), the DatabaseConnector is for you.  It's a pool of pools, containing a ConnectionPool for every host you connect to.
+
+The constructor arguments are:
+
+* module : database module, e.g. MySQLdb.  This is simply passed through to the ConnectionPool.
+* credentials : A dictionary, or dictionary-alike, mapping hostname to connection-argument-dictionary.  This is used for the constructors of the ConnectionPool objects.  Example:
+
+>>> dc = DatabaseConnector(MySQLdb,
+...      {'db.internal.example.com': {'user': 'internal', 'passwd': 's33kr1t'},
+...       'localhost': {'user': 'root', 'passwd': ''}})
+
+If the credentials contain a host named 'default', then the value for 'default' is used whenever trying to connect to a host that has no explicit entry in the database.  This is useful if there is some pool of hosts that share arguments.
+
+* conn_pool : The connection pool class to use.  Defaults to db_pool.ConnectionPool.
+
+The rest of the arguments to the DatabaseConnector constructor are passed on to the ConnectionPool.
+
+*Caveat: The DatabaseConnector is a bit unfinished, it only suits a subset of use cases.*
+
+.. automodule:: eventlet.db_pool
+	:members:
+	:undoc-members:

+ 5 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/debug.txt

@@ -0,0 +1,5 @@
+:mod:`debug` -- Debugging tools for Eventlet
+==================================================
+
+.. automodule:: eventlet.debug
+	:members:

+ 5 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/event.txt

@@ -0,0 +1,5 @@
+:mod:`event` -- Cross-greenthread primitive
+==================================================
+
+.. automodule:: eventlet.event
+	:members:

+ 6 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/greenpool.txt

@@ -0,0 +1,6 @@
+:mod:`greenpool` -- Green Thread Pools
+========================================
+
+.. automodule:: eventlet.greenpool
+	:members:
+

+ 5 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/greenthread.txt

@@ -0,0 +1,5 @@
+:mod:`greenthread` -- Green Thread Implementation
+==================================================
+
+.. automodule:: eventlet.greenthread
+	:members:

+ 5 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/pools.txt

@@ -0,0 +1,5 @@
+:mod:`pools` - Generic pools of resources 
+==========================================
+
+.. automodule:: eventlet.pools
+	:members:

+ 5 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/queue.txt

@@ -0,0 +1,5 @@
+:mod:`queue` -- Queue class
+========================================
+
+.. automodule:: eventlet.queue
+	:members:

+ 11 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/semaphore.txt

@@ -0,0 +1,11 @@
+:mod:`semaphore` -- Semaphore classes
+==================================================
+
+.. autoclass:: eventlet.semaphore.Semaphore
+	:members:
+
+.. autoclass:: eventlet.semaphore.BoundedSemaphore
+	:members:
+	
+.. autoclass:: eventlet.semaphore.CappedSemaphore
+	:members:

+ 93 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/timeout.txt

@@ -0,0 +1,93 @@
+:mod:`timeout` -- Universal Timeouts
+========================================
+
+.. class:: eventlet.timeout.Timeout
+
+    Raises *exception* in the current greenthread after *timeout* seconds::
+
+        timeout = Timeout(seconds, exception)
+        try:
+            ... # execution here is limited by timeout
+        finally:
+            timeout.cancel()
+
+    When *exception* is omitted or ``None``, the :class:`Timeout` instance 
+    itself is raised:
+
+        >>> Timeout(0.1)
+        >>> eventlet.sleep(0.2)
+        Traceback (most recent call last):
+         ...
+        Timeout: 0.1 seconds
+
+    In Python 2.5 and newer, you can use the  ``with`` statement for additional 
+    convenience::
+
+        with Timeout(seconds, exception) as timeout:
+            pass # ... code block ...
+
+    This is equivalent to the try/finally block in the first example.  
+    
+    There is an additional feature when using the ``with`` statement: if 
+    *exception* is ``False``, the timeout is still raised, but the with 
+    statement suppresses it, so the code outside the with-block won't see it::
+
+        data = None
+        with Timeout(5, False):
+            data = mysock.makefile().readline()
+        if data is None:
+            ... # 5 seconds passed without reading a line
+        else:
+            ... # a line was read within 5 seconds
+            
+    As a very special case, if *seconds* is None, the timer is not scheduled, 
+    and is only useful if you're planning to raise it directly.
+
+    There are two Timeout caveats to be aware of:
+    
+    * If the code block in the try/finally or with-block never cooperatively yields, the timeout cannot be raised.  In Eventlet, this should rarely be a problem, but be aware that you cannot time out CPU-only operations with this class.
+    * If the code block catches and doesn't re-raise :class:`BaseException`  (for example, with ``except:``), then it will catch the Timeout exception, and might not abort as intended.
+
+    When catching timeouts, keep in mind that the one you catch may not be the
+    one you have set; if you going to silence a timeout, always check that it's
+    the same instance that you set::
+
+        timeout = Timeout(1)
+        try:
+            ...
+        except Timeout, t:
+            if t is not timeout:
+                raise # not my timeout
+    
+    .. automethod:: cancel
+    .. autoattribute:: pending
+
+
+.. function:: eventlet.timeout.with_timeout(seconds, function, *args, **kwds)
+
+    Wrap a call to some (yielding) function with a timeout; if the called
+    function fails to return before the timeout, cancel it and return a flag
+    value.
+
+    :param seconds: seconds before timeout occurs
+    :type seconds: int or float
+    :param func: the callable to execute with a timeout; it must cooperatively yield, or else the timeout will not be able to trigger
+    :param \*args: positional arguments to pass to *func*
+    :param \*\*kwds: keyword arguments to pass to *func*
+    :param timeout_value: value to return if timeout occurs (by default raises
+      :class:`Timeout`)
+      
+    :rtype: Value returned by *func* if *func* returns before *seconds*, else
+      *timeout_value* if provided, else raises :class:`Timeout`.
+
+    :exception Timeout: if *func* times out and no ``timeout_value`` has
+      been provided.
+    :exception: Any exception raised by *func*
+
+    Example::
+
+        data = with_timeout(30, urllib2.open, 'http://www.google.com/', timeout_value="")
+
+    Here *data* is either the result of the ``get()`` call, or the empty string 
+    if it took too long to return.  Any exception raised by the ``get()`` call 
+    is passed through to the caller.

+ 31 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/websocket.txt

@@ -0,0 +1,31 @@
+:mod:`websocket` -- Websocket Server
+=====================================
+
+This module provides a simple way to create a `websocket
+<http://dev.w3.org/html5/websockets/>`_ server.  It works with a few
+tweaks in the :mod:`~eventlet.wsgi` module that allow websockets to
+coexist with other WSGI applications.
+
+To create a websocket server, simply decorate a handler method with
+:class:`WebSocketWSGI` and use it as a wsgi application::
+
+    from eventlet import wsgi, websocket
+    import eventlet
+    
+    @websocket.WebSocketWSGI
+    def hello_world(ws):
+        ws.send("hello world")
+    
+    wsgi.server(eventlet.listen(('', 8090)), hello_world)
+
+
+You can find a slightly more elaborate version of this code in the file
+``examples/websocket.py``.
+
+As of version 0.9.13, eventlet.websocket supports SSL websockets; all that's necessary is to use an :ref:`SSL wsgi server <wsgi_ssl>`.
+
+.. note :: The web socket spec is still under development, and it will be necessary to change the way that this module works in response to spec changes.
+
+
+.. automodule:: eventlet.websocket
+	:members:

+ 74 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/modules/wsgi.txt

@@ -0,0 +1,74 @@
+:mod:`wsgi` -- WSGI server
+===========================
+
+The wsgi module provides a simple and easy way to start an event-driven 
+`WSGI <http://wsgi.org/wsgi/>`_ server.  This can serve as an embedded
+web server in an application, or as the basis for a more full-featured web
+server package.  One such package is `Spawning <http://pypi.python.org/pypi/Spawning/>`_.
+
+To launch a wsgi server, simply create a socket and call :func:`eventlet.wsgi.server` with it::
+
+    from eventlet import wsgi
+    import eventlet
+    
+    def hello_world(env, start_response):
+        start_response('200 OK', [('Content-Type', 'text/plain')])
+        return ['Hello, World!\r\n']
+    
+    wsgi.server(eventlet.listen(('', 8090)), hello_world)
+
+
+You can find a slightly more elaborate version of this code in the file
+``examples/wsgi.py``.
+
+.. automodule:: eventlet.wsgi
+	:members:
+
+.. _wsgi_ssl:
+
+SSL
+---
+
+Creating a secure server is only slightly more involved than the base example.  All that's needed is to pass an SSL-wrapped socket to the :func:`~eventlet.wsgi.server` method::
+
+    wsgi.server(eventlet.wrap_ssl(eventlet.listen(('', 8090)),
+                                  certfile='cert.crt',
+                                  keyfile='private.key',
+                                  server_side=True),
+                hello_world)
+
+Applications can detect whether they are inside a secure server by the value of the ``env['wsgi.url_scheme']`` environment variable.
+
+
+Non-Standard Extension to Support Post Hooks
+--------------------------------------------
+Eventlet's WSGI server supports a non-standard extension to the WSGI
+specification where :samp:`env['eventlet.posthooks']` contains an array of
+`post hooks` that will be called after fully sending a response. Each post hook
+is a tuple of :samp:`(func, args, kwargs)` and the `func` will be called with
+the WSGI environment dictionary, followed by the `args` and then the `kwargs`
+in the post hook.
+
+For example::
+
+    from eventlet import wsgi
+    import eventlet
+
+    def hook(env, arg1, arg2, kwarg3=None, kwarg4=None):
+        print 'Hook called: %s %s %s %s %s' % (env, arg1, arg2, kwarg3, kwarg4)
+    
+    def hello_world(env, start_response):
+        env['eventlet.posthooks'].append(
+            (hook, ('arg1', 'arg2'), {'kwarg3': 3, 'kwarg4': 4}))
+        start_response('200 OK', [('Content-Type', 'text/plain')])
+        return ['Hello, World!\r\n']
+    
+    wsgi.server(eventlet.listen(('', 8090)), hello_world)
+
+The above code will print the WSGI environment and the other passed function
+arguments for every request processed.
+
+Post hooks are useful when code needs to be executed after a response has been
+fully sent to the client (or when the client disconnects early). One example is
+for more accurate logging of bandwidth used, as client disconnects use less
+bandwidth than the actual Content-Length.

+ 70 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/patching.txt

@@ -0,0 +1,70 @@
+Greening The World
+==================
+
+One of the challenges of writing a library like Eventlet is that the built-in networking libraries don't natively support the sort of cooperative yielding that we need.  What we must do instead is patch standard library modules in certain key places so that they do cooperatively yield.  We've in the past considered doing this automatically upon importing Eventlet, but have decided against that course of action because it is un-Pythonic to change the behavior of module A simply by importing module B.
+
+Therefore, the application using Eventlet must explicitly green the world for itself, using one or both of the convenient methods provided.
+
+.. _import-green:
+
+Import Green
+--------------
+
+The first way of greening an application is to import networking-related libraries from the ``eventlet.green`` package.  It contains libraries that have the same interfaces as common standard ones, but they are modified to behave well with green threads.  Using this method is a good engineering practice, because the true dependencies are apparent in every file::
+
+  from eventlet.green import socket
+  from eventlet.green import threading
+  from eventlet.green import asyncore
+  
+This works best if every library can be imported green in this manner.  If ``eventlet.green`` lacks a module (for example, non-python-standard modules), then :func:`~eventlet.patcher.import_patched` function can come to the rescue.  It is a replacement for the builtin import statement that greens any module on import.
+
+.. function:: eventlet.patcher.import_patched(module_name, *additional_modules, **kw_additional_modules)
+
+    Imports a module in a greened manner, so that the module's use of networking libraries like socket will use Eventlet's green versions instead.  The only required argument is the name of the module to be imported::
+    
+        import eventlet
+        httplib2 = eventlet.import_patched('httplib2')
+        
+    Under the hood, it works by temporarily swapping out the "normal" versions of the libraries in sys.modules for an eventlet.green equivalent.  When the import of the to-be-patched module completes, the state of sys.modules is restored.  Therefore, if the patched module contains the statement 'import socket', import_patched will have it reference eventlet.green.socket.  One weakness of this approach is that it doesn't work for late binding (i.e. imports that happen during runtime).  Late binding of imports is fortunately rarely done (it's slow and against `PEP-8 <http://www.python.org/dev/peps/pep-0008/>`_), so in most cases import_patched will work just fine.
+    
+    One other aspect of import_patched is the ability to specify exactly which modules are patched.  Doing so may provide a slight performance benefit since only the needed modules are imported, whereas import_patched with no arguments imports a bunch of modules in case they're needed.  The *additional_modules* and *kw_additional_modules* arguments are both sequences of name/module pairs.  Either or both can be used::
+    
+        from eventlet.green import socket
+        from eventlet.green import SocketServer        
+        BaseHTTPServer = eventlet.import_patched('BaseHTTPServer',
+                                ('socket', socket),
+                                ('SocketServer', SocketServer))
+        BaseHTTPServer = eventlet.import_patched('BaseHTTPServer',
+                                socket=socket, SocketServer=SocketServer)
+
+.. _monkey-patch:
+
+Monkeypatching the Standard Library
+----------------------------------------
+
+The other way of greening an application is simply to monkeypatch the standard
+library.  This has the disadvantage of appearing quite magical, but the advantage of avoiding the late-binding problem.
+
+.. function:: eventlet.patcher.monkey_patch(os=None, select=None, socket=None, thread=None, time=None, psycopg=None)
+
+    This function monkeypatches the key system modules by replacing their key elements with green equivalents.  If no arguments are specified, everything is patched::
+    
+        import eventlet
+        eventlet.monkey_patch()
+
+    The keyword arguments afford some control over which modules are patched, in case that's important.  Most patch the single module of the same name (e.g. time=True means that the time module is patched [time.sleep is patched by eventlet.sleep]).  The exceptions to this rule are *socket*, which also patches the :mod:`ssl` module if present; and *thread*, which patches :mod:`thread`, :mod:`threading`, and :mod:`Queue`.
+    
+    Here's an example of using monkey_patch to patch only a few modules::
+    
+        import eventlet
+        eventlet.monkey_patch(socket=True, select=True)
+         
+    It is important to call :func:`~eventlet.patcher.monkey_patch` as early in the lifetime of the application as possible.  Try to do it as one of the first lines in the main module.  The reason for this is that sometimes there is a class that inherits from a class that needs to be greened -- e.g. a class that inherits from socket.socket -- and inheritance is done at import time, so therefore the monkeypatching should happen before the derived class is defined.      It's safe to call monkey_patch multiple times.
+
+    The psycopg monkeypatching relies on Daniele Varrazzo's green psycopg2 branch; see `the announcement <https://lists.secondlife.com/pipermail/eventletdev/2010-April/000800.html>`_ for more information.
+
+.. function:: eventlet.patcher.is_monkey_patched(module)
+
+   Returns whether or not the specified module is currently monkeypatched. *module* can either be the module itself or the module's name.
+
+    Based entirely off the name of the module, so if you import a module some other way than with the import keyword (including :func:`~eventlet.patcher.import_patched`), is_monkey_patched might not be correct about that particular module.

+ 58 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/ssl.txt

@@ -0,0 +1,58 @@
+Using SSL With Eventlet
+========================
+
+Eventlet makes it easy to use non-blocking SSL sockets.  If you're using Python 2.6 or later, you're all set, eventlet wraps the built-in ssl module.  If on Python 2.5 or 2.4, you have to install pyOpenSSL_ to use eventlet.
+
+In either case, the the ``green`` modules handle SSL sockets transparently, just like their standard counterparts.  As an example, :mod:`eventlet.green.urllib2` can be used to fetch https urls in as non-blocking a fashion as you please::
+
+    from eventlet.green import urllib2
+    from eventlet import coros
+    bodies = [coros.execute(urllib2.urlopen, url)
+         for url in ("https://secondlife.com","https://google.com")]
+    for b in bodies:
+        print b.wait().read()
+ 
+ 
+With Python 2.6
+----------------
+
+To use ssl sockets directly in Python 2.6, use :mod:`eventlet.green.ssl`, which is a non-blocking wrapper around the standard Python :mod:`ssl` module, and which has the same interface.  See the standard documentation for instructions on use.
+
+With Python 2.5 or Earlier
+---------------------------
+
+Prior to Python 2.6, there is no :mod:`ssl`, so SSL support is much weaker.  Eventlet relies on pyOpenSSL to implement its SSL support on these older versions, so be sure to install pyOpenSSL, or you'll get an ImportError whenever your system tries to make an SSL connection.
+
+Once pyOpenSSL is installed, you can then use the ``eventlet.green`` modules, like :mod:`eventlet.green.httplib` to fetch https urls.  You can also use :func:`eventlet.green.socket.ssl`, which is a nonblocking wrapper for :func:`socket.ssl`.
+
+PyOpenSSL
+----------
+
+:mod:`eventlet.green.OpenSSL` has exactly the same interface as pyOpenSSL_ `(docs) <http://pyopenssl.sourceforge.net/pyOpenSSL.html/>`_, and works in all versions of Python.  This module is much more powerful than :func:`socket.ssl`, and may have some advantages over :mod:`ssl`, depending on your needs.
+
+Here's an example of a server::
+
+    from eventlet.green import socket
+    from eventlet.green.OpenSSL import SSL
+    
+    # insecure context, only for example purposes
+    context = SSL.Context(SSL.SSLv23_METHOD)
+    context.set_verify(SSL.VERIFY_NONE, lambda *x: True))
+
+    # create underlying green socket and wrap it in ssl
+    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    connection = SSL.Connection(context, sock)
+    
+    # configure as server
+    connection.set_accept_state()
+    connection.bind(('127.0.0.1', 80443))
+    connection.listen(50)
+    
+    # accept one client connection then close up shop
+    client_conn, addr = connection.accept()
+    print client_conn.read(100)
+    client_conn.shutdown()
+    client_conn.close()
+    connection.close()
+
+.. _pyOpenSSL: https://launchpad.net/pyopenssl

+ 96 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/testing.txt

@@ -0,0 +1,96 @@
+Testing Eventlet
+================
+
+Eventlet is tested using `Nose <http://somethingaboutorange.com/mrl/projects/nose/>`_.  To run tests, simply install nose, and then, in the eventlet tree, do:
+
+.. code-block:: sh
+
+  $ python setup.py test
+  
+If you want access to all the nose plugins via command line, you can run:
+
+.. code-block:: sh
+
+  $ python setup.py nosetests
+
+Lastly, you can just use nose directly if you want:
+
+.. code-block:: sh
+
+  $ nosetests
+
+That's it!  The output from running nose is the same as unittest's output, if the entire directory was one big test file.
+
+Many tests are skipped based on environmental factors; for example, it makes no sense to test Twisted-specific functionality when Twisted is not installed.  These are printed as S's during execution, and in the summary printed after the tests run it will tell you how many were skipped.
+
+.. note:: If running Python version 2.4, use this command instead: ``python tests/nosewrapper.py``.  There are several tests which make use of the `with` statement and therefore will cause nose grief when it tries to import them; nosewrapper.py excludes these tests so they are skipped.
+
+Doctests
+--------
+
+To run the doctests included in many of the eventlet modules, use this command:
+
+.. code-block :: sh
+
+  $ nosetests --with-doctest eventlet/*.py
+  
+Currently there are 16 doctests.
+
+Standard Library Tests
+----------------------
+
+Eventlet provides for the ability to test itself with the standard Python networking tests.  This verifies that the libraries it wraps work at least as well as the standard ones do.  The directory tests/stdlib contains a bunch of stubs that import the standard lib tests from your system and run them.  If you do not have any tests in your python distribution, they'll simply fail to import.
+
+There's a convenience module called all.py designed to handle the impedance mismatch between Nose and the standard tests:
+
+.. code-block:: sh
+
+  $ nosetests tests/stdlib/all.py
+  
+That will run all the tests, though the output will be a little weird because it will look like Nose is running about 20 tests, each of which consists of a bunch of sub-tests.  Not all test modules are present in all versions of Python, so there will be an occasional printout of "Not importing %s, it doesn't exist in this installation/version of Python".
+
+If you see "Ran 0 tests in 0.001s", it means that your Python installation lacks its own tests.  This is usually the case for Linux distributions.  One way to get the missing tests is to download a source tarball (of the same version you have installed on your system!) and copy its Lib/test directory into the correct place on your PYTHONPATH.
+
+
+Testing Eventlet Hubs
+---------------------
+
+When you run the tests, Eventlet will use the most appropriate hub for the current platform to do its dispatch.  It's sometimes useful when making changes to Eventlet to test those changes on hubs other than the default.  You can do this with the ``EVENTLET_HUB`` environment variable.
+
+.. code-block:: sh
+
+ $ EVENTLET_HUB=epolls nosetests
+
+See :ref:`understanding_hubs` for the full list of hubs.
+
+
+Writing Tests
+-------------
+
+What follows are some notes on writing tests, in no particular order.
+
+The filename convention when writing a test for module `foo` is to name the test `foo_test.py`.  We don't yet have a convention for tests that are of finer granularity, but a sensible one might be `foo_class_test.py`.
+
+If you are writing a test that involves a client connecting to a spawned server, it is best to not use a hardcoded port because that makes it harder to parallelize tests.  Instead bind the server to 0, and then look up its port when connecting the client, like this::
+
+  server_sock = eventlet.listener(('127.0.0.1', 0))
+  client_sock = eventlet.connect(('localhost', server_sock.getsockname()[1]))
+  
+Coverage
+--------
+
+Coverage.py is an awesome tool for evaluating how much code was exercised by unit tests.  Nose supports it if both are installed, so it's easy to generate coverage reports for eventlet.  Here's how:
+
+.. code-block:: sh
+
+ nosetests --with-coverage --cover-package=eventlet
+ 
+After running the tests to completion, this will emit a huge wodge of module names and line numbers.  For some reason, the ``--cover-inclusive`` option breaks everything rather than serving its purpose of limiting the coverage to the local files, so don't use that.
+
+The html option is quite useful because it generates nicely-formatted HTML that are much easier to read than line-number soup.  Here's a command that generates the annotation, dumping the html files into a directory called "cover":
+
+.. code-block:: sh
+
+  coverage html -d cover --omit='tempmod,<console>,tests'
+ 
+(``tempmod`` and ``console`` are omitted because they gets thrown away at the completion of their unit tests and coverage.py isn't smart enough to detect this.)

+ 30 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_sources/threading.txt

@@ -0,0 +1,30 @@
+Threads
+========
+
+Eventlet is thread-safe and can be used in conjunction with normal Python threads.  The way this works is that coroutines are confined to their 'parent' Python thread.  It's like each thread contains its own little world of coroutines that can switch between themselves but not between coroutines in other threads.  
+
+.. image:: /images/threading_illustration.png
+
+You can only communicate cross-thread using the "real" thread primitives and pipes.  Fortunately, there's little reason to use threads for concurrency when you're already using coroutines.
+
+The vast majority of the times you'll want to use threads are to wrap some operation that is not "green", such as a C library that uses its own OS calls to do socket operations.  The :mod:`~eventlet.tpool` module is provided to make these uses simpler.
+
+The optional :ref:`pyevent hub <understanding_hubs>` is not compatible with threads.
+
+Tpool - Simple thread pool
+---------------------------
+
+The simplest thing to do with :mod:`~eventlet.tpool` is to :func:`~eventlet.tpool.execute` a function with it.  The function will be run in a random thread in the pool, while the calling coroutine blocks on its completion::
+
+ >>> import thread
+ >>> from eventlet import tpool
+ >>> def my_func(starting_ident):
+ ...     print "running in new thread:", starting_ident != thread.get_ident()
+ ...
+ >>> tpool.execute(my_func, thread.get_ident())
+ running in new thread: True
+
+By default there are 20 threads in the pool, but you can configure this by setting the environment variable ``EVENTLET_THREADPOOL_SIZE`` to the desired pool size before importing tpool.
+
+.. automodule:: eventlet.tpool
+	:members:

BIN
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_static/file.png


BIN
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_static/minus.png


BIN
desktop/core/ext-py/eventlet-0.9.14/doc/_build/html/_static/plus.png


+ 4 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/authors.rst

@@ -0,0 +1,4 @@
+Authors
+=======
+
+.. include:: ../AUTHORS

+ 83 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/basic_usage.rst

@@ -0,0 +1,83 @@
+Basic Usage
+=============
+
+If it's your first time to Eventlet, you may find the illuminated examples in the :ref:`design-patterns` document to be a good starting point.
+
+Eventlet is built around the concept of green threads (i.e. coroutines, we use the terms interchangeably) that are launched to do network-related work.  Green threads differ from normal threads in two main ways:
+
+* Green threads are so cheap they are nearly free.  You do not have to conserve green threads like you would normal threads.  In general, there will be at least one green thread per network connection.
+* Green threads cooperatively yield to each other instead of preemptively being scheduled.  The major advantage from this behavior is that shared data structures don't need locks, because only if a yield is explicitly called can another green thread have access to the data structure.  It is also possible to inspect primitives such as queues to see if they have any pending data.
+
+Primary API
+===========
+
+The design goal for Eventlet's API is simplicity and readability.  You should be able to read its code and understand what it's doing.  Fewer lines of code are preferred over excessively clever implementations.  `Like Python itself <http://www.python.org/dev/peps/pep-0020/>`_, there should be one, and only one obvious way to do it in Eventlet!
+
+Though Eventlet has many modules, much of the most-used stuff is accessible simply by doing ``import eventlet``.  Here's a quick summary of the functionality available in the ``eventlet`` module, with links to more verbose documentation on each.
+
+Greenthread Spawn
+-----------------------
+
+.. function:: eventlet.spawn(func, *args, **kw)
+   
+   This launches a greenthread to call *func*.  Spawning off multiple greenthreads gets work done in parallel.  The return value from ``spawn`` is a :class:`greenthread.GreenThread` object, which can be used to retrieve the return value of *func*.  See :func:`spawn <eventlet.greenthread.spawn>` for more details.
+   
+.. function:: eventlet.spawn_n(func, *args, **kw)
+   
+   The same as :func:`spawn`, but it's not possible to know how the function terminated (i.e. no return value or exceptions).  This makes execution faster.  See :func:`spawn_n <eventlet.greenthread.spawn_n>` for more details.
+
+.. function:: eventlet.spawn_after(seconds, func, *args, **kw)
+   
+    Spawns *func* after *seconds* have elapsed; a delayed version of :func:`spawn`.   To abort the spawn and prevent *func* from being called, call :meth:`GreenThread.cancel` on the return value of :func:`spawn_after`.  See :func:`spawn_after <eventlet.greenthread.spawn_after>` for more details.
+
+Greenthread Control
+-----------------------
+
+.. function:: eventlet.sleep(seconds=0)
+
+    Suspends the current greenthread and allows others a chance to process.  See :func:`sleep <eventlet.greenthread.sleep>` for more details.
+
+.. class:: eventlet.GreenPool
+
+   Pools control concurrency.  It's very common in applications to want to consume only a finite amount of memory, or to restrict the amount of connections that one part of the code holds open so as to leave more for the rest, or to behave consistently in the face of unpredictable input data.  GreenPools provide this control.  See :class:`GreenPool <eventlet.greenpool.GreenPool>` for more on how to use these.
+
+.. class:: eventlet.GreenPile
+
+    GreenPile objects represent chunks of work.  In essence a GreenPile is an iterator that can be stuffed with work, and the results read out later. See :class:`GreenPile <eventlet.greenpool.GreenPile>` for more details.
+    
+.. class:: eventlet.Queue
+
+    Queues are a fundamental construct for communicating data between execution units.  Eventlet's Queue class is used to communicate between greenthreads, and provides a bunch of useful features for doing that.  See :class:`Queue <eventlet.queue.Queue>` for more details.
+    
+.. class:: eventlet.Timeout
+
+    This class is a way to add timeouts to anything.  It raises *exception* in the current greenthread after *timeout* seconds.  When *exception* is omitted or ``None``, the Timeout instance itself is raised.
+    
+    Timeout objects are context managers, and so can be used in with statements.
+    See :class:`Timeout <eventlet.timeout.Timeout>` for more details.
+
+Patching Functions
+---------------------
+    
+.. function:: eventlet.import_patched(modulename, *additional_modules, **kw_additional_modules)
+
+    Imports a module in a way that ensures that the module uses "green" versions of the standard library modules, so that everything works nonblockingly.  The only required argument is the name of the module to be imported.  For more information see :ref:`import-green`.
+
+.. function:: eventlet.monkey_patch(all=True, os=False, select=False, socket=False, thread=False, time=False)
+
+    Globally patches certain system modules to be greenthread-friendly. The keyword arguments afford some control over which modules are patched. If *all* is True, then all modules are patched regardless of the other arguments. If it's False, then the rest of the keyword arguments control patching of specific subsections of the standard library.  Most patch the single module of the same name (os, time, select).  The exceptions are socket, which also patches the ssl module if present; and thread, which patches thread, threading, and Queue.  It's safe to call monkey_patch multiple times.  For more information see :ref:`monkey-patch`.
+
+Network Convenience Functions
+------------------------------
+
+.. autofunction:: eventlet.connect
+
+.. autofunction:: eventlet.listen
+
+.. autofunction:: eventlet.wrap_ssl
+
+.. autofunction:: eventlet.serve
+
+.. autoclass:: eventlet.StopServe
+    
+These are the basic primitives of Eventlet; there are a lot more out there in the other Eventlet modules; check out the :doc:`modules`.

+ 4 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/common.txt

@@ -0,0 +1,4 @@
+.. |internal| replace::
+
+        This is considered an internal API, and it might change 
+        unexpectedly without being deprecated first.

+ 203 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/conf.py

@@ -0,0 +1,203 @@
+# -*- coding: utf-8 -*-
+#
+# Eventlet documentation build configuration file, created by
+# sphinx-quickstart on Sat Jul  4 19:48:27 2009.
+#
+# This file is execfile()d with the current directory set to its containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+import sys, os
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#sys.path.append(os.path.abspath('.'))
+
+# -- General configuration -----------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be extensions
+# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
+extensions = ['sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx.ext.coverage', 
+              'sphinx.ext.intersphinx']
+
+# If this is True, '.. todo::' and '.. todolist::' produce output, else they produce
+# nothing. The default is False.
+todo_include_todos = True
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix of source filenames.
+source_suffix = '.rst'
+
+# The encoding of source files.
+#source_encoding = 'utf-8'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = u'Eventlet'
+copyright = u'2005-2010, Eventlet Contributors'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+import eventlet
+# The short X.Y version.
+version = '%s.%s' % (eventlet.version_info[0], eventlet.version_info[1])
+# The full version, including alpha/beta/rc tags.
+release = eventlet.__version__
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of documents that shouldn't be included in the build.
+#unused_docs = []
+
+# List of directories, relative to source directory, that shouldn't be searched
+# for source files.
+exclude_trees = ['_build']
+
+# The reST default role (used for this markup: `text`) to use for all documents.
+#default_role = None
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#add_module_names = True
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+# Intersphinx references
+intersphinx_mapping = {'http://docs.python.org/': None}
+
+
+# -- Options for HTML output ---------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  Major themes that come with
+# Sphinx are currently 'default' and 'sphinxdoc'.
+html_theme = 'default'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#html_theme_options = {}
+
+# Add any paths that contain custom themes here, relative to this directory.
+#html_theme_path = []
+
+# The name for this set of Sphinx documents.  If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar.  Default is the same as html_title.
+#html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#html_logo = None
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#html_favicon = None
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+#html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_use_modindex = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+#html_show_sourcelink = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it.  The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = ''
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'Eventletdoc'
+
+
+# -- Options for LaTeX output --------------------------------------------------
+
+# The paper size ('letter' or 'a4').
+#latex_paper_size = 'letter'
+
+# The font size ('10pt', '11pt' or '12pt').
+#latex_font_size = '10pt'
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title, author, documentclass [howto/manual]).
+latex_documents = [
+  ('index', 'Eventlet.tex', u'Eventlet Documentation',
+   u'<eventlet contributors>', 'manual'),
+]
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+#latex_logo = None
+
+# For "manual" documents, if this is true, then toplevel headings are parts,
+# not chapters.
+#latex_use_parts = False
+
+# Additional stuff for the LaTeX preamble.
+#latex_preamble = ''
+
+# Documents to append as an appendix to all manuals.
+#latex_appendices = []
+
+# If false, no module index is generated.
+#latex_use_modindex = True

+ 113 - 0
desktop/core/ext-py/eventlet-0.9.14/doc/design_patterns.rst

@@ -0,0 +1,113 @@
+.. _design-patterns:
+
+Design Patterns
+=================
+
+There are a bunch of basic patterns that Eventlet usage falls into.  Here are a few examples that show their basic structure.
+
+Client Pattern
+--------------------
+
+The canonical client-side example is a web crawler.  This use case is given a list of urls and wants to retrieve their bodies for later processing.  Here is a very simple example::
+
+
+  urls = ["http://www.google.com/intl/en_ALL/images/logo.gif",
+         "https://wiki.secondlife.com/w/images/secondlife.jpg",
+         "http://us.i1.yimg.com/us.yimg.com/i/ww/beta/y3.gif"]
+  
+  import eventlet
+  from eventlet.green import urllib2  
+
+  def fetch(url):
+      return urllib2.urlopen(url).read()
+  
+  pool = eventlet.GreenPool()
+  for body in pool.imap(fetch, urls):
+      print "got body", len(body)
+
+There is a slightly more complex version of this in the :ref:`web crawler example <web_crawler_example>`.  Here's a tour of the interesting lines in this crawler. 
+
+``from eventlet.green import urllib2`` is how you import a cooperatively-yielding version of urllib2.  It is the same in all respects to the standard version, except that it uses green sockets for its communication.  This is an example of the :ref:`import-green` pattern.
+
+``pool = eventlet.GreenPool()`` constructs a :class:`GreenPool <eventlet.greenpool.GreenPool>` of a thousand green threads.  Using a pool is good practice because it provides an upper limit on the amount of work that this crawler will be doing simultaneously, which comes in handy when the input data changes dramatically.
+
+``for body in pool.imap(fetch, urls):`` iterates over the results of calling the fetch function in parallel.  :meth:`imap <eventlet.greenpool.GreenPool.imap>` makes the function calls in parallel, and the results are returned in the order that they were executed.
+
+The key aspect of the client pattern is that it involves collecting the results of each function call; the fact that each fetch is done concurrently is essentially an invisible optimization.  Note also that imap is memory-bounded and won't consume gigabytes of memory if the list of urls grows to the tens of thousands (yes, we had that problem in production once!).
+
+
+Server Pattern
+--------------------
+
+Here's a simple server-side example, a simple echo server::
+    
+    import eventlet
+    
+    def handle(client):
+        while True:
+            c = client.recv(1)
+            if not c: break
+            client.sendall(c)
+    
+    server = eventlet.listen(('0.0.0.0', 6000))
+    pool = eventlet.GreenPool(10000)
+    while True:
+        new_sock, address = server.accept()
+        pool.spawn_n(handle, new_sock)
+
+The file :ref:`echo server example <echo_server_example>` contains a somewhat more robust and complex version of this example.
+
+``server = eventlet.listen(('0.0.0.0', 6000))`` uses a convenience function to create a listening socket.
+
+``pool = eventlet.GreenPool(10000)`` creates a pool of green threads that could handle ten thousand clients.  
+
+``pool.spawn_n(handle, new_sock)`` launches a green thread to handle the new client.  The accept loop doesn't care about the return value of the ``handle`` function, so it uses :meth:`spawn_n <eventlet.greenpool.GreenPool.spawn_n>`, instead of :meth:`spawn <eventlet.greenpool.GreenPool.spawn>`.
+
+The difference between the server and the client patterns boils down to the fact that the server has a ``while`` loop calling ``accept()`` repeatedly, and that it hands off the client socket completely to the handle() method, rather than collecting the results.
+
+Dispatch Pattern
+-------------------
+
+One common use case that Linden Lab runs into all the time is a "dispatch" design pattern.  This is a server that is also a client of some other services.  Proxies, aggregators, job workers, and so on are all terms that apply here.  This is the use case that the :class:`GreenPile <eventlet.greenpool.GreenPile>` was designed for.
+
+Here's a somewhat contrived example: a server that receives POSTs from clients that contain a list of urls of RSS feeds.  The server fetches all the feeds concurrently and responds with a list of their titles to the client.  It's easy to imagine it doing something more complex than this, and this could be easily modified to become a Reader-style application::
+
+    import eventlet
+    feedparser = eventlet.import_patched('feedparser')
+    
+    pool = eventlet.GreenPool()
+    
+    def fetch_title(url):
+        d = feedparser.parse(url)
+        return d.feed.get('title', '')
+    
+    def app(environ, start_response):
+        pile = eventlet.GreenPile(pool)
+        for url in environ['wsgi.input'].readlines():
+            pile.spawn(fetch_title, url)
+        titles = '\n'.join(pile)
+        start_response('200 OK', [('Content-type', 'text/plain')])
+        return [titles]
+
+The full version of this example is in the :ref:`feed_scraper_example`, which includes code to start the WSGI server on a particular port.
+
+This example uses a global (gasp) :class:`GreenPool <eventlet.greenpool.GreenPool>` to control concurrency.  If we didn't have a global limit on the number of outgoing requests, then a client could cause the server to open tens of thousands of concurrent connections to external servers, thereby getting feedscraper's IP banned, or various other accidental-or-on-purpose bad behavior.  The pool isn't a complete DoS protection, but it's the bare minimum.
+
+.. highlight:: python
+    :linenothreshold: 1
+
+The interesting lines are in the app function::
+
+    pile = eventlet.GreenPile(pool)
+    for url in environ['wsgi.input'].readlines():
+        pile.spawn(fetch_title, url)
+    titles = '\n'.join(pile)
+
+.. highlight:: python
+    :linenothreshold: 1000
+
+Note that in line 1, the Pile is constructed using the global pool as its argument.  That ties the Pile's concurrency to the global's.  If there are already 1000 concurrent fetches from other clients of feedscraper, this one will block until some of those complete.  Limitations are good!
+
+Line 3 is just a spawn, but note that we don't store any return value from it.  This is because the return value is kept in the Pile itself.  This becomes evident in the next line...
+
+Line 4 is where we use the fact that the Pile is an iterator.  Each element in the iterator is one of the return values from the fetch_title function, which are strings.  We can use a normal Python idiom (:func:`join`) to concatenate these incrementally as they happen.

برخی فایل ها در این مقایسه diff نمایش داده نمی شوند زیرا تعداد فایل ها بسیار زیاد است