[minion-cvs] More work on multithreaded server, other goodies. It s...

[nickm@seul.org (Nick Mathewson)]

Update of /home/minion/cvsroot/src/minion
In directory moria.mit.edu:/tmp/cvs-serv10789

Modified Files:
	TODO setup.py 
Log Message:
More work on multithreaded server, other goodies.  It seems more stable now.

setup.py:
- Remove old mixminion/server/Queue.py where found

ClientMain, ServerMain:
- Don't use stderr if it isn't an error.

Common:
- Call 'Queue' 'MessageQueue' to avoid potential confusion.

Main: 
- Add a disclaimer to the main usage message

Modules, PacketHandler, ServerMain:
- Refactor how we pass packets around so as to move message decoding into the
  processing thread:  Objects are beautiful; tuples are kludgey.

ServerMain:
- Document more of the multithreaded stuff
- refactor
- make locking more sane
- handle queues with obsolete tuple-based messages.
- Remove old comment.
- Reset logs on sighup
- Check for thread liveness (this may be a bad idea)

ServerQueue:
- Simplify clean logic
- make locking more sane.
- Be more robust on erroneous queue state.

test, benchmark:
- Update tests to use new interfaces



Index: TODO
===================================================================
RCS file: /home/minion/cvsroot/src/minion/TODO,v
retrieving revision 1.62
retrieving revision 1.63
diff -u -d -r1.62 -r1.63
--- TODO	9 Jan 2003 06:55:11 -0000	1.62
+++ TODO	10 Jan 2003 20:12:04 -0000	1.63
@@ -24,20 +24,20 @@
                 - Read Base64-encoded messages and zbomb messages.
                 - Faster (adjustable?) timeout on client connect failure.
                 - Client-side pooling.
-        - Multithreaded server for better network behavior (no 10ms
+        . Multithreaded server for better network behavior (no 10ms
           hiccups when receiving; no pauses to shred files or deliver
           messages.)
                 o Make all C functions use Py_BEGIN_ALLOW_THREADS and
                   Py_END_ALLOW_THREADS properly.
                 o Make all commonly used Python code threadsafe.
                 . Make Queues threadsafe.
-                        . Design
-                        . Implement
+                        o Design
+                        o Implement
                         o Rename Queue.py to ServerQueue.py
                                 o Do it
-                                - Come up with install code to scrub
+                                o Come up with install code to scrub
                                   old Queue.py where found.
-                        - Come up with comments to explain why the
+                        o Come up with comments to explain why the
                           whole thing is threadsafe as used, and
                           explain what "as used" means.  Add asserts
                           to check "as used".
@@ -45,20 +45,26 @@
                 o Make shredding happen in a separate thread.
                 o Make delivery happen in a separate thread.
                 o Make packet processing happen in a separate thread. 
-                - Move message decoding into packet handler thread.
+                o Move message decoding into packet handler thread.
                 - Document
-                - Refactor until sane -- possibly along with
+                . Refactor until sane -- possibly along with
                   DeliveryQueue refactoring.
+                	o Refactor
+                        o Change tests to test new interfaces
+                        o Remove old interfaces
+                        . Re-test all modules by hand.
+                        - Unit tests for DeliveryPacket's more complex
+                          methods.
                 o Undaemonize threads; make them get shut
                   down explicitly so they can't die halfway.
-                - Test like crazy
+                . Test like crazy
         - Signals
-                . Cleaner shutdown on TERM.
+                o Cleaner shutdown on TERM.
                         o Implement
-                        - Test
+                        o Test
                 . Good reset handling for HUP.
 			o Don't die
-                        - Restart logs
+                        o Reset logs
                         - Check configuration file for changeable
                           things.  (First, decide what's changeable on
                           the fly.)
@@ -113,7 +119,8 @@
                 - Add an extra "this is anonymous" header.
         - COME UP WITH A REAL TESTING STRATEGY FOR PERFORMANCE AND
            CLI'S AND MULTI-SERVER SITUATIONS.
-        - 
+        - Tests for packet addressed to server with bogus IP.
+        - Tests for 
         - ????
 
 Required for "1.0":

Index: setup.py
===================================================================
RCS file: /home/minion/cvsroot/src/minion/setup.py,v
retrieving revision 1.37
retrieving revision 1.38
diff -u -d -r1.37 -r1.38
--- setup.py	9 Jan 2003 05:50:44 -0000	1.37
+++ setup.py	10 Jan 2003 20:12:04 -0000	1.38
@@ -193,6 +193,25 @@
                                  "python%s"%sys.version[:3],
                                  "site-packages"))
 
+if 'install' in sys.argv:
+    if os.environ.get("PREFIX"):
+        sp = os.path.join(os.environ.get("PREFIX"),
+                          "lib",
+                          "python%s"%sys.version[:3],
+                          "site-packages")
+    else:
+        sp = os.path.join(sys.prefix,
+                          "lib",
+                          "python%s"%sys.version[:3],
+                          "site-packages")
+
+    fn = os.path.join(sp, "mixminion", "server", "Queue.py")
+    if os.path.exists(fn):
+        print "Removing obsolete Queue.py"
+        try:
+            os.unlink(fn)
+        except OSError, e:
+            print "Couldn't unlink obsolete Queue.py: %s"%e
 
 INCLUDE_DIRS.append("src")