Difference between revisions of "Amu-chan"

Latest revision as of 23:37, 11 October 2008

Amu-chan is an asynchronous, distributed job queueing system for Microsoft Windows and Unix-like operating systems released under the GPL. It is written in Perl; source is available at Amu-chan/Source.

Run amu.pl on every machine you want to make available to run jobs. Then use amusend.pl to send a new job queue to one of the Amus. The Amus will then figure out among themselves who will do which jobs. Currently, the queueing algorithm is braindead-stupid (randomized first come, first serve). Also, Amu will not divide up jobs for you.

You can either start each Amu with a file in the same directory called amu_remotehosts.txt which contains a list of IP addresses (one per line) of other Amus, or (that's inclusive or) you can send one Amu a new list of IPs using amusend.pl (again using amu_remotehosts.txt) and they will be propagated automatically. You can also run amusend.pl with one argument to broadcast a new job queue. Amus currently working on jobs will ignore new job queues. A job queue consists of a semicolon-delimited Perl hash (key, value pairs). The key is the job name (can be anything), while the value is the actual command to execute.

Amus automatically update themselves to the latest version of the software they detect on the network. (Don't forget to increment $myversion every time you make a change you're reasonably confident won't break the auto-update system.)

Notes

• I don't know whether the collision resolution algorithm works because I've never seen a collision. I suspect that it does not work for the same reason I haven't observed a collision: we're talking sub-millisecond timing here. If two Amus claim the same job and manage to avoid learning about that until after they've both broadcasted the claim, then they will probably be "in sync" enough to play Janken with each other in sync to resolve the collision. However, I wouldn't count on it. Remember, the code has never been tested. Worst case scenario: two Amus perform the same job at the same time.
• Amu does not yet know what OS is it running under. At some point it would be cool to modify the job queue format such that each job is sent as a triplet, e.g. 1;ls;DIR, where the first field is the key, the second field is the Unix command, and the third field is the equivalent Windows command. Each Amu would then save an OS-specific version of the job queue after forwarding the original to all other known Amus.
• Amu may be slowly leaking memory. This is as difficult to test for as everyone says. I don't have the slightest idea where the leak(s) might be coming from. At the very least I know it is not producing zombie threads under OS X as of Amu version 44.
• Amu can only receive one command per TCP connection. This would probably be easy to fix, but I didn't think of it until a few seconds ago when I was trying to finish up amusend.pl.
• It would be cool to make the first character of new job queues the delimiter so we can use semicolons in jobs if needed.
• I forgot to put in job status reporting. Actually, there's no network status reporting of any kind yet in amusend.pl.