Process concurrency in perl.

Michael Brailsford brailsmt at yahoo.com
Fri Mar 9 10:19:04 MST 2007


I am trying to generate doxygen output on a very large code base, of over 20 million lines of code.  I tried to run it all as one big doxygen project, but doxygen tried to keep everything in memory (lol).  I have been back and forth with Dmitri and he is working on incremental doc builds, but I can't wait.  So, I instead wrote a perl script to build each seperate project on its own.  There are some 4000 projects.  I realized fairly quickly that I could write a few simple lockfiles and get some concurrency with several different invocations of the script each generating docs and skipping over those projects which have already been built by the other processes.  I started 5 or 6 of these and started to get deadlock, surprise I know.  Is there some better way in perl to coordinate N processes?  I know I can use semaphores, but I wanted to see if there was some better "perl way" of doing it first, before figuring out how to do semaphores in perl.

P.S.  It is not an option to write this in some other language.  My love is for ruby, but it simply isn't as ubiquitous as perl.

-Michael





More information about the PLUG mailing list