The Pulse lab project collects information from the Apache Software
Foundation's mailing lists, such as the numer of subscribers, who the
moderators are, and so on.  The information is loaded into a MySQL
database, and is used to generate reports and graphs for the Web site
(http://Pulse.Apache.Org/).  In addition, the information can be
queried via email requests.

The project is broken into several parts:

o Data collection
-----------------
This occurs on the Mail.Apache.Org system, under the username
'apmail'.  The '~apmailbin/collect-listinfo.sh' script collects the
info and records it in XML, and then sends it to
'collect-listinfo@apache.org' eddress.  Qmail forwards it according to
'~apmail/.qmail-collect-listinfo'.  Currently that points to
<pulse-load@Pulse.Apache.Org>.

o Analysis
----------
On Pulse.Apache.Org, the eddress <pulse-load> feeds the
'pulse/analysis/pulse-demail.pl' script, which strips the mail
information from the XML data and stores the latter in the
'pulse/data/queue/' directory.

A cron job running as the 'pulsestat' user runs the
'pulse/analysis/pulse-load.pl' script to load files from the queue
into the database.  After loading, the files are moved to the
'pulse/data/processed/' directory.

o Generation
------------
'Generation' only applies to the Web-based reports, and is centred on
the 'pulse/web-reports/' directory.  The key here is the 'mlists.php'
script, which is run each day after all queued data files have been
processed.  Also in this directory are the files that should be copied
to the Web document tree (such as the CSS files).

o Report upload
Uploading is performed by the 'mlists.php' script mentioned in the
previous section.  See the cron files for details.

o Email query
TBS

Files
=====
 o SVN Repository:
    https://svn.apache.org/repos/asf/labs/pulse/

 o Working directory
   /usr/local/labs/pulse
       /analysis                Scripts to unpack data and load them
                                into the DB
       /collection              Copies of scripts used on Mail.Apache.Org
       /cron.d                  Cron jobs
       /data                    
           /queue               XML files awaiting processing
           /processed           XML files after loading
       /email-query             Scripts used to answer email queries
       /test                    Stuff for testing
       /web-reports             Scripts for generating statistics for Web