The Apache Software Foundation
The Apache Software Foundation Incubator

Status of the clutch currently in incubation

Clutch is a tool which gathers details about the projects currently in incubation and re-generates the table below. It aims to encourage and nurture, to provide an overview of the state of establishment of various resources, and to provide quick access to those resources. See more notes below regarding purpose and interpretation.

  • The orange items indicate where more care and attention is needed. Anything vermilion is an issue that should be addressed ASAP. The bluish-green items indicate success and the yellow items are okay.
  • A: startDate
  • B: elapsedDays - <3 <12 <18 <24 >24 months
  • C: reportingMonthly (current report: July2017 )
  • D: reportingGroup - in which set of months do they report:
    group-1=Jan,Apr,Jul,Oct group-2=Feb,May,Aug,Nov group-3=Mar,Jun,Sep,Dec
  • E: hasReportingGroup - did podling remember to choose a group
  • F: hasStatusEntry in the main "Currently in incubation" table (i.e. i.a.o/projects generated from content/podlings.xml)
  • G: statusUpdateDate - the status page was last updated
  • H: statusAge - days since the SVN status page was last updated - <2 <4 <9 >9 months
  • I: statusUpdateCounts - number of commits to status file in the previous 2,4,9 months (cumulative)
  • J: numberCommitters - accounts have been established, total number to date
  • K: numberCommittersNew - new committers added since establishment (gathered from each project's Status page "News" section)
  • L: hasSvn - Subversion repository is created
  • M: hasTracker - has an Issue Tracker
  • N: hasMailListDev - has dev mail list and archives
  • O: hasMailListCommits - has commits mail list and archives
  • P: hasWebSite
  • Q: hasDistributionArea - has w.a.o/dist/incubator/... area (not necessarily yet a release)
  • R: hasKeys - has a PGP "KEYS" file in their distribution area
  • S: hasRelease - has a release in their distribution area
  • other persistent data files
  • other issues listed below for: ariatosca atlas commonsrdf distributedlog fluo madlib mynewt streams toree trafodion

Clutch last gathered: Wed Jul 19 22:17:17 2017 UTC.
Number of podlings in incubation: 53

Project Sponsor A B C D E F G H I J K L M N O P Q R S
Airflow Incubator 2016-03-31 False group-1 True True 2017-04-12 0,2,3 - 0 False True True True True True True True
Annotator Incubator 2016-08-30 False group-1 True True 2017-02-13 0,0,3 - 0 True True True True True False False False
AriaTosca Incubator 2016-08-27 False group-3 True True 2017-07-11 2,2,3 - 0 False True True True True True True True
BatchEE Incubator 2013-10-03 True group-1 True True 2015-01-06 0,0,0 8 0 True True True True True True True True
DataFu Incubator 2014-01-05 True group-1 True True 2017-04-20 0,2,2 17 4 True True True True True True False True
Edgent Incubator 2016-02-29 False group-2 True True 2017-05-22 1,1,2 8 0 False True True True True True True True
FreeMarker Incubator 2015-07-01 False group-1 True True 2017-07-17 2,3,7 - 1 False True True False True True True True
Gearpump Incubator 2016-03-08 False group-3 True True 2016-09-20 0,0,0 - 0 False True True True True True True True
Gobblin Incubator 2017-02-23 True group-1 True True 2017-03-16 0,0,3 - 0 False True True False False False False False
Gossip Incubator 2016-04-28 False group-1 True True 2016-11-13 0,0,2 6 0 False True True True True True True True
Griffin Incubator 2016-12-05 False group-3 True True 2017-03-29 0,3,4 10 0 False True True True True True True True
Guacamole Incubator 2016-02-10 False group-2 True True 2017-06-09 2,10,10 - 2 False True True True True True True True
HAWQ Incubator 2015-09-04 False group-1 True True 2017-07-11 1,1,5 52 6 False True True True True True True True
Heron Incubator 2017-06-23 True group-2 True True 2017-07-11 3,-,- - 0 False True True True False False False False
Hivemall Incubator 2016-09-13 False group-3 True True 2017-02-03 0,0,2 - 0 True True True True True False False False
HTrace Incubator 2014-11-11 False group-3 True True 2016-10-05 0,0,0 13 0 True True True True True True True True
Impala Incubator 2015-12-03 False group-2 True True 2017-07-17 3,6,14 22 0 False True True True True True True True
iota Incubator 2016-01-20 False group-2 True True 2016-01-31 0,0,0 - 0 False True True True True False False False
Joshua Incubator 2016-02-13 False group-2 True True 2017-04-19 0,1,4 9 0 True True True True True True True True
Juneau Incubator 2016-06-24 False group-1 True True 2016-11-02 0,0,1 5 0 False True True True True True True True
Livy Incubator 2017-06-05 True group-1 True True 2017-06-23 3,-,- - 0 False True True True False False False False
Milagro Incubator 2015-12-21 False group-1 True True 2016-04-12 0,0,0 - 0 True True True True True False False False
Mnemonic Incubator 2016-03-03 False group-3 True True 2017-06-12 5,9,12 - 0 False True True True True True True True
MRQL Incubator 2013-03-13 True group-3 True True 2015-06-22 0,0,0 12 0 True True True True True True True True
MXNet Incubator 2017-01-23 False group-1 True True 2017-07-14 3,4,13 - 0 False True True True True False False False
Project Sponsor A B C D E F G H I J K L M N O P Q R S
Myriad Incubator 2015-03-01 False group-3 True True 2015-12-03 0,0,0 10 1 True True True True True True False True
NetBeans Incubator 2016-10-01 True group-1 True True 2017-06-10 3,21,21 - 0 False True True True True False False False
ODF Toolkit Incubator 2011-08-01 False group-1 True True 2017-04-11 0,2,2 - 2 True True True True True True True True
Omid Incubator 2016-03-28 False group-3 True True 2016-04-07 0,0,0 - 0 False True True True True True True True
OpenWhisk Incubator 2016-11-23 False group-3 True True 2017-05-31 2,11,16 - 0 False True True True True False False False
Pony Mail Incubator 2016-05-27 False group-3 True True 2016-10-12 0,0,0 10 0 False True True True True True True True
PredictionIO Incubator 2016-05-26 False group-2 True True 2017-05-11 0,2,2 - 0 False True True True True True True True
Pulsar Incubator 2017-06-01 True group-3 True True 2017-07-18 6,-,- - 0 False True True True True False False False
Quickstep Incubator 2016-03-29 False group-3 True True 2016-04-19 0,0,0 - 0 True True True True True True True True
Ratis Incubator 2017-01-03 False group-2 True True 2017-02-02 0,0,3 17 0 False True True True False True True True
RocketMQ Incubator 2016-11-21 False group-1 True True 2017-01-02 0,0,7 - 0 False True True True True True True True
Rya Incubator 2015-09-18 False group-1 True True 2016-11-14 0,0,3 12 0 False True True True True True True True
S2Graph Incubator 2015-11-29 False group-2 True True 2016-05-10 0,0,0 13 0 True True True True True True True True
SAMOA Incubator 2014-12-15 False group-3 True True 2015-01-06 0,0,0 11 0 True True True True True True False True
SensSoft Incubator 2016-07-13 False group-1 True True 2017-01-11 0,0,1 10 0 False True True True True True False False
SINGA Incubator 2015-03-17 False group-3 True True 2015-05-20 0,0,0 11 0 True True True True True True True True
Slider Incubator 2014-04-29 False group-2 True True 2015-09-30 0,0,0 13 2 True True True True True True True True
Spot Incubator 2016-09-23 False group-3 True True 2016-10-28 0,0,2 - 0 False True True True True True False False
Superset Incubator 2017-05-21 True group-3 True True 2017-06-26 2,-,- - 0 False True True True True False False False
Tamaya Incubator 2014-11-14 False group-2 True True 2015-08-31 0,0,0 12 1 True True True True True True True True
Taverna Incubator 2014-10-20 False group-3 True True 2016-12-06 0,0,3 20 4 True True True True True True True True
Tephra Incubator 2016-03-07 False group-3 True True 2016-11-17 0,0,1 10 0 True True True True True True True True
Toree Incubator 2015-12-02 False group-2 True True 2016-10-10 0,0,0 8 0 True True True True True True True True
Traffic Control Incubator 2016-07-12 False group-1 True True 2016-08-29 0,0,0 - 0 False True True True True True True True
Trafodion Incubator 2015-05-24 False group-3 True True 2017-05-01 0,1,3 - 10 False True True True True True True True
Unomi Incubator 2015-10-05 False group-2 True True 2015-11-05 0,0,0 - 0 True True True True True True True True
Wave Incubator 2010-12-04 False group-3 True True 2016-12-20 0,0,1 28 0 True True True True True True True True
Weex Incubator 2016-11-30 False group-1 True True 2017-01-02 0,0,4 - 0 False True True True True True True True
Project Sponsor A B C D E F G H I J K L M N O P Q R S

Other issues

Occasionally there are other issues, usually regarding podlings not cleaning up after their "graduation".

  • ariatosca: Has a distribution filename missing the word "incubating/incubator". See help.
  • atlas: Has graduated, but still has remains on Incubator distribution mirrors. See help.
  • commonsrdf: Has graduated, but still has remains on Incubator distribution mirrors. See help.
  • distributedlog: Has graduated, but still has remains on Incubator distribution mirrors. See help.
  • fluo: Has graduated, but still has remains on Incubator distribution mirrors. See help.
  • madlib: Has graduated, but still has remains on Incubator distribution mirrors. See help.
  • mynewt: Has graduated, but still has remains on Incubator distribution mirrors. See help.
  • streams: Has graduated, but still has remains on Incubator distribution mirrors. See help.
  • toree: Has a distribution filename missing the word "incubating/incubator". See help.
  • trafodion: Has a distribution filename missing the word "incubating/incubator". See help.

Notes

Any Incubator committer can run 'clutch'. In the top-level of the "incubator/public/trunk" SVN, do: 'python3 clutch.py' which will re-generate some xml entities for the data. The normal site build process will automatically combine these with the "content/clutch.xml" source. Do 'svn diff' to see what changed. Then build and deploy the site as normal.

Clutch reads the source file for the "projects currently in incubation" table (i.e. content/podlings.xml), ensures that each new project has an entry, adds some more attributes. Clutch then gathers some data from each project's Status page, makes other guesses, and validates that certain facilities are established and steps are achieved.

The clutch is the set of eggs which have been laid. The mother hen (our Incubator PMC) needs to continually gather them, to ensure that none have rolled from the nest or moved to the outside, being forgotten or becoming cold. Clutch also means to grasp eagerly.

So please interpret this table from the point-of-view of encouragement and nurture.

Assist with Incubation Steps

Essentially Clutch is helping us all to follow the process from the "Acceptance" phase onwards (explained in Process Description and Incubation Policy and Mentor Guide and Other Guides and summarised at Podling Bootstrap). Any committer on a podling can help with these steps. Don't leave it to your mentors, although there are some steps that only they can do.

The first steps are the "Acceptance" phase. See doc and doc and doc.

The orange and vermilion items indicate where more care and attention is needed. The following notes for each column, expand on the column definitions above and attempt to link directly to the relevant process documentation:

  • Project - If your project is in incubation, but is missing from the Clutch table, then you have forgotten to add it to the podlings.xml file. (See doc and notes for "hasStatusEntry" below.)
  • Project - If this cell is highlighted with a border, then this project is in the process of graduating or retiring, and not yet finished their move. Hence resources (e.g. mail lists, svn) seem to disappear as they migrate. See Other issues below.
  • Sponsor - This information is gathered from the list of Projects in incubation summary table. The "not known" is because it is missing from that table. See notes for "hasStatusEntry" below.
  • A: startDate - This information is gathered from the list of Projects in incubation summary table. The "not known" is because it is missing from that table. See notes for "hasStatusEntry" below.
  • B: elapsedDays - Calculated from startDate.
  • C: reportingMonthly - Your project PPMC should manage your entry in the podlings.xml file
  • D: reportingGroup - in which set of months do they report.
  • E: hasReportingGroup - did podling remember to choose a group. See notes for "Project" above.
  • F: hasStatusEntry in the Currently in incubation table (i.e. generated from content/podlings.xml) and has an associated project Status page. (doc and doc and doc and doc and doc)
  • G: statusUpdateDate - the status page was last updated. Clutch does an 'svn log' query for the project's Status page.
  • H: statusAge - days since the SVN status page was last updated
  • I: statusUpdateCounts - number of commits to status file in the previous 2,4,9 months (cumulative). Clutch does an 'svn log' query for the project's Status page, and counts the edits for those time ranges.
  • J: numberCommitters - accounts have been established, total number to date. Clutch parses the relevant section of the ASF Committers by Project Modules page and counts the total, including mentors.
  • K: numberCommittersNew - new committers added since establishment (gathered from each project's Status page "News" section). Clutch parses the News section and detects the string "new committer", counting each matched line. See notes for "hasStatusEntry" above. (doc and doc and doc)
  • L: hasSvn - Subversion repository is created. (doc and doc)
  • M: hasTracker - has an Issue Tracker
  • N: hasMailListDev - Hints about mail list names are gathered from their Status page, then tries standard convention, then verifies that the mailing lists are established, then ensures that the mail archives are now available.
  • O: hasMailListCommits - see above.
  • P: hasWebSite -
  • Q: hasDistributionArea - has w.a.o/dist/incubator/... area (not necessarily yet a release)
  • R: hasKeys - has a PGP "KEYS" file in their distribution area. This utilises Henk's md5 checker which gathers the list of all KEYS files.
  • S: hasRelease - has a release in their distribution area. The test is whether they have a signature file (*.asc or *.sig ) associated with a source release artifact (*.tar.gz or *.tgz or *.bz2 or *zip) and the filename includes the word "incubating". (doc and doc)
  • otherIssues - When projects graduate or become retired, then they need to clean up.
  • Graduating: Update your project metadata in the content/podlings.xml file as you progress through the graduation phases. Do various other tasks listed in the docs, such as final tidy of Status page, tidy your old release distribution area, etc. (doc and doc and doc)
  • Retiring: In the Projects in incubation table, mark its status as "retired". (doc)

Mentors projects

This list is gathered from the Projects in incubation summary table. The Mentors are ASF members who have chosen to assist these projects. The best way to find mentors for a new project, is during your Discuss and Proposal phases on the general@ mail list.

  • Alan Cabrera: MRQL
  • Alan Gates: HAWQ, Omid, SAMOA, SINGA, Tephra
  • Andrei Savu: Spot
  • Andrew Bayer: Pony Mail
  • Andrew Purtell: Gearpump, HTrace, Mnemonic, PredictionIO, S2Graph, Tephra, Trafodion
  • Andy Seaborne: Taverna
  • Arun C Murthy: Slider
  • Ashutosh Chauhan: DataFu, SAMOA, Superset
  • Ate Douma: NetBeans
  • Benjamin Hindman: Myriad
  • Bertrand Delacretaz: NetBeans, Unomi
  • Bikas Saha: Livy
  • Billie Rinaldi: HTrace, Rya
  • Brian McCallister: Annotator, RocketMQ
  • Brock Noland: Impala, Livy, Spot
  • Bruce Snyder: RocketMQ
  • Carl Steinbach: Impala
  • Chris Mattmann: Joshua, SensSoft
  • Chris Nauroth: Airflow, Ratis
  • Craig Russell: Juneau
  • Danese Cooper: Myriad
  • Daniel Dai: Hivemall, SINGA
  • Daniel Debrunner: Edgent
  • Daniel Gruno: Annotator, Guacamole, iota, NetBeans, Traffic Control
  • Daniel J Debrunner: Taverna
  • Dave Fisher: Pulsar
  • David Blevins: Tamaya
  • Devaraj Das: Ratis, Slider, Trafodion
  • Drew Farris: Gossip
  • Edward J. Yoon: MRQL, Rya
  • Emmanuel Lecharny: NetBeans
  • Enis Soztutar: SAMOA
  • Enis Söztutar: Trafodion
  • Eric Covener: Traffic Control
  • Felix Meschberger: OpenWhisk
  • Flavio P. Junqueira: Omid
  • Francis Liu: Pulsar
  • Greg Trasuk: Guacamole
  • Hadrian Zbarcea: iota
  • Henri Yandell: MXNet
  • Henry Saputra: Mnemonic
  • Hitesh Shah: Airflow, Toree
  • Isabel Drost-Fromm: OpenWhisk
  • J. Aaron Farr: Traffic Control
  • Jacopo Cappellato: FreeMarker
  • Jacques Nadeau: Heron
  • Jake Farrell: Heron, HTrace
  • Jakob Homan: Airflow, AriaTosca, Ratis
  • James Taylor: Mnemonic, Omid, PredictionIO, Tephra
  • Jarek Jarcec Cecho: Gearpump, Spot
  • Jean-Baptiste Onofre: Guacamole, Gobblin, Livy, Unomi
  • Jean-Baptiste Onofré: BatchEE, Slider
  • Jean-Frederic Clere: FreeMarker
  • Jim Jagielski: Annotator, Guacamole, Gobblin, MXNet, Pulsar, Superset
  • Jochen Wiedmann: Juneau
  • John D. Ament: AriaTosca, Edgent, Juneau, Pony Mail, Tamaya
  • Josh Elser: Gossip, Rya
  • Julian Hyde: Quickstep
  • Julien Le Dem: Heron, Toree
  • Justin Erenkrantz: HAWQ
  • Justin McLean: RocketMQ
  • Justin Mclean: Edgent, iota
  • Kasper Sørensen: Griffin
  • Konstantin Boudnik: HAWQ, Quickstep
  • Lars Hofhansl: Omid, PredictionIO, Tephra, Trafodion
  • Lewis John McGibbney: Joshua, SensSoft
  • Lewis John Mcgibbney: HTrace
  • Luciano Resende: Edgent, Griffin, Livy, Myriad, PredictionIO, Toree
  • Luke Han: RocketMQ, Superset, Weex
  • Mahadev Konar: Slider
  • Mark Struberg: BatchEE, NetBeans
  • Markus Weimer: Hivemall, MXNet
  • Marlon Pierce: Taverna
  • Michael Stack: HTrace, Trafodion
  • Mohammad Nour El-Din: MRQL
  • Nick Burch: ODF Toolkit
  • Nick Kew: Annotator, Milagro
  • Olivier Lamy: BatchEE, Gobblin
  • P. Taylor Goetz: Gossip, Heron, Pulsar
  • Patrick Hunt: Mnemonic
  • Paul Ramirez: Joshua, SensSoft
  • Phil Sorber: Traffic Control
  • Ralph Goers: FreeMarker
  • Raphael Bircher: Weex
  • Reynold Xin: Gearpump, Hivemall, Toree
  • Roman Shaposhnik: DataFu, HAWQ, Quickstep
  • Ryan Blue: Toree
  • Sebastian Schelter: MXNet
  • Seetharam Venkatesh: S2Graph
  • Sergio Fernández: FreeMarker, OpenWhisk, S2Graph
  • Stephan Ewen: Weex
  • Sterling Hughes: iota, Milagro
  • Stian Soiland-Reyes: Taverna
  • Suneel Marthi: AriaTosca, MXNet, PredictionIO
  • Suresh Marru: Taverna
  • Suresh Srinivas: Taverna
  • Ted Dunning: DataFu, Myriad, SAMOA, SINGA
  • Thejas Nair: HAWQ, Omid, SINGA
  • Todd Lipcon: Gearpump, HTrace, Impala
  • Tom Barber: Joshua, ODF Toolkit
  • Tom White: Impala
  • Uma Maheswara Rao G: Ratis, Spot
  • Uma Maheswara Rao Gangumalla: Griffin
  • Upayavira: Wave
  • Venkatesh Seetharam: Rya
  • Vinod Kumar Vavilapalli: Slider
  • Willem Jiang: Weex
  • Willem Ning Jiang: RocketMQ
  • Xiangrui Meng: Hivemall, PredictionIO
  • Xuefu Zhang: Gearpump
  • Yegor Kozlov: ODF Toolkit

Data files

Data files are also generated so that other tools can easily re-use the metadata for the set of projects currently in incubation:

  • clutch.txt - list of projects, one per line. Comma-separated values (CSV).
  • clutch.pkl - a Python "pickle" file with various attributes (it is in the top-level of SVN incubator/public/trunk).
  • report-groups.txt - list of current projects showing their report groups.
  • One text file for each Reporting group listing the dev mail address of the group of projects due to report. (See report_due_[1-3].txt) This is used by other systems to send monthly marvin mail alert reminders.
  • Structured list of various details. Not yet happening - need to decide which format (e.g. RDFa, etc.) and which attributes to be stored.

Other Notes

Origin

The nest was becoming full, some podlings were forgotten, rolled into dark corners: new tool "clutch" assist oversight of all podlings 2008-10-19 (http://s.apache.org/clutch_start). There was prior list discussion. Also see INCUBATOR-78

Colour Scheme

The colours for the table were specifically chosen to assist with colour-blindness and general accessibility. See the email discussion that led to this: Re: Clutch color coding 2008-12-04 (http://s.apache.org/clutch_colour). and see some testing notes and demonstrations.

Thanks again to: "Color Universal Design (CUD) - How to make figures and presentations that are friendly to Colorblind people." Masataka Okabe and Kei Ito. (http://jfly.iam.u-tokyo.ac.jp/color/)

Other Summaries

A time-plot of Apache Incubator history.