Building and Testing

You can check out Giraph with git clone http://gitbox.apache.org/repos/asf/giraph.git. In order to build and test Giraph, you will need the following:

  • Java 1.8
  • Maven 3 or higher. Giraph uses the munge plugin, which requires Maven 3, to support multiple versions of Hadoop. Also, the web site plugin requires Maven 3.
  • One of the supported versions of Apache Hadoop:
    • YARN version: Apache Hadoop 2.0.3-alpha, other versions may work as well
    • Secure Hadoop versions: Apache Hadoop 0.20.203, 0.20.204, other secure versions may work as well
    • Unsecure Hadoop versions: Apache Hadoop 0.20.1, 0.20.2, 0.20.3. While we provide support for unsecure Hadoop with the maven profile 'hadoop_non_secure', we have been primarily focusing on secure Hadoop releases at this time.
    • Other distributions that included Apache Hadoop reported to work include: Cloudera CDH 4.4

Use the maven commands with secure Hadoop to:

  • compile, package and install: mvn -Phadoop_2 -fae -DskipTests clean install
  • test (i.e. mvn test) For testing, one can submit the test to a running Hadoop instance (i.e. mvn test -Dprop.mapred.job.tracker=localhost:50300)

For the non-secure versions of Hadoop, run the maven commands with the additional argument -Dhadoop=non_secure to enable the maven profile hadoop_non_secure. An example compilation command is mvn -Phadoop_2 -fae -DskipTests -Dhadoop=non_secure clean install.