Apache Incubator
Font size:      

How to build Apache Droids?

Download

Download the Droids code base (we refer to the resulting directory as $DROIDS_HOME):

svn co https://svn.apache.org/repos/asf/incubator/droids/trunk droids

Setting up the Environment

After downloading and extracting Droids, you need to add environment variables. The reason for this is so that the 'droids' command is available everywhere and it can locate its home directory and resources. It is beyond the scope of Droids to explain how to manage your operating system.

In Unix/Linux:

change directory to the top-level of the droids distribution and do ...

~/apache-droids$ export DROIDS_HOME=`pwd`

~/apache-droids$ export PATH=$PATH:$DROIDS_HOME/bin

Permanently Setting The Environment Variables for Linux/Unix

Export only changes the variables during that terminal session for that user, this is useful for testing. To permanently add the variable edit either /etc/bash.bashrc (for all users) or ~/.bash_profile or ~/.bashrc (for individual users). Add these lines to the end of the file you edit: 

      DROIDS_HOME=/full/path/to/droids
      export DROIDS_HOME
      
      PATH=$PATH:$DROIDS_HOME/bin
      export PATH

Windows 2000

Go to "My Computer", "Properties", "Advanced", "Environment Variables"

add a new variable DROIDS_HOME as C:\full\path\to\apache-droids

edit PATH as %PATH%;%DROIDS_HOME%\bin

In Windows XP:

Go to "My Computer", "Properties", "Advanced", "Environment Variables"

Create a New variable with name: DROIDS_HOME value: C:\full\path\to\apache-droids

Edit PATH by adding ;%DROIDS_HOME%\bin to the end of the current value.

Build

Before building droids, create a file called build.properties to customize your Droid. All properties that you can override can be found in the file default.properties.

The normal case is to implement the following properties with your custom values:

# The initial url to crawl (is likely the only property you do not want from default)
droids.initial.url=http://target-x.de/about.html
        
# The droid you want to use
droids.name=hello
        
# the regular expression file we want to use
droids.filter.regex=${droids.home}/regex-urlfilter.txt
        
# the root output dir for saving a crawl
droids.handler.save.dir=${droids.home}/export/

After you create the above file you can go ahead and build droids like follows. The command will first download all libaries that we need with the help of Apache Ivy and then generate a jar.

ant ivy.resolve droids.jar
...
[jar] Building jar: /home/thorsten/src/testing/droids/build/droids-initial.jar
BUILD SUCCESSFUL
Total time: 22 seconds

Build afer an update

If you are doing an update of the svn repository you need to build droids again. You normally can drop the resolving of the dependencies but if you want to make sure that everything is up to date do the following:

cd $DROIDS_HOME; svn up; ant ivy.resolve droids.clean droids.jar

First we make an update of the code (cd $DROIDS_HOME; svn up;). Then we resolve all dependencies (ant ivy.resolve). We clean then our old build (ant droids.clean) and create a new version of the droids.jar (ant droids.jar).