Packaging

Specifying And Referencing Packages
Packaging ZIPs
Packaging JARs
Packaging WARs
Packaging AARs
Packaging EARs
Packaging OSGi Bundles
Packaging Tars and GZipped Tars
Installing and Uploading
Packaging Sources and JavaDocs

For our next trick, we’re going to try and create an artifact ourselves. We’re going to start with:

package :jar

We just told the project to create a JAR file in the target directory, including all the classes (and resources) that we previously compiled into target/classes. Or we can create a WAR file:

package :war

The easy case is always easy, but sometimes we have more complicated use cases which we’ll address through the rest of this section.

Now let’s run the build, test cases and create these packages:

$ buildr package

The package task runs the build task (remember: compile and test) and then runs each of the packaging tasks, creating packages in the projects’ target directories.

The package task and package methods are related, but that relation is different from other task/method pairs. The package method creates a file task that points to the package in the target directory and knows how to create it. It then adds itself as a prerequisite to the package task. Translation: you can create multiple packages from the same project.

Specifying And Referencing Packages

Buildr supports several packaging types, and so when dealing with packages, you have to indicate the desired package type. The packaging type can be the first argument, or the value of the :type argument. The following two are equivalent:

package :jar
package :type=>:jar

If you do not specify a package type, Buildr will attempt to infer one.

In the documentation you will find a number of tasks dealing with specific packaging types (ZipTask, JarTask, etc). The package method is a convenience mechanism that sets up the package for you associates it with various project life cycle tasks.

To package a particular file, use the :file argument, for example:

package :zip, :file=>_('target/interesting.zip')

This returns a file task that will run as part of the project’s package task (generating all packages). It will invoke the build task to generate any necessary prerequisites, before creating the specified file.

The package type does not have to be the same as the file name extension, but if you don’t specify the package type, it will be inferred from the extension.

Most often you will want to use the second form to generate packages that are also artifacts. These packages have an artifact specification, which you can use to reference them from other projects (and buildfiles). They are also easier to share across projects: artifacts install themselves in the local repository when running the install task, and upload to the remote repository when running the upload task (see Installing and Uploading).

The artifact specification is based on the project name (using dashes instead of colons), group identifier and version number, all three obtained from the project definition. You can specify different values using the :id, :group, :version and :classifier arguments. For example:

define 'killer-app', :version=>'1.0' do
  # Generates silly-1.0.jar
  package :jar, :id=>'silly'

  # Generates killer-app-la-web-1.x.war
  project 'la-web' do
    package :war, :version=>'1.x'
  end

  # Generates killer-app-the-api-1.0-sources.zip
  project 'teh-api' do
    package :zip, :classifier=>'sources'
  end
end

The file name is determined from the identifier, version number, classifier and extension associated with that packaging type.

If you do not specify the packaging type, Buildr attempt to infer it from the project definition. In the general case it will use the default packaging type, ZIP. A project that compiles Java classes will default to JAR packaging; for other languages, consult the specific documentation.

A single project can create multiple packages. For example, a Java project may generate a JAR package for the runtime library and another JAR containing just the API; a ZIP file for the source code and another ZIP for the documentation. Make sure to always call package with enough information to identify the specific package you are referencing. Even if the project only defines a single package, calling the package method with no arguments does not necessarily refer to that one.

You can use the packages method to obtain a list of all packages defined in the project, for example:

project('killer-app:teh-impl').packages.first
project('killer-app:teh-impl').packages.select { |pkg| pkg.type == :zip }

Packaging ZIPs

ZIP is the most common form of packaging, used by default when no other packaging type applies. It also forms the basis for many other packaging types (e.g. JAR and WAR). Most of what you’ll find here applies to other packaging types.

Let’s start by including additional files in the ZIP package. We’re going to include the target/docs directory and README file:

package(:zip).include _('target/docs'), 'README'

The include method accepts files, directories and file tasks. You can also use file pattern to match multiple files and directories. File patterns include asterisk (*) to match any file name or part of a file name, double asterisk (**) to match directories recursively, question mark (?) to match any character, square braces ([]) to match a set of characters, and curly braces ({}) to match one of several names.

And the same way you include, you can also exclude specific files you don’t want showing up in the ZIP. For example, to exclude .draft and .raw files:

package(:zip).include(_('target/docs')).exclude('*.draft', '*.raw')

So far we’ve included files under the root of the ZIP. Let’s include some files under a given path using the :path option:

package(:zip).include _('target/docs'), :path=>"#{id}-#{version}"

If you need to use the :path option repeatedly, consider using the tap method instead. For example:

package(:zip).path("#{id}-#{version}").tap do |path|
  path.include _('target/docs')
  path.include _('README')
end

The tap method is not part of the core library, but a very useful extension. It takes an object, yields to the block with that object, and then returns that object.

To allow you to spread files across different paths, the include/exclude patterns are specific to a path. So in the above example, if you want to exclude some files from the “target/docs” directory, make sure to call exclude on the path, not on the ZIP task itself.

If you need to include a file or directory under a different name, use the :as option. For example:

package(:zip).include(_('corporate-logo-350x240.png'), :as=>'logo.png')

You can also use :as=>'.' to include all files from the given directory. For example:

package(:zip).include _('target/docs/*')
package(:zip).include _('target/docs'), :as=>'.'

These two perform identically. They both include all the files from the target/docs directory, but not the directory itself, and they are both lazy, meaning that the files can be created later and they will still get packaged into the zip package.

For example, when you use package :jar, under the hood it specifies to include all the files from target/classes with :as=>'.'. Even though this happens during project definition and nothing has been compiled yet (and in fact target/classes may not even exist yet), the .class files generated during compilation are still packaged in the .jar file, as expected.

If you need to get rid of all the included files, call the clean method. Some packaging types default to adding various files and directories, for example, JAR packaging will include all the compiled classes and resources.

You can also merge two ZIP files together, expanding the content of one ZIP into the other. For example:

package(:zip).merge _('part1.zip'), _('part2.zip')

If you need to be more selective, you can apply the include/exclude pattern to the expanded ZIP. For example:

# Everything but the libs
package(:zip).merge(_('bigbad.war')).exclude('libs/**/*')

Packaging JARs

JAR packages extend ZIP packages with support for Manifest files and the META-INF directory. They also default to include the class files found in the target/classes directory.

You can tell the JAR package to include a particular Manifest file:

package(:jar).with :manifest=>_('src/main/MANIFEST.MF')

Or generate a manifest from a hash:

package(:jar).with :manifest=>{ 'Copyright'=>'Acme Inc (C) 2007' }

You can also generate a JAR with no manifest with the value false, create a manifest with several sections using an array of hashes, or create it from a proc.

In large projects, where all the packages use the same manifest, it’s easier to set it once on the top project using the manifest project property. Sub-projects inherit the property from their parents, and the package method uses that property if you don’t override it, as we do above.

For example, we can get the same result by specifying this at the top project:

manifest['Copyright'] = 'Acme Inc (C) 2007'

If you need to mix-in the project’s manifest with values that only one package uses, you can do so easily:

package(:jar).with :manifest=>manifest.merge('Main-Class'=>'com.acme.Main')

If you need to include more files in the META-INF directory, you can use the :meta_inf option. You can give it a file, or array of files. And yes, there is a meta_inf project property you can set once to include the same set of file in all the JARs. It works like this:

meta_inf << file('DISCLAIMER') << file('NOTICE')

If you have a LICENSE file, it’s already included in the meta_inf list of files.

Other than that, package :jar includes the contents of the compiler’s target directory and resources, which most often is exactly what you intend it to do. If you want to include other files in the JAR, instead or in addition, you can do so using the include and exclude methods. If you do not want the target directory included in your JAR, simply call the clean method on it:

package(:jar).clean.include( only_these_files )

Packaging WARs

Pretty much everything you know about JARs works the same way for WARs, so let’s just look at the differences.

Without much prompting, package :war picks the contents of the src/main/webapp directory and places it at the root of the WAR, copies the compiler target directory into the WEB-INF/classes path, and copies any compiled dependencies into the WEB-INF/libs paths.

Again, you can use the include and exclude methods to change the contents of the WAR. There are two convenience options you can use to make the more common changes. If you need to include a classes directory other than the default:

package(:war).with :classes=>_('target/additional')

If you want to include a different set of libraries other than the default:

package(:war).with :libs=>MYSQL_JDBC

Both options accept a single value or an array. The :classes option accepts the name of a directory containing class files, initially set to compile.target and resources.target. The :libs option accepts artifact specifications, file names and tasks, initially set to include everything in compile.dependencies.

As you can guess, the package task has two attributes called classes and libs; the with method merely sets their value. If you need more precise control over these arrays, you can always work with them directly, for example:

# Add an artifact to the existing set:
package(:war).libs += artifacts(MYSQL_JDBC)
# Remove an artifact from the existing set:
package(:war).libs -= artifacts(LOG4J)
# List all the artifacts:
puts 'Artifacts included in WAR package:'
puts package(:war).libs.map(&:to_spec)

Compiling Assets

In modern web applications, it is common to use tools that compile and compress assets. i.e. Coffeescript is compiled into javascript and Sass compiles into CSS. Buildr provides support using a simple assets abstraction. Directory or file tasks can be added to the assets.paths configuration variable for a project and the contents will be included in the package.

Integrating CoffeeScript

target_dir = _(:target, :generated, "coffee/main/webapp")
source_dir = _(:source, :main, :coffee)

assets.paths << file(target_dir => [FileList["#{source_dir}/**/*.coffee"]]) do
  puts "Compiling coffeescript"
  sh "coffee --bare --compile --output #{target_dir} #{source_dir}"
  touch target_dir
end

Integrating Sass

target_dir = _(:target, :generated, "sass/main/webapp")
source_dir = _(:source, :main, :sass)

assets.paths << file(target_dir => [FileList["#{source_dir}/**/*.scss"]]) do
  puts "Compiling scss"
  sh "scss -q --update #{source_dir}:#{target_dir}"
  touch target_dir
end

Packaging AARs

Axis2 service archives are similar to JAR’s (compiled classes go into the root of the archive) but they can embed additional libraries under /lib and include services.xml and WSDL files.

package(:aar).with(:libs=>'log4j:log4j:jar:1.1')
package(:aar).with(:services_xml=>_('target/services.xml'),
                   :wsdls=>_('target/*.wsdl'))

The libs attribute is a list of .jar artifacts to be included in the archive under /lib. The default is no artifacts; compile dependencies are not included by default.

The services_xml attribute points to an Axis2 services configuration file called services.xml that will be placed in the META-INF directory inside the archive. The default behavior is to point to the services.xml file in the project’s src/main/axis2 directory. In the second example above we set it explicitly.

The wsdls attribute is a collection of file names or glob patterns for WSDL files that get included in the META-INF directory. In the second example we include WSDL files from the target directory, presumably created by an earlier build task. In addition, AAR packaging will include all files ending with .wsdl from the src/main/axis2 directory.

If you already have WSDL files in the src/main/axis2 directory but would like to perform some filtering, for example, to set the HTTP port number, consider ignoring the originals and including only the filtered files, for example:

# Host name depends on environment.
host = ENV['ENV'] == 'test' ? 'test.host' : 'ws.example.com'
filter.from(_('src/main/axis2')).into(_(:target)).
  include('services.xml', '==*==.wsdl').using('http_port'=>'8080',
                                              'http_host'=>host)
package(:aar).wsdls.clear
package(:aar).with(:services_xml=>_('target/services.xml'),
                   :wsdls=>_('target/==*==.wsdl'))

Packaging EARs

EAR packaging is slightly different from JAR/WAR packaging. It’s main purpose is to package components together, and so it includes special methods for handling component inclusion that take care to update application.xml and the component’s classpath.

EAR packages support four component types:

Argument	Component
`:war`	J2EE Web Application (WAR).
`:ejb`	Enterprise Java Bean (JAR).
`:jar`	J2EE Application Client (JAR).
`:lib`	Shared library (JAR).

This example shows two ways for adding components built by other projects:

package(:ear) << project('coolWebService').package(:war)
package(:ear).add project('commonLib') # By default, the JAR package

Adding a WAR package assumes it’s a WAR component and treats it as such, but JAR packages can be any of three component types, so by default they are all treated as shared libraries. If you want to add an EJB or Application Client component, you need to say so explicitly, either passing :type=>package, or by passing the component type in the :type option.

Here are three examples:

# Assumed to be a shared library.
package(:ear).add 'org.springframework:spring:jar:2.6'
# Component type mapped to package.
package(:ear).add :ejb=>project('beanery')
# Adding component with specific package type.
package(:ear).add project('client'), :type=>:jar

By default, WAR components are all added under the /war path, and likewise, EJB components are added under the /ejb path, shared libraries under /lib and Application Client components under /jar.

If you want to place components in different locations you can do so using the :path option, or by specifying a different mapping between component types and their destination directory. The following two examples are equivalent:

# Specify once per component.
package(:ear).add project('coolWebService').package(:war), :path=>'coolServices'
# Configure once and apply to all added components.
package(:ear).dirs[:war] = 'coolServices'
package(:ear) << project('coolWebService').package(:war)

EAR packages include an application.xml file in the META-INF directory that describes the application and its components. This file is created for you during packaging, by referencing all the components added to the EAR. There are a couple of things you will typically want to change.

display-name — The application’s display name defaults to the project’s identifier. You can change that by setting the display_name attribute.

description — The application’s description defaults to the project’s comment. You can change that by setting the description attribute.

context-root — WAR components specify a context root, based on the package identifier, for example, “cool-web-1.0.war” will have the context root “cool-web”. To specify a different context root, add the WAR package with the context_root option.

Again, by example:

package(:ear).display_name = 'MyCoolWebService'
package(:ear).description = 'MyCoolWebService: Making coolness kool again'
package(:ear).add project('coolWebService').package(:war), :context_root=>'coolness'

If you need to disable the context root (e.g. for Portlets), set context_root to false.

It is also possible to add security-role tags to the application.xml file by appending a hash with :id, :description and :name to the security_role array, like so:

package(:ear).security_roles << {:id=>'SecurityRole_123',
		:description=>'Read only user', :name=>'coolUser'}
package(:ear).security_roles << {:id=>'SecurityRole_456',
		:description=>'Super user', :name=>'superCoolUser'}

Packaging OSGi Bundles

OSGi bundles are jar files with additional metadata stored in the manifest. Buildr uses an external tool Bnd to create the package. Directives and properties can be explicitly passed to the build tool and buildr will provide reasonable defaults for properties that can be derived from the project model. Please see the bnd tool for documentation on the available properties.

The bundle packaging format is included as an addon so the build file must explicitly require the addon using using require "buildr/bnd" and must add a remote repository from which the bnd can be downloaded. A typical project that uses the bundle packaging addon may look something like;

require "buildr/bnd"

repositories.remote << "http://central.maven.org/maven2"
# uncomment the next version to override the version of bnd
# Buildr::Bnd.version = '0.0.384'

define 'myProject' do
  ...
  package(:bundle).tap do |bnd|
    bnd['Import-Package'] = "*;resolution:=optional"
    bnd['Export-Package'] = "*;version=#{version}"
  end
  ...
end

The [] method on the bundle package is used to provide directives to the bnd tool that are not inherited by sub-projects while the standard ‘manifest’ setting is used to define properties that inherited by sub-projects.

Defaults

The addon sets the following bnd parameters;

“Bundle-Version” defaults to the project version.
“Bundle-SymbolicName” defaults to the concatenation of the project group and project id, replacing ‘:’ characters with ‘.’.
“Bundle-Name” defaults to the project description if present else the project name
“Bundle-Description” defaults to the project description.
“-classpath” is set to the compile target directory and any compile time dependencies.
“Include-Resource” defaults to the dir project.resources.target if it exists.

Parameters

classpath_element

The user can also specify additional elements that are added to the classpath using the ‘classpath_element’ method. If the parameter to this element is a task, artifact, artifact namespace etc. then it will be resolved prior to invoking bnd.

...
define 'foo' do
  ...
  package(:bundle).tap do |bnd|
    # This dependency will be added to classpath
    bnd.classpath_element 'someOtherExistingFile.zip'
    # All of these dependencies will be invoked and added to classpath
    bnd.classpath_element artifact('com.sun.messaging.mq:imq:jar:4.4')
    bnd.classpath_element project('bar') # Adds all the packages
    bnd.classpath_element 'org.apache.ant:ant:jar:1.8.0'
    bnd.classpath_element file('myLocalFile.jar')
    ...
  end

  project 'bar' do
    ...
  end
end

classpath

The user can specify the complete classpath using the ‘classpath’ method. The classpath should be an array of elements. If the element is a task, artifact, artifact namespace etc. then it will be resolved prior to invoking bnd.

...
define 'foo' do
  ...
  package(:bundle).tap do |bnd|
    bnd.classpath [ project.compile.target,
                    'someOtherExistingFile.zip',
                    artifact('com.sun.messaging.mq:imq:jar:4.4'),
                    project('bar'),
                    'org.apache.ant:ant:jar:1.8.0',
                    file('myLocalFile.jar') ]
    ...
  end

  project 'bar' do
    ...
  end
end

Examples

Including non-class resources in a bundle

Bnd can be used to include non-class resources in a bundle. The following example includes all resources in ‘src/etc’ into the bundle.

define 'myproject' do
  ...
  package(:bundle).tap do |bnd|
    bnd['Include-Resource'] = project._('src/etc') + '/'
    ...
  end
end

Using bnd to wrap an existing jar

Bnd can be used to wrap an existing jar as an OSGi bundle. The following example wraps the OpenMQ JMS provider as an OSGi bundle.

...
# Add repository for OpenMQ
repositories.remote << 'http://download.java.net/maven/2'

desc 'OSGi bundle for OpenMQ JMS provider client library'
define 'com.sun.messaging.mq.imq' do
  project.version = '4.4'
  project.group = 'iris'
  package(:bundle).tap do |bnd|
    bnd['Import-Package'] = "*;resolution:=optional"
    bnd['Export-Package'] = "com.sun.messaging.*;version=#{version}"
    bnd.classpath_element 'com.sun.messaging.mq:imq:jar:4.4'
  end
end

Create an OSGi bundle with an Activator

The following example presents a basic buildfile for building an OSGi bundle with an activator.

...
# repository for OSGi core bundle
repositories.remote << 'https://repository.apache.org/content/repositories/releases'

desc 'Hello World bundle'
define 'helloworld' do
  project.version = '1.0'
  project.group = 'org.example'
  compile.with 'org.apache.felix:org.osgi.core:jar:1.4.0'
  package(:bundle).tap do |bnd|
    bnd['Export-Package'] = "org.example.helloworld.api.*;version=#{version}"
    bnd['Bundle-Activator'] = "org.example.helloworld.Activator"
  end
end

Inheriting parameters for bnd tool

The following example shows how you can use ‘manifest’ to define a bnd parameter that is inherited by all child sub-projects. The “Bundle-License” defined in the top level project is passed to the bnd tool when generating both the ‘fe’ and ‘fi’ sub-projects but the ‘fo’ sub-project overrides this parameter with a local value.

...
define 'myproject' do
  manifest['Bundle-License'] = "http://www.apache.org/licenses/LICENSE-2.0"
  ...
  define 'fe' do
    ...
    package(:bundle).tap do |bnd|
      ...
    end
  end

  define 'fi' do
    ...
    package(:bundle).tap do |bnd|
      ...
    end
  end

  define 'fo' do
    ...
    package(:bundle).tap do |bnd|
      bnd['Bundle-License'] = "http://www.apache.org/licenses/LICENSE-1.1"
    end
  end
end

Packaging Tars and GZipped Tars

Everything you know about working with ZIP files translates to Tar files, the two tasks are identical in more respect, so here we’ll just go over the differences.

package(:tar).include _('target/docs'), 'README'
package(:tgz).include _('target/docs'), 'README'

The first line creates a Tar archive with the extension .tar, the second creates a GZipped Tar archive with the extension .tgz.

In addition to packaging that includes the archive in the list of installed/released files, you can use the method tar to create a TarTask. This task is similar to ZipTask, and introduces the gzip attribute, which you can use to tell it whether to create a regular file, or GZip it. By default the attribute it set to true (GZip) if the file name ends with either .gz or .tgz.

Installing and Uploading

You can bring in the artifacts you need from remote repositories and install them in the local repositories. Other projects have the same expectation, that your packages be their artifacts.

So let’s create these packages and install them in the local repository where other projects can access them:

$ buildr install

If you changes your mind you can always:

$ buildr uninstall

That works between projects you build on the same machine. Now let’s share these artifacts with other developers through a remote repository:

$ buildr upload

Of course, you’ll need to tell Buildr about the release server:

repositories.release_to = 'sftp://john:secret@release/usr/share/repo'

If you have separate repositories for releases and snapshots, you can specify them accordingly. Buildr takes care of picking the correct one.

repositories.release_to = 'sftp://john:secret@release/usr/share/repo/releases'
repositories.snapshot_to = 'sftp://john:secret@release/usr/share/repo/snapshots'

This example uses the SFTP protocol. In addition, you can use the HTTP protocol — Buildr supports HTTP and HTTPS, Basic Authentication and uploads using PUT — or point to a directory on your file system.

The URL in this example contains the release server (“release”), path to repository (“user/share/repo”) and username/password for access. The way SFTP works, you specify the path on the release server, and give the user permissions to create directories and files inside the repository. The file system path is different from the path you use to download these artifacts through an HTTP server, and starts at the root, not the user’s home directory.

Of course, you’ll want to specify the release server URL in the Buildfile, but leave the username/password settings private in your local buildr.rb file. Let’s break up the release server settings:

# build.rb, loaded first
repositories.release_to[:username] = 'john'
repositories.release_to[:password] = 'secret'

# Buildfile, loaded next
repositories.release_to[:url] = 'sftp://release/usr/share/repo'

The upload task takes care of uploading all the packages created by your project, along with their associated POM files and MD5/SHA1 signatures (Buildr creates these for you).

If you need to upload other files, you can always extend the upload task and use repositories.release_to in combination with URI.upload. You can also extend it to upload to different servers, for example, to publish the documentation and test coverage reports to your site:

# We'll let some other task decide how to create 'docs'
task 'upload'=>'docs' do
  uri = URI("sftp://#{username}:#{password}@var/www/docs")
  uri.upload file('docs')
end

Uploading Options

For convenience, you can also pass any option of Net::SSH when configuring the remote repository.

If you need to enforce to use password-only authentication for example, you can set this option:

# Set password authentication only
repositories.release_to[:options] = {:ssh_options=>{:auth_methods=> 'password'}}

Packaging Sources and JavaDocs

IDEs can take advantage of source packages to help you debug and trace through compiled code. We’ll start with a simple example:

package :sources

This one creates a ZIP package with the classifier “sources” that will contain all the source directories in that project, typically src/main/java, but also other sources generated from Apt, JavaCC, XMLBeans and friends.

You can also generate a ZIP package with the classifier “javadoc” that contains the JavaDoc documentation for the project. It uses the same set of documentation files generated by the project’s doc task, so you can use it in combination with the doc method. For example:

package :javadoc
doc :windowtitle=>'Buggy but Works'

By default Buildr picks the project’s description for the window title.

You can also tell Buildr to automatically create sources and JavaDoc packages in all the sub-projects that have any source files to package or document. Just add either or both of these methods in the top-level project:

package_with_sources
package_with_javadoc

You can also tell it to be selective using the :only and :except options.
For example:

package_with_javadoc :except=>'la-web'

We packaged the code, but will it actually work? Let’s see what the tests say.