Building build files

A long time ago, I wrote a tool that generated ant build scripts for Eclipse projects. The tool, which I named “Smartbuild”, built a build script for a project by examining its “.project” and “.classpath” files, finding all of its dependent projects and libraries (and their dependent projects and libraries, recursively). Armed with this information, it generated a bare bones build script that contained the “usual suspects”: targets for “clean”, “init”, “compile” and “dist”. Smartbuild provided some nice functionality like merging manifests, setting up “Main” classes (for executable JAR’s), etc..

The nice things about Smartbuild were that it greatly simplified the grunt work of writing multi-project build scripts, but, more importantly, ensured that build scripts would stay in sync with my Eclipse projects. Each of my eclipse projects had a boilerplate build script that essentially was just about five lines long: all that the build script did was invoke smartbuild, have it generate the “real” build script, and run the generated script.

The “ugly” part about Smartbuild is that, like many projects that programmers create for themselves, it suffered from “organic” growth to the point that it became almost impossible to maintain. Also, since different parts of the code base reflected my then-current Java and OO skills, they became increasingly embarrassing to look at and debug as I became more skilled at my trade. Things got to the point where I decided that I needed to rewrite Smartbuild to continue using it. I will blog about my exercise of rewriting Smartbuild as I make progress.

Build Script generators and contributors

Build scripts are generated by generators with the help of one or more contributors. Generators are responsible for the overall organization of build tasks and how they depend on each other, while contributors are responsible for generating specific tasks.

Typically, users choose a generator depending upon the type of eclipse project that must be built, configure it with individual contributors to fine-tune the generated build script and finally, invoke the generator to obtain a build script. Example contributors are those that package source code (the “SourceCodeContributor”) and generate test builds (“JUnit contribitors”). Generators implement the IBuildScriptGenerator interface shown below:

 

package com.subhajit.tools.build;

import java.io.File;
import java.io.IOException;

import org.jdom.JDOMException;

import com.subhajit.eclipse.entities.EclipseMetafile;
import com.subhajit.eclipse.entities.EclipseProject;

public interface IBuildScriptGenerator {
    /**
     * Generates the build script for the selected Eclipse project.
     *
     * @return
     * @throws IOException
     * @throws JDOMException
     */
    String generateBuildScript() throws IOException, JDOMException;

    /**
     * Gets a list of all dependents of the selected Eclipse project, including
     * other projects on which the project depends on, third-party JAR files,
     * etc.
     *
     * @return
     */
    EclipseMetafile[] getDependents();

    /**
     * Returns the Eclipse project for which the build file is generated.
     *
     * @return
     */
    EclipseProject getEclipseProject();

    /**
     * Returns the build output directory.
     *
     * @return
     */
    File getBuildOutputDirectory();

    /**
     * Returns the distribution output directory.
     *
     * @return
     */
    File getDistOutputDirectory();

    BuildConfiguration getBuildConfiguration();
}

The stand alone build script generator

I provide a default implementation of the IBuildScriptGenerator that generates build-scripts to create stand-alone applications from Eclipse projects. I also provide a class named “ScriptGenerator” that contains a main method encapsulating the stand alone build script generator. This standalone program lets the user specify various inputs such as the project directory (for which the build script must be generated), the name of a “main” class, and a set of directories containing Eclipse project files. The last parameter must be specified if the eclipse project depends upon other eclipse projects which lie under a different directory; this option need not be specified if the project and all the projects it depends on happen to lie under the same directory.

Invoking the program without specifying any inputs shows its usage:

 buildgen-screenshot1

The following sample invocation shows how to generate a build file for the “tools” project, which contains the source code for the build generator:

buildgen-screenshot

The generated build file is:

<project default="build">
    <target name="clean">
        <delete dir="build" />
        <delete dir="dist" />
        <delete dir="testoutput" />
    </target>
    <target name="init">
        <mkdir dir="build" />
        <copy todir="build">
            <fileset dir="../base/src/main/resources">
                <include name="**/*" />
                <exclude name="**/*.java" />
            </fileset>
        </copy>
        <copy todir="build">
            <fileset dir="../base/src/main/java">
                <include name="**/*" />
                <exclude name="**/*.java" />
            </fileset>
        </copy>
        <copy todir="build">
            <fileset dir="../eclipse-model/src/main/java">
                <include name="**/*" />
                <exclude name="**/*.java" />
            </fileset>
        </copy>
        <copy todir="build">
            <fileset dir="../eclipse-model/src/main/test">
                <include name="**/*" />
                <exclude name="**/*.java" />
            </fileset>
        </copy>
        <copy todir="build">
            <fileset dir="../eclipse-model/src/main/resources">
                <include name="**/*" />
                <exclude name="**/*.java" />
            </fileset>
        </copy>
        <copy todir="build">
            <fileset dir="../base-xml/src/main/java">
                <include name="**/*" />
                <exclude name="**/*.java" />
            </fileset>
        </copy>
        <copy todir="build">
            <fileset dir="src/main/java">
                <include name="**/*" />
                <exclude name="**/*.java" />
            </fileset>
        </copy>
        <copy todir="build">
            <fileset dir="src/main/resources">
                <include name="**/*" />
                <exclude name="**/*.java" />
            </fileset>
        </copy>
    </target>
    <target name="compile" depends="init">
        <javac debug="true" destdir="build" source="1.5" target="1.5">
            <src path="../base/src/main/resources" />
            <src path="../base/src/main/java" />
            <src path="../eclipse-model/src/main/java" />
            <src path="../eclipse-model/src/main/test" />
            <src path="../eclipse-model/src/main/resources" />
            <src path="../base-xml/src/main/java" />
            <src path="src/main/java" />
            <src path="src/main/resources" />
            <classpath path="../lib-lite/lib/jdom/jdom-1.0.jar" />
            <classpath path="../lib-lite/lib/jdom/jaxen-core.jar" />
            <classpath path="../lib-lite/lib/jdom/jaxen-jdom.jar" />
            <classpath path="../lib-lite/lib/jdom/saxpath.jar" />
            <classpath path="../lib-lite/lib/junit/4.5/junit-4.5.jar" />
            <classpath path="../lib-lite/lib/ant/1.6.2/ant.jar" />
            <classpath path="../lib-lite/lib/log4j/1.2.15/log4j-1.2.15.jar" />
            <classpath path="../lib-lite/lib/jdom/saxon9he.jar" />
        </javac>
    </target>
    <target name="build" depends="compile">
        <mkdir dir="dist" />
        <delete dir="dist" />
        <mkdir dir="dist" />
        <copy todir="dist" file="../lib-lite/lib/jdom/jdom-1.0.jar" />
        <copy todir="dist" file="../lib-lite/lib/jdom/jaxen-core.jar" />
        <copy todir="dist" file="../lib-lite/lib/jdom/jaxen-jdom.jar" />
        <copy todir="dist" file="../lib-lite/lib/jdom/saxpath.jar" />
        <copy todir="dist" file="../lib-lite/lib/junit/4.5/junit-4.5.jar" />
        <copy todir="dist" file="../lib-lite/lib/ant/1.6.2/ant.jar" />
        <copy todir="dist" file="../lib-lite/lib/log4j/1.2.15/log4j-1.2.15.jar" />
        <copy todir="dist" file="../lib-lite/lib/jdom/saxon9he.jar" />
        <zip destfile="dist/src.zip">
            <zipfileset dir="../base/src/main/resources" />
            <zipfileset dir="../base/src/main/java" />
            <zipfileset dir="../eclipse-model/src/main/java" />
            <zipfileset dir="../eclipse-model/src/main/test" />
            <zipfileset dir="../eclipse-model/src/main/resources" />
            <zipfileset dir="../base-xml/src/main/java" />
            <zipfileset dir="src/main/java" />
            <zipfileset dir="src/main/resources" />
        </zip>
        <mkdir dir="build/META-INF" />
        <manifest file="build/META-INF/MANIFEST.MF">
            <attribute name="Class-Path" value="jdom-1.0.jar jaxen-core.jar jaxen-jdom.jar saxpath.jar junit-4.5.jar ant.jar log4j-1.2.15.jar saxon9he.jar" />
            <attribute name="Main-Class" value="ScriptGenerator" />
        </manifest>
        <jar destfile="dist/tools.jar" basedir="build" manifest="build/META-INF/MANIFEST.MF" />
    </target>
    <target name="compile-test" depends="init">
        <javac debug="true" destdir="build" source="1.5" target="1.5">
            <src path="../base/src/main/resources" />
            <src path="../base/src/test/java" />
            <src path="../base/src/main/java" />
            <src path="../base/src/test/resources" />
            <src path="../eclipse-model/src/main/java" />
            <src path="../eclipse-model/src/main/test" />
            <src path="../eclipse-model/src/main/resources" />
            <src path="../base-xml/src/test/java" />
            <src path="../base-xml/src/main/java" />
            <src path="../base-xml/src/test/resources" />
            <src path="src/main/java" />
            <src path="src/test/java" />
            <src path="src/main/resources" />
            <src path="src/test/resources" />
            <classpath path="../lib-lite/lib/jdom/jdom-1.0.jar" />
            <classpath path="../lib-lite/lib/jdom/jaxen-core.jar" />
            <classpath path="../lib-lite/lib/jdom/jaxen-jdom.jar" />
            <classpath path="../lib-lite/lib/jdom/saxpath.jar" />
            <classpath path="../lib-lite/lib/junit/4.5/junit-4.5.jar" />
            <classpath path="../lib-lite/lib/ant/1.6.2/ant.jar" />
            <classpath path="../lib-lite/lib/log4j/1.2.15/log4j-1.2.15.jar" />
            <classpath path="../lib-lite/lib/jdom/saxon9he.jar" />
        </javac>
    </target>
    <target name="build-test" depends="compile-test">
        <mkdir dir="dist" />
        <delete dir="dist" />
        <mkdir dir="dist" />
        <copy todir="dist" file="../lib-lite/lib/jdom/jdom-1.0.jar" />
        <copy todir="dist" file="../lib-lite/lib/jdom/jaxen-core.jar" />
        <copy todir="dist" file="../lib-lite/lib/jdom/jaxen-jdom.jar" />
        <copy todir="dist" file="../lib-lite/lib/jdom/saxpath.jar" />
        <copy todir="dist" file="../lib-lite/lib/junit/4.5/junit-4.5.jar" />
        <copy todir="dist" file="../lib-lite/lib/ant/1.6.2/ant.jar" />
        <copy todir="dist" file="../lib-lite/lib/log4j/1.2.15/log4j-1.2.15.jar" />
        <copy todir="dist" file="../lib-lite/lib/jdom/saxon9he.jar" />
        <zip destfile="dist/src.zip">
            <zipfileset dir="../base/src/main/resources" />
            <zipfileset dir="../base/src/test/java" />
            <zipfileset dir="../base/src/main/java" />
            <zipfileset dir="../base/src/test/resources" />
            <zipfileset dir="../eclipse-model/src/main/java" />
            <zipfileset dir="../eclipse-model/src/main/test" />
            <zipfileset dir="../eclipse-model/src/main/resources" />
            <zipfileset dir="../base-xml/src/test/java" />
            <zipfileset dir="../base-xml/src/main/java" />
            <zipfileset dir="../base-xml/src/test/resources" />
            <zipfileset dir="src/main/java" />
            <zipfileset dir="src/test/java" />
            <zipfileset dir="src/main/resources" />
            <zipfileset dir="src/test/resources" />
        </zip>
        <mkdir dir="build/META-INF" />
        <manifest file="build/META-INF/MANIFEST.MF">
            <attribute name="Class-Path" value="jdom-1.0.jar jaxen-core.jar jaxen-jdom.jar saxpath.jar junit-4.5.jar ant.jar log4j-1.2.15.jar saxon9he.jar" />
            <attribute name="Main-Class" value="ScriptGenerator" />
        </manifest>
        <jar destfile="dist/tools.jar" basedir="build" manifest="build/META-INF/MANIFEST.MF" />
    </target>
    <target name="test" depends="build-test">
        <mkdir dir="testoutput" />
        <delete dir="testoutput" />
        <mkdir dir="testoutput" />
        <junit printsummary="yes" fork="yes" haltonfailure="yes" haltonerror="no">
            <classpath>
                <fileset dir="dist">
                    <include name="**/*.jar" />
                </fileset>
            </classpath>
            <formatter type="plain" />
            <batchtest fork="yes" todir="testoutput">
                <fileset dir="src/main/java">
                    <include name="**/*Test.java" />
                </fileset>
                <fileset dir="src/test/java">
                    <include name="**/*Test.java" />
                </fileset>
                <fileset dir="src/main/resources">
                    <include name="**/*Test.java" />
                </fileset>
                <fileset dir="src/test/resources">
                    <include name="**/*Test.java" />
                </fileset>
            </batchtest>
        </junit>
    </target>
</project>

Using build generators programmatically

The following code snippet shows how the ScriptGenerator application uses the StandAloneAppBuildGenerator programmatically:

 

IBuildScriptGenerator genBuild = new StandAloneAppBuildGenerator(
        project);
genBuild.getBuildConfiguration().setSourceContributor(
        new SrcZipContributor(genBuild));
genBuild.getBuildConfiguration().setDistContributor(
        new ReferencedDependenciesContributor(genBuild));
genBuild.getBuildConfiguration().setExclusionFilter(
        new ExcludeTestCodeAndResourcesFilter());
ITestContributor testContributor = new JUnitContributor(genBuild);
testContributor.setDependsTaskName("build-test";
genBuild.getBuildConfiguration()
        .setTestContributor(testContributor);
genBuild.getBuildConfiguration().setSourceLevel(
        SourceLevel.ONE_FIVE);
genBuild.getBuildConfiguration().setTargetLevel(
        SourceLevel.ONE_FIVE);
if (mainClassName != null) {
    genBuild.getBuildConfiguration().setMainClass(mainClassName);
}

File buildOutput = new File(projectDirectory,
        "build.xml";
FileUtils.saveTextFile(buildOutput, genBuild
        .generateBuildScript());

The StandAloneAppBuildGenerator uses the following assumptions:

  • The eclipse project being built depends upon zero or more other eclipse projects.
  • The eclipse project may contain two types of source directories, of which one contains functionality and the other contains tests.
  • There are two possible ways to generate distributions for an eclipse project: one that contains functionality only, and another that contains both functionality and tests. It is assumed that distributions intended for release only contain functionality, while distributions intended for development include both functionality and code.

It generates a build script with the following features:

  1. There is a “clean” target that cleans, or removes, all build-output directories.
  2. There is a default “dist” target that produces a distribution suitable for release.
  3. There is a “test” target that depends upon a “build-test” target. The “build-test” target produces a development-time distribution. The “test” target effectively generates a development time distribution, followed by invoking tests.

Both release- and development-time distributions result in the generation of one or more JAR files. Of these, one JAR file (the “main” JAR) contains classes and resources contributed by the project being built and all of its dependent projects, recursively. The remaining JAR files are third-party libraries upon which the code in the main JAR depends (the “library” JARs). The main JAR’s manifest (the “META-INF/MANIFEST.MF” entry) is setup as follows:

  1. It contains an optional “Main-Class” attribute if the user specifies a “main” class.
  2. Its “Class-Path” attribute of its manifest points to the library JAR’s.

Next steps

I intend to pursue the following next steps:

  1. Create implementations of the IBuildScriptGenerator to generate build scripts for web applications.
  2. Create an eclipse plug-in to allow end users to generate build scripts (and, optionally, to invoke them) interactively within Eclipse.

Posted by Idle musings at 2/14/2010 12:57 PM
Categories: uncategorized
 

What did you think of this article?




Trackbacks
Trackback specific URL for this entry
  • No trackbacks exist for this entry.
Comments
  • No comments exist for this entry.
Leave a comment

Submitted comments will be subject to moderation before being displayed.

 Enter the above security code (required)

 Name

 Email (will not be published)

 Website

Your comment is 0 characters limited to 3000 characters.