Sample Clojure/Compojure Web Application with Maven and IntelliJ

After learning a bit of Clojure I decided to start a simple Clojure web application project to try more things out. I expected I would easily find tons of sample projects to start from, but at the end I couldn't find a single Getting Started for the type of project I wanted:
- I am too comfortable with Maven to want to change it with leiningen or anything else.
- Starting a quick Jetty server is great but I still want to have the traditional WAR.

I don't know, may be these are all things a hard-core Clojure developers laugh at, but that is what I feel comfortable with at the moment.

Googling I found this very complete and precise blog entry:
http://blog.kartikshah.com/2010/12/how-to-intellij-idea-for.html It all worked as described, using leiningen. I tweaked it a bit:
- mavenized it: removed leningen, edited the pom to add clojure compilation using the clojure-maven-plugin;
- removed the GAE stuff;
- renamed a few folders and packages.

See below the core Clojure class implementing HttpServlet and routing request using Compojure:

(ns pwc.core
  (:gen-class :extends javax.servlet.http.HttpServlet)
  (:use compojure.core
        ring.util.servlet)
  (:require [compojure.route :as route]))

(defroutes hello
  (GET "/" [] "Hello World Wide Web!")
  (GET "/user/:id" [id]
    (str "Hello user " id ))
  (route/not-found "Page not found"))

(defservice hello)

The Compojure routing is explained here

The project is available at git. Build a war from maven or import the pom.xml in IntelliJ and run the war artifact in a servlet container. I ran it in tomcat, starting conservative: in the run configuration I deployed the non-exploded war, running the package goal before launch.

**NOTE** One month later, I don't think this is the way to go if you want to dive into Clojure. I've joinde the leiningen/emacs way.

Staged Maven build in Hudson with shared workspace

Staged Builds

Continous Integration is all about providing feedback, the faster the better. It is all being said by Martin Fowler, I'll just quote a crucial sentence from the Staged Builds section:

Probably the most crucial step is to start working on setting up a staged build.

So true, I had the opportunity to think about it each time when I had to wait for 40 long minutes before I see the build turning red or green. These exhaustive cryptographic tests taking 8 minutes or these cool integration tests taking 6 minutes - let them run each time but not as a part of the commit build. Enough said, check the article

Hudson provides all needed parts to set up a staged maven build, but I wasn't able to find a clear guidance on how to glue a staged build together. I found some useful hints and even better comments here. There is definitely more than one way of setting up a staged build, here is how I put the puzzle pieces together:

The scenario
Let's start with an imaginary scenario to illustrate the point. Let there be 3 modules: A, B and C. Module B's tests take too long an are unlikely to fail. Module C is an assembly using A and B and on which some integration tests are run.

Let's see how to make a 3-stage build:
* stage1 compiles and tests A, only compiles B
* stage2 tests (the already compiled) B
* stage3 packages and tests C (using A and B snapshots).

The key is to let the 3 stages run in the same workspace, so that each stage picks up where the previous one stopped. Behavior can be coupled to a stage with a profile, so for this example I define 3 maven profiles - stage1, stage2 and stage3.

It is important that the profiles add, not suppress tasks. Then the profiles can be used in a additive way, i.e. -Pstage1,stage2 will execute all tasks linked to both stage1 and stage 2. to take tests as example, do no -DskipTests=true in a stage, but define empty test files set in one stage and extend it in the next stage.

The parent pom:

<modules>
  <module>A</module>
  <module>B</module> 
</modules>
<profiles>
  <profile>
   <id>stage3</id>
   <modules>
    <module>C</module>
   </modules>
  </profile>
</profiles>

Nothing special in the A's pom. The B's pom:

<build>
<plugins>
  <plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <configuration>
    <includes>
     <include>No default tests included</include>
     </includes>
    </configuration>
   </plugin>
  </plugins>
</build>

<profiles>
  <profile>
   <id>stage2</id>
   <build>
   <plugins>
    <plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-surefire-plugin</artifactId>
     <configuration>
      <includes>
       <include>**/Test*.java</include>
       <include>**/*Test.java</include>
       <include>**/*TestCase.java</include>
      </includes>
     </configuration>
    </plugin>
   </plugins>
  </build>
</profile>
</profiles>

 
Needed Hudson Plug-ins

The URL SCM plug-in lets you access archives from another build as SCM URLs. The trick is to let one stage provide the workspace for the next stage. Potentially this means passing huge archives between build nodes, more on that later. The M2 Extra Steps Plugin is needed to unzip the archive from the previous stage before starting the maven build.

Stage 1 Job

Define a Maven job that checks out the workspace from SCM. In the job settings, check 'Archive artifacts' and select the whole workspace (pattern **/*.*). The goal: clean, install.

Stage 2 Job

Copy Stage1 Job to a new job Stage2. In the Source Code Management section, select URL Copy. The URL to watch is something like http://hudson-server/job/stage1/lastSuccessfulBuild/artifact/*zip*/archive.zip. Go to Configure M2 Extra Build Steps and add an Execute Shell step: unzip archive.zip The goal: -Pstage2, install. Note that there is no clean. Finally, archive the workspace for the next stage.

Stage 3 Job

The Stage3 is much like stage2. The URL to watch will be http://hudson-server/job/stage2/lastSuccessfulBuild/artifact/*zip*/archive.zip. The goal will contain -Pstage2 but no clean.

Conclusion

This setup is pretty straightforward an pretty new for me, but I already notice a few points I am not really happy with:
- Packing, transferring and unpacking the workspace could cost a lot of time.
- The separate projects are too much aware about the build they will be part of. I'd prefer to be able to tweak the build without editing each particular project. Possibly inventing some profile names more descriptive than stage1, stage2 is the way to go.