brush clojure

Sunday, November 9, 2008

Browse sources and javadoc in maven

A few times I have tried to find the name of a library containing a particular class name. It is not something one needs often, but for those rare occasions I have not found a consistent way for such a search.

That shouldn't be too much work to do, I thought, and I haven't had a hobby project for a while. Two weekends and 50 GB of data later I'm launching a maven search site:
http://jaclaz.com/jacla/search.do

As an extra, all javadocs and sources can be browsed online if present in the repository.

PS: I underestimated how complex it is t keep a PC on 24/7 at home. The site is off-line.

PPS: On-line as http://alljavaclasses.com

Thursday, October 23, 2008

Measuring user session size with HtmlUnit and remote JMX

Keeping the user session small is a very important requirement for the application I am working on. The application architecture has tried to keep the size as small as possible, but how big is it eventually? The best way to know is to measure how much memory each new session costs. This turned out pretty straightforward with standard Java and open source tools:
- HtmlUnit provides a compact Web client. With little programming you can emulate some user action. An example follows which logs a new user.
- Start the Application Server in a separate JVM. Allow for remote JMX Monitoring.
- Start thousands of new sessions en follow what happens to memory usage in the server JVM using MemoryMXBean. JConsole is the easiest way to monitor the memory usage, but if you want to do some calculations or logging you'll have to gain access to the remote MXBean programmatically.

The method is independent of any server technology. You don't even need the application sources. A pic to illustrate what is going on:



and some implementation details:

Start the server and allow for remote JMX monitoring
Add the following to the java arguments:

-Dcom.sun.management.jmxremote.port=1444
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false


Program a simple web client to perform some user action

HtmlUnit's Webclient is a beauty. Here's an example of logging in in the Pluto portlet container an clicking on a link:


public void loginAndStartUseCase() throws Exception {
final WebClient webClient = new WebClient();
HtmlPage page = (HtmlPage) webClient.getPage("http://localhost:8080/pluto/portal/MyPage");
List forms = page.getForms();
boolean loginFormSubmitted = false;
for (HtmlForm htmlForm : forms) {
if (htmlForm.getActionAttribute().equals("j_security_check")) {
htmlForm.getInputByName("j_username").setValueAttribute("pluto");
htmlForm.getInputByName("j_password").setValueAttribute("pluto");
htmlForm.submit(htmlForm.getInputByName("login"));
loginFormSubmitted = true;
break;
}
}
if (!loginFormSubmitted) {
throw new RuntimeException("Login Form expected here");
}
page = (HtmlPage) webClient.getPage("http://localhost:8080/pluto/portal/MyPage");
page = clickLink(page, "MyUseCase");

page.getAllHtmlChildElements();
}

private static HtmlPage clickLink(HtmlPage page, String linkText) throws IOException {
LOG.debug("click " + linkText);
List anchors = page.getAnchors();
for (HtmlAnchor htmlAnchor : anchors) {
if (htmlAnchor.getFirstChild().asText().equals(linkText)) {
return (HtmlPage) htmlAnchor.click();
}
}
throw new RuntimeException("No link " + linkText);
}


Obtain a remote memory monitoring MXBean proxy

private static Object getMXBean(String beanType, Class beanClass, int port) throws MalformedURLException,
IOException, MalformedObjectNameException {
JMXServiceURL u = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:" + port + "/jmxrmi");
JMXConnector jmxConnector = JMXConnectorFactory.connect(u);
MBeanServerConnection beanServerConnection = jmxConnector.getMBeanServerConnection();

Hashtable props = new Hashtable();
props.put("type", beanType);
ObjectName on = new ObjectName("java.lang", props);

Set found = beanServerConnection.queryMBeans(on, null);
if (found.size() != 1) {
throw new RuntimeException("1 bean expected");
}

ObjectInstance objectInstance = found.iterator().next();

Object mxBean = ManagementFactory.newPlatformMXBeanProxy(beanServerConnection, objectInstance.getObjectName()
.toString(), beanClass);
return mxBean;
}


Start measuring!

Now you can start a new user session (or 10, or 1000), measure the memory impact and calculate whatever you want


MemoryMXBean memoryBean = (MemoryMXBean) getMXBean("Memory", MemoryMXBean.class, 1444);
for (many times) {
for(many_times) { loginAndStartUseCase(); }
memoryBean.gc();
long used = memoryBean.getHeapMemoryUsage().getUsed();
. . .



That's what my memory usage looks like after 10000 sessions:



This is an Excel chart built from comma-separated logging output.

The blue line shows the total memory usage - it usage grows consistently, it looks like no sessions have been released. The red line shows the calculated average session size. The average is pretty unreliable in the begin but stabilizes after a while - the cost of a new user just logging in and starting the use case seems to be around 3k.

Few remarks:
- Programming the Html WebClient for a more complex use case can be time consuming. Is there a good scripting alternative? Selenium was great, but starting a real browser for each user new session is not an option
- Make sure no sessions are released during the measurement. The test goes pretty fast, a normal session timeout of 15-30 min will be more than enough to reach your users limit.

Tuesday, October 7, 2008

Annotated controllers in Spring Portlet MVC

Usually trying out something new with the Spring framework goes smoothly. Reading the reference documentation provides me with the right amount of information to start and soon things work as expected - by me.

That is not what happened when I started using the annotated controllers in Spring Portlet MVC. I took me some time to get my first portlet working at all, and even more time to figure out how our team should write our portlets.

To sort things out, I started with a simple annotated controller that would allow me to collect a first and second name from the user, with few validations. A screenshot of two portlets on one Pluto page to get the idea:


That shouldn't be very difficult, right?

For me, it was. Here is a quick link to the final version.

The documentation is very clear about the design decision that all portlet details should be exposed to the portlet developer:

Most other portlet MVC frameworks attempt to completely hide the two phases from the developer and make it look as much like traditional servlet development as possible - we think this approach removes one of the main benefits of using portlets. So, the separation of the two phases is preserved throughout the Spring Portlet MVC framework.


Right, the portlet dispatcher has 2 phases, but the portlet annotations do not. You have to annotate your portlet's two phase controllers with the servlet annotation designed for one phase processing, so now and then you'll be short of words.

A small remark before we start: in your spring configuration, add the following interceptor to the DefalutAnnotationHandlerMapping:


<bean class="org.springframework.web.portlet.mvc.annotation.DefaultAnnotationHandlerMapping">
<property name="interceptors">
<bean class="org.springframework.web.portlet.handler.ParameterMappingInterceptor" />
</property>
</bean>


This will copy the action parameter from the ActionRequest to the ActionResponse, so that the same parameter is present in the subsequent RenderRequest.

Let's start with a simple controller:

@Controller
@RequestMapping("VIEW")
public class PersonController {

@RequestMapping(params = "action=editPerson")
public void action(@ModelAttribute("person")
Person person) {
. . .
}

@RequestMapping(params = "action=editPerson")
public ModelAndView render(@ModelAttribute("person")
Person person) {
return . . .
}
}
and a url to kick off the controller action:

<portlet:actionURL>
<portlet:param name="action" value="person" />
</portlet:actionURL>


Here comes the first surprise: only the action method is called in both phases, render is never called. And the first lesson learned is:

@RequestMapping - argument types determine the phase mapping

The implementation of AnnotationMethodHandlerAdapter gives the details: each annotated method can be marked as a 'render' and/or 'action' method, or none. Action methods do not compete to handle render requests and vice versa. A method is marked as an action method mapping, if one of its parameters is of type ActionRequest, ActionResponse, InputStream, Reader. The coresponding render method argument types are RenderRequest, RenderResponse, OutputStream, Writer.
So in the example above both methods are neither action nor render and the best match in both cases is the first method in the list: action.

To avoid such surprises, don't forget to *always* include one the above classes in the annotated function arguments. My personal rule is always to include a request argument. Here comes the next controller version:


@RequestMapping(params = "action=editPerson")
public void action(ActionRequest request, @ModelAttribute("person") Person person);

@RequestMapping(params = "action=editPerson")
public ModelAndView render(RenderRequest request, @ModelAttribute("person") Person person)


This is not very nice; it is not always clear why this finction argument is there and a perfectionist reviewer could even remove it. Support for portlet specifics in the annotation would be better:

@RenderMapping(params = "action=editPerson", phase="ACTION")
public void action(@ModelAttribute("person") Person person)


Let's do validations. The submitted form is handled in the action phase, all subsequent renders should show the same validation errors. The next controller version:


@Controller
@RequestMapping("VIEW")
public class PersonController {

@RequestMapping(params = "action=person")
public void action(ActionRequest request, Person person, Errors errors) {
ValidationUtils.rejectIfEmpty(errors, "firstName", "firstName.empty", "First name cannot be empty");
}

@RequestMapping(params = "action=person")
public ModelAndView render(RenderRequest request, Person person, Errors errors) {
return new ModelAndView("person");
}
}


The result is frightening: the error message is never shown from the following JSP fragment:


First Name
<form:input path="firstName"/>
<form:errors path="firstName"/>

The reason is the command object present in the arguments list of the render method; remove it an you'll see the validation warning. Here comes the next lesson:

Don't pass command objects as render parameters

Each time a command object argument is needed, it's bound again with the current request, no matter action or render request.
The next version of our controller is:


@RequestMapping(params = "action=person")
public ModelAndView render(RenderRequest request) {
. . .

With no command object as render parameter, the errors are preserved from the action phase to all subsequent render requests.

Now we have a non-empty implicit model which Spring holds for us in the session, hence the extra render parameter org.springframework.web.portlet.mvc.ImplicitModel=true in the request. Model data created in the ACTION phase is available in the RENDER phase.

Where to create your command object?

In the servlet example at http://static.springframework.org/spring/docs/2.5.x/reference/mvc.html#mvc-ann-requestmapping the bean is prepared in the GET request processing, bind and validate at following POSTs, in two separate methods. I like the idea of having two different methods for the first call when the bean is created, and for subsequent calls when it is filled and validated:

@RequestMapping(params = "action=startPerson")
public ModelAndView renderFormFirst(RenderRequest request) {
ModelAndView modelAndView = new ModelAndView("/WEB-INF/jsp/person.jsp");
Person person = new Person();
// initialize from model...
person.setFirstName("John");
modelAndView.addObject("person", person);
return modelAndView;
}

@RequestMapping(params = "action=person")
public ModelAndView renderForm(RenderRequest request, ModelMap modelMap) {
return new ModelAndView("/WEB-INF/jsp/person.jsp");
}


Having only a render handler for starting the process is fine: normally an action handler will set render parameters for the next render, like in:

@RequestMapping(params = "action=person")
public void action(ActionRequest request, ActionResponse response, Person person, Errors errors, SessionStatus sessionStatus) {
ValidationUtils.rejectIfEmpty(errors, "firstName", "firstName.empty", "Please, fill in your first name");
ValidationUtils.rejectIfEmpty(errors, "lastName", "lastName.empty", "Please, fill in your last name");
if (!errors.hasErrors()) {
response.setRenderParameter("action", "hello");
}
}


The next inconsstiency in the annotated portlet controllers:
@ModelAttribute methods called for both action and render phase

The @ModelAttribute annotated methods, as can be expected, is handled the same in both phases. Chances are you won't need most of them in the action phase, like the following example:

@ModelAttribute("countries")
public List getCountries() {
. . .
}


Once again, a better annotation would be:


@ModelAttribute("countries", phase="RENDER")
public List getCountries() {
. . .
}


As this is not available, my rule is: don't use Model attribute. Call it explicitly in the needed (probably render) method.

Wednesday, June 11, 2008

Generating IBM RSA Websphere projects with Maven

Rational Software Architect (RSA) may not be the best development environment, but when it comes to WAS-integrated IDEs, there are not that many choices. RSA is the familiar Eclipse packaged together with some plugins, which provide integration with Websphere. Starting with a Websphere Web application with RSA is easy - just choose the right project type and you can immediately run the application in Websphere from the IDE.

If you are working with Maven, however, there are very good reasons why you will not want to create any projects drom RSA. All of them boil down to the DRY principle - Don't Repeat Yourself. The Maven POM files already describe your projects and dependencies, why would you want to do this second time in RSA? You are going to make your release builds (and continuous integration) with Maven anyway, and any RSA project changes you make have no impact on the build. You may end up bulding something different than what you develop. Maintaining dependencies at two places - in maven poms and in your RSA project is not good, and Maven is much better in the dependency stuff.

Besides, why lock to RSA? If you can generate the IDE files for RSA/Websphere, so you can for Eclipse/Tomcat, for example.

The Maven Eclipse plugin generates the needed IDE files. The point is you want to be able to use them directly in RSA. It took me some time to get the POMs right, so I thought it is worth sharing.

Typically in a Websphere app, you have an EAR project, a Web project and several JAR projects. In this example, I start with 3 projects: ERA, WAR, JAR. Define them in the parent POM:



<modules>
<module>app-jar</module>
<module>app-web</module>
<module>app-ear</module>
<modules>


Further in the parent POM, declare you are using java 5.0:


<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>


N.B. Define 1.5 and not 5.0, this was fatal for my RSA installation!

There's nothing special about the JAR project, the generated eclipse project is fine. For the Web project, few more settings are essential. Assuming you have a maven webapp archetype project, include the following under build/plugins in the project's pom:


<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
<configuration>
<wtpversion>1.5</wtpversion>
<wtpContextName>myapp</wtpContextName>
<additionalProjectFacets>
<jst.java>5.0</jst.java>
<com.ibm.websphere.extended.web>6.1</com.ibm.websphere.extended.web>
<com.ibm.websphere.coexistence.web>6.1</com.ibm.websphere.coexistence.web>
</additionalProjectFacets>
</configuration>
</plugin>


The generated project will be a Dynamic Java project, and RSA wil recognise src/main/webapp as the web root folder. The web context name is also define in the plugin settings.

Similar setting are needed for the EAR POM:



<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
<configuration>
<wtpversion>1.5</wtpversion>
<additionalProjectFacets>
<jst.ear>1.4</jst.ear>
<com.ibm.websphere.extended.web>
6.1
</com.ibm.websphere.extended.web>
<com.ibm.websphere.coexistence.web>
6.1
</com.ibm.websphere.coexistence.web>
</additionalProjectFacets>
</configuration>
</plugin>


The generated project is recognized by RSA as an EAR project. Of course, you need a correct application.xml in src/main/application/META-INF. This project doesn't contain any code, but is still usefull for maven, as it will build the release EAR.

Once that all is done, go the the project root and run mvn eclipse:eclipse with any extra options you may need. Import the generated projects in RSA (File/Import/Existing projects int workspace). The resulting file structure is the well known maven webapp structure:



To run the application from RSA, drag the EAR project to 'Web Sphere Application Sever' in the Servers view and start te server.

There is one issue that has not been addressed here - how do you get all those WAS-specific IBM files. Update follows.

Profiles

It's a good idea to set all those settings in maven under a profile. For the next environment, add a new profile.

Hiccups

Yes, they are very present:
- Go to the app-web project, Properties, J2EE dependencies, Web libraries - all library dependencies are there but not the project dependency on app-dom, while it obviously should be. Add it. I couldn't find a solution for this, tips about it will be very appreciated.
- Make sure that the web module element in application.xml has no id attribute. The id runs out of sync each time project files are generated.
- Sadly enough, after fiddling with application and server deployment configuration, now and then I just have to restart RSA to make the web application running.

Sunday, April 27, 2008

Portlet development with Eclipse and Pluto

I had to set up a portlet development environment with Eclipse. The portlet is being built with Spring Portlet MVC 2.5.2 and will eventually run under Websphere, but Eclipse WTP and Websphere don't go together well. So I thought I'll try to run my portlet from Eclipse/WTP in a portlet container running under Tomcat. Pluto is such a container and does the job. It has its own peculiarities, it needs to generate its own web.xml using your web.xml and portlet.xml as input. Pluto provides an ant task to do this.

A good integration with eclipse and Maven should take care of the following issues:
- dependency management: download the needed Pluto ant task jar with all its dependencies.
- automatically generate web.xml for Pluto each time when needed - in Eclipse and during a Maven build.

The steps I took to get all nicely running:
1. Install Tomcat/Pluto bundle and configre it as a server in eclipse.
2. Tell Maven to save together all dependencies needed for the Pluto Ant Task.
3. Define an Ant build file with the Pluto task.
4. Add the execution of the Ant task in the Maven build.
5. Generate eclipse project, add the portlet application to the pluto server, run the server end configure a page with the portlet. It is fun to have more portlet instances on the same page.
6. Fix issues that prevent the portlet from running.
7. Define an Eclipse Custom Builder to update the web.xml automatically.

Once this is done, start Tomcat/Pluto from eclipse, log in as admin and add your portlet to a Pluto portal page. After that, just start the server and go to your portal page.

The details of how to perform the above steps:

Install Tomcat/Pluto bundle and configre it as a server in eclipse

I installed pluto 1.1.6 based on tomcat 5.5 - pluto-current-bundle.zip from http://archive.apache.org/dist/portals/pluto/

In the Eclipse Servers tab, define a new Apache Tomcat 5.5 server. IMPORTANT: In the server configuration page, select 'Use Tomcat installation (takes control of tomcat installation)' under Server locations. I don't understande why, but otherwise pluto doesn't see the portlet.


Tell Maven to save together all dependencies needed for the Pluto Ant Task.


First of all, the org.apache.pluto:pluto-ant-tasks dependency must be present in your (root) pom.xml. Then, add the following, preferably under a new profile pluto:


<profile>
<id>pluto</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>generate-resources</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>target/libs-for-pluto</outputDirectory>
<includeArtifactIds>
pluto-ant-tasks,pluto-util,pluto-descriptor-api,pluto-descriptor-impl,commons-logging,castor,xercesImpl
</includeArtifactIds>
</configuration>
</execution>
. . .


This way, all dependencies needed for the Pluto ant task will be copied in target/libs-for-pluto folder where they can be used by the ant task.

Define an Ant build file with the Pluto task

In a new file, e.g. generate-pluto-web-xml.xml, define the task which generates the web.xml file needed by pluto in the web-inf folder. A place is needed for the original web.xml that will be procerssd by the task, I've placed it in src/main/config.

The contents of update-web-xml.xml:

<project name="pluto-update-web-xml" basedir="." default="generate-web-xml">
<path id="pluto.task.classes" >
<fileset dir="target/libs-for-pluto" includes="**/*" />
</path>

<typedef name="passemble"
classname="org.apache.pluto.ant.AssembleTask" classpathref="pluto.task.classes" />

<target name="generate-web-xml">
<echo>Generating Pluto web.xml</echo>
<passemble webxml="src/main/config/web.xml"
portletxml="src/main/webapp/web-inf/portlet.xml"
destfile="src/main/webapp/web-inf/web.xml" />
</target>

</project>

At this point, you should be able to run this ant task from the command line. It is better to do this from maven, see next step.

Add the execution of the Ant task in the Maven build.
In pom.xml, add the following under the profile pluto:

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>generate-resources</phase>
<configuration>
<tasks>
<ant antfile="${basedir}/generate-pluto-web-xml.xml">
<target name="generate-web-xml" />
</ant>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>


At this point, running manually
mvn eclipse:eclipse -P pluto
will produce a web.xml as needed for pluto. The original web.xml stays in src/main/config. Needless to say, you must have a good portlet.xml in src/main/webapp/WEB-INF. Don't forget to make it a Web project.

Configure your portlet in Pluto
That is pretty straightforward - go to the Admin tab in pulto, and add your portlet to a new or existing page. The portlets in the application you deployed on pluto should be visible in the Portlet Application list box on the admin page.
Fix issues

When I first started the page with the portlet, I got the following stack trace:

java.lang.ClassCastException: org.springframework.web.servlet.support.JstlUtils$SpringLocalizationContext cannot be cast to java.lang.String
org.apache.taglibs.standard.tag.common.fmt.BundleSupport.getLocalizationContext(BundleSupport.java:134)
org.apache.taglibs.standard.tag.common.fmt.SetLocaleSupport.getFormattingLocale(SetLocaleSupport.java:251)
org.apache.taglibs.standard.tag.common.fmt.FormatDateSupport.doEndTag(FormatDateSupport.java:116)
org.apache.jsp.WEB_002dINF.themes.pluto_002ddefault_002dtheme_jsp._jspx_meth_fmt_formatDate_0(pluto_002ddefault_002dtheme_jsp.java:602)


Whatever the reason, I just removed the tag causing the error: a tag in pluto-default-theme.jsp and voila, it worked.

Define an Eclipse Custom Builder to update the web.xml automatically

You can define a custom bulder to update your web.xml in src/main/webapp/WEB-INF automatically each time you build the project or change web.xml in src/main/config. That's not really necessary, unless you are going to change web.xml often.

Alternatively, run mvn generate-resources manually each time you change web.xml.

To define a custom builder:

- In Eclipse, go to Project properties, Builders. New, Ant Builder.
In the Main tab, select the ant file and the working folder.


In the Refresh tab, select Specific Resources an click on Specify Resources. Select src/main/webapp/WEB-INF/web.xml. After each build, eclipse will refresh this resource.

In the Build Options tab, check 'Specify working set of relevant resources'. Click on Specify resources and select main/webapp/config/web.xml.

Now each time you change this file you see the ant task output in the console and the file is updated.

Tuesday, April 15, 2008

Open Session In View with JSF and Spring

I expected that a quick google session would immediately deliver the standard solution for this situation, but it didn't go this way.

The problem: I want Open Session in View for an JSF/Spring application. Ok, I know there is something fishy about open session in view, but believe me, for this application this is just fine. There is a chance that the application will become a portlet eventually, so I didn't want to have to handle portlet and servlet filters issues too. Instead, I want to use the convenient hooks provided by JSF to open and close the hibernate session - phase listeners. Happily, the Spring-provided OpenSessionInViewFilter reveals the technicalities of how Spring deals with the hibernate session factory.

Here is the result:

public class HibernateRestoreViewPhaseListener implements PhaseListener {

public void afterPhase(PhaseEvent event) {
}

protected SessionFactory lookupSessionFactory() {

FacesContext context = FacesContext.getCurrentInstance();
ServletContext servletContext = (ServletContext) context
.getExternalContext().getContext();
WebApplicationContext wac = WebApplicationContextUtils
.getWebApplicationContext(servletContext);
return (SessionFactory) wac.getBean("hibernate-session-factory",
SessionFactory.class);
}

public void beforePhase(PhaseEvent event) {
SessionFactory sessionFactory = lookupSessionFactory();
if (!TransactionSynchronizationManager.hasResource(sessionFactory)) {
Session session = getSession(sessionFactory);
TransactionSynchronizationManager.bindResource(sessionFactory,
new SessionHolder(session));
}
}

public PhaseId getPhaseId() {
return PhaseId.RESTORE_VIEW;
}

protected Session getSession(SessionFactory sessionFactory)
throws DataAccessResourceFailureException {
Session session = SessionFactoryUtils.getSession(sessionFactory, true);
session.setFlushMode(FlushMode.MANUAL);
return session;
}
}

The session gets closed when Render Response phase is finished:


public class HibernateRenderResponsePhaseListener implements PhaseListener {

public void afterPhase(PhaseEvent event) {
SessionFactory sessionFactory = lookupSessionFactory();

SessionHolder sessionHolder = (SessionHolder) TransactionSynchronizationManager
.unbindResource(sessionFactory);
closeSession(sessionHolder.getSession(), sessionFactory);
}
. . .



Don't forget to register the listeners in faces-config.xml

<lifecycle>
<phase-listener>
...HibernateRestoreViewPhaseListener
</phase-listener>
<phase-listener>
...HibernateRenderResponsePhaseListener
</phase-listener>
</lifecycle>



I have the feeling this is not the final solution and this post will be edited soon.