I’m looking forward to testing out Google’s recent release of their App Engine for Java. Whilst it may not have the a full JRE there (check the
whitelist to see the supported classes) there is plenty there to build a full-featured application (persistence, email, scheduling, etc.) We’re looking to make a prototype version of one of our marketing search applications which should test out a lot of the features there.

The SDK comes with a nice Eclipse plugin (nice, if you use Eclipse as we do) which looks as if it makes development and deployment very easy. I’ll post more as I get through the prototype development and the rosy hue begins to fade from my spectacles :-)

Iteration 1 of the prototype was just to put together some simple objects and persist them in GAEJ’s datastore. There’s no relational database exposed via the app engine, instead persistent storage is implemented via
Datanucleus and accessed using JDO or JPA.

After fiddling around with JPA, I chose JDO as it seemed easier for the purposes. Don’t go thinking that you can use JPA and then switch your Hibernate code over, Hibernate isn’t really supported.

I created a simple class Search which owned a list of SearchResults and used annotations to mark up the attributes I wanted to persist. Then it’s just a case of using a PersistenceManager to manage the object’s lifecycle. I encapsulated all of these calls in to a SearchDataService to make defining transaction boundaries a bit easier. Having said that I did find issues in persisting relationships within a transaction which I wasn’t sure whether was down to dodgy configuration on my part or support for those features in the JDO implementation GAE4J. There’s a good tutorial on the Datanucleus JDO implementation here.

All went well until I wanted to change my List of SearchResults in to a Set (in order to make sure I didn’t store the same result twice).
At present, Datanucleus doesn’t seem to support Sets. When I implemented SearchResult as a Set I got a “Could not extract parent key from query expression” error. This is documented further at posting on the Google Group for GAE4J

It’s a bit of a limitation, but not the end of the world. It just means you have to do a bit of work before saving the results. Other than that it is pretty easy to get up and running persisting objects and deploying them to the app engine in the wild. The Admin Console that comes with
GAE4J is basic but allows you to browse and manipulate the data you’ve uploaded.

Next on the to-do list is using the app engine to connect to an HTTP API and parsing XML results.

Next up was to enhance the prototype app by sending an HTTP request for an Atom output, parsing, and storing details from the results.

Sending the HTTP request was straightforward and used the java.net libraries. There’s plenty of resources out there about that so I’m not going to go in to it.

More “interesting” is the inevitable fun of dealing with XML in Java. In order to find the elements I was after in the Atom output I decided to use XPath as it’s a quick and fairly intuitive way of navigating a document and beats the getChildren() API calls of other Java libraries.

XPath support is within the javax.xml.XPath libraries. The specification requires abstract classes only, but Sun’s JRE provides an implementation in the form of com.sun.org.apache.xerces.internal.impl.xpath. However this library isn’t present in the Google JRE in the App Engine. This led to a runtime error
(when deploying to the App Engine but not in development) where it couldn’t instantiate an XPathFactory
(“XPathFactory#newInstance() failed to create an XPathFactory”).

In order to solve this, I needed to do the following:

• Include xalan in the project by copying the xalan jar to the /war/WEB-INF/lib directory
• Update the Eclipse build path to include this library
• Get a new XPathFactory by directly instantiating the Apache Xalan class:
  XPathFactory factory = new org.apache.xpath.jaxp.XPathFactoryImpl();
• Tell the App Engine to load the Xalan implementation by adding the following to the
  war/WEB-INF/appengine-web.xml file:
  

It’s slightly frustrating, but anyone who has tried to do anything using Java and XML will be familiar with the class loading hoops you have to jump through, so it’s not really an App Engine issue.

Anyway, following that I was able to retrieve and navigate the Atom output and store key pieces of information in the GAEJ datastore.

Next on the list will be scheduling and email …

The app engine makes scheduling and email really easy. I wanted to take our news feed search, run it daily and then create a mailer that would send a digest to consumers each evening.

GAEJ has a cron-type scheduler built in, and creating scheduled jobs is as easy as adding a file called cron.xml to the classpath. Here is an
example file. You’ll see the syntax is much friendlier than the normal cron entries.

<cronentries>
	<cron>          
		<url>runSearches</url>
		<description>Run any searches required.</description>
		<schedule>every 12 hours</schedule>
	</cron>
	<cron>
		<url>/sendDigest?id=2001</url>
		<description>Send daily digest for West Ham news</description>
		<schedule>every 24 hours</schedule>
	</cron>
</cronentries>

Email support is taken care of through the standard javax.mail packages. There are plenty of good tutorials out there, like
this jGuru one on javasoft, so I won’t go in to the details.

A couple of things to note though. You don’t need to specify the mail host, but instead obtain it from the GAEJ properties:

Properties props = new Properties();

Session session = Session.getDefaultInstance(props, null);

…

Message msg = new MimeMessage(session);

Also, you need to make sure that you use the Google account you registered for GAEJ as the from address:
msg.setFrom(new InternetAddress("mygoogleaccountgmail.com"));@

And it’s as easy as that!

Last on the list of things to look at is using the Memcache API that GAEJ provides in order to cache content.

Caching

The final part of the prototype aggregator was to test out the caching capabilities. Caching for its own sake can cause pitfalls but in an environment like GAEJ, where every disk read costs, caching expensive data makes sense – even more so as GAEJ provides a robust cache implementation for you.

I wanted to cache the results from our news aggregator so the daily search results could be held in memory after the first retrieval. In order to do this I just needed to alter my data service to check the cache before going to disk. GAEJ uses the Memcache Java API which supports the standard JCache JSR-107 interface.

The nice thing about the implementation is that there’s no configuration needed. Each deployment in the GAEJ has access to a
javax.cache.CacheManager and you create a new cache through the following:

cache = CacheManager.getInstance().getCacheFactory().createCache(Collections.emptyMap());

Then it’s just a case of putting to and getting from the cache … and identifying the lifecycle events for your cache items! But you’d have done that already, right ?!

In conclusion

The app engine is a great way to deploy and run Java web applications. The Eclipse plugin makes development and deployment very easy . It has a rich set of services under the hood. Caching, email, and scheduling are a breeze. The image manipulation is a little more involved, but still
much much easier than using other libraries or (perish the thought) writing it yourself. The persistence layer feels a little creaky after using Hibernate but is good enough for simple models. There’s a good management console and best yet, it’s free under certain limits! Google have just lowered these but we didn’t run in to any issues. I’d recommend trying it out … we certainly plan to develop products that can be deployed to the app engine. I’ll post to let you know how we get on with that.

I was happy to see this blog posting which announces incoming email support for the App Engine. It definitely makes it possible to create more powerful applications with better legacy integration. We’ll be trialling out shortly and post here with our findings.

I was caught out the other day whilst writing a data service when I got a JDOException telling me I was accessing a field (in this case it was a serialized Collection) that hadn’t been detached.

This happens when you try to access an object, or collection within that object, which is still under the control of the PersistenceManager.

The PersistenceManager manages the persistence lifecycle of datastore objects, making sure they are synchronised with the underlying store. This is great in the data service layers, but most web applications will want to manipulate and display objects outside of this control before putting them back in to persistence-land. This is what detaching allows you to do.

The JDO spec is well worth a read as it describes the lifecycle in detail (a lot of detail :-)

There are several ways (including doing it manually) how you can handle the persisted object’s lifecycle and detaching in particular, but as we use transactions in our app the easiest was to use the JDO feature detachAllOnCommit which detaches an object and its collections. The code snippet below shows how.

Incidentally, although you can set detachAllOnCommit in JDO properties I prefer the explicit approach as it doesn’t hide potentially complex behaviour in configuration.

pm = jdoPersistenceHelper.getPersistenceManager();

pm.setDetachAllOnCommit(true);

tx = pm.currentTransaction();				

tx.begin();					

p = pm.getObjectById(Participant.class,participantId);				

tx.commit();					

pm.close();

...
// manipulate the paricipant here, for example
MailManager.invite(p.getEmail());

One final word about using Transactions in the Google App Engine. You can only enlist objects in a transaction that belong to the same EntityGroup. This means an object and its children (even objects of the same type are in different EntityGroups). We designed our application so that this would be easy for us, but it required careful thought and an app engine specific design. There’s more about transactions in the Google App Engine here which also addresses other ways to detach.

We’ve just finished a presentation on tips and guidelines we’ve learned whilst working with the GAEJ.

It covers issues problems detaching objects, embedding associations, when to denormalise, and gives some advice on data modelling and when GAEJ may or may not be a good choice.

Take a look and we’d be really interested to hear your tips and experiences too.

This (unimaginatively titled) post explains how to set up remote debugging for Tomcat6 running as a Windows service using Eclipse and first, why you might want to.

Why remote debug?

Once an application grows over a certain size it becomes more difficult to work out what’s going on from watching its behaviour – e.g. error messages and stack traces. This is equally true for small but layered or componentised applications (and doubly true for large layered and componentised applications!).

One approach is to add log statements outputting debug information like variable names, loop counter statuses, null checks, but this quickly become more time-consuming than it’s worth, and being able to debug in the IDE gives you more control and ultimately saves a lot of time.

You might not believe me, but try a few rounds of entering log statements and restarting and then we’ll talk. If you’re not even doing that and entering System.out.println then shame on you!

How to set it up for Tomcat6/Eclipse

This article covers setting it up for Tomcat6 only because it’s slightly different. There are undoubtedly resources out there for your favourite web server / IDE combination so take a look.

In order to remotely debug the IDE needs to connect to another port where your app server is running (another meaning one than the port on which it accepts HTTP connections).

There are two parts to making this happen:

1. Telling the JVM on the web server which port to use for remote debugging
2. Telling your IDE which host and port to listen to

Tomcat6 is usually set up as a service on Windows and that makes setting the JVM arguments a little more complicated.

1. Open a command prompt in your /bin folder, e.g. “C:\Program Files\Apache Software Foundation\Tomcat 6.0\bin”
2. Use the tomcat6.exe utility to update the existing Tomcat6 service to add the JVM arguments. The arguments begin with -X and are separated by semi-colons.
   

   tomcat6 //US// —JvmOptions -Xdebug; -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n

The first argument tells the JVM to remotely debug and the second to accept a socket connection on port 8000. You don’t have to choose this port, any sensible free port will do.

That’s the web server set up. Now you need to set up Eclipse.

1. Go to Run>Debug Configurations and click the icon at the top of that screen to create a new one
2. Enter the host (in the picture I’m running my server locally hence localhost) and the port that you configured it on.

That’s all the set up. Now you just need to start the web server, set a breakpoint (e.g. in your login.jsp) and navigate to the login page on your development server. You should see your breakpoint hit.

Good luck!

I was at a webinar last night about AWS Elastic Beanstalk, Amazon’s newly released web app cloud deployment platform.

The timing of the webinar (10 a.m. PST == dinner time in the UK) meant that I had to leave before the end, but it was a pretty interesting introduction and left me wanting to check out.

AWS Elastic Beanstalk is a Java-Tomcat stack that allows easy deployment of JEE applications (via AWS itself, via an Eclipse toolkit, via the command line, or SDK/API). Perhaps the best thing about deployment is that you can change a bunch of configuration items a lot easier than having to redeploy.

The usual but incredibly useful stuff about scalability and growth being taken care for you holds just as it does for Google App Engine for Java. As does the fact that there’s a console with which you can monitor your apps – again via the AWS dashboard.

It comes with database support (MySQL, SQL*Server, Oracle, DB2). JRuby support is also in there and the stack seems more standard that the GAEJ stack. That coupled with the fact that you have root access so can go low-level with root access means a lot more flexibility than GAEJ gives.

AWS have a sample app that can be deployed so you can see although I’m not sure I get the point of that really (won’t anybody inclined to or capable of deploying will probably have their own sample Java web apps they’d like to try out anyway?).

Other platforms are planned, but Amazon were coy about even mentioning let alone committing to a roadmap on the webinar. I figured they just hadn’t decided yet but that’s no matter.

We’re planning to try it out in the next month or so with a JRuby port of Jair so will post up our findings as we go.

I’m currently working on a client project where we’re making web service calls via JAX-RPC. Web Services are hugely important and useful but I can’t be the only person to think they’re a royal pain to develop and deploy!

If you’re using Apache Axis then there are some inbuilt tools that can help work out what’s going wrong when things go wrong. The most useful is tcpmon which acts as an “app-in-the-middle” and displays your SOAP request and response.

Here’s a quick how-to in setting up tcpmon in order to intercept your development web services calls and responses. Once you start using it the rest is pretty obvious but I’ve also included some screenshots of the input and output just so you can see.

The instructions assume Windows but replace the command prompts and environment variables with the usual in *nix and it all should work the same.

1. Open a command prompt and set the Java classpath to pick up the Axis classes (tcpmon is bundled with them)
2. set CLASSPATH=CLASSPATH;[my axis classpath]/wsif.jar;[my axis classpath]/axis-1.4.jar;[my axis classpath]/axis-ant-1.4.jar;[my axis classpath]/commons-logging-1.0.4.jar;[my axis classpath]/commons-discovery-0.2.jar;[my axis classpath]/wsdl4j-1.5.1.jar
3. Start tcpmon java org.apache.axis.utils.tcpmon
4. Enter the port where you want to run tcpmon (in the example below I use 1976 but check with netstat to find a free port)
5. Make the webservice call from your test client (the code snippet below is from a JUnit test method)

	// Note that the port is the tcpmon port and it will be forwarded to the web service port
	String endpoint = "http://localhost:1976/axis/services/PortingPort";

	Service service = new Service();
	Call call = (Call) service.createCall();

	// Prepare the call for the Web service
	call.setTargetEndpointAddress(new java.net.URL(endpoint));
	call.setOperationName("myService");
	MyParameterRequestType por = new MyParameterRequestType();
	por.setGreeting("Hello");
	call.addParameter("requestParam",new QName("http://mywsdl.endpoint.for.service/Service"),ParameterMode.IN);
	call.setReturnType(new QName("myParameterResponse"));
	call.invoke(new Object[] {por});

Below is the tcpmon “console” where you set the port that tcpmon listen on.

… and this is the request and response intercepted by tcpmon after making the test call