I’m looking forward to testing out Google’s recent release of their App Engine for Java. Whilst it may not have the a full JRE there (check the
whitelist to see the supported classes) there is plenty there to build a full-featured application (persistence, email, scheduling, etc.) We’re looking to make a prototype version of one of our marketing search applications which should test out a lot of the features there.

The SDK comes with a nice Eclipse plugin (nice, if you use Eclipse as we do) which looks as if it makes development and deployment very easy. I’ll post more as I get through the prototype development and the rosy hue begins to fade from my spectacles :-)

Iteration 1 of the prototype was just to put together some simple objects and persist them in GAEJ’s datastore. There’s no relational database exposed via the app engine, instead persistent storage is implemented via
Datanucleus and accessed using JDO or JPA.

After fiddling around with JPA, I chose JDO as it seemed easier for the purposes. Don’t go thinking that you can use JPA and then switch your Hibernate code over, Hibernate isn’t really supported.

I created a simple class Search which owned a list of SearchResults and used annotations to mark up the attributes I wanted to persist. Then it’s just a case of using a PersistenceManager to manage the object’s lifecycle. I encapsulated all of these calls in to a SearchDataService to make defining transaction boundaries a bit easier. Having said that I did find issues in persisting relationships within a transaction which I wasn’t sure whether was down to dodgy configuration on my part or support for those features in the JDO implementation GAE4J. There’s a good tutorial on the Datanucleus JDO implementation here.

All went well until I wanted to change my List of SearchResults in to a Set (in order to make sure I didn’t store the same result twice).
At present, Datanucleus doesn’t seem to support Sets. When I implemented SearchResult as a Set I got a “Could not extract parent key from query expression” error. This is documented further at posting on the Google Group for GAE4J

It’s a bit of a limitation, but not the end of the world. It just means you have to do a bit of work before saving the results. Other than that it is pretty easy to get up and running persisting objects and deploying them to the app engine in the wild. The Admin Console that comes with
GAE4J is basic but allows you to browse and manipulate the data you’ve uploaded.

Next on the to-do list is using the app engine to connect to an HTTP API and parsing XML results.

Next up was to enhance the prototype app by sending an HTTP request for an Atom output, parsing, and storing details from the results.

Sending the HTTP request was straightforward and used the java.net libraries. There’s plenty of resources out there about that so I’m not going to go in to it.

More “interesting” is the inevitable fun of dealing with XML in Java. In order to find the elements I was after in the Atom output I decided to use XPath as it’s a quick and fairly intuitive way of navigating a document and beats the getChildren() API calls of other Java libraries.

XPath support is within the javax.xml.XPath libraries. The specification requires abstract classes only, but Sun’s JRE provides an implementation in the form of com.sun.org.apache.xerces.internal.impl.xpath. However this library isn’t present in the Google JRE in the App Engine. This led to a runtime error
(when deploying to the App Engine but not in development) where it couldn’t instantiate an XPathFactory
(“XPathFactory#newInstance() failed to create an XPathFactory”).

In order to solve this, I needed to do the following:

• Include xalan in the project by copying the xalan jar to the /war/WEB-INF/lib directory
• Update the Eclipse build path to include this library
• Get a new XPathFactory by directly instantiating the Apache Xalan class:
  XPathFactory factory = new org.apache.xpath.jaxp.XPathFactoryImpl();
• Tell the App Engine to load the Xalan implementation by adding the following to the
  war/WEB-INF/appengine-web.xml file:
  

It’s slightly frustrating, but anyone who has tried to do anything using Java and XML will be familiar with the class loading hoops you have to jump through, so it’s not really an App Engine issue.

Anyway, following that I was able to retrieve and navigate the Atom output and store key pieces of information in the GAEJ datastore.

Next on the list will be scheduling and email …

The app engine makes scheduling and email really easy. I wanted to take our news feed search, run it daily and then create a mailer that would send a digest to consumers each evening.

GAEJ has a cron-type scheduler built in, and creating scheduled jobs is as easy as adding a file called cron.xml to the classpath. Here is an
example file. You’ll see the syntax is much friendlier than the normal cron entries.

<cronentries>
	<cron>          
		<url>runSearches</url>
		<description>Run any searches required.</description>
		<schedule>every 12 hours</schedule>
	</cron>
	<cron>
		<url>/sendDigest?id=2001</url>
		<description>Send daily digest for West Ham news</description>
		<schedule>every 24 hours</schedule>
	</cron>
</cronentries>

Email support is taken care of through the standard javax.mail packages. There are plenty of good tutorials out there, like
this jGuru one on javasoft, so I won’t go in to the details.

A couple of things to note though. You don’t need to specify the mail host, but instead obtain it from the GAEJ properties:

Properties props = new Properties();

Session session = Session.getDefaultInstance(props, null);

…

Message msg = new MimeMessage(session);

Also, you need to make sure that you use the Google account you registered for GAEJ as the from address:
msg.setFrom(new InternetAddress("mygoogleaccountgmail.com"));@

And it’s as easy as that!

Last on the list of things to look at is using the Memcache API that GAEJ provides in order to cache content.

Caching

The final part of the prototype aggregator was to test out the caching capabilities. Caching for its own sake can cause pitfalls but in an environment like GAEJ, where every disk read costs, caching expensive data makes sense – even more so as GAEJ provides a robust cache implementation for you.

I wanted to cache the results from our news aggregator so the daily search results could be held in memory after the first retrieval. In order to do this I just needed to alter my data service to check the cache before going to disk. GAEJ uses the Memcache Java API which supports the standard JCache JSR-107 interface.

The nice thing about the implementation is that there’s no configuration needed. Each deployment in the GAEJ has access to a
javax.cache.CacheManager and you create a new cache through the following:

cache = CacheManager.getInstance().getCacheFactory().createCache(Collections.emptyMap());

Then it’s just a case of putting to and getting from the cache … and identifying the lifecycle events for your cache items! But you’d have done that already, right ?!

In conclusion

The app engine is a great way to deploy and run Java web applications. The Eclipse plugin makes development and deployment very easy . It has a rich set of services under the hood. Caching, email, and scheduling are a breeze. The image manipulation is a little more involved, but still
much much easier than using other libraries or (perish the thought) writing it yourself. The persistence layer feels a little creaky after using Hibernate but is good enough for simple models. There’s a good management console and best yet, it’s free under certain limits! Google have just lowered these but we didn’t run in to any issues. I’d recommend trying it out … we certainly plan to develop products that can be deployed to the app engine. I’ll post to let you know how we get on with that.

I was happy to see this blog posting which announces incoming email support for the App Engine. It definitely makes it possible to create more powerful applications with better legacy integration. We’ll be trialling out shortly and post here with our findings.

The last post a good month and a half ago spoke about porting our revenue management product Jair to JRuby and deploying on Amazon’s Elastic Beanstalk.

Well, we’ve ported the app and apart from a few hiccups with gem versions not being supported in JRuby – we had to upgrade from the CSV to FasterCSV gem which I guess is no bad thing! – all went smoothly.

I really like JRuby as there’s something satisfying about writing in Ruby but deploying on to Tomcat. Even more satisfying is when the application actually runs up on Tomcat!

I’m not yet fully convinced as on shutdown Tomcat issues dire warnings about memory leaks being “likely to happen” but we’re going to do some application profiling soon so we’ll see.

Rather than use Elastic Beanstalk (only available in Amazon’s US regions at the moment) we just deployed on to a customised Bitnami Tomcat stack on AWS EC2. Again, this seemed a lot easier than the last time I tried although perhaps that’s down to having the right tools installed beforehand(!).

I’ve got a bunch of tips / things that we learned from the porting exercise and also AWS deployment so I’ll post those up in the next few days.

For us, AWS rather than GAEJ, is the way forward. Although there are different services on GAEJ (can only think of caching and IM offhand) so it could be useful for some apps, but how long before AWS offer the same? As well as that with AWS there is the freedom to install things like memcached etc.

I do feel a bit guilty though every day at 4p.m. when the True North chatbot (one of our apps on the Google App Engine) pings me on IM … maybe it’ll be its turn for porting next.