The last post a good month and a half ago spoke about porting our revenue management product Jair to JRuby and deploying on Amazon’s Elastic Beanstalk.

Well, we’ve ported the app and apart from a few hiccups with gem versions not being supported in JRuby – we had to upgrade from the CSV to FasterCSV gem which I guess is no bad thing! – all went smoothly.

I really like JRuby as there’s something satisfying about writing in Ruby but deploying on to Tomcat. Even more satisfying is when the application actually runs up on Tomcat!

I’m not yet fully convinced as on shutdown Tomcat issues dire warnings about memory leaks being “likely to happen” but we’re going to do some application profiling soon so we’ll see.

Rather than use Elastic Beanstalk (only available in Amazon’s US regions at the moment) we just deployed on to a customised Bitnami Tomcat stack on AWS EC2. Again, this seemed a lot easier than the last time I tried although perhaps that’s down to having the right tools installed beforehand(!).

I’ve got a bunch of tips / things that we learned from the porting exercise and also AWS deployment so I’ll post those up in the next few days.

For us, AWS rather than GAEJ, is the way forward. Although there are different services on GAEJ (can only think of caching and IM offhand) so it could be useful for some apps, but how long before AWS offer the same? As well as that with AWS there is the freedom to install things like memcached etc.

I do feel a bit guilty though every day at 4p.m. when the True North chatbot (one of our apps on the Google App Engine) pings me on IM … maybe it’ll be its turn for porting next.

I’m writing a bot to place bets using Betfair’s API. The smarts about what bets to put on will be provided by bet-guru “Meffa”. The easier bit will be the robot. It’s just a “fun” project (disclosure: secret hope is that will be on a beach in Antigua next year :) but a good opportunity to try something other than Rails-style projects using JRuby.

Betfair provide a full API and more importantly really excellent documentation and a full-featured demo app. Other API providers (and I include ourselves) please note. The API comes in different flavours, but I’m using the Java one which makes web service calls through Apache Axis.

I’m going to chronicle the steps to build our “botfair” bot although I’ll leave out the secret sauce of Meffa’s algorithms otherwise that beach is going to get crowded.

The first step is making a simple API call using from the interactive JRuby prompt.

Ingredients

• A betfair account. Sign up here
• The betfair Java sample extracted in to a directory, e.g. c:\dev\betfair-java-sample
• JRuby

Logging in and checking markets

Once you’ve extracted the sample application. You’ll need to build it and the WSDL stubs.

We’ll be calling methods from the \betfair-java-sample\src\demo\APIDemo.java class so it’s worth opening up this source file to take a look.

Build the sample and stubs
Before we can do anything we need to compile the classes and regenerate the web services stubs. You’ll need Java 5 for this.

Navigate to the directory and then run ant to build the class files and ant stubs to regenerate the WSDLs.

Set the classpath
You’ll need to make sure that all of the JARs needed by the sample are on your classpath. There are a lot, so I wrote a Ruby script to generate the classpath statement. Run it from the lib file to create you a batch file to do it:

jars=Dir.glob("*.jar")
f=File.new("set-classpath.bat","w")
jars.each do |j|
	f.puts("set CLASSPATH=%CLASSPATH%;c:\\dev\\betfair-java-sample\\lib\\#{j}")
end
f.close

Login and look at markets
In this part we login to Betfair from the interactive JRuby prompt, jirb. You’ll need your Betfair username and password.

1. Start a command prompt in the compiled classes directory (where Ant created your classes).

2. Set the classpath by running the file we generated above. This will make sure that all of the JARs are available (if anyone has a better way of doing this then please let me know!)

3. Type jirb to create a JRuby interactive session and then run each of the commands below separately in the session

require "java"
include_class 'demo.util.APIContext'
include_class 'demo.handler.GlobalAPI'
context=APIContext.new
GlobalAPI.login(context,'your_username','your_password')
types = GlobalAPI.getActiveEventTypes(context)
types.each do |e|
puts e.name
end

What did we do?
The result of all that should have been a list of markets that you can view or bet on using other features of the API.

This isn’t particularly useful as a bot to bet with but does act as a nice proof-of-concept to show that it’s pretty easy to rig up a betfair client using JRuby and Betfair’s API.

Next time …

The next stage will be to poll a particular market (will pick English football whilst the season is still going) and report back on potential opportunities.

Last week we ran up a JRuby contacts tracking app which we deployed on AWS. It’s small but beautifully-formed – insert your own joke now – and is a lot better than our having to exchange emails with contacts and leads.

Despite that, yesterday I got fed up of copying contacts from LinkedIn to the app so I thought I’d add a way the user can browse and add their LinkedIn contacts. This was straightforward enough to do in a day, but also had a few gotchas along the way so after a quick bit of background I thought I’d share those.

LinkedIn provide both a JS and a REST API to their
services and split the API in to three primary domains:

• Profile API – to get information about people (e.g. name, role, company, etc.)
• Connections – provides a list of connections for a user
• People Search API – which mimics the search capabilities when you log in to LinkedIn

There are a couple of other APIs around Invitations and Sharing updates etc but those were less of interest to what we were trying to achieve.

Our goal was pretty simple – to save data entry by accessing our connections from LinkedIn and copying their name, role, company, and LinkedIn profile to our contacts application.

This broke down in to 5 technical tasks:

1. Register our contacts application with LinkedIn (needed for the authentication
   step)
2. Authenticate using OAuth and the tokens provided from registration
3. Retrieve connections
4. Pull profile information from each connection
5. Save contact information in to the contacts database

LinkedIn uses OAuth to authenticate and authorize and having had fun in the past with OAuth and Google I was glad this time to see that Wynn’s API not only wrapped OAuth but also had some examples that worked
out of the box.

I’m not going to replicate the code here so I recommend people take a look.

With authentication, the only issue I had was with the OAuth gem under JRuby. It was throwing an exception “Wrong # of arguments (3 for 2)” in the digest.rb class. All that it needed was a different Digest object to be instantiated. I put the following code in to a module called digest.rb and all was well:

require 'oauth/signature/base'
require 'digest/hmac'
require 'openssl'
#
# Hack/fix to allow oauth to be used with JRuby / Tomcat.
# The digest class doesn't work so use the OpenSSL digest class instead
#
#
module OAuth::Signature::HMAC
  class Base < OAuth::Signature::Base
    private
      def digest
        self.class.digest_class Object.module_eval("::Digest::#{self.class.digest_klass}")
        digest  = OpenSSL::Digest::Digest.new('sha1')
        OpenSSL::HMAC.digest(digest, secret, signature_base_string)
      end
   end
end

Once authenticated then it’s just a case of using the methods in the gem’s
Client class to fetch back the objects. Not all of the API is supported but there was enough for what we needed to do. Again, the detail is in the gem source and examples, and the only more “complex” thing that we did was to take the “headline” field (e.g. my headline is Co-Founder at True North) and split that in to a role and position.

# most are --role-- at --company-- but will not fit all
posn_company = lic.headline.split(" at ")
posn=posn_company[0]
company=posn_company[1]||""

Not everybody’s is in this format but enough are to make it not worth going any more complex.

Finally it was a case of deploying on to our running version of contacts on AWS. We used a modified Ubuntu/Tomcat/MySQL bitnami stack again as it’s been good to us before and is pretty straightforward to set up.

This is where the final gotcha
got me. The version of Ubuntu didn’t have some of the XML libraries needed and I ended up with “Could not open library xml2 libxml2.so”. This was just a case of installing those libraries on the machine.

Again, the joy of AWS and the bitnami AMI we could just run
sudo apt-get install libxml2 libxml2-dev libxslt1-dev. I made a us a new AWS AMI with the patch in (so I don’t have to remember again).

For us that was it. The code itself was quick to do – some of the issues took time but by the end of the day we were all happily importing from LinkedIn.

Ping me on http://twitter.com/truenorth_sid if you want to know more or are interested in a cloud-based contacts tracker with LinkedIn integration!

Finally, in order to protect the innocent, the screenshot above is test data generated by Benjamin Curtis’ Faker gem rather than our real contacts!