Challenge Mashup #32 - Geocode Data

Hi mashers,

well here's the next installment in my challenge to build a working example of all 34 (well I'm sure I counted 37 but what's a mashup or two between friends) mashup patterns suggested in Mike Ogrinz's book "Mashup Patterns - Designs and Examples for the Modern Enterprise" (which you can find out more about here).

   Assemble/Location Mapping: Geocode data for location mapping or address verification

In New Zealand during the late summer months, dairy farmers have to manage and control a disease called Facial Eczema. The disease is caused by the spores of a particular fungus that contain a toxin. Animals ingest the spores when they graze. The toxin  eventually causes the skin on the cows to peel when exposed to sunlight. The disease can be managed by spraying pastures and administering zinc salts when the risk is high during hot humid weather. Counts of spores help to give farmers an indication of the level of risk.

My mashup idea was to take spore count data that we collect for various locations, geocode the data and plot it on Google maps. Sounds like a pretty simple task!

The first steps was simple. Capture and publish the spore count data. For this I loaded the sample date, location, spore count, and source name into an Excel spreadsheet and published it using the Presto Excel Connector. Later we might get a bit more sophisticated and get it from a database but that will have to be version 2.0. That turned out to be the easiest bit.

Next I needed to geocode the locations so that I had a mappable reference point for the spore count data. For this I used the Google Maps HTTP based geocode service API and wrote an emml macro to take a location and return a geocoded response. That wasn't too hard either.

I had the raw spore count data and some way to geocode it, but I now needed to transform that data into something usuable with Google maps. This was to be the main mashup. It would generate a KML (Keyhole Markup Language) file which I could then use as part of a Google Map search. Oh boy, did this turn in to a major challenge. Constructing the KML in the mashup wasn't all that hard once I had got to grips with the syntax and figured out how to deal with the various levels of detail returned by the geocoding process.

What proved to be the most challenging aspect in building this script was creating the balloons that appear on the map when you click on a pin/icon. In the KML file, when you specify the balloon, you can include HTML in the text node, but it must be enclosed in a CDATA section. The problem is that a CDATA section is also valid within emml. When the emml is parsed, the CDATA tagging is removed. You also can't nest CDATA sections. So how do you include a CDATA in the mashup output without having it removed during processing. This kept me busy for a day.

The solution was to include a Groovy script I wrote within the mashup and have it process the KML output once constructed and replace two easily identifiable markers with proper CDATA opening and closing tags.This had to be almost the final step in the mashup because any subsequent processing that involved XPath  would remove the CDATA tags.

Using the Groovy script I could make <text><![CDATA[ ... some enclosed HTML ... ]]></text> from <text>##CDATAbegin# ...some enclosed HTML ...##CDATAend#</text> without violating the CDATA rules or having it removed by the emml parsing process.

The last step was to write the constructed KML out to a file that could be used with Google maps. Finally ... success.

  Here's a link to show you the final outcome. It's just a test version though.

  http://maps.google.com/maps?q=http://sites.google.com/site/nzbluefish/kml/fespore.kml

I will post the code for anyone who is interested. The CDATA solution may be helpful to someone else when trying to generate XML output that needs to include a CDATA section.

To borrow a phrase from Chef Gordon Ramsay then ... "geocoding mashup ... done!"

Cheers and happy mashing everyone,

Innes