News Mapper: A mashup experience with the Google Mashup Editor

Now in the two weeks after my successful graduation and before starting to work I found some space for doing some programming experiments. :) What I planned already for a longer time now has been to do some JavaScript coding since it becomes an always more important aspect in modern web technologies such as AJAX and so on. My experience till now was quite limited by just coding in a try-and-see fashion since I've never really learned JavaScript.
Since the best way is to learn-by-doing, I started to develop a project which I had already in mind for quite a long time now: "mashing up" news and maps. The basic idea is to display news on a Google Map depending on their location. I already did such a project during the Internet Technologies II course at the University with some study colleagues. At that time we created an ASP.NET application (with an IIS webserver) for displaying the map and the news on it and a local C#.NET deamon which was intended to run at the server for collecting, geocoding and storing all of the news in the web application's database. The project outcome was quite cool. This time however I wanted to realize the application without a server-side part by just using client-side JavaScript. That was the main challenge.

Google Mashup Editor
About half a year ago now, Google launched a new product called the Google Mashup Editor. The aim was to provide an environment for easily developing, hosting and publishing Google Mashups as there are quite a lot available around the web. Till now I was quite busy with my thesis project wherefore I never had the opportunity to play with their new product which is currently just available to about a thousand developers. Being one of those I now wanted to make use of it and build my mashup project on top of the GME API. Moreover this promised me a lot more users since the project will be promoted on the official Google Code pages (see image on the right).

News Mapper - Intro
The development of the application itself was quite straightforward once I understood the basics of the Google Mashup Editor. The majority of the information is well documented on the Google Mashup main website. The page documents the basic structure of a mashup application and gives a lot of samples for getting started. Anyway during the development undocumented problems may occur. In this case I suggest you to contact the developers over the forum. I had to do that several times and I was also able to contribute a little to the editor by submitting a bug. Taking this occasion I would like to say that the Mashup development team is for sure one of the most responsive Google teams. I submitted several questions and got an answer even within the same day. This holds especially for Jason Cooper (Google Mashup team member) which does a great job by supporting early mashup developers (including myself) by supplying useful tips and sample projects and often also still undocumented and hidden features. His hints helped a lot in the development of my News Mapper. This responsiveness is in my eyes also one of the main reasons why many of the Google products have such a great success. Building a stable user community is essential for such services since you get a lot of usage statistics, bug reports and suggestions which help to improve the product. Moreover it is nice - being myself a software developer - to see that one is able to contribute to such products by providing bug reports or feature suggestions.

News Mapper - How does it work?
I'd like to explain the News Mapper system with the following figure. As you probably imagine ;) I like those "free-form" diagrams. They illustrate the system in an informal way and help to understand how things are correlated.
I will explain here just briefly the main and most interesting things of the News Mapper application. Other things such as generalities about the GME can be read in its documentation.

The first and probably most important aspect is how the system interacts with the outside world, meaning how it retrieves the data. GME applications can use two different data sources: external feeds (RSS or Atom) and application-internal built-in data sources such as ${app}, ${test} and ${user} which are also represented as XML feeds. More about that can be read on this page. At the moment the News Mapper makes just use of the external data sources in the form of RSS feeds. These are taken from the Reuters web-page. A typical entry's body of the Reuters RSS feed looks as follows:

BAGHDAD (Reuters) - A battle between al Qaeda in Iraq and a major Sunni Arab insurgent group killed at least 16 militants on Friday near the ancient city of Samarra, a senior security officer told Reuters on Saturday.

As you can see at the beginning of each feed there is the location given as the name of the city ("BAGHDAD" in this case). My idea was therefore to cut the city string out of the whole text by using the JavaScript split(...) function and to replace then the string-pattern "(Reuters)" with the empty string. This clean string containing the name of the city ("Baghdad") could be sent to the GMap2 geocoding service for getting the lat/lng data. Although this worked quite fine it was a little inconvenient since in the placing of the markers and displaying of info-windows had to be done programmatically and I couldn't make use of the Google Mashup API's map capabilities which requires the lat/lng data to be encoded directly in the RSS feed as XML elements or attributes. This problem however was quickly solved when I found the geonames.org's webservice RSS to GeoRSS. In this way I could get rid of the geocoding stuff which was taken by that service and moreover it seems that geonames.org does some more sophisticated geocoding by analyzing the entry's content (in this case the article preview). For storing the received, geocoded data the gm:list has been used. This is then linked to the map module as follows:

<gm:map id="map" control="large" height="550px" data="${sourceFeed}"
latref="geo:lat" lngref="geo:long" maptypes="true" infotemplate="mapInfoWindowTemplate">
<gm:handleevent src="sourceFeed" event="select">
</gm:handleevent></gm:map>

The "data" attribute contains the id of the data-feed represented as gm:list in the case of the News Mapper. The attributes "latref" and "lngref"contain the GPath expression for retrieving the XML elements containing the location data.

The second thing I'd like to explain is the JavaScript part of the application, especially the one which is responsible for processing the retrieved RSS entries. This was necessary since the Reuters feed contained some HTML code for displaying advertisement and links. This caused some mess when displaying the information in the map-bubbles. For truncating the HTML code out of the RSS entries I used the JavaScript regular expression engine for identifying and replacing the part. The main challenge was to find the right point when to start the processing since it has to be done after the feed has been loaded completely. Here Jason Cooper (GME developer) helped by indicating me that there exists the "repaint" event (details are at the moment still undocumented) which is thrown after the list has finished its loading. In this way, as soon as the gm:list has finished its loading, the JavaScript updates its entries by truncating all HTML code.

The last point which could be interesting is the dynamic changing of the source feed. This is realized with a proper JavaScript function.
For the News Mapper I wrote the following lines of code:

/*
*  to load different feeds
*/
function changeSourceFeed(feedURL, feedTitle){
feedCleaned = false;

//constructing URL of new feed
var newFeed = "http://ws.geonames.org/rssToGeoRSS?feedUrl=" + feedURL;
debug("URL of newly loaded feed: " + newFeed);

//setting title of feed
var titleContainer = document.getElementById("feedTitle");
titleContainer.innerHTML = feedTitle;

//retrieving a reference to the gm:list
var listModule = google.mashups.getObjectById("sourceFeed");
listModule.setData(newFeed);
}

I think the code is self-explanatory since I also added the appropriate comments. This JavaScript method is then called when the user clicks on the links (see figure on the right). This is done as follows:

<a href="javascript:changeSourceFeed('http://feeds.reuters.com/reuters/topNews', 'Top News');">Top News</a>

News Mapper - Debugging
Debugging is one of the points that makes JavaScript programming sometimes a difficult task. A first important point is that you have the possibility to printout some intermediate results (as the Java System.out.println() statements). For doing so, one of the best ways is to define a container in the HTML code as follows:

<div id="debugger"></div>

Then a JavaScript function has to be defined that writes to this container for displaying information to the programmer:

   /*
   *  For printing debugging information
   */
   function debug(message){
     if(printDebugInfo){
       var debugContainer = document.getElementById("debugger");
       debugContainer.innerHTML = debugContainer.innerHTML + "
" + message;
     }
   }

This function can then be called from any place in the JavaScript code passing to it the message that has to be displayed. Another interesting tool that is a must for any web/JavaScript programmer is FireBug (Firefox extension). You should take a look at it.

News Mapper - The final outcome
The final outcome can be seen at

---

Questions? Thoughts? Hit me up on Twitter