What’s Up Next?

I have been planning to revamp my old website and only had time to do it during christmas break.  I had this idea of making it a portal of sorts for me. The rational for me is that I now rely on several “community” accounts like Multiply, Flickr and WordPress for my needs instead of creating everything on my own. The approach I decided on was to display a timeline out of the combined RSS feeds from my different accounts – and I wanted to do this without any backend coding.

To do this, I had to do the following steps:

1. Get RSS feeds from my accounts (Multiply, Flickr, WordPress and Plurk)
2. Combine the RSS feeds into one single feed using Yahoo Pipes
   1. Do some search and replace on the feed titles (like remove the date from my Multiply entries)
   2. Sort entries by date in descending order
   3. Truncate the entries to the first 30 entries only
   4. Return the feed in JSON format
3. Use jQuery (javascript library that does AJAX and JSONP) to retrieve my feed from Yahoo Pipes
4. Write a javascript routine to layout the entries timeline in correct order and spacing

RSS Feeds
Majority of today’s website support RSS feeds, although RSS feeds have differences in format, Yahoo Pipes is able to handle the more common differences. What are the feed URLs?

• http://xxxxx.multiply.com/feed.rss
• http://www.plurk.com/xxxxx.xml
• http://xxxxx.wordpress.com/feed

The “xxxxx” is your account name. You just need to know what the URLs are as input to Yahoo Pipes.

Yahoo Pipes

Yahoo Pipes is a website that allows you to post process RSS feeds (or organized HTML). It has a  drag and drop interface where you can specify the various operations your feed can go through until the final output is achieved. Some used it to cross reference Google Map and an RSS feed with location information (longtitude, latitude) to generate result like finding  apartments near your location. One of its more common use is for combining multiple feeds into a single one and there are a lot of tutorial out there already. You can specify the format used for the output feed (RSS or JSON).

One caveat here is to use the “y:published” date in your pipe operations (like sorting)  instead of the “pubDate” field of the RSS feed as different RSS feeds would have different date formats while “y:published” returns the date as processed by Yahoo Pipes.

jQuery

“jQuery is a fast and concise JavaScript Library that simplifies HTML document traversing, event handling, animating, and Ajax interactions for rapid web development.” as quoted from its website. One ability of this library is to use JSONP for calling cross-domain APIs. Normal AJAX requires that you only call your own server for security reasons. JSONP is a convention between the client and the server to bypass this restriction using script tag injection. More explanation here. It is this convention that allows us to call Yahoo Pipes directly from javascript without the use of any backend server code (at least, in your own server). Note that JSONP requires that the server supports this convention for it to work.

JSONP requires passing along a callback function in your query parameter and for accessing Yahoo Pipes, this parameter must be named as “_callback” (another name will not work, as I had to find out the hard way).

Calling Yahoo Pipes from jQuery

// call yahoo pipes where XXXXX is your pipe application id, asking yahoo to return it as json and passing the callback function
$.getJSON("http://pipes.yahoo.com/pipes/pipe.run?_id=XXXXX&_render=json&_callback=?", function(data){
// data here is the JSON object with count, value (feed entries are inside the value property)</code>

$.each(data.value.items, function( i, item) {
// this function will be called for each entry
// i is the count starting from zero, item is the feed entry
// e.g. item.title will return the feed title and so on
});

//  do some cleanup and exit
});

I also did some clean up like removing image tags from the item.description before using them since I choose to display the description on my website as tooltip popups. To remove the image tags, I used javascript regular expressions to match the tags and replace them with a space.

var re = /<img(\s+)(.+?)>/ig;
var desc = item.description.replace(re, " ");

In Yahoo Pipes, I choose to prefix my RSS feeds (with “Multiply: “, “Flickr: “, etc) in order to identify its source (of course, you can also check the item.link) and these prefixes are replaced with HTML code for displaying the mini icons for each account type.

The Timeline Effect

I used CSS absolute position in order to plot the timeline. This basically translates into a graph ( I used 20 pixels to display for one day horizontally, for example).  Since the RSS is already sorted from latest to oldest, I just loop through the entries, extract the date and multiply it by my spacing (factor  in the year and month in the same manner) and that’s the horizontal position.

var position = ((((firstYear-entryYear)*12) + (firstMonth-entryMonth)) * 20 * 31) + ((31 - entryDay) * 20) - ((31-firstDay) * 20);
// where 31 days (in a month) and 20 pixels spacing (for a day)
// firstYear, firstMonth and firstDay is the date from the first entry
// entryYear, entryMonth and entryDay is the date of the current entry

The issue I had here was the vertical positioning:

• there can be multiple entries on the same day and overlap each over
• there might be two entries from two different days but overlapping each other side by side ( consider also different lengths of title text)

The solution I finally decided on was quite simple: Store the horizontal tail position (end of title text, not the start position) of the titles that are already displayed in an array with the row as index. For each new title entry,  loop through the array to check that the tail position is not overlapping with the current starting position (in other words, check  for the first available non-overlapping row). If available, then use that row, if not, add a new row at the bottom.

(Side note: A friend told me that horizontal scrolling is annoying – so I added an option to view the information as a list.)

The Finished Product

Presenting my own web 2.0/mashup/portal website (just the first page; the other pages… to follow when I have time ).