Shane O'Sullivan's technical blog… really ties the room together

Using Yahoo Query Language (YQL) to count XML elements

Posted by Shane O'Sullivan on April 19, 2010

A quick little YQL tip. I recently had the need to count the number of items in an RSS feed, but without downloading the feed itself as the application is bandwidth sensitive.  So I figured I’d use the Yahoo Query Language, which lets you define SQL queries for accessing all kinds of public data on the web, including of course RSS and Atom feeds.

However it’s implementation of the SQL count() function seems to be buggy, as you can’t simply do

select count(*) from xml where url=”http://path/To/Rss.xml” and and itemPath=”/rss/channel/item”

as it throws and error.  Putting “item” in the count() doesn’t work either.  However, if you try to select the text inside an element, it just returns a single text value, but you still get the count of rows returned, e.g.

select * from xml where url=”http://path/To/Rss.xml” and itemPath=”/rss/channel/item/title/text()”

returns JSON that looks like

 "query": {
  "count": "233",
  "created": "2010-04-19T10:50:42Z",
  "lang": "en-US",
  "results": "5 stars by Anonymous on February 26, 2010"

meaning you save a huge amount of bandwidth but still can get the count of the items in the RSS stream. Check out an example HERE, just press the Test button. Cool huh?


3 Responses to “Using Yahoo Query Language (YQL) to count XML elements”

  1. David said

    I haven’t used YQL so this is just curiosity… Is “itemPath” like xPath? If it is xPath, “/rss/channel/item/title/text()” should return a concatenation of all of the text nodes that match the selection. But here it’s returning the equivalent of “/rss/channel/item/title[1]/text()”

    A bug in YQL, perhaps?

  2. Hi David,

    Yes, itemPath is meant to be xpath, but YQL has quite a few quirks/bugs with its implementation. For example, the xpath count() function doesn’t work, which I would have like to have used. This text() query does also seem to be a bug, but what the hell, it works for me 😉

    In general though, YQL can be a fantastic tool for a web or mobile developer. Particularly on smartphones where you often have to deal with very slow connections and data transfer fees, YQL can do a lot of the heavy lifting of aggregating data, and saves you from having to implement many different custom web services yourself.

  3. […] Using Yahoo Query Language (YQL) to count XML elements « SOS […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: