While the following post uses #opendata as an example subject to pull information from Twitter, the methodolgy is generic.
The background is that I love to know what is happening in the world of Open Data, but there are so many people doing so many things in a diffuse environment that it is difficult to know where to look in order to keep up with the latest developments.
Separately, I’ve been playing with IFTTT (If This Then That) which is an incredibly powerful way of plumbing web accounts together to do some clever things.
So, here is an experiment that I’ve been playing with. If you want to follow it, to make your own project then you’ll need three things:
All are free, and the links above will take you to the sign-up pages. You can be up in running in minutes. Once signed up, you can follow the steps below.
My first task was to create task in IFTTT to search all or Twitter and find who is tweeting about #opendata. Luckily someone had already set up a similar task in IFTTT. On that site, shared tasks are called recipes, and you can copy a pre-existing recipe.
Here is the recipe you want: http://ifttt.com/recipes/7540
Once you’re logged into IFTTT, just follow the recipe, and, under Step 1, enter the search term you’re interested in. In my case that was #opendata. Then, under step two, leave the first field as suggested then go to the second field and enter the name of the Twitter list you want to use.
Once you click Create Task, the event will fire every 15 minutes and each time, if it finds a new user tweeting about #opendata it will add them to your Twitter list.
So, you’re now building up a list of people who tweet about a specific topic – whcih might in itself be useful for those setting up hashtags for events, hackdays or conferences.
At this stage, before we move one, you might want to go to your Twitter account and add a user manually to the list just so there is something there to work with.
Now that that is set up and working you can set about searching the Twitter stream for that list and pulling out what people are saying about Open Data. To do that you need to grab the RSS feed of the list. This is more difficult than you might expect – and certainly harder than it used to be with the ‘old’ Twitter.
However, someone has been through the pain for you already. You’ll find the details here on David Calhoun’s blog.
I found the method suggested by Pete Carrol on that page the most straightforward.
Go to a link such as this: http://api.twitter.com/1/lists.xml?screen_name=scibella
but change the screen_name to your own.
From that you’ll get back some XML with one or more <list> items. If there is more than one <list>, choose the ID of the one you want.
then use this link:
http://api.twitter.com/1/lists.xml?user_id=51496296 but substituting the ID for the one for you list
That is the feed for your Twitter list identified. Now, copy that link and head back to IFTTT and to another recipe – this time one that I created to search a Twitter list for a specific hashtag, and push the results to a Readitlater account. Here it is: http://ifttt.com/recipes/19356
Follow the instructions there and you’re done.
You’ve now be searching for the hashtag that you’re interested, from the Twitter list you set up to monitor who chats about it on Twitter. The results will be available in your Readitlater account.
Since I set this up yesterday the experiment has worked. My Twitter list of those who’ve tweeted about #opendata is growing. Also, the task search the Twitter list for mentions of #opendata is working, and the tweets have been pushed to my Readitlater account allowing me to read them on my mobile phone. That said the IFTTT log shows that its been throwing up some errors as API seems to have failed a few times over night. I’ll have to keep an eye on it.
Also, you could adapt the latter recipe and push the alerts to mail, SMS or to other clipping services using the built in channels in IFTTT.
If you find this useful – or have comments on it, please leave comments below.