Author name: chipx86

Penny Arcade and Control-Alt-Del are great comics. I love to read them and do so when I’m not feeling really lazy. See, I pretty much live inside my Google Calendar, GMail, Remember the Milk, Google Reader, and Netvibes tabs, for the most part. I’m pretty lazy when it comes to some forms of content, and if a comic doesn’t show up in Google Reader, I’ll typically ignore it until someone points out a particularly good strip to me.

While playing around with Yahoo! Pipes, I realized I could finally do something about this. I began to play around with a couple of pipes to read in the RSS feeds, look for all comic entries, and change the content. The trick was to copy the location the feed item was pointing to (which would contain the actual comic image within the page) to the description, and then apply a regular expression to the description to turn it into an <img> tag pointing to the image itself.

I got lucky. To my knowledge, there is currently no way to fetch content from any arbitrary HTML page and do something with a piece of that page. I suspect their Fetch Data module might let me, but I haven’t managed to get it to work just yet. I was able to pull this off since the comic image was stored with a predictable path based on the date of the comic, and the page being linked to also contained the date. A regular expression was all that was needed to parse out the date and rebuild the path.

Anyway. the end result is that I now have inline comics in my Penny Arcade and Control-Alt-Del RSS feeds! You can add them to your RSS reader below, or take a look at how they were made.

• [Pipe] [RSS] Penny Arcade News and Inline Comics
• [Pipe] [RSS] Penny Arcade Inline Comics
• [Pipe] [RSS] Control-Alt-Del Inline Comics

Yahoo is doing a lot of interesting things these days. While Google gains a lot of the attention when it comes to search and web applications, Yahoo should not be ignored. del.icio.us and Flickr are of course two widely popular services, but they have a few useful utilities floating around their developer site.

There’s one tool in particular that I found tonight that has already proven useful. Pipes. Pipes allows a user to quickly put together a simple set of pipeline filters for turning various forms of data into an RSS feed and accompanying JSON file.

Such forms of data include user input (validated as a date, geographical location, number, text or URL) and web-based input (Flickr pictures, RSS2/Atom feeds, JSON/XML data, Google Base listings, Yahoo! Local searches, and Yahoo! search results). This data can be fed through several layers of pipes (including back into another Flickr pipeline and such as query input). The pipes can transform the data, walk through each feed item and modify or extract data, combine data together, sort, remove duplicates, apply regexes, translate languages, and so on.

This can be pretty powerful. While still a young project, many users have already published pipes, myself included. With the increase in API-enabled web services, I can only expect this to become more powerful, with work. It’s just a little tricky coming up with actual useful applications.

So I played around a bit and started to experiment with what could be done. I ended up with a couple of simple, but very useful pipes. One thing I have wanted for the longest time was a way to see feeds from several Planets in one listing (for Netvibes, since space is precious), without having to deal with duplicate entries. Pipes made this all too simple.

I’m feeding several feeds into a Unique operator, saying to filter based on the title. I then output that. That’s all it takes. You can see the results and even add the RSS feed.

I then took this one step further and decided to write a quick pipe for searching through the planets. Now, pipes are reusable, so I was able to incorporate the Unique Planets pipe into this. This was fed into a Filter, using a couple of text inputs (for a text string and a name) as parameters to the filter. The screenshot below will clarify this. The result is the ability to quickly search four planets by name or content.

You can play with the results. Go ahead, give it a try.

Pipes can be published for other people to use, or they can be used privately. Private pipes are great when you want to deal with data that can’t easily be queried, such as your Twitter feed or your own Flickr feed.

Pipes are also quite useful when you have a small web application that needs to deal with several other feeds, filtering results or combining data from multiple sources. Sure, you could write this all yourself, but it’s far easier to change and maintain a Pipe than a whole bunch of code.

If you want to play with pipes, I recommend just jumping in and playing. Also take a look at some other people’s pipes, and you may want to browse the tutorials. For some starter ideas, try making a pipe that searches your local area for sales or singles or something using Google Base and Craigslist, or one that searches all your favorite blogs for a certain keyword, or maybe something that keeps track of your friends’ blogs and Flickr posts.

Now, pipelines are hardly a new concept. Several programs offer them, including some development environments that rely solely on pipelines for development in order to quickly produce simple programs. What makes Yahoo!’s Pipes interesting is that they make it very easy for almost anybody to quickly build a pipe to modify or search all kinds of data on the web that people actually use. This makes them more immediately useful to many people, and of course Yahoo makes it dead simple to start out.

What would be useful in the future, aside from adding native support for more services, would be to output data in other formats or somehow easily lay out information onto a page from one or more feeds. The project seems pretty young though, so I’m sure in time, this will mature into a much more useful project, both to developers and (certain) end users.