Typical Yahoo Pipes
Type of site
|Created by||Pasha Sadri, Ed Ho, Jonathan Trevor, Kevin Cheng and Daniel Raffel|
|Launched||7 February 2007|
Yahoo! Pipes is a web application from Yahoo! that provides a graphical user interface for building data mashups that aggregate web feeds, web pages, and other services, creating Web-based apps from various sources, and publishing those apps. The application works by enabling users to "pipe" information from different sources and then set up rules for how that content should be modified (for example, filtering). Other than the pipe editing page, the website has a documentation page and a discussion page. Documentation page contains information about pipes, a user guide on pipe edition and a troubleshooting guide. The discussion page enables users to discuss the pipes with other users. The site is currently in beta.
Yahoo! Pipes was released to the public in beta on 7 February 2007. It was built by Pasha Sadri, Ed Ho, Jonathan Trevor and Daniel Raffel of Yahoo! It is described by its creators as:
…a hosted service that lets you remix feeds and create new data mashups in a visual programming environment. The name of the service pays tribute to Unix pipes, which make it easy to chain simple utilities together on the command line.
— Official Yahoo! Pipes Blog, Pipes Blog
The purpose of Yahoo! Pipes is to create new pages by aggregating RSS feeds from different sources. Yahoo! Pipes has many modules which can be used either to grab data from sources or to edit the data that is grabbed from the sources. These modules are grouped into categories. These categories are sources, user inputs, operators, URL, string, date, location and number.
Creating and editing a Pipe
To create or edit a pipe, user has to sign up with a Yahoo! ID. Creation and editing of the pipes are completely online; the user doesn't have to download a plug-in or a program. User selects the "Create a pipe" option to open the Pipe Editor. Pipe editor is composed of three panes which are the canvas, the library and the debugger. The pipe is created using these panes. After creation, pipe is saved and run. Then user gives a name to the pipe and writes a short description of it. If the creator of the pipe decides to publish the pipe, it becomes visible for everyone. Other users can clone the pipe. Then they can use and edit their copy for their own use.
Canvas is the main pane where the editing of pipes is done. It is in the center of the page. Modules that are selected from the Library pane are dragged on this pane and connected together. After that all the modules are wired in the desired order the pipe is ready to be used.
The Library is the place where the modules are selected to be dragged on the Canvas. These modules are grouped by their functions. The library pane is on the left hand side.
In this category, there are modules which are used to grab data from one or multiple sources on internet.
The modules in this category enables user to add an input in the pipe. By using the modules in this category, user can add date, location, number, text or URL input to the pipe.
The modules in this category are used either to filter or to transform the data that is flowing in the pipe. It contains filter, count, location extractor, loop, regex, rename, reverse, sort, split, sub-element, tail, truncate, union, unique and web service modules.
There is only URL builder module in this category. Pipes need to have URLs of RSS to get the content. This URL builder enables users to create URLs instead of typing the URL address. It uses a base URL and query parameters to generate other URLs.
The modules in this category are used to either to change or combine the strings. String builder, string regex, string replace, sub string, term extractor, and translate are the modules of this category.
There are only two modules in this category. They are date builder and date formatter. Date builder module converts a text to a date when there is text in the date format such as "yesterday" or "tomorrow". Date formatter module takes a date as input date and changes it to the desired format.
This category contains only location builder module. This module recognizes the strings which are the description of a location and converts them to geographical location. Its input is in string form and output in location form.
This category only has the simple math module. It applies simple math operations to the number input it takes and outputs the result of math operation. Both the input and the output are numbers.
This is the pane which shows the output of the selected module on the pipe. While designing a pipe, when user clicks on a module on the canvas, the debugger pane shows the output of the selected module. It shows both the title and the content of each item.
A Web Mashup is a Web application that combines data from more than one Web data source into a single integrated Web application. Just as Unix pipes are often used to quickly combine several different data sources but are generally not sufficient to create a useful application, Yahoo! Pipes is a data mashup tool rather than a complete Mashup editor.
- Google Mashup Editor (Deprecated, since January 14, 2009)
- Mozilla Ubiquity
- Yahoo! query language
- Scratch programming language
- Open Mashup Alliance
- WSO2 Mashup Server
- Mozilla Jetpack
- Pipes Official Site
- Tim O'Reilly blog article
- Yahoo! Pipes Adds Geo Data Support - including export of data in KML