Mark Dudlik

What’s that you say? Dapper pipes sounds like fancy plumbing and you want nothing to do with this article now? Well, give it a second.

With how today’s economy is going, it’s increasingly difficult to find jobs. A lot of time and effort goes into the job search itself, and it’d be nice to have a easy way to check all those many job sites at once, making it easy for your unemployed butt to do more important things, like work on your website, or finish off that last row of oreos.

Dapper.net lets you turn any page into an RSS feed or widget for your site, through finding common elements of different web pages, adapting them into similar categories and exporting them to RSS. I’ve found the site extremely useful for other things too, such as creating a feed for a webcomic that doesn’t have RSS. And, in the context of this article, taking the table-based simple page of a Phoenix job listing site, and exporting it’s non-RSS enabled content into a feed that I can put together in Pipes.

Yahoo Pipes allows you to combine RSS feeds into one stream, then sort, filter and export them into a single feed. In the example of my comics feed, I took the feeds of a bunch of webcomics, sorted them by date and exported that feed. When I view it in netvibes, I have a feed of comics that update daily, without having to jump around to 10-15 sites. It makes my slacking that much more quickly satisfying and prompt.

I’m going to walk through setting up a feed in Dapper, and then how to combine the newly created feed with other feeds to create an always updating, sorted by date job search feed, that enables you to keep track of new job postings in whatever RSS reader you work with.

(Clicking on the images in this will make them fullsize.)

FIrst, creating a feed with Dapper.net

Go to Dapper.net, on the left there is a section where you can plug in the address of the site you want to create a feed from, to search if there are already feeds set up by other users. For this example just click the “or Create a new Dapp” link below the search button.

Once on the start screen, select RSS feed and paste the link to the page you’d like to Dapp. In the example, I’m using a page for ad2phoenix.com’s job list page. You can use any link, from what I’ve seen, for instance a search result for Graphic Design from phoenix.craigslist.com.

(You may be prompted with tutorial videos to watch, as your start the dapp, which I recommend watching. Here however, I just ignored those and jumped right into adding pages to the basket. For this example, there is only one page, so click Add to basket, then click “Next Step”

This part is the most important part of the feed creation. You need to select the areas that you want the feed to recognize. For this page, its Date, Position and Company. Clicking on a date will highlight most of the page, so you need to un-select areas that you don’t want to be part of that section. In the image, I select the first date listed, then deselect position and company and the column title. It eventually realizes which specific part of the page you want. Once its correct, hit “Save Field.”

Save the field as shown below, for Date, select date. The item title works best when you select the link to the post itself. For instance, I made the link to the position description page the title for this Dapp. This allows it so that in the RSS reader, you jump to the post itself, instead of to the main page.

Once you’ve selected all 3 areas, click “Next” This will bring you to a page displaying what your feed will look like. If its all good, click next step again.

Now, you will save the Dapp. You do not need to bother with those bottom three selections.

Below shows the page once the Dapp has been created. On the right side select RSS from the options. The menu asks which item you’d like to be the “title.” You can use the Position field for this. Then select the 3 items, if you’d like them all to show up in the text of the feed post, and then, though its optional, you should choose “date” as the Date Published field.

Hit “update input” to see the feed results. At this point, there is an RSS feed address ready for you to take and subscribe to. Save the link, and then create other Dapps as you see fit, before moving onto Pipes.

Now, once you have those feeds, its time to jump into Pipes to put them together:

This is the homepage of pipes.yahoo.com, once you log in with a yahoo ID, click “Create a pipe”

Once it loads, the is the screen you are faced with. The left side is the variable, item loading modules space with the right being devoted to the actual “pipes.”

The pipes act as little nodes that connect to each other with the final “pipe output” node showing the end result.

You’ll see this final node once you add a module.

For this example, pull a “Fetch Feed” module, drag it onto the workspace. You’ll see that by doing so, you’ll have automatically added a pipe output node as well.

Add the feeds that you created into the Fetch Feed node.For this example, I have ad2phoenix, creativehotlist Dapps, as well as Indeed.com and Simplyhired.com feeds as well (these both have an RSS feed option available once you search, as do some other sites.)

If you click on the fetch feed node and click refresh in the footer results screen, you’ll see how the RSS feed would look, if you connected it to the pipe output node. The problem is, that its only displaying the results in the order of the feeds placed in the note, its not sorting them in any way.

To address this, select Operators from the left side menu and pull a “Sort” node onto the workspace.

Select item.pubDate and Descending from the menu.

Then, if you’ll notice, there are circles at the top/bottom of the Sort node. Click and drag the circle at the bottom of the Fetch Feed node to the top of the Sort node. Then, click and drag the bottom of the Sort node to the Pipe Output node, resulting in what you see below. If you click on the Pipe Output node, and hit refresh, you can see the RSS results have no adjusted so that they display based on the dates of the individual feed posts.

Save the pipe.

This shows the screen, once you’ve saved and titled a pipe. To edit it, add/remove RSS feeds, etc, you simply click Edit Source.

To subscribe to the resulting RSS feed for your own RSS reader,  select “More Options” and then Get as RSS.

If you’re like me, and use Netvibes, A. You’re cool, and B. you should see an often updated, easily accessible RSS feed helping you with your job search.

Dapper and Pipes are incredibly useful tools. This example alone should show that, and there are many other possibilities of combining these two services towards creating feeds to gather information and help reduce that constant page to page that happens when browsing. This example uses Phoenix based job sites, but there are many other opportunities for this type of resource. You can create a Dapp for your local craigslist job page, or if you have a city that people actually post jobs to for Coroflot and Krop and others, you can access those as well (Phoenix, unfortunately, never seems to list anything there)

This could also be used on freelance job sites, to see what new opportunities have been posted in your area, so you can get a jump on them before others.

I hope you find this helpful, and if you have any questions, clarification points or update requests, please let me know, we’re going to change this to a page resources soon. Also, if you liked this kind of post, please let us know too!

Happy job hunting.

(p.s. no, I’m not going to give you the end RSS link, you can do it yourself now!)