Accessing private feeds using Superfeedr (and hiding those feeds from the world)

Update: Because superfeedr is basically just a PubSubHubbub this post is also a great example for subscribing to any PubSubHubbub using Java.

I've been working on JOPPFM in the last few days and I made a few important decisions:

• No more AppEngine. It's not because of the recent pricing changes! It's because I'm not allowed to do the things the way I want using AppEngine (SMTP, long-lived HTTP requests and the like).
• Heroku! Really easy deployment. git push and you're done. I love that!
• I'm now using superfeedr for almost everything applicable. Why? Because it's dead-simple and cheap. Still, it provides "real-time" notifications about changed feeds.

But migrating to superfeedr wasn't that straight forward, because I have a really weird use case: I want to access feeds that are only visible for a user that's authenticated with a service like Twitter or Facebook. This isn't what superfeedr is made for actually.

Fortunately superfeedr's team is uber-responsive and replied within a few hours (thanks Julien) with an interesting solution I didn't really understand at first: "Proxy the feeds". Two mails later I understood what Julien meant and I was amazed. I couldn't have figured out such an elegant solution myself. Maybe I should use my brain more often more intensively...

One disadvantage of this method is you're not able to save bandwidth. Everytime superfeedr wants to access the feed, you have to authorize, fetch the feed and return it to superfeedr. Fortunately I don't care very much about bandwidth, because it doesn't matter at Heroku. :D

And here comes the, in my opinion, biggest advantage: No need to bother about polling. Never ever. No background workers, cron jobs or anything like that.

Before I'm going to show you sample code there's one more important thing you should know: Because superfeedr wasn't originally built for private feeds, ALL feeds running through superfeedr are searchable by every other superfeedr user out there. However, Julien (again) told me how to secure your user's data: Add a "superfeedr.private" parameter when subscribing to a feed.

Ok, let's take a look at some code: https://gist.github.com/1212398


You need Apache's HTTP Client v4 in order to get this working. Also, you need to implement the servlet waiting at feedUrl. As noted in the code, this servlet should return a (private) feed.

That's it? That's it!