No More Oodles of Craig’s Listings
Fresh from the Oodle blog - Craigslist content is no longer being included in the Oodle index. Evidently, Craigslist requested that its content not be crawled by Oodle going forward. It’s not yet clear if Craig is opting-out in response to some especially naughty act by team Oodle or if Craigslist just decided Oodle’s approach was more of a threat than a helpful traffic source. Without knowing the motivation, we don’t see how this is good for consumers, but we can clearly see why it is good defensive move for Craigslist and bad news for Oodle.
A lot of folks are asking “why block Oodle and not the other aggregators (e.g., Indeed and SimplyHired) or even Google / Yahoo?” Indeed (no pun intended), the other aggregators must be asking this question and hoping that Craigslist doesn’t extend the blockade. As our last post suggested, it presents a huge problem (at least in the near-term) for the aggregators if the major boards opt-out of their indices – especially Craigslist given that it’s the biggest classifieds site.
The discussion on John Battelle’s Searchblog is also interesting. One reader speculates on the reason why Craigslist may be blocking Oodle:
“If one scraper is using too much from one source it DOES become infringement and competition. I’m not familiar with Craigslist’s specific beef with Oodle, but I suspect that’s it.”
Since the job-specific aggregators aren’t indexing all of Craigslist’s classifieds – only the job listings – perhaps Oodle will be the only man thrown overboard.
And, in case you were wondering, YorZ has no intention of blocking any of the aggregators from crawling our site at this time. 