(2003 to 2005)

Blogs are being scraped as a part of a social experiment..

Thu Feb 12 22:10:00 UTC+1100 2004


This just popped into my head, but it seems pretty clear that it is the case to me.

The whole blogging / RSS thing is pretty interesting. You can see interesting things that you can use to model relationships.

You know the whole 6 degrees of separation thing right? Well basically I'm convinced that somewhere, someone is using RSS feeds from peoples blogs to analyse social models. I can see in my weblogs that there are several bots that routinely scrape my feed. I've never really bothered figuring out what magic they are doing with it, but I'm sure someone out there is modeling the stuff.

The thing about a URL is that it is an excellent primary key. When two people point to the same primary key then you can infer a connection or a relationship. You can look at these relationships and get a great deal of information. You could even take samples and extrapolate the results to some extent. I think it's pretty interesting stuff, but at the same time, I'm pretty sure that at the end of the day it will be used by clever marketing regimes that use it to more effectively cram shit down your throat.

That about does it for social insights from me for the evening.


Copyright © 2003-2005 John Elliot