These are the old pages from the weblog as they were published at Cornell. Visit www.allthingsdistributed.com for up-to-date entries.

March 02, 2004

The Rise of the Feed Readers

While waiting for some other log processing to finish, I had a moment to review the software agents that pull the rss & atom feeds from the weblog. In past two months there were 141 unique agents that requested one of the 3 feeds. Of the unique 'users' (for some definition of unique) 75% of them use one of the big 6 aggregators.

The numbers on all the 141 agents can be found in the extended posting.

Agent Percentage

SharpReader

25.25638

NewsGator

13.83471

NetNewsWire

13.63362

Mozilla

11.42168

FeedDemon

8.184195

RssBandit

7.701589

Shrook

2.312487

Feedreader

1.769556

NetNewsWire+Lite

1.508144

Googlebot

1.28695

RssReader

0.764126

Bloglines

0.744018

AmphetaDesk

0.723909

Aggie

0.643475

FeedOnFeeds

0.643475

Oracle+Ultra+Search

0.603258

nntp

0.522823

Green+Research

0.502715

Java

0.442389

Radio+UserLand

0.402172

Liferea

0.341846

msnbot

0.261412

BottomFeeder

0.261412

UltraLiberalFeedParser

0.241303

larbin_2.6.3+larbin2.6.3@unspecified.mail

0.241303

Awasu

0.201086

RPT-HTTPClient

0.180977

Wildgrape+NewsDesk

0.180977

Straw

0.160869

Java1.3.1

0.14076

TurnitinBot

0.14076

QuepasaCreep+(+crawler@quepasacorp.com+)

0.120652

UniversalFeedParser

0.120652

Wget

0.100543

Python-urllib

0.100543

Jakarta+Commons-HttpClient

0.100543

FAST-WebCrawler

0.100543

Syndirella

0.100543

libwww-perl

0.100543

JetBrains+OmniaMea+(EAP)

0.100543

IXE+Crawler

0.080434

Feedster+Crawler

0.080434

Lynx

0.080434

FeedValidator

0.080434

Teleport+Pro

0.080434

vspider

0.080434

WSB+WebCrawler+V1.0+(Beta)

0.080434

ia_archiver

0.080434

Diff-Engine+Liang.Lu@cern.ch

0.060326

Mozilla(IE+Compatible)

0.060326

intraVnews

0.060326

FeedRover+1.0;+Headlines+Archive

0.060326

Vox+Lite(1.0.12.0)

0.060326

Frontier

0.060326

rawdog

0.060326

Yahoo-VerticalCrawler-FormerWebCrawler

0.060326

MagpieRSS

0.060326

TranSGeniKBot+http:

0.060326

larbin_2.6.3+admins@uptime.at

0.060326

effnews

0.060326

PluckExplorerBar

0.060326

http:

0.060326

NaverBot-1.0+(NHN+Corp.+

0.060326

Beaver(0.4.3.35480)+-+mailto_etafix_yahoo_com;

0.060326

MovableType

0.060326

EARTHCOM.info

0.040217

augurnfind+V-1.8

0.040217

blink

0.040217

Opera

0.040217

daypopbot

0.040217

PocketFeed

0.040217

lmspider+lmspider@scansoft.com

0.040217

curl

0.040217

User-Agent:+Mozilla

0.040217

AOLserver-Tcl

0.040217

Abilon

0.040217

lwp-trivial

0.040217

BlogStreetBot

0.040217

PubSub.com+RSS+reader+-+http:

0.040217

Baiduspider+(+http:

0.040217

sherlock_spider+jimfan@163.com

0.040217

Syndic8

0.040217

NaverBot_dloader

0.040217

XML::RSS::uptimeRSSFetch

0.040217

Desktop+Sidebar+v1

0.040217

larbin_2.6.3+pimenas@softnet.tuc.gr

0.040217

348NorthNews

0.040217

Microcomputers+Etc.

0.040217

Microsoft+URL+Control+-+6.00.8862

0.040217

effnews+1.0.15+(http:

0.040217

Java(TM)+2+Runtime+Environment

0.040217

Internet+Ninja+5.0

0.040217

blink+aggregator

0.040217

BlackMoss-1.0

0.040217

larbin_2.6.3+larbin-crawler@un.bewaff.net

0.040217

Wotbox

0.020109

Raggle

0.020109

BlogBot

0.020109

ASPseek

0.020109

Pears

0.020109

fetch+libfetch

0.020109

Test1.0

0.020109

blagg

0.020109

MyWireServiceBot

0.020109

Thames+Research+of+Manchester

0.020109

larbin+UserAgent

0.020109

WebCopier+v4.0

0.020109

BravoBrian+BStop

0.020109

MultiText

0.020109

grep+News+rss

0.020109

http+generic

0.020109

ping.blo.gs

0.020109

fastbuzz.com

0.020109

TestApp

0.020109

lwp-request

0.020109

LWP::Simple

0.020109

Program+Shareware+1.0.3

0.020109

NutchOrg

0.020109

PHP

0.020109

NationalDirectory-WebSpider

0.020109

blogrunner-reaper+(+http:

0.020109

Missigua+Locator+1.9

0.020109

YahooFeedSeeker

0.020109

HericomBot+1.0

0.020109

Offline+Explorer

0.020109

Blogdigger

0.020109

CFMX+Agent

0.020109

sagg.urlqueue

0.020109

FLpro+1.1

0.020109

BravoBrian+SpiderEngine+MarcoPolo

0.020109

Waypath+Scout+v2.5+(devel)+-+info+at+waypath+dot+com

0.020109

Wildgrape+NewsDesk+Pro

0.020109

PictureOfInternet

0.020109

NetAnts

0.020109

RSS+FeedReader+Web+Part

0.020109

k2spider

0.020109

FeederService

0.020109

NIF

0.020109

NPBot+(http:

0.020109

BlogAggregate

0.020109

SauceReader-webreader-1.0b3

0.020109

Posted by Werner Vogels at March 2, 2004 10:02 PM
TrackBacks

The Rise of the RSS Feed Readers
Excerpt: All Things Distributed reports, "While waiting for some other log processing to finish, I had a moment to review the software agents that pull the rss & atom feeds from the weblog. In past two months there were 141 unique agents that requested one of t...
Weblog: Lockergnome's RSS Resource
Tracked: March 3, 2004 01:11 PM
All Things Distributed: The Rise of the Feed Readers
Excerpt: All Things Distributed: The Rise of the Feed Readers 141 different unique feed tools knocked on this weblog within the last two months! Check out the graph.
Weblog: Educational Weblogs
Tracked: March 3, 2004 02:14 PM
Popular RSS Readers
Excerpt: Werner Vogels posted some statistics regarding which RSS Readers hit his site most often. My first thought was that it is interesting to see which programs seem to be the popular ones. My next thought was that, from a content producer's standpoint, it ...
Weblog: Information Management Weblog
Tracked: March 3, 2004 11:28 PM
Feed 리더들
Excerpt: [via All Things Distributed] Werner Vogels의 The Rise of the Feed Readers에서 보여주는 통계자료. 어떤 Feed Reader들을 사용하는지를 알 수 있는 재미있는 통계자료입니다. Feed Reader들의 춘추전국시대임과 동시에 그 가능성을 짐작할 수 있는 자료라 생각됩니다...
Weblog: HOLLOBLOG (별주부뎐)
Tracked: March 4, 2004 05:23 AM
Welke RSS Reader worden er gebruikt?
Excerpt: Mocht je je ooit afvragen hoeveel RSS Readers er reeds op de markt zijn en welke er het meest gebruikt word wel op Werner Vogels Weblog All Things Considered - kan je een overzicht zien van welke Readers het meest gebruikt zijn, inclusief een lijst...
Weblog: RSS Nieuws
Tracked: March 4, 2004 06:04 AM
Rise of the Feed Readers
Excerpt: Interesting graph and discussion from Werner Vogel on the the RSS-readers used to access his site. I’m part of the few, the proud, the 8% who use FeedDemon, although I don’t completely like it. I’ve also looked at SharpReader and...
Weblog: Timely Snow
Tracked: March 13, 2004 12:32 PM
IzyNews - News, Blogs, and RSS Feeds in your eMail
Excerpt: RSS News surprisingly easy - no reader software
Weblog: IzyNews - News, Blogs, and RSS Feeds in your eMail
Tracked: May 28, 2004 02:12 AM
Comments

Hi Werner,

May I ask what your "definition of unique" is? Unique IP-Address I guess? This would unfortunately negatively impact the results of web-based aggregators. I believe bloglines puts the number of actual subscribers to your feed in the useragent - don't know if others do the same...

Either way, I'm glad my aggregator is #1 for your blog - us Dutchies in exile got to stick together! :-)

-----
PS: after clicking the PREVIEW button, the following error appeared under "Previous Comments":

MT::App::Comments=HASH(0x10985d4) Use of uninitialized value in sprintf at C:\Inetpub\MT\lib/MT/Template/Context.pm line 1187.

looks like an MT 2.661 bug, but I thought I'd let you know just in case you made some changes to Context.pm yourself...

Posted by: Luke Hutteman on March 3, 2004 10:23 AM

Hi Luke, I picked IP-Adress+feed+agent as the test for uniqueness. So if you are running two aggregators concurrently on the same machine you are counted as two users. Also if you are pulling in both an rss and the atom feed you also count as 2. I know the IP address approach is imperfect. A laptop carried from work to home is likely to have different IPs. Some DHCP engines do not re-assign the same IP when a machine has been switched off over night. On the other hand, a set of users behind a NAT will have the same IP.

I did manually compensate Bloglines, they only had a few ip-address, but indeed carry the #subscriptions.

I'll look into the MT thing.

Posted by: Werner on March 3, 2004 11:02 AM