These are the old pages from the weblog as they were published at Cornell. Visit for up-to-date entries.

March 18, 2004

Feed Pull Periods per Software Agent

Is there a noticeable difference between the way the different software agent are configured in terms of polling frequency? Below are graphs for the inter-pull periods (per user) dissected by the most popular feed agents. First graph is the CDF for all agents which shows you that there is quite a bit of variance in configuration up to the point of 1 hour, by that time most of the agents have issued 80-90% of their requests.

Exception is FeedDemon, which uses a different model from most of the other aggregators. In FeedDemon you organize the feeds into channel-groups and only the group that is currently visible will be automatically fetched. The default fetch period is 3 hours. Other channels-groups are fetched as soon as you switch to them, or when you issue a 'update-all-channel-groups' request. This non-automated behavior gives feed-demon a very different pull behavior than all the other aggregators, as it is more user driven and less automated (unless of course you have all your feeds in one channel group). This gives you classical open vs. closed feedback loop distributions.

Radio is the other aggregator that shows a clearly different behavior, with almost no request coming within less than an hour.

The other two graphs show the histograms per agent, but they are not as clear as the CDF. It works better if you plot those per single agent instead of grouping more on a chart.

Posted by Werner Vogels at March 18, 2004 02:07 PM