« September 11, 2005 | Main | On being overly cautious »

September 18, 2005

New Tool: fastzolver

I've been using The Webalizer for my weblog analysis for some time, and the April 2005 issue of Linux Magazine has my feature article on "Tracking Traffic with Webalizer" (it seems that registration is now required - ugh).

While I was writing the article, I became unhappy with the very slow manner in which webazolver resolved IP address to DNS names, so I wrote my own program, fastzolver, which uses the excellent adns asynchronous DNS resolver library: it's dramatically faster. The existing code sends a request for DNS resolution, and waits: a dead or unreachable DNS resolver might take 60 seconds to timeout, so it burns a full minute doing absolutely nothing while it waits. The extant solution is to run multiple subprocesses, but this is not very efficient.

Using an asynchronous solution means that we send a request and make a note that we're waiting for a response, then send the next request. As responses or timeouts occur, they rendesvous with their requests, but this allows a single thread to handle many more outstanding requests without burning time waiting for I/O that won't ever happen.

I write a fair number of communications controllers, and asynchronous I/O is the only way to scale a system up, and I was able to apply this to The Webalizer to good effect.

Unixwiz.net Tool: Very fast Asynchronous DNS resolver for Webalizer

Posted by steve at September 18, 2005 06:42 PM

Trackback Pings

TrackBack URL for this entry:
http://www.unixwiz.net/mt/trackback/49

Comments