« August 2005 | Main | October 2005 »

September 26, 2005

On being overly cautious

Most of us have least heard of buffer overflows, and all C programmers ought to pay attention to the size of target arrays and never attempt to put more in them than they have capacity for. Using something like: 

strcpy(targetbuf, ptr);        // might be dangerous

is — in general — asking for trouble because one doesn't know whether the pointed-to string (ptr) will actuallyfit in the target space. So many developers use size-limited functions that insure there won't be any overflow.

One of these is snprintf, which performs a printf-like copy to a buffer, but the buffer's size is included: 

snprintf(targetbuf, targetsize, "message %d %s", i, p);  // safer!

and it guarantees that it won't write beyond that. It's a wise technique, but one can be overly cautious about it. This was seen in code today (multiple times): 

snprintf(result_string, BUFSIZE-1, "%s", "U");

I'll leave it as an exercise to the reader to find better ways to handle this particular circumstance.

Posted by steve at 09:42 AM | Comments (1) | TrackBack

September 18, 2005

New Tool: fastzolver

I've been using The Webalizer for my weblog analysis for some time, and the April 2005 issue of Linux Magazine has my feature article on "Tracking Traffic with Webalizer" (it seems that registration is now required - ugh).

While I was writing the article, I became unhappy with the very slow manner in which webazolver resolved IP address to DNS names, so I wrote my own program, fastzolver, which uses the excellent adns asynchronous DNS resolver library: it's dramatically faster. The existing code sends a request for DNS resolution, and waits: a dead or unreachable DNS resolver might take 60 seconds to timeout, so it burns a full minute doing absolutely nothing while it waits. The extant solution is to run multiple subprocesses, but this is not very efficient.

Using an asynchronous solution means that we send a request and make a note that we're waiting for a response, then send the next request. As responses or timeouts occur, they rendesvous with their requests, but this allows a single thread to handle many more outstanding requests without burning time waiting for I/O that won't ever happen.

I write a fair number of communications controllers, and asynchronous I/O is the only way to scale a system up, and I was able to apply this to The Webalizer to good effect.

Unixwiz.net Tool: Very fast Asynchronous DNS resolver for Webalizer

Posted by steve at 06:42 PM | Comments (0) | TrackBack

September 11, 2005

September 11, 2005

.. has more than the obvious meaning for me. In addition to being the fourth anniversary of the terrorist attacks on the United States, today also marks the 20th anniversary my consulting career's start in Southern California.

I started V-Systems with school buddy David Droman on September 11, 1985, originally to do the same kind of consulting I do now; software development and system administration (though we also did hardware sales, something I don't do as a consultant). In the late 1980s David stumbled upon computer facsimile, and the VSI-FAX system was born. This was one of the first, and still the most popular, commercial UNIX fax system.

I really preferred the variety of consulting, so with David's blessing left the company to take care of our original customers. I nevertheless did a fair amount of fax software development — all the modem drivers, and most of the port to Windows NT — and our informal partnership continued until V-Systems was sold to Esker Software a few years ago.

But throughout all this, I kept with my consulting customers. I've gotten to work in many different industries and with many different technologies, and this has really contributed to what I hope is a decent depth and breadth of experience.

Our first customer was MG Bookkeeping Services, which did accounting for ARCO am/pm Mini Marts on AT&T 3B2 computers, but they've long since changed their focus to payroll under the name Telepayroll. We've had some rough times over the years — the ARCO contract evaporating was particularly memorable — but it remains a delightful relationship to this day.

I still have several other customers from the eighties and early nineties, but nothing gives me as much pride as the endorsement of consulting to one customer for twenty years.

Michael, Martin: it's been an honor.

Posted by steve at 08:08 AM | Comments (7) | TrackBack

September 08, 2005

Dumb software: CPAN

CPAN is the Comprehensive Perl Archive Network, and it's a huge repository of perl modules covering an enormous array of computing needs. It's an amazing resource and one of the best features of perl.

The client side is the cpan command, which can find and install requested modules, and it knows about and offers to install dependencies: 

# cpan
cpan shell -- CPAN exploration and modules installation 
cpan> install HTML::Foo

I'm taking a look at the RT ticketing system, and it has a long list of requirements. This is exactly what CPAN was meant for. But as I installed package after package, it kept trying to install the junoscript-perl package each time. Huh? This package is used for talking with Juniper routers, and is reportedly excellent, but I'd not requested it. Hmmm.

We do use Juniper routers, so perhaps this is stuck in a cache somewhere, so I looked all over the system and killed any reference to junoscript. It kept coming back. WTF?

Several hours later I figured it out: in addition to the shell version, one can launch an install directly on the command line, and I'd been using: 

$ cpan install HTML::Foo        surprise!

Well guess what: install is not a keyword on the command line, it's the name of a package or module, and cpan located a package in its directory that provides the install module. This directory is found in the .cpan/sources/modules/02packages.details.txt file, and it contains: 

...
INSPEC::BibTeX     0.01 V/VP/VPLA/INSPEC-0.01.tar.gz
INSPEC::Retriever  0.01 V/VP/VPLA/INSPEC-0.01.tar.gz
install           undef J/JU/JUNOS/junoscript-perl-6.4I0.tar.gz
integer            1.00 N/NW/NWCLARK/perl-5.8.6.tar.gz
interface          0.02 S/SW/SWALTERS/interface-0.02.tar.gz
...

So junoscript is providing an install module, and this is what I was asking for. Over and over. For hours. The proper syntax is: 

$ cpan HTML::Foo                correct

This was technically my fault — I used the wrong syntax — but this is a clear violation of the Principle of Least Astonishment, and I can't believe how frustrating this was. A bit of google shows that I'm not the only one who's had this problem either.

Why isn't this a keyword, or at least have the install module removed from Junoscript?

Posted by steve at 05:42 AM | Comments (5) | TrackBack

September 03, 2005

Finally: new website design

It's only taken months of effort, and I can't believe how much work it was, but I've finally deployed my new website design. The actual creative part was done by the great Chris Mospaw, but since there was no way I could essentially go offline while I converted the whole site en masse, so I had to organize this in a way compatible with routine site updates.

I maintain my site content in the excellent Perforce source code control system, and I created a separate branch for the new content. The heavy lifting of any new design is in the CSS, but I also have a lame, home-grown markup processor written in perl that translates .web to .html. It's very lame, but functional enough.

So over the last several months, I've been migrating the common markup to be compatible with both old and new designs, minimizing the number of files that actually differed. This required visiting almost 200 content pages to migrate to the new markup, and the tedium was compounded by the difficulties due to not knowing CSS that well.

Finally I've gotten it deployed and off my list. There are still a few glitched (the lower right corner doesn't look right in IE), and I'll probably never stop fooling with it, but I don't think it's any worse than the old layout. I've even gotten my weblog to partly share the look.

Those finding things horribly wrong are encouraged to let me know.

Thanks again, Chris.

Posted by steve at 10:08 PM | Comments (1) | TrackBack