Technologists.com tidbits 2003

about archives notes (RSS) music vidconf.net

Technologists.com

home > tidbits

(12/24) Getting Away From SPAM?

After I wrote my lengthy "End of Two Weeks of SPAM Purgatory!?", I almost discarded (did not publish) it because I thought it was too long and not that interesting. In retrospect, it is obvious that my perspective was distorted, since there has been significant subsequent positive impact:

Two ex-colleagues from the 80s, both of whom I still think of as friends, are trying to lead the fight against spam, but didn't even know of the other's efforts. I've been privileged to get them engaged in an intense dialogue and read what they have to say to each other. If that was all my efforts produced, that would be enough.
Every now and then, Chris Pirillo finds one of my "tidbits" worth republishing to his audience, which is orders of magnitude larger than the routine audience for my postings. While I feared that what I had written was too long and boring, Chris obviously thought otherwise, since he reproduced it in his Lockergnome Windows Fanatics feed day before yesterday. I've been so busy that I haven't even been keeping up with my usual RSS feeds, so I started getting queries in response to Chris' republishing before I knew of the republishing!
My ex-colleagues, who are much more spam-fighting experts that I am, seem to have concluded that my simplistic approach is more effective and reasonable than they would have thought without empirical evidence.

What I do really is simplistic. I am surprised (delighted) that it works as well as it does because I know so many ways the spammers could defeat it. Like many good programmers, I am basically lazy in the sense that I try to get the best results with the least amount of effort. Of course, that attitude is not limited to programming. For example, the late, great Israel Kamakawiwo`ole, in his video "IZ: The Man and His Music" talking about making music, says "basically ... what I do it's minimum effort but maximum pleasure, and that's part of being Hawaiian".

This "tidbit" is even more technically presumptive than the predecessor

. I'm hoping it will be helpful to a broad audience, yet definitive enough that I can get back to some of the other topics I keep saying I'm going to bring to completion, e.g., nt4eol and mod_auth++.

My intention here is to explain my practices in enough detail that anyone who runs their own mail server can adopt (with or without modifications) my practices. By far, the biggest assumption is that the mail server is a Unix oriented machine. (And to make it easier for me to get this written, I make some very weak assuptions that the server is running something similar to recent Red Hat releases or Fedora.) I've not even thought about doing similar things on a Windows-based mail server:

The environment is radically different.
Microsoft and others are attempting far more ambitious approaches for Exchange servers. (Somebody must have a good reason for trying to run a Microsoft-based mail server that doesn't use Exchange, but I've never heard one. From my perspective, you either use Exchange or a Unix-oriented environment. Before there is any backlash from Mac advocates, (a) Macs have yet to be established as significant in the server competition, and (b) I think of OS X as another flavor of Unix.)

Though everything I've done has only been run on recent Red Hat Linux or Fedora, I assume that my approaches would work with any of the BSD flavors and any of the vendor proprietary Unix flavors, but I don't even have easy access to most of those. (As those who have read my past tidbits know, I am very proud of what my team did in creating Dell Unix V.4 Version 2.2 and I still have a machine that can run Dell Unix. But that is irrelevant in a production environment. I also have a machine that can run Solaris 9 X86 or FreeBSD 5.1, but I haven't found the time to work with either of those. When that machine is powered on, it is most likely running Fedora or some flavor of Windows.) Finally, in terms of clients, what I have has mostly been exercised with Outlook 2000 for POP and Outlook Express 6 for IMAP.

I hope that is enough background. I am assuming that anyone who reads further has already gleaned the basic strategy from the prior posting

and is ready for more detail. One of my challenges in describing things is that my personal usage has been strictly IMAP oriented, but I expect that most people are more interested in POP.

Assume an e-mail gets in far enough that this discussion is relevant. I'm assuming that the default Red Hat/Fedora mechanisms are already in effect, plus all spam-oriented options in sendmail.mc are enabled, for example, sendmail.mc has

dnl FEATURE(`accept_unresolvable_domains´)

I am pretty sure, based on my server's log files, that such settings are pretty important. On the other hand, I don't have any evidence one way or the other whether spamassassin as supplied/configured by Red Hat does any good. In my experience, Red Hat has good judgement on such things, so I accept their judgement when I don't make the effort to make my own assessment.

All of the above could/should be seen as disclaimers. The substance of what I do is best visible at http://technologists.com/~procmail/.procmailrc and the referenced files visible as links in http://technologists.com/~procmail/.

Notes:

When I started this two years ago, I had no procmail experience. I looked through many examples of procmail-based spam fighting. I should be giving credit to the examples that influenced me most, but it was so long ago I don't remember who/what deserves the credit and thanks.
The above links show a very generic POP setup. But what I use in production for IMAP for myself and my wife isn't all that different from what you see in those links.
I depend on renattach to neutralize potentially hazardous attachments.
rc.suspect4pop is really not the reference version -- when I see an address or id that seems suspicious, I run virc.spam, which changes rc.suspect (the version I use for IMAP) and the perl expression embedded in virc.spam derives rc.suspect4pop from rc.suspect.
Because I am trusting my "white lists" rc.fromaddressbook and rc.exempt, I am brutal in rc.devnull and more brutal in rc.suspect. All the spammers reading this should immediately realize that my biggest vulnerability is forged "from" addresses.
Whenever I see something suspicious, I run virc.spam and change rc.suspect (and thus change rc.suspect4pop). On rare occassions I find something so obviously spammish that I change rc.devnull
mkfromaddressbook.pl is a simplistic way to create rc.fromaddressbook from Outlook "Contacts" exported as comma separated values.
rc.suspect4pop is adding an "X-Suspect: [Suspect]" header to the message. The client must be looking for this header to put the mail wherever suspect mail should go. For example, with Outlook, the "Rules Wizard" can be used to put mail with this header in a folder named "Suspect".
I used to have some domains in rc.suspect that I would really like to have left in there. For example, except for my monthly bill, anything I get from att.com is almost certainly forged. But some of the most important users of my mail server get lots of genuine mail from att.com. So I had att.com in rc.suspect, but took it out to make things right for the majority of the users of my mail server.

I hope the above is enough to help people use these tools for themselves.

Happy Holidays!

(12/21) End of Two Weeks of SPAM Purgatory!?

Background

This will be longish, definitely not a "tidbit". I hope you will find it worth reading. It concerns spam, spam filtering solutions, and ISP customer service experience. If those topics do not interest you, you need not read further. Some of this will seem very technical to some of the e-mail recipients, but I will try to explain the technical aspects as I write.

Spam is frustrating to all of us. Some say that more than half of e-mail is now spam. It seemed like spam started escalating dramatically after the 9/11 tragedy. My wife and I seemed to be victims of the early escalation of undesired e-mail two years ago, presumably because we had made our e-mail addresses very visible publicly, especially on our web sites. Starting in early 2002 I have been crafting a custom solution that has been satisfactory for the two of us.

Technical Issues: There are two primary Internet e-mail protocols for picking up mail: POP and IMAP. Most people use POP (Post Office Protocol). POP stores the mail on the client, so (unless you tell it otherwise) it deletes the mail from the server when your client gets it. If you only use one computer, that's fine. But if you use more than one computer, POP can be frustrating. My wife and I use IMAP (Internet Message Access Protocol) because it stores the mail on the server in such a way that it is the same regardless of what client computer you use. Originally, my spam solutions only worked reasonably with IMAP. (On the other hand, IMAP is inefficient and can be frustratingly slow...)

A good friend, very astute technically, called a few weeks ago and asked about using what I had done. Then the answer was wishy-washy, since he wanted to continue to use POP. Week before last, one of my client's people and my client complained to me about spam. They all use POP, with Outlook 2000. I told them I would make him a guinea pig for a modified version of what my wife and I use. I spent midnight to 4:30 a.m. that night reworking what I had done to make that possible, making a coordinated IMAP and POP version, got some more sleep, then spent much of the afternoon tweaking/testing what I had done earlier.

I applied it to my client's account and he seems happy with the changes. I think what I have done is immediately applicable to anyone who uses my mail server.

Stepping Back

First, what is spam? Some think it is any unsolicited e-mail. My wife likes to get e-mail telling her how to enlarge her penis!? My male friends don't!? More seriously, if you've ordered stuff from Amazon and they suggest you buy something similar, is that spam? Some say yes, some say no. If an outfit you've never heard of tries to sell you Vicodin, we probably all would call that "spam", even if Hormel wishes we wouldn't.

Second, in some sense the spammers are winning. They're tricking a lot of people. If you get spam and it gives you a "take me off this list" link, the last thing you want to do is click on that link. Spammers are looking for viable e-mail addresses. Most of the stuff they send goes to invalid addresses. If you click on a "take me off this list" link, they've suddenly discovered a valid address and will add your address to their list of viable addresses, exactly the opposite of what they said and you wanted.

Third, the e-mail protocols were designed without thinking about this problem. Unless/until those protocols change, which is not easy, there is no 100% solution. It is very easy to forge e-mail addresses. Spammers have lots of other tools at their disposal. The most we can hope for is to make spam no more annoying than the junk paper mail we receive and recycle.

Server vs. Client solutions: Ideally, this would all be dealt with at the e-mail server. That way, your dial-up connection wouldn't waste the time of downloading a virus you didn't want in the first place. (I'm not going to try to distinguish between spam and viruses. They're different, but I don't want either of them, and I use coordinated mechanisms to keep them at bay.) However, many of the commercial solutions, and there are some very good ones, deal with things at the e-mail client (i) because there can be more control at the client and (ii) maybe they can make more money selling solutions per client than solutions per server.

Open Source vs. Commercial Solutions: There are many good efforts both from the free software advocates and those trying to make money. (1) I didn't want to spend money or time sorting through all of the options and (2) I wanted to understand as best I could how to deal with the problems directly. It turns out that everything I use is either free software or stuff I've crafted myself. However, my client's request forced me to look at how to make what I did work with commercial software, specifically Microsoft Outlook, and I think I have done so.

Perfection: If you're looking for a perfect solution, stop reading. I don't have one. What I have is good enough for me, good enough for my wife, and, I hope, good enough for everyone who uses my mail server. Because of all the problems listed above, any attempted solution is going to fail to some extent, either by throwing away mail you want to see, or making you look at mail you don't want to see. My bias is to try to never throw away good mail, even if bad mail gets through. (I have a strategy for neutralizing viruses in bad mail, so even if bad mail gets through, it is unlikely to harm the computer.)

My Basic Strategy

First, I use an automatically generated "white-list" - anyone that I (or other user of my mail server) says they want to receive mail from gets to send me (or the other user) mail. If George W. Bush (probably forged, since he said he stopped using e-mail entirely when he took office) wants to tell me how to enlarge my penis, and G-dub is in my white-list, the mail gets to me. Part of what I have done is to make it easier to make this "white-list" be based on addresses the user has put in their address book. Second, anyone not in my white list who has VIAGRA or Vicodin or similar words or common mis-spelling of those words in their subject line gets their mail thrown away. They can be clever with mis-spellings and get the mail through. Every day, I (and other users of my spam filters) get a list of who had their mail thrown away, so if someone I really wanted to hear from wrote me, I can write them back and say "so sorry, my spam filter threw your mail away". Third, I have a growing list of "suspect" domains and addresses. Anything from those lists gets re-routed to a "Suspect" folder, in the IMAP case, or gets an X-Suspect header in the POP case. Either way, the "suspect" mail is in a separate folder and can be quickly scanned, when/if it seems worthwhile. 95%+ of what goes in my Suspect folder is immediately deleted. Finally, anything that doesn't pass/fail the above tests ends up in my inbox.

My Purgatory

Most of the above is excerpted and abstracted from an e-mail I sent to clients, friends, and family December 14. The youngest recipient was my niece just turned 15, so I didn't think I would offend her or my sister with the word "penis" and so forth. About 15% of the recipients had addresses at one of the largest ISPs, which I will refer to as XYZ hereafter. I think everything I am saying is factual, and there are only two reasonable interpretations of "XYZ" but I am trying to avoid offending either one of them. My telephone conversations with XYZ have intended to be polite and constructive, in spite of XYZ severely trying my patience and forgiveness. Anyway, the December 14 mail got through to all of the recipients, even with the potentially offensive content.

December 16 I sent a family-letter, to the same addressess, and the ISP (XYZ) rejected all of the copies going to their clients. The rejection message was very unclear and truncated. For my personal account with XYZ, the rejection said:

    ----- The following addresses had permanent fatal errors -----
    <chsauer@xyz.com>
        (reason: 554 TRANSACTION FAILED:  (HVU:B1) The URL contained in your 
	email to XYZ members has generated a high volume of complaints.?? 
	Per our Unsolic)

This is literally what it said, except that I have substituted XYZ for the ISP's domain name. (I assume they intended to say "Unsolicited" and continue further, but the many rejection mails I got all stopped at that same spot.)

This made absolutely no sense. If my spam descriptive e-mail got through, including potentially offensive words, why was this rejected? (A slightly excerpted version of the e-mail is visible at quarterdecademilestoneletterexcerpted.html.)

What URL could be the problem? Certainly not the one for the Methodist Church, http://nwhillsumc.org/. And seemingly unlikely any of the http://technologists.com/ URLs. I sent e-mail to the postmaster at the ISP and got no response. Surprise. So I started calling their customer support numbers. I probably spoke to 20 people, most of whom were seemingly not competent for the discussion at hand. They would give me a ticket number and say they were transferring me to someone who could help. Half of those transfers were disconnects!

Finally, I got a toll-free number for the postmaster's office. I called that number, waited on hold for an hour and 20 minutes, then finally spoke to someone who seemed to have a resaonable idea of how to diagnose the problem. The first thing he did was have me forward the rejected e-mail to an address at Yahoo.com! (XYZ is not Yahoo!) When he read the message, he couldn't see any reason why it was rejected. He gave me a new ticket number, admitted they were swamped with technical problems, and said that someone would resolve. He couln't say how long that would take.

Since I knew that most messages I sent to my XYZ correspondents were getting through, I realized there was an obvious workaround: put the e-mail on my web-site, password protect it, and tell the XYZ recipients where to find it and give them that id/password. That worked. So besides clumsiness/frustration, all of the problems were solved.

Last night I received an e-mail from my pastor, who is very computer savvy and aware of what was going on, saying "Thank God I don't use XYZ. A friend just upgraded to their latest software and now his system is unusable."

This afternoon, when I was intending to write this, I suddenly realized there were 3 URLs, not two that might be offending XYZ. I was pretty sure that http://nwhillsumc.org/ was not the problem, and I doubted that anything related to http://technologists.com/ was an issue, but there was a third domain name in the e-mail. In the postscript of the e-mail, I had said

P.S. This is not the end of my project, just a milestone. I still want more family e-mail addresses to add to the lists. I still want more photos. I'm also beginning to make MP3s of my out-of-print LPs, etc. One of my accomplishments last week was to help the Red Clay Ramblers make CDs of out-of-print albums they recorded! I'm astonished that I could help them in this regard to help them recover lost recordings of their own music. The MP3s are in a separate password protected directory to avoid copyright violations.

In doing so, I had given the URL for the Red Clay Ramblers web site, http://members.tripod.com/~RedClayRamblers/. Note that I am not making this a hyperlink, because that is the URL XYZ is rejecting. There is no sense in this at all that I can recognize. http://members.tripod.com/ was one of the first, after XYZ, to inundate their users with pop-up/pop-under windows. With that exception, I know of no reason why XYZ should be blocking references to http://members.tripod.com/~RedClayRamblers/.

When I realized all of this, and made tests that proved to me conclusively, that I had diagnosed the issue, I called the XYZ postmaster toll-free number, expecting to be put on hold for an hour. To my delight, a human answered immediately, seemed to understand what I was saying, said she was adding the info to my trouble ticket and that even though they are horribly back-logged, they should fix this problem in about a week.

(12/17) Quarter Decade Photo Project; MP3s; Simplistic Spam Solutions

Quarter Decade Photo Project

Somehow it always seems like I spend my time on things other than my plans. Sometimes this is logical, sometimes it is serendipity.

For 2 1/2 years I've been working on archiving family and friends' photographs, for a number of reasons:

If the physical photsgraphs get lost/destroyed, the digital versions are much better than ashes or whatever. In my case, this strikes close to home, since my father's birthplace had a bad fire over a decade ago, and many family treasures were lost. On the other hand, I have photos of my mother's mother's mother's family.
Many of these photos I had never seen before. If I had never seen them, then it is likely that other/younger family members had never seen them.
The digital versions allow for editing/enhancement that is impractical for those without a convential darkroom. (I used to have a darkroom in the 60s and had access to one in the 70s. Now I depend on what I can do with scanners and software.)
I've reached a major milestone, having scanned almost 1700 photos/slides/negatives in the last 2 1/2 years. Not only are these available on the web to most family members, I've made paper copies for those who are too old to want to learn to use a browser.

(The URL for the photos is http://technologists.com/photos/.)

Rise and Fall of MP3.com

One of my real thrills of 1998-99 was the emeregence of MP3.com, and the ability to make Caroline's 70s/80s/90s recording available to a broad audience. We were both thrilled in 1999 when some of her songs hit the top of the charts on MP3.com, not only in folk/country genres, but even her tribute to Bob Marley Tuff Gong and some of her other songs e.g., Lonely Man being promoted by mp3.com. Unfortunately, it looks like only the domain name "mp3.com" will survive, and all of the 250,000 artists' music will disappear unless/until something is done to make it available elsewhrere. Fortunately, all of Caroline's MP3s are vislble at http://kaybuena.com/songs/.

Simplistic SPAM Filtering

The other thing I've been doing is making my simplistic spam filtering solutions useful to all who use my mail server. If they endose what I've done, I'll tell more, but the biggest limitation is that my solutions only work for those who receive mail on my server.

(12/1) Making Peace With Windows XP

A while back, I admitted that I needed to make peace with Windows XP. I had three main issues with XP:

The new "Start Menu" seemed to slow me down, especially on notebooks and other machines with limited pixel layouts.
I had been unable to get my WiFi card to work with XP.
Some administrative tasks seemed unnecessarily harder than with 2K.

(Others have other issues with XP, e.g., the "activation" requirement. Those issues do not particularly bother me.)

I've newly started attending services at a neighborhood church. The senior pastor called me and suggested a 1-1 meeting. At the end of that meeting, I volunteered to help with any computer problems at the church, other churches and/or non-profit organizaitons. Bill, the pastor, immediately said he was having problems making his WiFi connection as secure and functional as he would like. It turns out that Bill has been working with computers about as long as I have, and has been working with PCs longer than I have! Though he's quite adept with managing his own and the church's computers, sometimes he gets stuck, as we all do.

When I arrived the next day to follow-up, I found out that (a) Bill's notebook was running XP and (b) the church had 802.11g equipment, with capabilities beyond my obsolescent 802.11b stuff. At first I was stumped, and didn't get things working much better that they already were. I installed XP on my notebook, yet again, with several significant differences from before:

I installed XP SP1 before trying anything else.
I read the knowledge base articles on the WiFi manufacturer's web site.
I set XP for the "Classic Start Menu" and made the other user interface tweaks that I routinely make when I setup a Windows machine for myself.

Though clumsier than my experience with Windows 2000, I did get my 802.11b stuff working with XP, including enabling WEP.

Then I went ahead and fetched the church's 802.11g equipment, got everything working the way I thought it should, including enabling WPA. I've taken 802.11g stuff back to the church and have it working well there.

So now my attitude toward XP is similar to my attitude toward Outlook -- in general I'm not a fan of Outlook, but for some situations it is the tool of choice. All things being equal, given a choice, I'd use Windows 2000 before using Windows XP. However, there is at least one thing I can do easily with XP, enable WPA, that I can't do easily with Win 2K.

So I think I've reconciled with XP at least as well as I have with Outlook.

The church's primary server is running NT4 -- yet another motivation for me to get back to nt4eol.

(11/21) Disks STILL Fail (Sometimes Catastrophically)

Those of us who remember computing before the last decade probably remember the great improvement in disk drive reliability that occurred in the early 90s. Before then, disk drives seemed to be the most failure-prone component of computers. "Everyone" was concious of "head crashes" (when a recording head hits the spinning magnetic platter, usually destroying both of them). Backups, mirroring, "Redundant Arrays of Inexpensive Disks" (RAID) and other strategies were emphasized to cope with the failures.

Seemingly overnight, disk manufacturers dramatically improved reliability. At a time when disk drives seemed to last a couple of years, manufacturers started quoting "Mean Time Between Failures" (MTBF) of close to 30 years! It is critical to realize that this is predicted average behavior, and that any given disk can fail at any time. Still, it is very easy to lull oneself into thinking that disk drives last forever. They don't!

Between my own premises, other commercial premises, and residential premises, I probably control forty to fifty disk drives. They do fail. I think I'm well prepared for failure of the most important drives. (I'm usually obsessive about backups and redundancy.) However, I got caught this week.

In my experience in the last decade, when a disk drive fails it is almost always gradual, not catastrophic. Presumably, the magnetic material fails in spots, and sectors of the drive become unusable. Depending on the circumstances, this may go un-noticed, but more often than not, even the in-experienced user will notice that something is wrong and at least ask for help. However, this Tuesday I saw the first catastrophic disk failure I can remember in over 10 years. Unfortunately, it happened to the disk drive that is most important to me, the primary drive on my Linux production server.

My NT4 production server was designed to be a rack-mount server, has a built-in RAID system and good monitoring software. As long as I keep an eye on the monitoring software, any significant problem is very unlikely. (One drive failure would probably only be noticed by me and the warranty service person.) However, my Linux production server was really designed to be a desktop machine and has had minimal disk redundancy. I had been planning to institute much more formal mirroring when I upgraded that machine from RH 9.0 to Fedora, probably Thanksgiving weekend. I still intend to institute the mirroring, but right now I am humbled and embarassed that that machine failed Tuesday, with a small loss of data and an outage of several hours.

It could have been worse. I was on premises and noticed the problem within an hour. My existing redundancy strategies worked as expected so that the loss of data was minimized. I decided to go ahead with Fedora on Tuesday, since I needed to do a complete OS install in any case. That went well. I had been out of town three of the previous four days and would have had much more of a challenge fixing things remotely. (I believe I could have done so reasonably, with one of my hot spare machines and backups. I don't think there would have been any worse loss of data, but the problem would have not been recognized so quickly and the recovery would have taken longer.)

For those of you in the U.S., Happy Thanksgiving!

(11/12) Fedora's Fine; nt4eol; mod_auth++

Fedora's Fine

So far, I have no complaints about Fedora. It feels like a good successor to Red Hat 9.0. The only obvious omission is tripwire. I created my own ad hoc, simplistic analog of tripwire in 1998, before I knew of tripwire, and have continued to maintain it. So the omission of tripwire probably is a concern to others, but doesn't directly affect me. I have Fedora installed on all of my Linux machines except for the production machine that is still running RH 9.0 (and the museum machine that runs Red Hat 5.2). Assuming things go as I expect, Fedora will replace 9.0 on the production machine in a couple of weeks.

Of course, the big questions revolve around updates, business practices and other potential changes as Red Hat proceeds with Fedora. For now, I'll hope that those questions are resolved positively.

Tangentially, I have learned a lot more about multi-booting many of the operating systems in my museum. In other words, I've spent many frustrating hours installing and reinstalling many of those operating systems. The big problem seems to be that they make different, incompatible, assumptions about disk geometry. I won't rant about that the way I might want to, but I will say that NT4's "Disk Administrator" tool was my best friend in resolving the problems.

NT4 Server End of Life

All the above and other activities have impeded my nominal plans. I'm filling in my experiments and experiences in nt4eol, but have much more to do.

mod_auth++

Because of the above, no new news about mod_auth++. However, I plan to use Fedora to test/fix/enhance mod_auth++ before I put Fedora on my production Linux server.

(11/06) Brave New World: NT4 2004 Edition

NT4 Server End of Life

Huxley probably wouldn't notice, but 2004 is when we'll have to deal with the real demise of NT4 Server. I've started nt4eol to describe my experiments and experiences. Right now there are four placeholders for additional pages I plan to add.

"and all those things" (mod_auth++, Fedora)

Except for the citations in the October 30 and November 5 editions of the Lockergnome IT channel (thanks Chris!), I don't have much to add about mod_auth++. I continue to use it, test it, and recognize bugs, but I need to allocate time to fixes/enhancements.

In the Linux world, there's lots of news, especially the Core 1 release of Fedora and Novell's acquisition of SUSE. I got the Fedora ISOs quickly, thanks to BitTorrent, and am beginning to assess Fedora as a replacement for Red Hat 9.0. Obviously, there are going to be many assessing/wondering this, e.g., Red Hat's Fedora released - the upgrade path for the rest of us?. My assessment so far is definitely "thumbs up". I think Red Hat has done the right thing. Technically, Fedora feels to me like an incremental Red Hat Linux release. I probably grumbled more about the changes between RHL 7.0 and RHL 7.1 than I will grumble about what has changed between RHL 9.0 and Fedora. My evaluation of a new Red Hat release goes through three stages:

Install "everything" on a machine that doesn't matter and look for obvious problems. I've done that with Fedora. No obvious problems.
Install on my "hot spare" server. That server is intended to be able to take over if either my Linux or my NT4 server fails. I've just started installing Fedora on my hot spare server.
Install on my production Linux server.

Fedora feels more like an incremental Red Hat Linux release than something new. If I were a product manager at Red Hat, I would be grumbling about all of the places Fedora still seems like Red Hat Linux 9.x from a business perspective. For example, on one of the early pages, it says "Welcome to Fedora Core 1 ... If you have purchased Official Fedora Core, be sure to register your purchase through our web site, http://www.redhat.com/." Since you can't purchase Fedora, this is nonsense. But the similar message that existed with shrink-wrap Red Hat Linux was apropos. Anyway, so far I am very pleased with Fedora both from a technical and a business perspective.

(10/31) mod_auth++ Beta; "it's the end of NT4 as we know it"

(I was going to post this 10/30, but how could I not wait for more burnt orange on Halloween?)

1. mod_auth++ Beta

I've solved the biggest problems I had with mod_auth++. Let's call the current version "Beta". I'm expanding my production use of mod_auth++. If you're curious, (and willing to assume any risk involved) please give it a try. The usual disclaimers apply -- I take no responsibility if something goes wrong. A special thanks to Matthew Gregg at the mod_auth_any project for telling me of their approach to avoiding the "browser close/reopen" problem.

2. "it's the end of NT4 as we know it"

Microsoft is bringing Windows NT4 Server to "end-of-life". See Retiring Windows NT Server 4.0: Changes in Product Availability and Support. My interpretation is that there will be no new fixes, except for security issues, after this year. Security fixes will stop a year later, after January 1, 2005.

This seems perfectly reasonable. NT4 is ancient. Microsoft has released two successor products, Windows 2000 Server and, now, Windows 2003 Server. (Of course, there are sub-versions of both 2000 and 2003 Server.) However, there are lots of production NT4 servers going strong. My two production servers run NT4 and Linux, respectively.

The biggest problems in the upgrading are the directory issues. Because of the radical changes between NT4 and the successors, there is no easy answer. Here are some possibilities:

(Ostrich mode) Pretend there is no problem. This might actually be viable for my production NT4 server, since it has minimal directory issues and is behind a firewall. However, I'm assuming that by January 1, 2005 it will be running some flavor of Windows 2003 Server.
Samba 3+ on Linux (or some other Unix-like environment). This is plausible. I've experimented with the latest Samba build (Samba 3.0.1pre1) and see much promise. However, I'm not as optimistic as Samba 3.0 Does Windows Even Better.
Windows 2000 Server
Windows 2003 Server

I used to be proficient in dealing with NT4 directory issues, but had gotten out of practice. I've given myself a refresher course. Soon I plan to add a new section to this site devoted to all of the above, plus, LDAP, which is even more important than I realized before.

(10/20) mod_auth++ "and all those things"

mod_auth++

"mod_auth++" started with my frustration with existing authorization mechanisms that are available with standard browsers (IE, et al) and web servers (Apache, IIS, et al). I wanted to be able to control access to web cams, photographs, and other static content on my web servers in ways that seemed impractical with the commonly used mechanisms. After investigating and thinking, I believed I knew how to do so.

I think I have successfully prototyped what I envisioned, at least with IE and Apache, and believe what I call "mod_auth++" will also work with other browsers and servers. There is a first draft document at mod_auth++ which describes what I've done, how mod_auth++ might be used, and the limitations and problems I've recognized.

"and all those things"

A friend who read that I was making 12-year-old software and hardware work asked if I was a "masochist". I said "no, I am a historian". I took his comment as a challenge and brought my Dell 320N+ 386SX 20MHz back to life running Windows for Workgroups 3.11, including an alpha version of Mosaic 2.0.

A different friend said I had created a museum and should make it visible on the Internet. I wish I could. Unfortunately, 12-year-old software (and lots of more recent software) would be very vulnerable in the currently dangerous state of the Internet.

I did install Windows 2003 Server on a couple of machines, but have not done much more than that. Perhaps for good reasons, Windows 2003 Server is much less friendly to multi-booting other operating systems (Microsoft and non-Microsoft) than previous Windows Server versions. In particular, on one machine that already had Windows 98 and Red Hat 9.0 installed on it, the Windows 2003 Server install disabled the Windows 98 and corrupted the Linux install. However, on a different machine that had Windows NT 4.0 Server, Red Hat 9.0 and Windows 2000 Professsional, installing Windows 2003 Server did no harm to any of the existing systems. So I have to assume that the Linux corruption on the first machine was not intentional. The Windows 98 disabling clearly was intentional.

(10/6) "If Tomorrow Wasn't Such A Long Time"

When I said "Diving In", I thought I would stop updating this page for a month or so. I had no idea it would be 8 months! But everything always takes longer than you think it will. The words of one of my main muses, Bob Dylan, have resonated with me as I've tried to overcome bloggers' block and get back to writing. (I don't really think of this as a daily blog, but I have meant to write something every few days, not allowing lapses of months and months.)

What have I been doing?

Spending my time with personal and family challenges and blessings. In particular, June 21st I was father of the bride. Not only was my daughter's wedding a blessing, I tremendously enjoyed my role and helping/seeing it happen. (I was not at all like Spencer Tracy or Steve Martin in the movie renditions.)
Pursing the "unifying access control approach that will be both secure and usable". So far, this has worked out fairly well. This was the nominal reason for taking the writing hiatus and "diving in". I have alpha+/beta- code working with Apache. I've used some of the new capabilities for my own production purposes for a couple of months. One of my next steps is to finish and document what is visible at http://technologists.com/mod_auth++/ -- what is visible there now is mostly incomprehensible unless you look at what I've done to mod_auth.c.
Becoming a self-taught expert regarding Microsoft Active Directory, as implemented in Windows 2000 Server. This ties back to my interest in making LDAP usable for non-experts, since Active Directory is based on LDAP. However, Active Directory is at least as intimidating as plain LDAP. Next I plan to go back to plain LDAP and also explore the reported improvements in Active Directory in Windows Server 2003.
Expanding my already eclectic interests in alternate operating evironments. What if SCO really puts a damper on Linux? I've been looking at Solaris and FreeBSD much more closely, understanding how they work on their own and how they fit with Windows and Linux. What if Samba 3 really is a satisfactory replacement for a Windows NT4 Server?
Both because of this expansion of interests and my desire to preserve my access to ancient environments, I've been setting up some multi-boot machines that allow me to run any of the following, though not all at the same time. (These are listed in approximate order of the age of the OS, oldest first. These are in addition to my usual operating/testing environments: Windows 2000 Professional, Red Hat Linux 9.0, and Windows 2000 Server.)
1. Dell Unix V.4 Version 2.2, which, 11 years ago, was the best x86 implementation of Unix. It was based on the latest AT&T SVR4 and included many extras, notably the Roell X-server (pre-cursor to XFree86) and lots of useful public source packages.
2. Windows 95 (OSR2) with IE 5.5. (I'm tempted to bring up a Windows 3.1 environment that works with TCP/IP -- I've got a 20MHz 386sx notebook that only knows NETBEUI and IPX/SPX right now. We'll see.)
3. Windows NT4 Workstation with IE 5.5.
4. Red Hat Linux 5.2
5. Windows 98 with all the latest Microsoft updates.
6. Windows NT4 Server with all the latest Microsoft updates.
7. Solaris 9 X86
8. FreeBSD 5.1
9. (Soon to come) Windows 2003 Server.
1 through 4 are on a 12-year old Dell 450 DE/2 DGX! Part of what started this all was seeing if I could get the DGX running again, and to see if I could get Linux running on that machine. It turned out that 5.2 is the most recent Red Hat release that I could get to work with a machine that old.

There are at least two serious omissions from this list:
- Windows XP. I've tried it numerous times on different machines and just don't like it. One of these days I'll have to make peace with XP, just as I had to make peace with Outlook, but that took several years.
- Macs. I have two ancient, non-functional Macs that might be cobbled together into one functional system. What I really need to do is buy a modern Mac. But I haven't bought a modern PC for myself in quite a while, so I'll probably get a Centrino notebook before I get a new Mac. (I do have NextSTEP 86 and compatible hardware, since the X86 port was developed on prototypes of the Dell 450 DGX, but getting that working again seems much less important that a modern Mac.)

More later.

(2/6) Valuable Distractions and Discoveries: Diving In

I've not written one word of my intended requirements document. Rather, I've been pursuing a "unifying access control approach that will be both secure and usable":

I discovered a grant solicitation that seemed closely related to what I'm working on, so I submitted a funding proposal. This may have seemed a distraction, but the thinking and learning were very valuable even if my proposal is not funded.
I think I have come up with a secure scheme for new authentication and access control mechanisms that will integrate nicely with existing web browsers and servers. It seems to fit nicely with the Apache web server. There seems to be a natural way to do equivalent things with Microsoft's IIS. This is a meaningful discovery if, as it seems, there is a way to provide improved authentication and access control mechanisms that fit well with existing code. It will be a victory for software architecture if this works without a huge coding effort.
Now it is time to (i) dive in to the details of the existing Apache authentication modules and (ii) build new modules with new capabilities. Having never even built Apache from source code before now, there is probably much to learn. However, I've already found what looks like a minor bug in one of the existing authentication modules, and think I have a fix for the bug, so the next step is to build the repaired module and test.

[koko] knowing and accepting limitations
February 6, 2024
[koko] keeping warm
August 7, 2023
[koko] still learning
June 18, 2023
Roe is gone, one more round
June 28, 2022
“just as good as Caruso” – props for Kim Wilson & Charlie McCoy
May 5, 2022
Mel West, engaging people to help people in Nicaragua
April 25, 2022
Glimpses from the Vulcan, 1969-70
February 14, 2022
[koko] MISP 2022
Janary 10, 2022
Why I continue to serve — I remember Nicaragua
December 13, 2021
Making private 1960s and 70s recordings public
August 21, 2021
Jimmie Vaughan set w/ Storm track I recorded
August 4, 2021
Celebrate Ramblin' Jack Elliott's ~~90th~~ ~~91st~~ 92nd birthday!
August 1, 2021
[koko] LP digitizing milestone approaching
May 18, 2021
remembering Denny Freeman
April 28, 2021
[koko] Dell Unix sustainable!
January 19, 2021
Computer Systems Performance Modeling
August 25, 2020
Remembering RESQ
August 25, 2020
[koko] (welcome to …) eight Jurassic O.S. on 1992 Dell 486D/50
September 26, 2019
[koko] reviving timbl's WorldWideWeb browser
July 1, 2019
[koko] exploring NEXTSTEP 486
July 1, 2019
1992 JAWS demo for Stewart Cheifet
May 17, 2019
Let's start at the very beginning... 801, ROMP, RT/PC, AIX versions
March 8, 2017
NeXT, give Steve a little credit for the Web
October 8, 2011
Mainstream Videoconferencing available again
February 14, 2008
A brief history of Dell UNIX
January 10, 2008