Technologists.com tidbits 2005

about archives notes (RSS) music vidconf.net

Technologists.com

home > tidbits

(9/10) The Really Difficult Parts; More iBook Struggles

"Let Me Keep My Metadata!" (July 22, 2005)
"They Took My JPEGs! & won't give 'em back!" (July 16, 2005)
"They Took My Kodachrome!" (July 2, 2005)
"Don't take my Kodachrome away" - Paul Simon (1973)
"They took our jobs!" South Park (April 28, 2004)

The Really Difficult Parts

The previous notes in this series were relatively easy to write. I started to write this one right after the last one, but became consumed by other things.

However, both back in July and now, I find it hard to take the next steps, to start writing about identifying/sorting/searching/sharing digital photos. I think I understand lots of the pieces, well enough to

Be very conscious of the limitations and problems with what I have done so far for my family photo sharing site.
Quickly dismiss most of the commercial approaches I have examined.
Ponder all of the limitations in Flickr yet admire how much better Flickr seems than the alternatives.

Putting all the pieces together is difficult.

Some of the requirements include:

Storing vast amounts of data. (My family photos site of 1800 photos is about 8GB, so anything with commercial scalability is many terabytes, if not more than a few petabytes of data.)
Privacy controls to restrict (or not restrict) access to images.
Accessibility to images ranging from thumbnails to original sizes.
Preservation of metadata on all accessible images.
Facilities for adding/editing of image metadata.
Flexibility in organizing and searching, including many views:
- Date (ranges)
- Subjects (including lists of specific people, etc.)
- Locations
- Photographer
- Expressions allowing combinations of the above and other info.

What I have done previously allows for pieces of the above and has seemed useful for a first attempt. But that original approach is really very simplistic. I am pondering how and whether to

Continue the status quo.
Attempt incremental changes.
Attempt radical changes.

During my hiatus from thinking/working on this topic, I did notice Philip Greenspun issuing a related specification. See http://philip.greenspun.com/images/tools/slide-shows-spec.txt.

I found it interesting that Greenspun's spec does not seem to assume a SQL database would be involved in any way. Greenspun is an expert on databases and web sites. In particular, his book Database-backed Web Sites was very meaningful to me when I read it almost a decade ago. I have been assuming that if I attempt radical changes, they will involve some use of a SQL database to facilitate organizing and searching of photos. (I assume the database would not contain large photo images.)

This is probably the last piece, for a while, on this topic until I take the time to experiment and ponder.

More iBook Struggles

My iBook was seemingly doing just fine. I closed the lid one morning a week ago, expecting it to go to sleep mode. When I reopened, it wouldn't wake up. I tried every trick I could think of to get it to power cycle, but no signs of life. I suspected that the small power circuit board that was replaced under warranty in November had failed again, but both the system and the replaced board are out of warranty. (The original warranty expired after 1 year in February, and, coincidentally, the 90 day repair warranty also expired in February.)

At first, I was ready to abandon it, but a friend recommended a local Apple specialty store (not the Apple store at the shopping mall). That turned out to be good advice.

For a $45 diagnosis fee, they sort of brought the machine back to life. They worked on it for a couple of hours to accomplish this, so I feel like I got a bargain. However, the backlight on the built-in LCD has clearly failed. They recommend sending to Apple for flat fee repair ($395 less the $45).

My first reaction was total disbelief at the diagnosis and the price. But I trusted the person I was talking to and was able to reconcile almost everything. The reason I was shocked at the price was recollection of an incident in 1993 when I was at a Microsoft conference in Anaheim. In a crowded dark auditorium, I stepped on my Dell notebook, breaking the backlight tube (a miniature fluorescent bulb). I was no longer with Dell, but a friend still there graciously Fedexed me a couple of replacement backlight tubes. In the hotel room, with only a pocket knife and fingernail clippers for tools, I successfully replaced the backlight tube. But of course it seems that everything Apple costs more, and iBooks seem incredibly difficult to repair. Back to the iBook diagnosis, I remember thinking that the LCD was dimmer than it should be when it came back from Apple repair last year.

Except when I would take it out of the house, the normal place for the iBook was on a desk next to a dual input (DVI/VGA) LCD. A Windows machine uses the DVI input, but the VGA input was unused. So now the iBook is plugged into the VGA input and seemingly working just fine, ignoring the built-in LCD.

However, I don't think I could ever trust this iBook again as a portable machine. I'm still thinking it is likely to fail in some other way, just sitting on the desk. So it seems very unlikely that I will get the backlight repaired.

I have picked out the Dell Latitude I think I want, am still pondering some of the specifics, but will probably order before the end of the month. I don't expect to travel until October, so I don't have urgent need for a notebook. If I had to suddenly travel, I could probably live with my 6 year old Latitude.

(7/22) Let Me Keep My Metadata!

"They Took My JPEGs! & won't give 'em back!" (July 16, 2005)
"They Took My Kodachrome!" (July 2, 2005)
"Don't take my Kodachrome away" - Paul Simon (1973)
"They took our jobs!" South Park (April 28, 2004)

If you buy a (conventional audio) CD, you expect to find basic information about the music printed on the disc and/or the paper insert:

Year published (copyright date)
Song titles
Performers/composers
(maybe) lyrics and other descriptive information

If you buy an MP3 file, most of this metadata should be embedded in the file, in addition to the sounds. Programs like iTunes and Musicmatch that organize libraries of MP3s depend on the metadata. Sometimes these programs ask the user to provide some of the metadata. (These programs usually mostly ignore the file name. "MyFavoriteSong.mp3" is not a reliable indication of what song is in the file!)

50 years ago, if you took a snapshot, you could expect the photo-processor to put the processing date on the prints and/or slides. You might pencil a description on the back of the print or the border of the slide. Today, if you take a digital photo, you can expect your camera to put all sorts of metadata in the JPEG file: date, time, exposure settings, focus settings, pixel counts (e.g., 2048x1536), etc. Sometime in the future, you or someone else might want to know the names of people in the picture, where it was taken, etc.

MP3 and JPEG files are analogous:

Both depend on esoteric compression algorithms to reduce huge data files to relatively small files, without loss of quality in the perception of the non-expert.
Both are immensely popular on the Internet, in portable devices, personal computers, et al.
Correspondingly, both have inspired prodigious numbers of software projects, from the obscure individual programmers to the mega-multinational companies, e.g., Microsoft.

Beyond the obvious difference that MP3 is for audio and JPEG is for still images, a friend argues that the second biggest difference is that the JPEG metadata must be provided by individuals (though as more and more individuals create MP3s, e.g., for podcasts, this difference fades). The second biggest difference, in my opinion, is that MP3 efforts have been enormously successful (ignoring the copyright wars) and JPEG software has been relatively (but not totally) unsuccessful.

Why? MP3 software is targeted at typical end users. How many people listen to music? JPEG software seems mostly targeted at very sophisticated users. How many people understand F-stops and ISO film speeds? Metadata, the arcane additional information stored in an MP3 file to describe the audio and stored in a JPEG file to describe the image, has been treated radically, and unnecessarily, differently in the MP3/JPEG worlds:

With MP3, the metadata (ID3 and its competitors) started out simplistically, almost too simplistically. ID3 and its competition have evolved slowly enough to become de facto standards. The initial ID3 totaled 128 bytes, allowing for
- Track Name - e.g., the song title
- Artist Name
- Album Name
- Year
- Comment
and one byte for genre: "blues", "classic rock", "country", etc.
On the other hand, metadata for JPEG started more comprehensively and leapt forward without apparent consensus. Where a small set of analogs of the above would be a great starting point, there are a plethora of fields consuming as many bytes as needed. For example, in lieu of the "Track Name" there are fields for "Headline", "Location", multiple variants of "Title" and other fields that potentially name the image. There are just shy of a dozen ways to describe the "Creator". There are at least two fields for "Description". Get the "picture"? See JPEG captions and more, EXIF, and IPTC IIM for some of the background. If you do pursue these sources, pay attention to the complexity and redundancy.
As a consequence of (1), MP3 software tends to be relatively consistent in handling of metadata. Further, there are lots of utilities readily available for manipulating MP3 metadata. On the other hand, JPEG software often ignores the metadata entirely. The few programs that attempt to notice the metadata do so in different ways making the metadata unreliable at best and useless at worst.
Digital cameras typically include reasonable starting values for the metadata and store them in the photo file. Things start to fall apart when the file leaves the camera.

I have tried numerous pieces of photo-oriented software (Windows, Mac and "open source") and several photo-oriented web sites. With few exceptions, these experiences have been very disappointing. The software and web sites not only ignore existing metadata when they could make good use of the metadata, they usually discard the metadata. Aargh!

The two most promising sources of commercial software for JPEG metadata seem to be Adobe and JASC. Even these make it harder to find/edit the metadata than I would like, but at least they have some usable provisions for handling metadata. Google's Picasa seems to comprehend some of the more interesting metadata, but then seems to store changes to the metadata in a private database, rather than the JPEG file.

Flickr seems to recognize some interesting metadata when a file is uploaded, but after that point seems to do things its own way. It does appear that the paid subscription version of Flickr does allow for downloading of the original files, contrary to what I said before

. I have not yet paid for a subscription, so I have not tested.

Some programming languages have classes or other support for JPEG metadata. Of these, PHP seems to be the most comprehensive and interesting. Since PHP is widely used for web sites, the PHP support seems especially encouraging. See Metadata Toolkit Example.

Consistently handling metadata seems key to capabilities to identify/sort/search/share digital photos. Those are the next things to talk about.

(7/16) They Took My JPEGs! & won't give 'em back!

"They Took My Kodachrome!" (July 2, 2005)
"Don't take my Kodachrome away" - Paul Simon (1973)
"They took our jobs!" South Park (April 28, 2004)

I think my most useful experiments/thoughts/plans regard (mis-)handling of the metadata contained in JPEG files (to be picky, I should probably be referring to JFIF, not JPEG) and the opportunities for using the metadata and other information to identify/sort/search/share digital photos on the Internet.

But first things first. It makes no sense to discuss helping the typical point and shoot digital photographer (and those who would see their work) to do more with their photos if their photos are so likely to get lost. Of course, traditional photographs easily get lost, as well. They're picked up from the 1-hour photo processor, viewed once, and stuck in some "safe" place, never to be seen again.

Digital photos allow for better solutions to that scenario, but for now let's consider some familiar, not so good practices:

Depending on prints as primary storage. Though prevalent, this is not good for traditional photography or digital photos.
- With film-based photography, the negative or slide is a better version, probably stored in a better (no worse?) environment than print(s) and least likely to deteriorate. But these original media versions are most likely to be lost, if, for no other reason, because they are small.
- However, century old black and white prints seem to have survived pretty well. (See http://technologists.com/images/sm1890.jpg, for example.)
- Half-century old color prints are more likely to have degraded, but some are still pretty good. See http://technologists.com/images/sm1956.jpg for one example. There are lots of counter-examples of faded 20th century prints, but there are enough existing good quality mid-20th century prints to blame sunlight and other environmental conditions for the counter-examples.
- Let's stipulate that commercially generated prints from digital sources will last at least as well as last century's commercial prints.
- But what about the non-commercially generated prints from digital sources? Can anyone reliably predict whether/when they will degrade?
The above discussion intentionally ignores the art of creating prints from negatives/slides/digital files. That art is a gift in the right hands but, in my opinion, a disastrous overindulgence of almost all software for manipulating digital images. Using an enlarger skillfully to enhance a traditional print or Photoshop to do analogous things with digital photographs can be wonderful. But 80% of those who would use Photoshop either ignore the potential, or worse, misuse the abundant facilities.
Depending on the camera's memory or the computer hard disk for storage. These are probably the two most vulnerable places to save digital photos, but are likely the most prevalent. The plethora of problems, and opinions on solutions, might fill a tome. Also, safely preserving digital photo files on hard disk is a specific instance of the bigger issue of safely preserving files on hard disk, so I'll not say much more about this now.
Having just disclaimed the hard disk preservation topic, consider the most likely preservative practice: copying files from disk to writable CDs or DVDs. Depending on where the optical disks reside (sitting in the sun? locked in a safe-deposit box?), this may be a very good scenario. However, we do not have enough experience with the longevity of CDs and DVDs to really know. Will a newly created DVD deteriorate or not in the next decades?
There is a seemingly preferable but minimally realized scenario: uploading the JPEG (or another file format) to appropriate web sites and depending on those sites to "do the right things".

Unfortunately, with the exception of Flickr, none of the existing commercially oriented sites (of those that I have tried) come within shouting distance of my perspective of the "right things". Rather, the primary emphasis is on selling conventional prints. Providing conventional prints is a valuable service, but not near the top of my list. The biggest problem with all of the sites I have tried, including Flickr, is I see no way to get back the files the customer uploaded. Thus the title of this tidbit -- these sites will accept my JPEG files but won't give them back to me.

More on "doing the right things" in the future.

(7/2) They Took My Kodachrome! Medical Addendum

"Don't take my Kodachrome away" - Paul Simon (1973)
"They took our jobs!" South Park (April 28, 2004)

67 years after introducing the reference standard color film, Kodak discontinued Kodachrome 25 in 2002. (It is my understanding that the other two versions, Kodachrome 64 and 200, are also being discontinued.) When I was a child, first learning about photography and film, Kodachrome 25 was exotic. I mostly used black and white film and borrowed darkrooms because I could not afford color film or commercial processing. I never became more than an amateur, having neither artistic gifts nor great fascination with photographic technology. But I took lots of photos and accumulated lots more family pictures -- I maintain a family photos web site with about 1800 photos, some dating back to the 19th century.

To be honest, I've rarely used Kodachrome, mostly opting for Ektachrome (to enable indoor photos without flash) and print film (Kodachrome was/is primarily for slides). Though not really the end of traditional photography, the end of Kodachrome symbolizes the end of film-based photography, ignoring disposable cameras and (semi-)professional photographers. Low cost, automated digital cameras let the average person, the amateur photographer, replace a bulky camera, scattered negatives and disorganized collections of prints with pocket-sized and television-like alternatives. (It literally took years for me to collect, organize and digitize the prints, slides and negatives that are the basis for my family photos site.)

Digital photography has tremendous benefits:

Instant gratification. With most digital cameras, you can immediately view the photo, maybe even immediately print the photo, depending on where you are. Even pervasive 1-hour processing of film-based photography can't match that.
Zero-cost/minimal-effort to delete unsuccessful efforts. You don't have to tell the 1-hour processing clerk which prints you want.
Integration with television sets and computers. Most digital cameras come with cables that allow you to view your photos on your TV. All digital cameras are designed for easy transfer of photos to your personal computer. More and more, this can be a cable-free proposition -- remove the flash memory card from the camera and insert it in the computer.

There are many more currently realized benefits of digital photography, but what concerns/interests me more are the hazards of digital photography and the unrealized potential of digital photography.

One example of a hazard: catastrophic loss of photos seems more likely to me with common practice today than common practice with film-based photos. With film-based photos, one usually had both negatives and prints, often in separate rooms if not further apart, so it would take a major physical disaster, e.g., a house fire, to lose the photos. With digital photos, common practice is to transfer photos to a computer and delete from the camera. If the computer fails, because of infection, a disk failure, whatever, the only copy of the photo is lost. (There might be a printed copy, but even then, if made on a home printer, that print might deteriorate more rapidly and/or be of noticeably lower quality that commercial prints.)

One example of an unrealized benefit: digital images, computers and web sites allow for far more ways to identify, organize and present photos than traditional envelopes and albums of prints. But in practice, the software and web sites for digital photos seem mostly oriented toward selling prints of digital photos, doing little different than film-based photography in allowing sharing and viewing of photos.

With digital photos, it feels like we're where personal computers and/or the Internet were in the mid-1980s. The potential is thrilling, but the reality is not. As I've said before, I'm trying to figure out how to leverage what others have done with incremental improvements I might suggest.

Medical Addendum

First, I am delighted with my wife's recovery after her hip replacement. Just seven weeks and a day after her surgery, she is so much more mobile, and so relatively free of pain, that we are both in somewhat of a state of disbelief. But her recovery is real and seemingly faster than what we were lead to expect.

However, her experience and another family member's experience after multiple glaucoma surgeries have made me very conscious of the importance of following the doctor's orders after surgery. My wife was given three primary rules of things she was not supposed to do for the first six weeks after surgery. She was very careful to follow those rules, and her caution, along with a great surgeon and a very good physical therapist, are some of the main reasons she is doing so well now. The surgeon, his staff, and the written instructions she received all emphasized the importance of those three rules (which were formally termed "precautions").

In the case of the glaucoma surgery, perhaps the surgeon was not so emphatic in discussing the post-surgery precautions. In any case, the family member was not carefully following the restrictions and had a frightening setback. After a couple of weeks of realization that this was seemingly the primary problem, including a second opinion from a different surgeon, the restrictions are being followed. Yesterday, the surgeon said the eye is "well on its way to total recovery".

Bottom line, I omitted a critical issue in Navigating Modern Medicine: the importance of understanding and strictly following post-surgery directions from the surgeon. That is the patient's responsibility!

(5/23) Navigating Modern Medicine, Miscellany

"Well, Jane, it just goes to show you. It's always something. If it's not one thing, it's another." - Roseanne Rosannadanna

I've been spending hardly any time on the things I expected I would be a month ago. But I am not about to complain. Rather, I thank God for the magic of modern medical technology, and my late mother, a nursing professor, for preparing me so well to cope with the complexity of modern medical practice. (Besides thanking my mother for teaching me so much about nursing, I have to thank her and my father, who is nearing 95-years-old, for giving me a wonderful sister who became an M.D. and my most trusted medical advisor.)

I'm going to mostly focus on my wife's condition and treatment, comment on the challenges of being a patient and a caretaker where we live. There will be a little discussion of computer-oriented things, but expect this to be mostly different from what I usually write about.

My wife developed "avascular necrosis" in her right hip, the same condition that ended Bo Jackson's football career. With an artificial hip and rehabilitation, Jackson was able to resume playing major league baseball. That was almost 15 years ago. Hip replacement has progressed so far since then. It is one of the two surgeries with the highest rate of patient satisfaction, the other being cataract removal. (I'm basing this on what an anesthesiologist told us. I assume he both knew what he was talking about and had no reason to be biased.) We originally scheduled surgery for May 24. The surgeon asked that we move the date up to May 13. Given how much Caroline was suffering, I would have voted for an earlier date.

Caroline's orthopedic surgeon is probably the best in Austin. The hospital, next door to his office, is probably the best in Austin. Yet, I tremble when I think about those patients who don't have the kind of facilitation I was able to provide, and regret the one night I chose to sleep at home instead of in her room.

The 24x7 effectiveness of a hospital is almost entirely dependent on the nursing staff. We encountered mostly excellent, dedicated nurses, some not quite so good, and a few that needed discipline and re-education. I knew beforehand that good nurses are scarce and overworked. But I didn't know that viscerally until the weekend Caroline spent in the hospital. I didn't have a clue about the degree of overwork.

Which gets back to me as a facilitator. I was able to remind the nurses of things that had been forgotten or delayed. I was able to handle some of the tasks that were really the nurses' responsibility. I was able to talk to the nurses in the terminology and abbreviations they are used to. And, except for the one night I slept at home, I was able to prevent the less than excellent nurses from making mistakes, large and small. This gets me back to thanking my mother and my sister. I'm very good at teaching myself, but without the basic training from my mother and the counsel from my sister, I could not have learned what I needed to learn, could not have done what needed to be done.

The surgeon recommended that Caroline's rehabilitation be at home (vs. a much longer hospital stay). Since she hates being in the hospital, since hospitals are full of germs, and since the surgeon believed we could succeed with home rehabilitation, the decision was easy. (In general, everything the surgeon and his assistants have said has so far proved to be correct, so my attitude has been to trust him entirely.) Caroline has been home for a week, as of today, and all seems to be going as we were led to expect.

There are lots of computer oriented things I could talk about from the hospital experience, but I'll pick one piece of "low hanging fruit". One of the devices used for post-surgical patients is a PCA ("Patient Controlled Analgesic") which either continuously or on patient push of a button introduces intravenous narcotic. The PCA Caroline had was a very sophisticated device, but totally baffling to almost all the nurses, even the most technologically sophisticated nurses. By the time it was removed, I had figured out how to understand its display, and I knew what it was supposed to be doing, but I certainly would not have wanted to be responsible for programming the PCA.

A couple of unrelated tidbits:

Yesterday, I upgraded my VAIO to Windows Media Center 2005. So far, I haven't noticed much that is different. However, I was dismayed at the number of reboots it took to get the upgrade accomplished, the number of personalized settings that had to be reset, etc.
My Uncle Hugh, who will turn 90 later this year, has just started using email and web browsers, under his daughter's tutelage. So he is now the oldest recipient of this distribution.

Back to digital photography, I have been able to use digital photographs of Caroline's incision to show medical professionals how the incision is healing, not infected, etc., without Caroline having to go through the discomfort of those professionals removing/replacing the dressing.

(4/23) Digital Photos, PHP, FC3, Dead Fans

"Come together right now over me" - John Lennon

I wouldn't dream of comparing my writing to Jean-Paul Sartre, but this may seem like stream of consciousness, so please bear with me. I hope it will all come together by the bottom of the page.

This week it seems to me that there's nothing like being a first time grandfather (April Rose was born this past Monday) to make one conscious of digital cameras, the huge benefits of digital cameras (see April) but also the unnecessary discrepancies between different digital cameras and deficiencies in the associated software. This reinforces my motivation to document the benefits and problems of digital photography and to pursue software to make things better. I expect I'll have lots more to say about this in the future, based on thoughts and draft documents I'm not ready to reveal.

I've been looking at all of the (free, bundled and/or affordable) photo software I can get my hands on, and have lots of opinions. Bottom line, I don't think any of the software is close to getting things "right", though some is much closer than other software. In the process, I've discovered that PHP probably has better primitives for dealing with the problems than any other web environment. Since PHP has so many other advantages and advocates, it was easy for me to conclude that anything besides the two heavyweight contenders (from Microsoft and Sun) could not compete with PHP.

Unfortunately, my own Apache modifications had broken PHP on my own Linux servers. I realized quickly that there was no inherent conflict, just my naive approach of entirely rebuilding Apache to add mod_auth++. So I resolved that quickly on my test servers.

That brought me back to whether I was going to upgrade my production Linux server to Fedora Core 3, now that FC2 is "legacy". The only plausible answer was "yes", but when? And what do I do about the unnecessarily manual process, I'd go through to configure Fedora after install?

To exacerbate things, the processor fan in my Linux server developed bad bearings. As much as I like the "medium desktop" Dell Optiplex chassis design and all of the improvements and variations that have appeared over the last decade, my production server was early vintage and it looked like the fan was one of the hardest components to replace. That server normally sits in a minimally temperature controlled closet with a newer, fully loaded Optiplex running Windows 2000 Server and a well-loaded Mac G4. Between the three of them, they generate lots of heat, so I knew I had to make hardware changes before summer.

Fortunately, I have a half-dozen of the right vintage Optiplex desktops, so I could swap hardware easily. I developed a collection of useful scripts for configuring Fedora after install. So all that was left was to be brave and put in place the things I'd been contemplating/prototyping. I did that today. So far, so good. So now I can get back to digital photography software.

(2/28) (disc)centricity: Solaris X and Fedora in a Windows world

"It was 20 years ago today
Sgt. Pepper taught the band to play." - Lennon/McCartney

I wrote before about my frustrations and concerns with Fedora and my intentions to explore alternatives, especially Solaris X. In my recent explorations, I've puzzled about what Sun has released and wondered how serious they are about Solaris on X86. Having been involved with Solaris on X86 since the very beginnings, I would be delighted to enjoy Solaris on X86 and see it have some success. (In 1991 or '92, before Sun said anything publicly about Solaris on X86, four Sun executives paid me a surprise visit at Dell to talk about collaboration on putting Solaris on X86.)

Before starting to install Solaris X on my favorite test machine (an older Dell Optiplex with a 733MHz PIII), I installed larger disks to have plenty of space for Solaris, multiple Fedora releases, and Windows 2003 Server. Suddenly, I seemed mired in all of the arcane details of disk partitioning that trace back to the 1983 introduction of the IBM PC/XT with its "large" 10-megabyte disk. (The disk really was large, both in capacity, and in physical size, at that time.) Today, with disk drive capacities thousands of times larger in much smaller physical packaging, the "PC architecture" still reflects decisions made back then.

Much has been written about the shortsighted memory parameters ("who will ever need more than 640KB of memory") of the original IBM PC. By the mid-80s, the PC world was struggling even more seriously with the limitations of 16 bit addressing, just as the PC world is beginning to struggle with the limitations of 32 bit addressing today.

Disk capacities have increased roughly comparably with physical memory, but there has seemingly been no de facto standard for extending the disk partitioning parameters. Until recently, I naively assumed that Microsoft was/is able to set the de facto standard. Sun (Solaris) and Red Hat (Fedora) seem to disagree. Worse, Sun and Red Hat seem to have changed their own partitioning assumptions between Solaris 9 and Solaris X, and between Fedora Core 2 and 3. (Fortunately, Windows seems to accept any of these partitioning setups.)

On a disk with Microsoft established partitioning and Windows 2003 Server, I installed Fedora Core 2. Then I tried to install Solaris X. (On this same machine, with a smaller disk, I'd previously had Windows NT4 Server, Solaris 9, Free BSD and FC2 all on the one smaller disk.) Solaris X told me the disk partitioning was invalid and that if I wanted to install Solaris X, I'd have to re-initialize the partition table, and, in doing so, delete everything on the disk. Grumble. Before accepting that, while ruminating about workarounds, I tried Fedora Core 3. It told me essentially the same thing, that it would have to re-initialize the partition table!

I'll skip most of the subsequent arcane tribulations I experienced. I was able to create a partition table acceptable to FC2 and Solaris X (and Windows). Then the Solaris X install told me it would not allow Linux on the same disk (machine?) and it would delete the Linux partition before I could proceed! This made me think back to mid-80s battling amongst the vendors of all the different versions of Unix. SCO dominated market share on PC hardware and Sun dominated market share on workstations. There were lots of other Unix versions in the mix. Sun, and to a lesser extent, SCO, seemed unaware of how the rapidly increasing dominance of Microsoft, Novell and Apple products would leave little room for even one version of Unix. That the battles went on between the different versions of Unix, especially since there were so many arbitrary and unnecessary incompatibilities between the versions, made it impossible for any of them to thrive. Today, it seems those lessons have been forgotten. Only to the extent that the various Linux distributions remain more or less compatible with each other, then Linux on server-like machines seems a realistic alternative to Windows.

I managed to get Solaris X, FC2, FC3, and Windows Server 2003 barely coexisting on the same machine, using three disks. The Solaris graphical user interface wouldn't start because Solaris X installation had mis-configured the "Xorg" X-windows server instead of the "Xsun" server. (Apparently, this is a common experience, based on my search for a solution. The solution buried in http://forum.sun.com/thread.jspa?threadID=22723&messageID=73851 worked for me.) Solaris needs to be obviously better than Linux to even be in the competition (while hoping that Apple doesn't release their Unix, OS X, on PC hardware). So far, Solaris X has frustrated me more than it has engaged me.

I'll probably gradually learn more about Solaris X, but not now. Paying attention to Linux and Windows (and OS X) seems much more valuable. Red Hat continues to indicate better attention to Fedora. I've figured out workarounds for all of the problems I'd been having with Fedora Core 3 and may even use it on my production Linux server soon.

(2/3) Closing Out 2004, Planning 2005 Research

Let's see, I could start on income taxes. No, I still haven't received a couple of 1099s. Whew! It's not much more fun to admit that many of the things, I've tried to work on the last year or so have led me to disappointing technical conclusions. "You can't always get what you want ... you get what you need". (That could be attributed to Mick Jagger/Keith Richard, but I'd rather think of the Biblical basis.)

Anyway, this is intended to be a brief recap on various technical topics, approximately in order of most disappointing first, and a glimmer of where I hope research will take me this year.

Replacing/Preserving NT4 Server

There are "lots" of organizations still using NT4 Server, even though it has reached "end-of-life" in Microsoft's perspective. Since I manage a few of these servers myself, and the owners cannot easily migrate to Windows 2003 Server, as Microsoft would want, I'd hoped to come up with strategies that are viable for either staying with NT4 or switching to Samba. However, my reluctant conclusion is that neither of these are good solutions:

The biggest problem I see is that Windows XP clients running SP2 do not "play well" with NT4 domains, in my experience. I assume this is a problem for Samba, as well, but have not bothered to test. To the extent XP SP2 works poorly in NT4 and Samba domains, this is a showstopper for NT4 and Samba, in my opinion.
Though NT4 servers may be adequately protected from external intrusion by independent firewalls, they are still vulnerable to intrusions from local machines (unless the NT4 servers have their own firewalls).
The big security scourge has become PUS (potentially unwanted software, in Microsoft terminology), more commonly known as "spyware" or worse. This makes it increasingly hazardous to use Internet Explorer on an NT4 console.
Though I use Samba casually, it doesn't (yet) seem ready for production use in a network dominated by Windows clients.

The NT4 Server machine I had here was replaced with one running Windows 2000 Server. (The NT4 machine was the one decimated by lightning.)

LDAP For General Use

Unfortunately, I have no better report than that of my "LDAP Angst". The more I learned, the more OpenLDAP seemed incomplete, and the more Samba seemed incomplete, due to dependence on LDAP. But I'll continue to learn more about both, hoping that new releases will bring both closer to being complete.

Fedora

Ambiguous, conflicting messages are coming from Red Hat regarding Fedora. http://fedora.redhat.com/ seems to indicate that all is going according to plan. However, news reports quoting Red Hat sources admit "mistakes" and suggest changes are coming.

I hope so. I've seemingly wasted much time and had allowed myself to get very frustrated with Fedora Core 3. After extensive testing on several other machines, I tried to switch my production Linux machine to FC3. I could never even get it to boot after the install! I submitted detailed bug reports with Bugzilla. As far as I can tell, those reports were never even read. Further, even new FC2 kernel updates seem to have significant problems. They "panic" in the disk driver at boot time on my best test machine. (That machine happily runs NT4, Solaris 9, FreeBSD, ... as well as older FC2/FC3 kernels.)

For most purposes, I'm backing away from new Fedora releases until I see what changes, if any, are made in the overall Fedora strategy. My production Linux server should be happy with FC2 for the foreseeable future, even though FC2 is going to "Legacy" status soon. However, I have some plans for new FC3/FC4 experiments. I'll also be looking at alternate Linux distributions and Solaris X.

Secure Wireless; Spam

I don't think I've written anything about these topics in a long time, basically because I think I have good solutions in hand. Between WPA, SSL for picking up e-mail, SSH for sending e-mail, and other usage of SSL and SSH, I think pretty much everything I do using wireless connections is encrypted at least at one level if not multiple levels. Spam continues to be an annoyance, but my simplistic solutions still seem to keep things under control.

Looking Forward

So what next? SBC's intent to purchase AT&T and discovery of an ancestral tie to the Wright brothers have made me ponder the formerly dominant commercial research labs in the U.S. a few decades ago. The main three I think of are

AT&T Bell Labs, the birthplace of the transistor and Unix. (Of course, Bell Labs is part of Lucent, not AT&T, these days.)
IBM Thomas J. Watson Research Center, the birthplace of RISC processors and my first employer after graduate school. (Like the Wright brothers, Thomas J. Watson, Sr. was from Dayton, OH.)
Xerox PARC, birthplace of Ethernet and graphical user interfaces.

Perhaps one or more of these will spring forth with more breakthroughs and/or modern analogs, such as Microsoft Research, will do likewise. But none of these currently have the cachet that say, Bell Labs, once had.

One of the bureaucratic, yet effective, procedures at T.J. Watson used to be annual production of "Research Orders" that documented what a group had accomplished and why it should continue to be funded. Sort of like a grant proposal, except that it is easier to justify continuing successful efforts than to compete for external funds.

I'm thinking I should at least sketch out something like a research order or a grant proposal for the things I want to work on. I think I am on the verge of a one paragraph introduction, which might be something like:

"Computer-based photographs, from digital cameras and scanned conventional media, have become pervasive. Major companies, notably Google, have produced a variety of (free) software and services (e.g., Blogger.com, Hello, and Picasa) to facilitate Internet communication and sharing of photographs. Yet, these have seen miniscule use, in comparison to email, web browsing and other more established Internet capabilities. This research will identify barriers to broader acceptance and attempt to overcome these barriers."

[koko] tales of sensory power in today’s world
November 26, 2024
USA choice: self-obsession or beacon of hope?
November 4, 2024
always a technician – thanks to Mom & Uncle Clint
July 8, 2024
[koko] rarely one to avoid controversy…
May 28, 2024
[koko] knowing and accepting limitations
February 6, 2024
[koko] keeping warm
August 7, 2023
[koko] still learning
June 18, 2023
Roe is gone, one more round
June 28, 2022
“just as good as Caruso” – props for Kim Wilson & Charlie McCoy
May 5, 2022
Mel West, engaging people to help people in Nicaragua
April 25, 2022
Glimpses from the Vulcan, 1969-70
February 14, 2022
[koko] MISP 2022
Janary 10, 2022
Why I continue to serve — I remember Nicaragua
December 13, 2021
Making private 1960s and 70s recordings public
August 21, 2021
Jimmie Vaughan set w/ Storm track I recorded
August 4, 2021
Celebrate Ramblin' Jack Elliott's ~~90th~~ ~~91st~~ ~~92nd~~ 93rd birthday!
August 1, 2024
[koko] LP digitizing milestone approaching
May 18, 2021
remembering Denny Freeman
April 28, 2021
[koko] Dell Unix sustainable!
January 19, 2021
Computer Systems Performance Modeling
August 25, 2020
Remembering RESQ
August 25, 2020
[koko] (welcome to …) eight Jurassic O.S. on 1992 Dell 486D/50
September 26, 2019
[koko] reviving timbl's WorldWideWeb browser
July 1, 2019
[koko] exploring NEXTSTEP 486
July 1, 2019
1992 JAWS demo for Stewart Cheifet
May 17, 2019
Let's start at the very beginning... 801, ROMP, RT/PC, AIX versions
March 8, 2017
NeXT, give Steve a little credit for the Web
October 8, 2011
Mainstream Videoconferencing available again
February 14, 2008
A brief history of Dell UNIX
January 10, 2008