Sunday, October 27, 2013

Retrocomputing with SCO OpenServer 5.0.5


It seems like everybody hates SCO these days. But, there was a time... Way back, when they did good work, or at least work that enabled me to do good work. My understanding of SCO's history is a little fuzzy, but as I remember it, they spun off of Microsoft when Microsoft decided not to pursue Xenix. Either that or they bought Xenix from Microsoft. Something like that. Then they went on to develop a full fledged Unix for x86 and owned that market until Linux matured. In the mid-to-late 90's, Caldera began to challenge them with their Linux offering. SCO's technology lagged but they had a huge customer base. Eventually SCO proper got out of the OS business, changed their name to Tarantella and went on to continue development and support of their Tarantella platform. Caldera basically bought their customer base and ended up with their OS development business in the deal. Then leadership changed at Caldera. Caldera renamed themselves The SCO Group to capitalize on the name equity and began suing people left and right. That didn't go so well for them and I'm not exactly sure what happened next. They either renamed themselves Xinuos or Xinuos bought them, or something.

At any rate, before all of the politics, when I was a kid, my Dad used to run companies off of a PC, SCO Xenix (and later Unix), a Computone Intelliport (or later Intelliserver), and several dozen Wyse terminals. When I grew up, I wrote more software for those systems than I can even begin to remember.

Those were the days.

For a while, SCO Unix was pretty awesome too. When OpenServer came out in the early 90's, it had shared object libraries, a journaling filesystem, a Motif-based graphical desktop, graphical system administration tools, drivers written by the hardware vendors themselves and back-compatibility all the way to Xenix 286. Linux was still using coff binaries, libc4, ext2, athena widgets and you had to be really careful what hardware you tried to run it on. For a while, you could buy a single-user development version of OpenServer for like $100 too. I continued to use SCO, basically until the Linux distros started including ext3 and Gnome.

Between development versions and swag from old companies, my Dad and I ended up with several versions of SCO Unix and Xenix and software that ran on them. We used them for a while but for years now, the CD's and license keys have been sitting in boxes, collecting dust. He recently dug through all that though and came up with a copy of OpenServer 5.0.5, circa 1999.

5.0.4 and 5.0.5 were both very popular versions of OpenServer. Earlier versions were buggy and though later versions added DHCP and USB support, those features just weren't all that important on the server. Later versions suffered from competition with UnixWare, Solaris x86 and Linux too, and from general anti-SCO sentiment.


The first challenge with any of these older OS'es is finding an emulator that will actually boot them. Fortunately SCO OSR 5 is semi-supported by VMware. For a while, it was even one of the OS'es you could select from the OS list during installation.

I used an 8g disk and 128m of RAM, which I believe were the defaults and selected Other from the OS list.

The CD wasn't bootable though. Instead I had to mount it, find the boot disk image at images/boot/install.img and copy it out. The bootloader loaded fine but there is a trick to getting it the installer to actually run. I'm not sure where I found the trick, but I had it in some notes from the last OSR 5.0.5 install I did, umpteen years ago. Basically it needs a contiguous block of at least 32M of RAM, probably to load a ramdisk into.

The magic bootstring:

defbootstr mem=1m-64m

That actually allocates 63m of contiguous RAM. I might have gotten away with a 1m-33m parameter but 1m-64m was in my notes, so I used that.

OSR 5.0.5 - 1 boot

Apparently (according to my old notes), 8g drives are supported by default but if you want to use a drive larger than 8g then you have to add a biosgeom=(cylinders,heads,sectors) flag as well. cylinders=(#ofgig*1024*1024/63/255*2)-5. So, for a 30g disk, (30*1024*1024/63/255*2)-5=3911. You'd use:

defbootstr mem=1m-64m biosgeom=(3911,255,63)

Ha, I actually remember having to upgrade the bios in many of my old computers to support disks larger than 8g.

Anyway, after supplying the proper boot string, the installer started up.

I always thought the installer was attractive, as text-based installers go anyway.

It is a little quirky. You use arrows to move around, enter to accept an options and space to change an option. The cdrom drive isn't auto-detected, you have to specify whether it's on the primary or secondary bus and whether it's the master or slave. The "System name" prompt wants the host name. You can mostly accept the defaults.

One thing I always did though is turn off Bad Tracking, otherwise the installer will scan the drive and look for bad sectors. This takes eons and isn't necessary. Another quirk in the installer though... After turning Bad Tracking off, the installer still reports that its still on. It's not. There's just a bug in the installer.

OpenServer 5.0.5 doesn't support DHCP so you have to give it an IP address. It does support the emulated PCNet card though.

For video and graphics, use VESA SVGA (the default).

Oddly, the mouse is not configured by default and must be set to "Low Resolution Keyboard Mouse".

After all that, and otherwise accepting defaults, the installer will run unattended.

OSR 5.0.5 - 2 install

And when it's done, after a reboot, you get a fairly attractive graphical login.

OSR 5.0.5 - 3 first login

At one of the companies I used to work for, we named our servers after scientists. We'd swap that SCO logo for a photo of them and set the background to something appropriate as well. It was loads of fun.

Logging in revealed the Motif-based desktop.

OSR 5.0.5 - 4 desktop

State of the art for when it was released in 1995. Actually, I that's when I first saw it, in OpenServer 5.0.0. SCO released their OpenDesktop product back in 1990. It's possible that it came with the same desktop. I'm not sure.

A Post-Install Tweaks

Post-installation, as usual, I had to tweak a few things.

First, I created a user for myself with:

useradd -m

Next, I disabled the graphical login and I wouldn't be needing it in the future.

scologin stop scologin disable

Then I finished the network setup. The installer prompts for an IP address, netmask, hostname and domain name but fails to prompt for a gateway and DNS information.

Setting up DNS is straightforward. Create /etc/resolv.conf with something like:

domain nameserver

Adding a gateway is less straightforward. I remember from setting up 5.0.2 (Internet FastStart) that there is some file, somewhere in the system, that if you put the IP address of the gateway in it, it will automatically be set at boot. It's not in an intuitive place though and I had an impossible time finding it. For all I know, it only worked on 5.0.2. What we always did was edit /etc/tcp and look for a line like:

/etc/route add 0 > /dev/null 2>&1

Then add a line after it like:

/etc/route add default > /dev/null 2>&1

So that's what I did. After a reboot, I had networking.

I also fixed the man page system.

A little digressive background on the OpenServer man page system...

In OpenServer, man pages are actually served by a small http server called scohttpd. In theory, this is a really cool idea.

First, you can read the man pages by pointing a browser at port 457 and drilling down. However, if you want to crash a modern browser, go ahead and try that. The man pages are compressed with the "compress" format (which predates gzip) and apparently modern browsers think they can handle that and then die trying.

Second, you could install all the man pages on a "man server" and point the other systems at it using the MANPORT and MANSERVER environment variables rather than having to install man pages on every system in the network. In the late 90's this would have saved an appreciable amount of disk space. SCO software packages are even structured to make this possible. SCO never promoted doing this though, for some reason, and sadly, the feature went largely unused.

The man system has a bug though. The scohttpd server runs early in the boot process and some time later in the boot process, the socket files that it uses get deleted.

A quick fix is to edit /etc/rc and append:

/etc/scohttp stop /etc/scohttp start

...and reboot.

OSR 5.0.5 - 5 text login

Woohoo! Text login and working man pages.

Next, I installed the SCO OpenServer Development System and SCO Optimizing C Compiler from the distribution CD.

SCO's package manager is an interactive program called "custom". As in, to "customize" your system.

It, along with the rest of the system-administration tools, is written in TCL and, depending on whether you're running in a graphical environment or not, displays a graphical or text-based UI. The UI's look very similar and function identically. I was always impresssed with that technology.

To install the dev tools, I ran custom and selected Software->Install New->From osr505 (the name of my system)->CD-ROM Drive 0. I unselected Microsoft Client Software, selected SCO OpenServer Development System and SCO Optimizing C Compiler and selected Install.

If I may digress again...

Another very cool feature of OpenServer is the "From osr505" option that I mentioned above. CD-ROM drives in the late 90's were still really, really slow. If you had a software package installed on another system in the network, instead of installing from a CD on the local machine, you could tell custom to install it from another machine, across the network. The packages were structured to keep the original copy of config files too, so if you did a network install from another machine, you wouldn't get a modified version of the config files. With fast internet, apt-get, yum and friends, this approach is obsolete these days, but in the 90's, it was awesome.

Every OpenServer release was followed up with a bunch of patches and ultimately a release supplement roll up. Sometimes there were several consecutive Release Supplements. Patches and Release Supplements are all listed at and available directly at I installed RS505A, which patched the OS and the Development System.

SCO packages are quirky. They are typically distrbuted as a tar or cpio file containing a set of files named VOL.000.000, VOL.001.000, VOL.002.000, etc. which are themselves, cpio files containing the software. To install them, you must extract the file, tell custom to install "Media Images" and point it at the directory containing the VOL files. I think there is a non-interactive way to use custom, but I never figured it out. I don't think there's any way to get it to dig into a tar or cpio file though. So, package installation has always been a manually intensive process.

I didn't at this point, but if you attempt an OpenServer installation, I recommend installing the oss646b supplement as well, to save yourself lots of headaches later. Ironically, it's missing from the list of supplements, but it's absolutely critical if you want to use gcc rather than the native C compiler. It's available at

Remote Access

Telnet and ftp are the only built-in way to access an OSR 5.0.5 machine but ssh is available via "Skunkware".

Over the years, SCO downloaded, patched, compiled and made installable packages for a bunch of open source software. They called this project Skunkware, an allusion to Lockheed Skunkworks, I guess. These packages are available at and they put out "Skunkware" CD's too. Apparently they'd been doing this since the Xenix days, but I'm not sure how they distributed the software back then.

Skunkware packages for OSR5 in particular are available at The openssh package is among them.

Actually, I had to install zlib, prngd and openssh. I also had to tweak /etc/init.d/RMTMPFILES and add:

rm -f /usr/local/var/prngd/prngd.lock

Otherwise prngd would leave a lock file lying around and wouldn't restart after a reboot.

I tried to get X11 forwarding working. In /usr/local/etc/sshd_config, I uncommented and set:

X11Forwarding yes

I eventually discovered that the path to xauth was hardcoded into the program too, at /usr/X11R6/bin, which doesn't exist on OSR5 as OSR5's X Windows is based on X11R5, not X11R6.

I made the appropriate directories and added a symlink:

mkdir /usr/X11R6
mkdir /usr/X11R6/bin
cd /usr/X11R6/bin
ln -s /opt/K/SCO/XServer/5.2.2a/usr/bin/X11/xauth xauth

Unfortunately, X11 forwarding still failed with "X11 connection rejected because of wrong authentication." I still haven't figured it out. It's possible that xauth from X11R5 just isn't compatible with modern X. X11R6 is available for OSR5. Maybe I'll try that some day.

What's up with that crazy path to xauth?

Another fairly innovative thing that SCO did with the OpenServer series is install literally everything under /opt and /var/opt and create symlinks from the files there into /bin, /lib, /usr/bin, /usr/lib, /etc and so forth.

Internally, their package manager would just extract a new package into /opt and /var/opt and run the "fixmog" program to update the links.

Patch management was easy for the installer too. The patch was installed under /opt and /var/opt and making the patch available just involved updating the links. Rolling the patch back was simple too. Remove the patch, update the links.

The real power of this approach though, SCO never promoted. You could install software on one machine, NFS-export the directories for that package under /opt and /var/opt, mount them on another machine and run fixmog on that machine. The software was then immediately available on the other machine without having to fiddle with PATH's or LD_LIBRARY_PATH's. We did this a lot. It was quirky though, in that the packages showed in custom as having been installed locally, and if you didn't know they were NFS-distributed and deleted one, it would delete it from the app server. Maybe I can see why SCO never promoted NFS-exporting their software.

The whole links-everywhere thing seemed like a good idea, but it turned out to be confusing as hell. The output of "ls -l" had a gazillion symlinks to long paths and looked terribly cluttered. Just running "l" appended a @ to the filename if it was a link, which was also confusing. I think there were issues with scripts that ran ls -l or l as well. In the end, the dpkg/rpm approach was superior.

But I digress...

With OpenSSH installed, I could ssh into the VM. Yay, no more working on the console.

I could also scp and sftp.


Sadly, OSR505 comes with compress/uncompress but doesn't come with gzip, bzip2, zip or unzip and its tar program lacks the ability to uncompress on the fly. Fortunately GNU tar and the rest of those programs are available via Skunkware as well.


OSR505 comes with Netscape Communicator 4.05 which had a laughably difficult time rendering anything. I was able to install lynx and wget from Skunkware. Maybe someday I'll work on getting a semi-modern graphical browser working.

On the server side, 5.0.5 comes with the Netscape FastTrack Server. I bet you didn't even know Netscape made a web server. They did. I used it for years and years.

If you run a "ps -efa | grep http" on a running OSR505 system, it will reveal a confusing array of web servers, actually:

root 544 1 0 Oct-18 ? 00:00:00 /var/scohttp/scohttpd -d /var/scohttp
nouser 467 466 0 Oct-18 ? 00:00:00 /usr/internet/etc/ncsa_httpd -d /usr/internet/admin
root 411 1 0 Oct-18 ? 00:00:00 /usr/internet/ns_httpd/admserv/ns-admin -d /usr/internet/ns_httpd/admserv
root 413 411 0 Oct-18 ? 00:00:00 /usr/internet/ns_httpd/admserv/ns-admin -d /usr/internet/ns_httpd/admserv
root 415 413 0 Oct-18 ? 00:00:00 /usr/internet/ns_httpd/admserv/ns-admin -d /usr/internet/ns_httpd/admserv
nouser 420 1 0 Oct-18 ? 00:00:00 ./ns-httpd -d /usr/internet/ns_httpd/httpd-80/config
root 466 1 0 Oct-18 ? 00:00:00 /usr/internet/etc/ncsa_httpd -d /usr/internet/admin
nouser 421 420 0 Oct-18 ? 00:00:00 ./ns-httpd -d /usr/internet/ns_httpd/httpd-80/config

What the heck?

The ns-httpd processes are the Netscape FastTrack server - the web-server proper. The scohttpd server is for serving man pages. The ns-admin servers run a web-based configuration interface for the Netscape FastTrack server. The ncsa_httpd process runs a web-based configuration interface for configuring networking and other things on the local system (very quirky). If you enable secure httpd on the FastTrack server, a second set of FastTrack servers will start up and a second set of servers will start up for administering it. Gazillions of web servers on an OSR5 system. They used them for everything.

I aimed a browser at the various servers to verify that they were working but I didn't play with them other than that. Maybe later.


Yes. This is what I'm always really up to.

The native compiler had a hilariously difficult time making sense of my software. Maybe one day I'll tweak it to work with the native toolchain, but I figured I'd have an easier time with gcc.

In retrospect, it might have been easier to modify my software.

Several different versions of gcc are available via Skunkware and one via a "GNU Tools Supplement" available at None of them worked. This seemed strange because I know confidently that I used gcc for years on various OSR 5.0.X platforms.

It took hours and hours of installing, uninstalling, patching and rolling back to figure out how to get everything working but here's a summary of the challenges and solutions:

  • The linker ld, as shipped by default and updated by RS505A doesn't support -static or -R options but libtool insists that it does.
    • The -static issue expresses itself with lots of "relocations remaining" errors at link time.
    • Actually ld does support -R but it means something totally different than libtool thinks it does, it's not the rpath.
  • oss499a adds -static support (or somehow works around it) but occasionally "relocations remaining" errors persist.
  • oss646b fixes all of these issues but causes the Skunkware gcc's to error out at link time with a _fini error.
  • The gcc from gnutools fixes the _fini error.
  • So, you have to use oss646b and gcc from gnutools, there's no other reliable option.

I didn't install all of gnutools, only GCC. Some of the other packages are duplicates (or newer versions of) tools I'd already installed and some of them require a different package called gwxlibs. In fact, during intallation it warns about:

Some of the selected software depends on other software that is not currently installed on your system.

...but it's safe to ignore the warning if you're just installing gcc.

I also installed perl, python, php4, pth (a portable pthreads package required by python), readline, bash, cvs and gmake from Skunkware. I'd swear that Java 1.1.3 was once available for OSR5 too, and there are even allusions to this on the SCO site but I can't find it anywhere anymore.

I tried installing openssl and postgresql from Skunkware but as it turns out, openssl only provides static libraries and postgresql requires the FSU-pthreads package which causes relocation errors at runtime.

I could probably build openssl from source and maybe postgresql too, against pth, but I'll save that for another day.

After all that, and a few tweaks to my code, I was eventually able to build and install everything. The SQL Relay installation was a little anticlimactic because I didn't have any database software installed to run the server against, but the client ran fine.

$ uname -a
SCO_SV osr505 3.2 5.0.5 i386
$ sqlrsh -host fedora19 -port 9000 -user test -password test
SQLRShell - Version 0.53
Connected to: fedora19:9000 as test

type help; for help.

0> select banner from sys.v_$version;
Oracle Database 12c Enterprise Edition Release - 64bit Production
PL/SQL Release - Production
CORE Production
TNS for Linux: Version - Production
NLSRTL Version - Production
Rows Returned : 5
Fields Returned : 5
System time : 10000


Ironically I originally developed the precursor to SQL Relay on an SCO OpenServer system to run against Oracle 7. These days I don't have a copy of Oracle 7 and even if I did, I long removed support for it. OpenServer has been relegated to client-only status.

Oh well, times change.


Aside from the quirks I mentioned above, I ran into a slew of other oddities while fiddling with OSR505. Some can be attributed to its age, others to SCO's commitment to back-compatibility. Others, I'm just not sure about.

  • At boot there are options for entering single-user mode and setting the date/time. There are timeouts for both prompts. However, hitting enter at the initial boot prompt disables the timeouts and makes it sit on the prompt to enter single-user mode forever.
  • On the command line, hit delete instead of ctrl-C.
  • Home directories are created under /usr.
  • Root's home directory is /.
  • There is no "which" command.
  • PATH is set in /etc/default/login for console logins but ssh appears to hardcode its own PATH before running .profile.
  • /usr/bin/X11 isn't in the PATH by default.
  • ./ is in the PATH.
  • The shell doesn't know about ~.
  • The package manager automatically selects the first package. So, if you want to uninstall something, you have to make sure to unselect the first package or it will get uninstalled too.

And, the quirk that confounds automated remote scripting over ssh:

The .profile runs a tset command that guesses your terminal type and then prompts you for whether it was correct or not. This makes ssh'ing remote commands a pain because you have to hit return manually to accept the terminal type. Without this command, the TERM type appears to generally get set correctly. Just comment it out in the .profile:

#eval `tset -m scoansi:${TERM:-scoansi} -m :\?${TERM:-scoansi} -r -s -Q`


I had a lot of fun installing OSR505 and it was a nice walk down memory lane, but I haven't yet gotten to do everything I wanted to.

It would be cool to:

Get the webserver working and serving CGI's.

Find Java and get my java code working with it.

Get xauth working, maybe by installing X11R6.

Get postgresql working, maybe by building from source.

Get my software to compile with the native toolchain and optimizing compiler.

SCO supported IPX/SPX and NetBEUI, had an Advanced File and Print Server that predated SAMBA and had a Volution Messaging server that was compatible with older versions of Exchange Server. I think all of that is on the install CD. It'd be neat to get all of that working.

Earlier versions of OSR5 came with Wabi and Merge. Wabi means Windows ABI and was an environment similar to Wine that allowed you to run Windows 3.1 apps on SCO. It even came with a graphical desktop similar to 3.1. It was dog slow in the 90's but might run OK now if I can find it. Merge was the predecessor to Win4Lin. Basically Win4Lin was a port of Merge to Linux. The Merge that came with OSR only supported Windows 3.1 though, I think. It'd be cool to locate that software and play with it.

My Dad has floppies for Microsoft Basic for Xenix, Microsoft Word for Xenix, Foxplus and maybe even Foxpro for Unix. It'd be fun to intall those and play around with them. We wrote so much code in Basic and Fox, you can't begin to imagine.

Get Oracle 7 working again. We once had the installation CD's for 7.4.3 and 7.4.4. I might be able to dig them up.

Ahhh, some day... Some day.