Sunday, December 28, 2014

SQL Relay 0.58 is out

SQL Relay version 0.58 is now available!

Windows Support

The most significant feature of this release is preliminary support for the SQL Relay server on Windows.  The client API's and command line programs have long supported Windows, but in this release, it's possible to run the SQL Relay server on Windows as well.

This is a preliminary release,it currently only supports Oracle, and there are some quirks, but it's a good first step.

Running the SQL Relay server on Windows is much the same as on Linux or Unix, just run sqlr-start as described in the documentation.  The biggest quirk is that it doesn't run as a service yet.  You must currently run it from a console window and if you close the console window, it kills the processes.  Other quirks: sqlr-status doesn't work yet and the various loadable modules such as password encryption, authentication and logging don't work yet.  Also, only Oracle databases are currently supported.  You can't run SQL Relay on Windows, against Sybase, DB2, MySQL, PostgreSQL, SQLite, MS Access or MS SQL Server yet.

Building SQL Relay on Windows is tricky and currently requires the right versions of things to be in the right places (as described in the documentation) so if you want to try SQL Relay on Windows, I recommend the SQL Relay Binary Distribution for Windows, available on the downloads page for a small fee.

Also, if you are running a 64-bit OS then use the 64-bit version of SQL Relay.  The 32-bit version doesn't perform well on 64-bit systems.  The issues it has appear to affect other applications as well though, and don't appear to be specific to SQL Relay.

Configuration and Init Improvements

Another significant new feature is support for an sqlrelay.conf.d directory.  As of this release, in addition to the sqlrelay.conf file, all configuration files in the sqlrelay.conf.d directory are also processed.  This makes it possible to keep instance configurations separate.

The init script and sqlr-start program have also been updated.  Rather than running sqlr-start with each instance listed in /etc/sqlrelay or /etc/sysconfig/sqlrelay, the init script now just runs sqlr-start without specifying an instance.  If run without specifying an instance, sqlr-start now digs through the configuration files and starts all instances with enabled="yes" attributes.

Together, these updates make configuration much cleaner, but in particular make it much easier for third party applications to install a configuration and cause an instance to start at boot.

Threaded Listener

The sqlr-listener can run in either multi-process or multi-thread mode and now runs in multi-thread mode by default, which is much faster and lighter weight.

PHP Improvements

PHP ini files are now installed to enable the PHP and PHP PDO modules.  It's no longer necessary (on modern platforms, at least) to manually enable them.

PHP PDO Improvements

The PHP PDO driver features many improvements.

The connect-string now includes resultsetbuffersize, dontgetcolumninfo and nullsasnulls parameters.  The debug connect-string parameter can be set to a file name to send debug information to a file instead of to the browser.

Output bind variables can be bound to streams now.  It is, for example, possible to fetch a blob directly into a file, using an output bind variable.

Samat Yusupov contributed a set of driver-methods for ending, suspending and resuming sessions.  They have been incorporated and improved a little, as described in the documentation.

Perl DBI Improvements

The Perl DBI driver is much improved as well.

Support for the maxrows parameter to fetchall_arrayref has been added.  The begin_work method works now.  The get_info method works properly now.

RowCacheSize and RowsInCache attributes work correctly now.  The entire result set is now buffered by default (as is the case with other SQL Relay client API's) unless the RowCacheSize is set.

A custom ext_SQLR_Debug database handle attribute now allows debug to be turned on or off or sent to a file.

Type, length, precision and scale attributes can now be set for bind variables.  Output bind BLOBs and CLOBs  work now.

The ParamValues, ParamTypes and ParamArrays attributes work correctly now.

The driver works correctly with very old versions of Perl DBI now too.

Lots of fixes.

SAP ASE Support

SAP/Sybase ASE 16.0 is supported now.  The configure script detects it properly.  Version 16.0 revealed a few bugs that have been fixed too.

PostgreSQL Improvements

It is now possible to connect to PostgreSQL databases over SSL using the sslmode option.

JMeter  Support

The ODBC driver has been improved sufficiently to work with JMeter via the JDBC-ODBC bridge.  JMeter can now be used to benchmark all tiers of the application directly.  This is helpful in determining whether a performance problem lies in the database, SQL Relay, or the app.

Changes Under the Hood

Internally, quite a few things were restructured.  A lot of tightly coupled code has been decoupled and refactored into frameworks with plugins.  This ought to be invisible to the end user, but should enable much more modular development in the future.

Bug Fixes

A slew of bugs have been fixed.

Some are significant:
  • Oracle JDK 7 and 8 are now detected correctly on Debian/Ubuntu systems.
  • Blobs work when faking input binds now.
  • Everything ought to compile cleanly with clang now.
  • DB2 output bind BLOBs aren't truncated any more.
Others only occurred in obscure circumstances, but I guess they're still significant if they happened to you:
  • sqlrsh doesn't lose the timezone when binding dates any more.
  • The PostgreSQL drop-in replacement library had buggy implementations of PQreset, PQresetStart, PQresetPoll.  Those are fixed.
  • The MySQL drop-in replacement library had implementations of mysql_row_seek, mysql_row_tell., mysql_stmt_row_seek and mysql_stmt_row_tell that didn't work on platforms with 32-bit void pointers.  Those are fixed.
  • Subtle bugs related to keeping track of the total row count in the sybase and freetds connections have been fixed.
  • Old versions of Perl (5.00x) are supported now.
  • An obscure bug that caused DB2 output bind integers to be copied out incorrectly sometimes has been fixed.
  • A very obscure bug that could cause Oracle connections to crash sometime later if an output bind cursor was used to fetch blob data has been fixed.

Rudiments 0.49 is out

Rudiments version 0.49 is now available.

Plenty of good stuff in this release: much improved support for Windows, some refactoring of the signal-handling classes, unification of the passwdentry and shadowentry classes into a single userentry class, improvements when compiling with clang, and a slew of subtle bug fixes.

Full changelog follows:
  • combined passswdentry and shadowentry classes into userentry class
  • updated file::sync() to use FlushFileBuffers directly
  • implemented sys::sync(), reboot(), halt() and shutdown() for windows
  • implemented various sys::getXXX() methods for windows
  • added "detached" option to process::spawn()
  • crash-related methods in process class catch SIGABRT, SIGFPE, SIGILL, SIGBUS, SIGIOT, SIGEMT and SIGSYS in addition to SIGSEGV now
  • shutdown-related methods in process class catch SIGQUIT and SIGHUP in addition to SIGINT and SIGTERM now
  • on windows, signal classes now support catching, sending/raising SIGABRT, SIGFPE, SIGILL, SIGINT, SIGSEGV and SIGTERM and sending/raising SIGKILL
  • disabled threads outright on OSR5
  • fixed threads detection for solaris 2.6
  • codetree bails properly at end of string now
  • fixed a break-related condition that could cause codetree to loop indefinitely
  • added support for a beginning-of-line character to codetree
  • added support for recursive breaks to codetree
  • made charstring::compareIgnoringCase null-safe for platforms that don't have str(n)casecmp
  • improved Windows support in signal classes
  • -Wno-overloaded-virtual is now set when clang is used
  • fixed some subtle kqueue-related bugs
  • added sys/byteorder.h and htonll/ntohll for solaris 11
  • fixed endian-detection macros for solaris 10 and older
  • added support for SIGLARM and signalmanager::alarm() on windows
  • improved windows implementation of semaphoreset
  • added sys::signalsInterruptSystemCalls()
  • added process::supportsFork()

Thursday, November 20, 2014

Sudo on Minix 3.1.8

Minix 3.3.0 came out a while ago and I had all kinds of fun playing with it.  I'd long been running 3.2.1 but never gave 3.1.8 a try at all.  The fun I had with 3.3.0 got me motivated though and a few weeks back I gave it a shot.

It was fairly straightforward, for the most part.  A few things here and there reminded me of 2.0 but the overall user experience was similar to 3.2.1.  Getting various bits of software running required many of the same tweaks that 3.3.0 and 3.2.1 required, but the most difficult bit by far was sudo.

Sudo 1.7.something-or-other is available via pkgin but it dumps core whenever you do anything with it.

Sudo version 1.6.9p23 has been reliably portable for me though, so I gave it a try.  It took a while to work around various issues, but eventually I got it working.  If anyone out there is interested, here are the tricks:

The code requires a little hackery.

Add the following line after the includes in interfaces.c


Edit and comment out:

# include <search.h>

it should look like this when you're done:

/*# include <search.h>*/

Minix 3.1.8's implementation of getgroups always returns -1.  This causes a few problems for sudo.  To solve them, edit set_perms.c and make 3 modifications.

Make the declaration of runas_groups (around line 376) look like:

static GETGROUPS_T *runas_groups = NULL;
Update a section of the function runas_setgroups() as follows:

if (runas_ngroups <= 0) {
pw = runas_pw ? runas_pw :;
if (initgroups(pw->pw_name, pw->pw_gid) < 0)
log_error(USE_ERRNO|MSG_ONLY, "can't set runas group vector");

if ((runas_ngroups = getgroups(0, NULL)) > 0) {
runas_groups = emalloc2(runas_ngroups, sizeof(GETGROUPS_T));
if (getgroups(runas_ngroups, runas_groups) < 0)
log_error(USE_ERRNO|MSG_ONLY, "can't get runas group vector");

} else {

Update the function restore_groups() as follows:

static void
if (user_ngroups > 0) {
if (setgroups(user_ngroups, user_groups) < 0)
log_error(USE_ERRNO|MSG_ONLY, "can't reset user group vector");

That third modification is actually a workaround for a bug (or feature) in setgroups().  It goes completely haywire if user_ngroups is -1.  I'll bet that this is the problem that the version of sudo available via pkgin has too.  There might be a libc update that fixes this issue but for now we'll just work around it.

Now build and install.  Some options needs to be enabled in the header files with the _POSIX_SOURCE macro, so we'll pass that in.  I also like to install it under it's own directory tree, so I use the --prefix option for that.

CPPFLAGS="-D_POSIX_SOURCE" ./configure --prefix=/usr/local/sudo-1.6.9p23
make install

The default permissions are wrong for Minix though, so they need to be set manually:

chmod 555 /usr/local/sudo-1.6.9p23/bin/sudo
chmod u+s /usr/local/sudo-1.6.9p23/bin/sudo

Now add a line for your user (dmuse in my case) to /etc/sudoers.


Yes, the sudoers file is installed under /etc independent of the --prefix option used earlier.  The NOPASSWD option makes it so you don't have to enter a password.

And the last step is to add /usr/local/sudo-1.6.9p23/bin to your PATH in ~/.ashrc

export PATH=$HOME/bin:/usr/local/bin:/bin:/sbin:/usr/bin:/usr/sbin:\

And that's it.  Re-login and sudo should work as expected.

$ sudo ls -l /
total 44
drwxr-xr-x 2 root operator 2880 Sep 4 2010 bin
drwxr-xr-x 3 root operator 384 Sep 4 2010 boot
drwxr-xr-x 2 root operator 29120 Sep 20 02:47 dev
drwxr-xr-x 3 root operator 2304 Nov 20 22:53 etc
drwxrwxrwx 5 bin bin 320 Sep 20 02:55 home
drwxrwxrwx 2 root operator 128 Sep 20 02:45 mnt
drwx------ 2 root operator 768 Sep 20 04:09 root
drwxr-xr-x 2 root operator 896 Sep 4 2010 sbin
drwxrwxrwt 2 root operator 640 Nov 20 23:52 tmp
drwxrwxrwx 23 bin bin 1536 Sep 20 03:27 usr
drwxr-xr-x 3 root operator 256 Sep 20 02:45 var

Woohoo!  Sudo on Minix 3.1.8.

Wednesday, November 12, 2014

Windows Progress...

SQL Relay has classically run on Unix and Linux but not Windows. A few years ago I got the C/C++ API working on Windows and added a C# API and .NET data provider. I got a few of the command-line programs working too, but that was about all you could do on Windows for quite a while. A few months ago when I got the Perl, Python and PHP API's working on Windows along with the rest of the command line clients, but still, only the client side software worked.

Since then I've made quite a bit of progress getting the server working too. And tonight things really started to come together...

That's the SQL Relay server running on Windows, connected to a database running on Linux, accessed by sqlrsh, also running on Windows.

And it can be accessed remotely too.

That's the same SQL Relay server running on Windows, accessed by sqlrsh running in a separate Linux VM.

There are a number of things that don't work yet like sqlr-stop and various plugins, and there are several things that just haven't been tested yet, but the big things are coming along nicely. I expect to be able to announce preliminary support for the SQL Relay server software on Windows in the next release.

Tuesday, October 28, 2014

oracle-jvm error...

I'm not sure how long ago this happened, but there are now lots of different JVM's available for Debian and Ubuntu systems: openjdk-6, openjdk-7, openjdk-8, oracle-jvm-7 and oracle-jvm-8. Possibly more...

SQL Relay 0.57 has no trouble detecting the openjdk's, but doesn't properly detect oracle-jvm's and the build fails. I ran into this error trying to build on a Raspberry PI running Raspbian, but a few people have reported the same error on Debian and Ubuntu sytems.

There are 3 workarounds:

Disable Java

If you're not using SQL Relay with java then you can add --disable-java to the configure command and SQL Relay won't even try to build against oracle-jvm.

Replace Oracle JVM with OpenJDK

This may be tricky. I'm not sure which packages have a hard dependency on oracle-jvm as opposed to just needing some version of Java. Your mileage may vary.

Patch the Configure Script

Edit the configure script in the SQL Relay source distribution and look for a line like:

   for i in `ls -d /usr/java/jdk* /usr/java/j2sdk* /usr/local/jdk* 2> /dev/null` /usr/java 
/usr/local/java `ls -d /usr/local/openjdk* /usr/pkg/java/openjdk* 2> /dev/null` `ls -d /usr/lib64/jvm/java 2> /dev/null` 
`ls -d /usr/lib64/jvm/java-1.8* 2> /dev/null` `ls -d /usr/lib64/jvm/java-1.7* 2> /dev/null` `ls -d /usr/lib64
/jvm/java-1.6* 2> /dev/null` `ls -d /usr/lib/jvm/java 2> /dev/null` `ls -d /usr/lib/jvm/java-1.8* 2> /dev/null` `ls -d 
/usr/lib/jvm/java-1.7* 2> /dev/null` `ls -d /usr/lib/jvm/java-1.6* 2> /dev/null` /System/Library/Frameworks
/JavaVM.framework/Versions/Current /usr /usr/local

and replace it with these lines:

   for i in `ls -d /usr/java/jdk* /usr/java/j2sdk* /usr/local/jdk* 2> /dev/null` \
    /usr/java \
    /usr/local/java \
    `ls -d /usr/local/openjdk* /usr/pkg/java/openjdk* 2> /dev/null` \
    `ls -d /usr/lib64/jvm/java 2> /dev/null` \
    `ls -d /usr/lib64/jvm/java-1.8* 2> /dev/null` \
    `ls -d /usr/lib64/jvm/java-1.7* 2> /dev/null` \
    `ls -d /usr/lib64/jvm/java-1.6* 2> /dev/null` \
    `ls -d /usr/lib64/jvm/jdk-7-* 2> /dev/null` \
    `ls -d /usr/lib64/jvm/jdk-8-* 2> /dev/null` \
    `ls -d /usr/lib/jvm/java 2> /dev/null` \
    `ls -d /usr/lib/jvm/java-1.8* 2> /dev/null` \
    `ls -d /usr/lib/jvm/java-1.7* 2> /dev/null` \
    `ls -d /usr/lib/jvm/java-1.6* 2> /dev/null` \
    `ls -d /usr/lib/jvm/jdk-7-* 2> /dev/null` \
    `ls -d /usr/lib/jvm/jdk-8-* 2> /dev/null` \
    /System/Library/Frameworks/JavaVM.framework/Versions/Current \
    /usr \

One of those three solutions should work.

The source has been updated and it's fixed in CVS. The next release will include the fix too.

Thursday, October 23, 2014

Retrocomputing with Solaris 2.6 x86


If you search eBay for "Solaris" you'll find a lot of people selling old copies of 2.6, but all of these copies are for the SPARC platform. 2.6 for Intel was alleged to exist, but after leaving a fruitless search going for months and months, I started to wonder if it was a myth.

And then, one day, out of the blue, something turned up. It was too good to be true. I didn't believe it at first. But there it was. Solaris 2.6 for Intel.


When it came in the mail, I was even happier. The packaging might have been reused from something else, but boy was it cute.

solaris 2.6 - 1. packaging

Also, the shipper included a little note on cat-themed stationery thanking me for my purchase, AND refunding me my change from shipping! kri-bri - best seller ever.

I was so wrapped up admiring the packaging that I almost forgot about what was inside. But there it was, the mythical Solaris 2.6 for Intel, shiny and still shrink-wrapped.

solaris 2.6 - 2. box

It almost felt wrong to open it. "Fortunately" I didn't have to right away. I was really busy at the time so all I could do for a week or more was put it on the shelf and wait for a break.

VM Configuration

(Two Weeks Later)

Not terribly long ago I'd managed to get the unexpectedly less-rare Solaris 2.5.1 x86 running in VMware 8.0.6. It had required jumping through a few hoops. Even longer ago I'd installed Solaris 7 x86, and in comparison, it had required jumping through flaming hoops while riding a unicycle. I was curious where 2.6 would fall on this spectrum and a little nervous that it might require some set of new gymnastics.

Turned out there were few surprises.

Through trial and error I discovered that 2.6 only supports about a 4G disk. Or, at least that a 4G disk image works but an 8G does not. You can install to an 8G disk, but you can't boot from it after installation. Or at least I couldn't get that to work. So, 4G disk.

128mb of memory seemed to be enough. I told VMware that the guest OS was Solaris 8. I removed the sound card, and aimed the virtual floppy at the Solaris Device Configuration Manager image.


Right away I got an odd error:

Loading driver blogic.bef
Failed to initialize host adapter

It looked like Solaris wasn't too happy with VMware's Buslogic implementation. My virtual drives were on the virtual IDE controller though, and it turned out I could just hit return to keep going.

The rest of the installation was straightforward-ish. It was a lot like other Solaris installations, with some of the same quirks. It didn't ask for DNS info or a default route. Several times it tried to get me to run kdmconfig and I had to either Bypass or Bypass and Suppress it.

The oddest thing was disk partitioning, but it was no odder than usual. In fact it was exactly the same as 2.5.1 and 7. You have to uncheck and re-check c0d0 (c-zero d-zero) or the installer will think that it has partitions defined even though it doesn't. Also, the default filesystem layout includes a small root and large /export/home and the editor for removing /export/home and resizing root isn't very intuitive.

But I was used to that, did that, and pretty soon it looked like this:

solaris 2.6 - 2. install

The main installer ran for a while, then it appeared to install some updates. I'd told it to not to auto-reboot so I'd be able to remove the floppy. When it was done it dropped me to a root prompt, I removed the floppy, halted the system and rebooted.

solaris 2.6 - 3. first boot

All right!

Post-Install Tweaks

Everything wasn't perfect yet though.

Even though I'd bypassed and suppressed configuration of the graphical environment it still tried to start anyway, so I disabled it for real.

dtconfig -d

Fortunately the emulated PCNet NIC was recognized (unlike in 2.5.1), but the installer didn't ask for a default route or DNS info so I configured that manually too.

Default route:

echo "" > /etc/defaultrouter




hosts: files dns

I rebooted after that step to make sure everything network-related came back up correctly on restart, and it did.

The default path didn't include some important stuff, so I updated /etc/profile


I'd told the installer not to create a /export/home partition, but if you don't create the partition then the directory doesn't even get created. This is where home directories are created by default though, so I added it.

mkdir /export/home

I also added a user for myself.

useradd -m -d /export/home/dmuse dmuse
passwd dmuse

And I added /usr/local and /opt/sfw because I intended to install some software in those locations later.

mkdir /usr/local
chmod 777 /usr/local
mkdir /opt/sfw
chmod 777 /opt/sfw

As with Solaris 2.5.1 and 7, software was difficult to come by. There's plenty of source code out there but what was I going to compile it with?

Also as with 2.5.1 and 7, the solution was to build a compiler targeted for 2.6 on a different version of Solaris. This time I built gcc 2.95.3 on Solaris 9. I first tried building on 11 and 10 but the versions of gcc that I had installed on those systems appeared to be too new to build 2.95.3.

On Solaris 9, I ran:

./configure --prefix=/usr/local/gcc-2.95.3 --host=i386-pc-solaris2.6 --target=i386-pc-solaris2.6 --enable-shared
gmake install

Then I tarred up /usr/local/gcc-2.95.3, ftp'ed it over to the 2.6 VM, untarred it in /usr/local and added /usr/local/gcc-2.95.3/bin to my PATH.

As if by magic, I could now build software in my 2.6 VM.



Now that I had a working compiler, I rebuilt gcc natively and installed it under /opt/sfw.

CC=gcc ./configure --prefix=/opt/sfw --enable-shared
make install

I removed /usr/local/gcc-2.95.3, removed it from my path, added /opt/sfw/bin to my path (and /opt/sfw/lib to my LD_LIBRARY_PATH) and used the new gcc to build a bunch of other stuff: gzip-1.3.12, bash-1.14.7, sudo-1.6.9p23, perl-5.003_07, openssl-0.9.6h, zlib-1.2.8, openssh-3.1p1, GNU tar-1.13, apache-2.0.65, lynx-2.8.7, wget-1.5.3, cvs-1.11.23, GNU make-3.82, vim-7.4... Maybe some other stuff too.

I configured everything to install under /opt/sfw and just about everything compiled cleanly.

openssh required line 58 of openbsd-compat/bsd-snprintf.c to be commented out:

/*# undef HAVE_VSNPRINTF*/

I think that was all though.

Remote Access

It's fun to work on the console for a while, but it does get old. After building and installing openssh, I created an init script for it at /etc/init.d/ssh


case "$1" in
killall sshd
echo $"Usage: $0 {start|stop}"
exit 1

exit 0

...and set it up to run at boot:

chmod 755 /etc/init.d/sshd
cd /etc/rc3.d
ln -s ../init.d/sshd S90sshd

I tried enabling X-Forwarding in /opt/sfw/etc/sshd_config

X11Forwarding yes

...but for whatever reason it never worked.

But, after starting ssh

/etc/init.d/sshd start

I could log in remotely.

solaris 2.6 - 4. ssh

I could also scp files back and forth.

The Web

I'd built and installed lynx and wget earlier and they both worked as expected.

Netscape 4.78 was available and worked but struggled hilariously with modern websites.

solaris 2.6 - 5. netscape

It did support PNG's though, which I didn't expect.

The hotjava web browser was also available but it struggled even more. Side note though... I think Solaris 2.6 was the first version to support java (version 1.1.3 no less), or at least to ship with support for it.

On the server side, I was able to get Apache working but I had to do the same Group trick that I did under Solaris 7. By default, Apache is configured to run as Group #-1. On most systems, group id's are 16-bit, -1 translates to 2^16-2, and the "nobody" group has that group id. Apparently, on Solaris, group id's are 32-bit, "nobody" has an id like 60001, and it couldn't make any sense of #-1. This fixed it though:

#Group #-1
Group nobody

I added an init script for it too, at /etc/init.d/httpd


case "$1" in
/opt/sfw/bin/apachectl start
/opt/sfw/bin/apachectl stop
echo $"Usage: $0 {start|stop}"
exit 1

exit 0

...and configured it to run at boot.

chmod 755 /etc/init.d/httpd
cd /etc/rc3.d
ln -s ../init.d/httpd S90httpd
X Windows

Solaris 2.6 comes with Sun's X server but it doesn't support VESA framebuffers so under VMware it's limited to 16-color VGA mode at 640x480.


Binaries of XFree86 4.7.0 are available for Solaris 2.6 at And they work!

They do take a little coercion though.

To get it working, I downloaded the following files:


Then I ran:

sudo sh

The prompts were intuitive and I installed all optional components but I told it not to create links for opengl or rstart.

I also added /usr/X11R6/bin ahead of other paths in the PATH variable and added /usr/X11R6/lib ahead of other paths in the LD_LIBRARY_PATH variable.

Configuring XFree86 (xf86config) took some trial-and-error but I eventually got it. The important settings are:

  • mouse protocol: 1. Auto
  • monitor: 2 31.5 - 35.1; Super VGA, 800x600 @ 56 Hz
  • vertical sync range: 4 40-150
  • yes, look at a card database
  • card: 0 * Generic VESA compatible
  • video memory: 3 1024K
  • color depth by default: 4 16 bits (65536 colors)

The color depth appears to have to be the same as the depth of the host display.

After configuration the mouse didn't work, but editing /etc/X11/XF86Config manually and commenting out the following line seemed to fix it:

#Option "Device" "/dev/mouse"

After all that I was able to get a simple X session running with startx.

Running openwin gave me an OPENLOOK session.

solaris 2.6 - 7. openwin

I was able to get a CDE session by creating an .xinitrc with:


...and running startx.

solaris 2.6 - 8. cde

A graphical login was a little more difficult.

It turns out that the dtlogin program runs /usr/openwin/bin/Xsun directly, and passes it an option that the XFree86 server doesn't understand, so I replaced it with a script that removes the option:


ARGS=`echo $@ | sed -e "s|-nobanner||g"`

/usr/X11R6/bin/X $ARGS

I'd had the same issue with Solaris 7 though, so the solution was familiar.

Enabling dtlogin was simple.

dtconfig -e

It didn't work perfectly though. As on Solaris 7, I could login and run a CDE session, but I had to use ctrl-backspace to exit the session and get back to the login.


I had previously built and installed all the tools I usually use to build my software, so I gave it a try.

Rudiments required one modification. I'd added a hack at some point for FSU pthreads on SCO something-or-other, but then later disabled FSU pthreads support but forgot to remove the hack. The hack caused a problem on Solaris 2.6 and removing it solved it.

Everything else built cleanly and I was able to run the SQL Relay client software.

solaris 2.6 - 9. sqlrsh

I'd really like to get the SQL Relay server running on Solaris though.

I'm still looking for a version of Oracle or Sybase or DB2 or something that'll run on older Solaris x86. I remember seeing Oracle 8.0.5 on eBay once, but only once. I guess I should have jumped on it then.

I probably should try to build MySQL, PostgreSQL, Firebird and SQLite. I'm not sure how much work that would be but if I could get it going then it'd at least be something.

Eh, something to work on later.

Endearing Features

Solaris 2.6 has the standard set of Solaris quirks: you have to use delete for backspace on the console, home directories are created in /export/home, the default shell doesn't understand ~/, filesytem performance is not so good under VMware (untarring and rm -rf'ing take a while) and setting the domain in /etc/resolv.conf requires the "domainname" keyword rather than "domain".

2.6 also only supports 4g disks, which make sense, given its vintage - 1998. Unfortunately it also makes host cpu run at 100%. I can't say that was unexpected either though, as both 2.5.1 and 7 do the same.

So there you have it. Solaris 2.6 x86. About what I expected. Not too surprising. Not too disappointing.

I'll have to try to get more software running on it though - databases, maybe some old java. Yeah, that would be fun.

Saturday, October 11, 2014


It appears that I forgot to run "make distclean" before building that last round of tar.gz and zip files for both Rudiments and SQL Relay. So, if you downloaded last week (the week of October 6th, 2014), or the week before, then you may have had trouble building. Configure probably ran, but make might not have made anything.

Who knows what platform the binaries that were in there ran on? Fedora 20 x64 maybe... Maybe not though.


I just updated the tar.gz and zip files with clean versions. I also downloaded and verified them this time. They ought to be correct.

Thanks Florin... If you hadn't pointed it out, I might never have noticed.

Saturday, October 4, 2014

SQL Relay - 0.57 is out

SQL Relay 0.57 is now available!

Windows Support

Support for Windows is much improved in this release. The command line clients all work now - sqlrsh, sqlr-export, sqlr-import, etc. The C/C++, C# and ADO.NET API's have worked for a while, but are now joined by native API's for Perl, Python, PHP and Java as well as drivers for Perl DBI, Python DB and PHP PDO.

It is possible to build all of this on Windows using MS Visual Studio or MS Visual Studio Express but it is currently quite difficult and requires specific versions of things to be installed in the right places. PHP is especially difficult to work with. In fact, I'm honestly not sure how 3rd party developers are expected to develop drivers for Windows. I had to shoehorn the headers in manually from the source code.

However... The SQL Relay Binary Distribution for Windows is now available (for a few bucks) and provides installers for binary versions of the above-mentioned software.

So... in addition to just accessing SQL Relay from command line clients, SQL Relay applications, including web-based applications, can now be developed for Windows in C, C++, C# (and other .NET languages), Perl, Python, PHP and Java.

Give it a try, let me know if you run into trouble.

Oracle "describe table" Improvements

In sqlrsh if you run "describe tablename" it returns info about the columns of the table - name, type, size, etc. This is accomplished by a call to the getColumnList() method of the C++ API. All SQL Relay API's support this method, and it has been improved.

For oracle tables, if a column is a primary, unique or foreign key, that information is now given. In previous releases it was not. This is a fairly expensive operation, as it turns out, so there is a config parameter to disable looking up keys. See disablekeylookup=yes/no in the oracle connection string in the SQL Relay Configuration Reference for more info.

Also, if you run "describe" on a synonym to a table in another schema, it will now return information about that table. In previous releases, only tables could be described.

Rudiments Improvements

The Rudiments library that SQL Relay depends on has been updated to randomize the list of hosts returned by DNS server when using round-robin DNS. If you are using round robin DNS to distribute SQL Relay clients over a pool of SQL Relay servers, then the behavior may different than in previous releases. In particular, if one host is down, then rather than slamming the next host in the list with all of the downed host's traffic, load will be distributed over the remaining hosts. This feature was added to work around an issue with many recent-ish implementations of getaddrinfo().

Rudiments has also been updated to support kqueue, epoll, port_create, /dev/poll, poll and select in its listener class. There should be no concerns now about large numbers of clients queuing up on the listener, or with listener performance with regard to using select internally.

See the rudiments release announcement for more information on these updates.

Bug Fixes and Minor Improvements

This release features a few bug fixes and minor improvements too.

The Python API's getRowDictionary method had a bug where None's were being returned as 0's when getNullsAsNones was set. That's fixed now.

Sensible errors are now returned when the format of a numeric bind variable is incorrect when using mysql, firebird or oracle. An error was fixed that could cause mysql connection to loop up if alphanumeric bind variable names were used instead of numbers. A bind-variable translation bug was fixed that could cause variables not to be translated when multiple formats are used in the same query and one of them is the correct format too.

The --with-system-libtool configure option was broken but it's fixed now.

Minix 3.3.0 is now supported. At least the client side is. Ie. you can access Oracle from Minix 3.3.0 using sqlrsh and SQL Relay running on another system. The server-side stuff doens't work yet. The Minix kernel doesn't support semaphores by default and Minix 3.3.0 ships without kernel source. I'm not sure if it supports semaphores at all though, so maybe that's moot.

The drop-in replacement library for MySQL had bugs with its implementations of mysql_row_seek() and mysql_row_tell(). Those are fixed now.

The configure script properly detects Maria DB on Ubuntu 14.04.1 now.

Give it a try and report any bugs that you find!

Friday, October 3, 2014

Rudiments - 0.48 is out

Rudiments 0.48 is now available!

This release has a few fairly important features.

Random Number Generation

The randomnumber class has been updated to support CryptGenRandom (on Windows), arc4random, random_r, rand_r, lrand48_r, random, rand and lrand48. It will use whichever of those it can find, in that order. The class has static and non-static members now. When the non-static members are used, the class can be seeded once and numbers (or scaled numbers) can be generated over and over without explicitly re-seeding.

Inet Socket Clients

Several improvements have been made to the inetsocketclient class.

Internally, inetsocketclient uses either getaddrinfo() or some variant of gethostbyname(), depending on what's available.

First, a bug was fixed that caused timeouts not to work on systems that don't have getaddrinfo().

Second, it seems that the behavior of getaddrinfo() has changed over the years and a workaround has been introduced to deal with some issues that the new behavior causes...

When round-robin DNS is used, gethostbyname() and getaddrinfo() fetch the entire list of IP addresses that the host name resolves to. The inetsocketclient's connect() method gets this list and tries to connect to each of these IP's, in the order that they were returned by gethostbyname()/getaddrinfo(). When round-robin DNS is used, the order of the IP's in the list is rotated with each DNS request. This can be used to implement a simple load-balancing scheme.

"Recent" (I'm not sure how recent) implementations of getaddrinfo() have begun sorting the IP's, defeating round-robin DNS entirely. Some implementations have a /etc/gai.conf file that allow a sortv4=no directive, but many do not. Some implementations randomize the list of IP's. Some implementations return them in the order that the DNS server returned them in.

Rudiments has no way of knowing what the behavior of getaddrinfo() is, so now, by default, the list of IP addresses is randomized. For the sake of consistency, it's randomized whether it uses getaddrinfo() internally or some variant of gethostbyname(). Methods have been added to the class to enable or disable this behavior too.

Randomization provides a benefit over round-robin DNS as well. With round-robin DNS, when a host goes down, the next host in the list will receive all of the downed host's traffic. With randomization, traffic is distributed evenly over the hosts that are still up.

listner class

The listener class has been refactored entirely to match the paradigms established by kqueue, epoll, /dev/poll, etc. It now supports kqueue, epoll, port_create, /dev/poll, poll and select internally as well, and will use whichever is available, in that order.

In the previous release, support was added for kqueue, epoll and poll but the implementation still did some inefficient things like rebuilding the list of file descriptors to listen on before each iteration. The list is only rebuilt if a file descriptor is added or removed now.

Minix 3.3.0

Minix 3.3.0 came out since the last release and it's now supported.

General Bug Fixes and Improvements

Some general bug fixes and improvements were implemented as well. Cygwin builds ought to work correctly now. Setting file permissions ought to work natively on Windows now too.

Friday, September 19, 2014

Minix 3.3.0 x86


It's all over the web... Minix 3.3.0 is out! And it's got a long list of improvements over 3.2.1, not the least of which are support for shared object libraries and ARM platforms.

In case you don't know much about Minix, Wikipedia can do a much better job describing it and microkernels and the famous debate over microkernels vs. monolithic kernels than I can. So I'll leave that to them.

I love new software, almost as much as I love old software, so for that past few days, I've been giving Minix 3.3.0 a spin.


The 3.3.0 iso is available at I downloaded it from there, aimed my (admittedly aging) VMware 8.0.6 at it, and let the magic happen.

The VM configuration was straightforward. I gave it a 2G disk image, 256M of ram, and removed the devices I didn't intend to use, like the floppy, sound card and printer.

The Minix installation was as simple as powering on the VM and following the directions. Log in as root (with no password), run "setup", choose "automatic mode"... the directions are explicit and the process is intuitive.

Minix 3.3.0 x86 - 1. install

The only marginally tricky step is if you manually configure ethernet, make sure to give the fully qualified domain name (eg. rather than just minix303x86) or later if you install apache, it will complain that it can't determine the fully qualified domain name.

Upon reboot, I seemed to have a working system.

Minix 3.3.0 x86 - 2. first boot

Post-Install Tweaks

...but it turned out that I did have to configure a few things manually.

The installer didn't ask me for my domain name and didn't extract it from the hostname, so I added a search command to /etc/resolv.conf.

# Use Minix nonamed for now

The system didn't appear to know its hostname either, which turned out to be the result of duplicate entries in /etc/hosts. Commenting out the line that correlated the system's IP with %nameserver fixed that.

# %nameserver %nameserver #DNS 1

I also gave root a password, updated pkgin, installed openssh...

# passwd
Changing local password for root.
New password:
Retype new password:
# pkgin update
# pkgin install openssh

...and rebooted again to enable ssh.

Adding a User

Adding a user was straightforward:

useradd -m dmuse
passwd dmuse

I could log in as me now and access the VM via remotely but I can't live without sudo and pkgin didn't have any idea what I was talking about when I tried to install it.

# pkgin install sudo
sudo is not available on the repository
calculating dependencies... done.
nothing to do.

Hmmm... Yeah, I remember that from 3.2.1 too.

I've built sudo from source on a bunch of systems though, perhaps it could be built on Minix too. I never tried on 3.2.1, I just used root for everything. Sad, I know.

Turns out, yes, you can build sudo from source, but it takes a little effort.

First, you need a compiler. The documentation alleges that gcc and clang are both available but gcc isn't on the CD and pkgin didn't appear to know anything about a native gcc toolchain. Surprisingly, it knew about a mingw32 cross-compiler, but nothing native. It did know about clang though, so I gave it a try.

# pkgin install clang
# pkgin install binutils

Sudo version 1.6.9p23 has worked well for me on various old RedHat and Solaris platforms, so I tried building it. It needed a little coaxing, but only a little.

First off, configure had never heard of clang so I created some symlinks.

# cd /usr/pkg/bin
# ln -s clang cc
# ln -s clang++ c++

I also had to edit interfaces.c and add a line like:

#define IFF_LOOPBACK 0x8

after the includes. Minix doesn't define an IFF_LOOPBACK flag and after looking at the code, it appeared harmless to define it. After that, sudo built and installed smoothly.

$ ./configure --prefix=/usr/local/sudo-1.6.9p23
configure: Configuring Sudo version 1.6.9
checking for cc... cc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
$ make
cc -c -I. -I. -O2 -D_PATH_SUDOERS=\"/etc/sudoers\" -D_PATH_SUDOERS_TMP=\"/etc/sudoers.tmp\" -DSUDOERS_UID=0 -DSUDOERS_GID=0 -DSUDOERS_MODE=0440 check.c
cc -c -I. -I. -O2 -D_PATH_SUDOERS=\"/etc/sudoers\" -D_PATH_SUDOERS_TMP=\"/etc/sudoers.tmp\" -DSUDOERS_UID=0 -DSUDOERS_GID=0 -DSUDOERS_MODE=0440 env.c
cc -c -I. -I. -O2 -D_PATH_SUDOERS=\"/etc/sudoers\" -D_PATH_SUDOERS_TMP=\"/etc/sudoers.tmp\" -DSUDOERS_UID=0 -DSUDOERS_GID=0 -DSUDOERS_MODE=0440 getspwuid.c
cc -c -I. -I. -O2 -D_PATH_SUDOERS=\"/etc/sudoers\" -D_PATH_SUDOERS_TMP=\"/etc/sudoers.tmp\" -DSUDOERS_UID=0 -DSUDOERS_GID=0 -DSUDOERS_MODE=0440 gettime.c
cc -o sudo check.o env.o getspwuid.o gettime.o goodpath.o fileops.o find_path.o interfaces.o logging.o parse.o set_perms.o sudo.o sudo_edit.o tgetpass.o zero_bytes.o sudo_auth.o passwd.o lex.yy.o alloc.o defaults.o -lcrypt
cc -c -I. -I. -O2 -D_PATH_SUDOERS=\"/etc/sudoers\" -D_PATH_SUDOERS_TMP=\"/etc/sudoers.tmp\" -DSUDOERS_UID=0 -DSUDOERS_GID=0 -DSUDOERS_MODE=0440 visudo.c
cc -o visudo visudo.o fileops.o gettime.o goodpath.o find_path.o zero_bytes.o lex.yy.o alloc.o defaults.o
$ su
# make install

And I added /usr/local/sudo-1.6.9p23/bin to the PATH environment variable in /etc/profile and added a line for myself at the end of /etc/sudoers...


...but it didn't work. Turns out on Minix, programs need read and execute permissions to run. Execute alone isn't good enough.

# cd /usr/local/sudo-1.6.9p23/bin
# chmod 555 sudo
# chmod u+s sudo

Ha! That made it work.

$ sudo ls /
bin boot_monitor home proc service usr
boot dev lib root sys var
boot.cfg etc mnt sbin tmp


Remote Access and File Transfer

Minix 3.3.0 was looking pretty good from a remote access and file transfer standpoint, at least from the command line.

I could ssh in and out, transfer files with scp, sftp and regular ftp. Wget and lynx were available from pkgin. Tar, gzip, bzip2 and unzip were all installed by default. Zip wasn't but it was available from pkgin.

Ping was installed by default, but oddly telnet wasn't and it also wasn't available from pkgin. It's difficult to track down some connectivity issues without telnet, but I hoped I wouldn't need it.

The Web

Apache was available via pkgin, version 2.0.65nb2 to be precise. I installed it but it wouldn't start. After digging through the logs I eventually discovered that it needs the directory /usr/pkg/var/run to exist. After creating that, it ran just fine.

I did have trouble getting it to run cgi's. I configured it the same way I've configured it dozens of times before, but it just wouldn't work. The mod_cgi module didn't appear to be installed and didn't appear to be linked in statically either.

Between this and gcc, I got to looking around in the pkgin repository a little. Several apache modules were in there, but many more were missing. Other odd combinations of things were in there too. For example, various firefox language packs were available, but firefox itself wasn't. Hmm... This release just came out. I'll bet that some packages still need a little patching and so they and everything that depends on them isn't getting built yet. Maybe in a few weeks or months more will be available.

X Windows

X was a total loss. It wasn't installed as part of the base distribution, no x11 package was available and attempts to aim pkgin at the 3.2.1 repository failed pretty badly too.

I guess I could have tried building from source. Maybe later. Or maybe if I wait a few weeks or months a package will just appear one day.


I had working C and C++ compilers, and Minix 3.3.0 alleged to support shared object libraries.

I was itching to try and get my software running.

Almost immediately I ran into trouble though. CVS wasn't available through pkgin. Fortunately version 1.11.23 built cleanly from source. Then there were issues with libtool, and clang throws a lot more warnings that gcc, and there were goofy problems with threads, and Minix doesn't like it if you use ioctl to set blocking/nonblocking IO on sockets, you have to use fcntl.

That sounds like a lot but as usual, though it took a while to track down solutions, in the end it just took some minor tweaks to the Rudiments library and a few even more minor tweaks to my build script to get the SQL Relay client software running.

Minix 3.3.0 x86 - 3. sqlrsh

Look at that, accessing an Oracle database from Minix. Who'd have thought it was possible?

And look at that:

$ ldd /usr/local/firstworks/bin/sqlrsh
-lsqlrutil-0.57.1 => /usr/local/firstworks/lib/
-lrudiments-0.48.1 => /usr/local/firstworks/lib/
-lmthread.0 => /usr/lib/
-lpthread.20 => /usr/pkg/lib/
-lc.12 => /usr/lib/
-lssl.1.0.0 => /usr/pkg/lib/
-lcrypto.1.0.0 => /usr/pkg/lib/
-lpcre.1 => /usr/pkg/lib/
-lcrypt.1 => /usr/lib/
-lc++.1 => /usr/lib/
-lm.0 => /usr/lib/
-lsqlrclient-0.57.1 => /usr/local/firstworks/lib/
-lreadline.6 => /usr/pkg/lib/
-lterminfo.1 => /usr/lib/
-lhistory.6 => /usr/pkg/lib/

Dynamic linking! On Minix!

PostgreSQL was available from pkgin too, so I installed it and SQL Relay compiled against it just fine, but I couldn't get the SQL Relay server to run because the kernel wasn't compiled with support for semaphores. Shared memory, yes. Semaphores, no.

$ ipcs

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status

kernel not configured for semaphores


This caused problems on reboot too, when the PostgreSQL server tried to run.

I figured I'd try recompiling the kernel with support for semaphores, but I couldn't find the kernel source anywhere. It wasn't installed, it's not hidden away on the CD, and it's not available via pkgin. How does one get a hold of the Minix 3.3.0 kernel source? If somebody knows, please clue me in.


Minix 3.3.0 seems to run a bit more smoothly than 3.2.1. Now that I think about it, I can't remember exactly what problems I had with 3.2.1, but it seems like I had problems running certain programs after ssh'ing in and then su'ing and also running out of memory from time to time. I didn't run into any trouble like that at all with 3.3.0. For certain, 3.3.0 is at least as stable as 3.2.1.

I was able to tune it down to 96M of RAM too and everything continued to run fine, even software builds. I might even be able to tune it down more.

Lack of X, gcc and other packages was a drag, but I expect that they'll be available eventually. Fuel for another article, I guess.

Lack of built-in support for semaphores and lack of kernel source was a serious drag. The source has got to be available somewhere though. Maybe I'll find it if I dig a little bit.

Of course, every OS has it's share of endearing features. Here are a few that I ran into while playing around.

  • tab-completion doesn't work after su'ing
  • the route command is implemented as 3 commands: add_route, del_route, pr_routes
  • setsockopt errors for various unsupported options are displayed on the screen from time to time
  • clang requires -Wno-overloaded-virtual to work with rudiments socketclient
  • clang requires -Wno-mismatched-tags to compile sqlrelay c api
  • libtool doesn't know that Minix can create dynamic libs yet, must configure with --with-system-libtool
  • must include -lmthread before -L/usr/pkg/lib -lpthread (or use -pthread instead of both)
  • FIONBIO can't be used on sockets with ioctl, must use fcntl instead

Arguably the clang issues are bugs in my software. I guess I shouldn't count those. The libtool thing is just because this version of Minix is so new and the libtool patches haven't gone mainstream yet.

The mthread thing is a little weird.

It's a brand new release though, with lots of changes under the hood. A few endearing features are to be expected.

I hope X and gcc are available soon, I can't wait to play around with them.

In the mean time though, it's stable enough and doesn't slam the host CPU. I think I'll keep a Minix 3.3.0 VM running and add it to my build farm.

Thursday, August 28, 2014

Retrocomputing with Coherent 4.2.10


Back in the 80's and 90's, there was this OS floating around called Coherent. Don't call it Coherent Unix because it's not, it's just Coherent. It is very unix-like but as far as I know, it's a clean-room implementation of a unix'ish system, not a derivative of any prior work. From what I've heard, early versions were less unix'ish than later versions, but I've never seen any of those, so who knows?

I'm not sure where I first saw Coherent. I know that I never saw it running anywhere. I think I saw a set of installation disks at a company I worked for in New Orleans in the 90's, but that could be a false memory. I know that it's impossible, but it seems like I've just always known about it. Funny how memory works.

I know for sure that I've run into it archived at The Unix Heritage Society. I was always a nervous about giving it a run it though, as distribution would require permission but it was unclear whether that had ever actually been given. There's open-source software in the archive and updates which Robert Chalmers got permission to distribute, but no word about the install disks themselves...

Until recently.

In 2012 Hans Bezemer tracked down the right guy (Bob Schwartz) and got permission to put together a redistributable root image. Sort-of. He's allowed to distribute it. But just him. Not you!

From The Beez's post:

I've had permission from Robert Swartz himself to distribute the
first perfectly legal Coherent 4.2.10 QEMU image as long as you
comply with the following conditions:

You may use the software on this image free of charge for personal,
non-commercial use. You may NOT redistribute this image or the software it
contains without written permission from the copyright holder. The software
is provided on an "as is" basis without warranty of any kind.

Well, maybe you could distribute it if you got permission too.

I wasn't looking to go bugging people for that though. I just downloaded the image and started playing with it.

That was like a year ago or more.

It ran great, but it was a little small, 42mb I think. I wanted to play with X and gcc. X barely fit on the root and I only got gcc to install after making a copy of the image, mounting it at /u1 and deleting all of the files. There was barely any room left on root, the kernel didn't appear to support more than 2 disks, no support for cross-partition links or symlinks... Red lights at every turn.

There had to be a better way.

Making More Space

I tried in vain to create a larger disk image but for some reason, every time I'd run a mkfs it would hose the root. I'm not sure if I was doing something wrong, or if there were bugs in QEMU back then, but a few days ago I got the urge to fiddle with it again, and, under QEMU 2.1.0, it worked great.

So, to make a bigger root, here's what you do...

Create a 512m disk image on the VM host.

dd if=/dev/zero of=coherent-bigger.img bs=1024 count=524288

I had all kinds of problems with larger disks, but 512m worked fine.

Boot qemu with the public image and the bigger disk image.

qemu -hda coherent.public.img -hdb coherent-bigger.dd -boot c -m 32

Log in as root with password rootroot

Partition the bigger drive and, while you're at it, install a master boot record.

fdisk -b /conf/mboot /dev/at1x

The -b option specifies the a master boot record to use. /conf has a bunch of bootloader images in it. mboot is the master boot record. /dev/at1x refers to the second drive. /dev/at0x is the first drive. The "x" means "whole drive" as opposed to individual partitions which are "a","b","c" and "d".

fdisk is interactive and friendly but very different from fdisk on other unixes.

You must:

  • zero the partition table
  • change active partition to partition 4
  • change one logical partition (partition 4)
  • specify partition sizes in cylinders
  • yes, create a coherent partition
  • accept defaults for base cylinder and partition size
  • make note of the number of blocks in the partition
  • quit
  • write the partition table

Now that the partition is defined, it's accessible via /dev/at1a. Format it!

mkfs /dev/at1a 1032129

What's that crazy number off the end there? That's the number of 512-byte blocks in the partition. It ought to still be on the screen from fdisk. Yes, it's essential to provide that number. Yes, it will hose all kinds of things if you use too large of a number.

Now, mount the new partition, copy everything from the root over to it, then sync and unmount it.

mount /dev/at1a /mnt
tar cvfp mnt/root.tar .exrc .lastlogin .profile autoboot bin coherent* conf dev etc f0 f1 home lib lost+found tboot tmp tmp.pipe update usr
cd mnt
tar xvfp root.tar
rm root.tar
mkdir mnt
chmod 777 mnt
cd /
umount /dev/at1a

About the boot process... Coherent has a three-stage boot. The master boot loader looks for a bootable partition and runs whatever boot loader is installed in the first sector. This secondary boot loader looks on the root file system for /tboot and runs it. tboot (the tertiary boot loader) looks for autoboot which is usually a link to the current coherent kernel and runs that. At various points you have the option to specify a different partition or kernel to boot, but that's the general idea.

fdisk installed the master boot loader, but we still need the secondary boot loader on the primary partition. Fortunately that's easy to install.

dd if=/conf/boot of=/dev/at1a count=1

Also fortunately (for me), the man pages for "boot" and "fdisk" described all of this very clearly.

And that's it. Now we just need to shut down...

cd /
shutdown halt 0

...and restart qemu with the larger disk as hda.

qemu-system-i386 -hda coherent-bigger.img -boot c -m 32


Coherent 4.2.10 - 1. first boot

And look at all that space!

Coherent 4.2.10 - 2. df
Post-Install Tweaks

The Beez created a user for himself in the public image but I'm not him, so I wanted to get rid of him and add myself. Turns out that you have to manually edit /etc/passwd, /etc/shadow and /etc/group to do this and then run passwd on the command like to set the new user's password. You also have to manually remove and create the home directories too, and there's no /etc/skel to copy into the new home.

Also, the chown command doesn't support setting the group, so you have to run chown and chgrp to properly set permissions for the home directory.

Ok! I did all of that, and I could now log in as me and poke all around the system.

There was a C compiler, an older vi clone (elvis), some games in /usr/games (but sadly no advent), uucp, and the standard set of unixy commands.

Disk access was exceptionally slow though. Exceptionally. File caching on the host helped a lot, but things were slow until it kicked in, and then got slow again as blocks got pushed out. Seems unlikely that Coherent would have run that way on real hardware. I imagine that QEMU's virtual disk controller doesn't emulate some particular thing that Coherent's looking for to decide whether to use DMA or something. Ehh. What are you going to do? Try SCSI instead of IDE? Seems like I tried that once, long ago. Might be worth checking into again though.

X Windows


I really wanted to get X and gcc installed. Both were tricky. After much trial and error, here's what I ended up with...

Install X first. The gcc installer asks you to swap the linker post-install and the new linker causes problems for the X installer. Also, X ONLY supports serial mice. Fortunately QEMU does too, but you have to shutdown Coherent and restart QEMU using:

qemu-system-i386 -hda coherent-bigger.img -boot c -m 32 -chardev msmouse,id=msmouse -device isa-serial,chardev=msmouse

The X installer disk images are available in the TUHS Archive.

The installation process is simple except for 1 thing. You have to tell the installer the name of the package you want to install. You can't just tell it "install everything on the disk". There's no apparent way to do that at least. How does one determine the name of the package provided by the set of floppies? Maybe it was written on the outside of the physical media. It's definitely not in any of the FAQ's or any other obvious place. I randomly discovered the names of the X and gcc packages when googling for something else, unrelated. Some guy had torn his hair out figuring them out and happened to mention them in a post about something else. As far as I know, that post is the only place on the internet that the package names are given.

But! Once you know them, the installation process is straightforward.

install CohX /dev/fva0 2

CohX is the name of the X package. /dev/fva0 is floppy drive 0. 2 is the number of disks in the set.

Use the QEMU monitor to insert and swap disk images when prompted.

The installation is interesting. It relinks some binaries and builds others from source. When prompted, select a Microsoft Serial Mouse, at 1200 baud. Uncompress the manual pages.

When the installation is complete, add /usr/X11/bin to the PATH in /etc/profile. Then edit /usr/X11/lib/Xconfig and remove "640x480" from the two lines that it's found in. Then log out and back in to reset your PATH.

To get an X session, run:

Coherent 4.2.10 - 3. X11R5


Backspace doesn't work properly in the xterm, nor does vi, but xcalc does and xeyes does and so do various other X Windows distractions.

This is X11R5 though, and a seemingly stripped down version of it. is still around though, so maybe some day I'll try to get some other software running.

The color X server doesn't work, but if you want to try it, remove /usr/X11/bin/X and create a hard link to X386color.

cd /usr/X11/bin/X
rm X
ln X386color X

Maybe it can be made to work under bochs or some other emulator. Something to try later.


The GCC installer disk images are also available in the TUHS Archive.

GCC was a little trickier to install than X. CohGCC is the name of the package. Knowing that, the install command was straightforward:

install CohGCC /dev/fva0 4

However disks 3 and 4 are mislabeled. Disk 4 is Disk 3 and vice versa. Ha!


When the installation is finished, replace the default linker with a gcc-compatible linker.

mv /bin/ld /bin/ld.pregcc
mv /bin/ld.pre11 /bin/ld

Add /u1/gnu/bin to PATH in /etc/profile, relogin and bam!

Coherent 4.2.10 - 4. g++


Unfortunately there's not too much to compile. There are some programs under /usr/src, but ideally I'd want to try to get some modern software running.

File Transfer

Hmmm... According to the FAQs, Coherent supports NE2000 ethernet cards, TCP/IP and SLIP. If it does though, the version I have doesn't. The kernel doesn't appear to and there don't appear to be any networky programs installed anywhere. If any of that was available on the original 4.2 distribution, it didn't get installed. I don't imagine it was though, being only 4 floppies and all.

So, what options did I have?

The Beez recommends a floppy image formatted for MSDOS. Coherent has various utilities for reading those, you can loopback mount the image under most unixes, and QEMU allows you to insert and eject the image easily. But what if I want to transfer 15 meg of data. No way I'm tarring, splitting, dd'ing, copying and catting those together.

Coherent appears to come with various serial-line file transfer solutions though. Taylor UUCP was on there, but UUCP is a nightmare to set up. Kermit was also available though. Hmmm...

I set up a getty on com2 by editing /etc/ttys and updating the second line:


That cryptic jumble means: run a login (1), local device (l), 38400 baud (S), device /dev/com2l (com2l). 38400 is unfortunately the fastest speed available.

Then I re-ran qemu with a second serial port redirected to a telnet server on port 10000.

qemu-system-i386 -hda coherent-bigger.img -boot c -m 32 -chardev msmouse,id=msmouse -device isa-serial,chardev=msmouse -chardev socket,id=socket,host=,port=10000,ipv4,nodelay,server,nowait -device isa-serial,chardev=socket

And then I could use kermit on linux and ckermit on Coherent to transfer files. It's a little tricky and long-winded to explain, but basically you connect over, log in, run kermit over there, start a receive process, escape back locally, start a transfer, and wait...

Coherent 4.2.10 - 5. kermit

And I do mean wait.

No way I'm waiting that long...

It was cool for small files, but not for anything big.

In the end, it was faster to tar up a bunch of files, reboot QEMU using the tar file as the second disk...

qemu-system-i386 -hda coherent-bigger.img -hdb files.tar -boot c -m 32 -chardev msmouse,id=msmouse -device isa-serial,chardev=msmouse

...and then untar it in Coherent as root.

tar xvf /dev/at1x

Unfortunately this requires a reboot every time you want to transfer a bunch of files, but it's faster than any of the alternatives.


I really hoped to get bash working and a newer version of vi, and maybe eventually some of my own software, but it was a losing battle from the start. Even bash 1.14.7 requires a lot of tweaking, and I still haven't gotten it to fully compile. There's a 14 character filename limit too. C++ source has to have a .C extension rather than .cpp ...


Not the downhill run I'd hoped for. But, on the bright side, it's something to look forward to tinkering with later.

Endearing Features

Coherent has a lot of these.

  • QEMU doesn't detect the idle loop and slams the host cpu.
  • It only appears to support 2 drives.
  • Disk access is incredibly slow.
  • Symlinks are not supported.
  • Cross-device links are not supported.
  • X only supports serial mice.
  • backspace and vi don't work in xterms.
  • The shutdown command is odd - shutdown halt|reboot
  • Gcc requires .C extensions on C++ sources.
  • There's a 14 character filename limit.
  • The filesystem appears to be quite sensitive. Any improper reboot hoses the filesystem. Fsck is not automatically run at startup. When you run it manually, it ends up asking you to remove corrupted files almost every time.
  • Y2K issues galore.

There were more too but those were the most significant. Sounds like a lot, but given the age of the distribution, I expected even more.

I guess it's also worth mentioning that The Mark Williams Company was a small outfit, tiny compared to SCO and the Linux community with whom they ultimately had to compete. That they were able to build a unix-like OS for PC with the features that it had and maintain it for so long is quite a feat.

And Finally...

All right! I finally have a working Coherent installation. It seems pretty stable. There's plenty of space to play with. Lots of stuff works and lots of opportunities remain for tinkering. Hopefully I'll figure out color graphics and get some more modern software to compile soon. Maybe I'll even find some vintage software to run on it. I'd ideally like to get some kind of networking going too. I might have to wait on The Beez to finagle that though.

Monday, July 14, 2014

SQL Relay - 0.56 is out

SQL Relay version 0.56 is now available!

You might want to get this one. A subtle memory leak is fixed in this release. If you really can't upgrade and need a fix for a previous release, contact me and we can work something out.

Amazing new features:

Perl, PHP, Python and Java API's all build and work on Windows. You still have to build them from source though, and it's a pain. If you need Windows binaries, contact me and we can work something out.

Intelligent defaults have been added for a lot of the configuration parameters. The sqlrelay.conf file can be much more stripped down now. I also added a Configuration Guide that walks you through various configurations from minimal to complex. Enjoy.

The PHP PDO API now has a custom statement attribute PDO::SQLRELAY_ATTR_GET_NULLS_AS_EMPTY_STRINGS that can be set to true or false to cause it to return NULL values as NULLs or empty strings.

New parameters have been added for connecting to MySQL via SSL. See the Configuration Reference for more detail.

Amazing bug fixes:

The PHP PDO driver handles float values correctly now. Previously it was converting them to strings. It also handles NULL integers correctly now rather than converting them to 0's.

There was a bug in the sqlr-scaler that could cause it to use a noticeable amount of the CPU when sitting idle. That's fixed now.

Some installations of Oracle instant client might not have been properly detected before, but they are now.

There was a bug that caused problems with DB2 7.X. It's fixed now.

sessionhandler="thread" and listenertimeout!="0" ought to work now. This is still sketchy and experimental though.

Internal stuff:

A lot of code has been reorganized internally. Many things have been modularized and simplified. Frameworks have been built and some things have been re-implemented using them. This is ongoing. More of this will be done over the next few releases.

Full ChangeLog follows:

  • removed VERSION from perl api bootstrap
  • reorganized and simplified perl api code
  • got perl api working on windows with ActivePerl
  • perl api uninstall cleans up better now
  • got python api working on windows with ActivePython
  • consolidated php include tweaks
  • got java api working on windows
  • updated pdo driver to convert float fields to strings rather than ints
  • updated pthread test to match rudiments pthread test
  • moved everything done by children of sqlwriter interface into individual translations
  • added plugin-based authentication framework
  • reorganized code tree a bit
  • reworked default values to enable more minimal configurations
  • updated configuration docs and added configuration guide
  • fixed cast issue with firebird 1.5
  • added result set translation framework and re-implemented date translation using it
  • fixed millisecond->nanosecond confusion in sqlr-scaler
  • fixed subtle oracle instantclient version detection bug
  • added null input bind support to sqlrsh
  • fixed php pdo driver to return null rather than 0 for null integer fields and bind null rather than 0 for null integer input binds
  • added attribute to return nulls as nulls or empty strings to php pdo
  • added parameters for connecting to mysql via ssl
  • added a few tweaks to support DB2 < 8.0
  • fixed bad return value type in python getConnectionPort method
  • fixed sessionhandler="thread" with listenertimeout!="0"

Rudiments - 0.47 is out

For immediate release!

Rudiments 0.47 is now available.

This update to rudiments fixes a few obscure things including a memory leak in the datetime class that could cause SQL Relay to chew up memory (very slowly) and a potential null-dereference in the xmldomnode class. Not sure how I ever missed those bugs before, given how often I use those classes, but at least they're fixed now.

Another significant update in this release is internal support for poll, epoll and kqueue. In the past, the listener class used select exclusively, but select has some issues. Most notably, attempting to listen on "large numbers" (more than 1024 on most systems) of file descriptors could cause undefined problems (references beyond the end of a 1024-member array). Now, if the platform supports epoll (like most modern linux systems) or kqueue (like most modern BSD systems) then those will be used in favor of select. If the platform supports neither epoll nor kqueue, but supports regular poll, then poll will also be used in favor of select. /dev/poll is not yet supported on solaris yet, but it's on the TODO list. Also, the poll/epoll/kqueue implementations aren't optimized yet. They still do inefficient things like rebuild the list of file descriptors every time, which has to be done with select, but they are a bit faster than select, and safe for large numbers of file descriptors.

Two classes have been renamed to be more consistent - variablebuffer is now bytebuffer and rawbuffer is now bytestring. Hey, it's pre-1.0 software. Things get renamed.

The linkedlist class has been refactored and a singlylinkedlist class has been added. Both now have sort methods now too: selectionSort() and heapSort(). Writing them took me back to my data structures class in college. selectionSort() is slow but uses no additional storage. heapSort() is much faster but uses N additional storage. Choose!

I also fixed an obscure bug on windows involving file::open() with O_CREAT but not O_EXCL.

Solaris 2.5.1 is also supported, just in case anyone still uses that!

Full ChangeLog follows:

  • fixed possible null-dereference in xmldomnode::safeAppend
  • added charstring::inSetIgnoringCase
  • xmldomnode::setAttributeValue does nothing when the null node is referenced now
  • fixed a codetree bug that could cause indentation to attempt to go negative when using an unsigned number
  • added a configure test to see if -Wno-format is needed
  • configure tests for mlockall/munlockall attempt link now
  • added sys/types.h before sys/un.h in sys/un.h test for older freebsd
  • implemented stubs for pure virtual methods of client and server classes so instances of them could be created to attach already-open file descriptors to
  • unified usage of select/poll and prefer poll
  • pushed all select/poll-related code into listener class
  • removed problematic and unused useListener-related methods in filedescriptor class
  • epoll is used in place of select/poll for systems that support it
  • kqueue is used in place of select/poll for systems that support it
  • consolidated getpagesize() calls to use sys::getPageSize()
  • added detection and support for __vsnprintf for platforms that have that instead of vsnprintf
  • added cancel and raiseSignal methods to the thread class
  • fixed datetime memory leak
  • added xmldomnode::clone method to clone a node
  • renamed variablebuffer to bytebuffer
  • renamed rawbuffer to bytestring
  • refactored linkedlist
  • added singlylinkedlist class
  • added sort, detach, move and insert methods to linkedlist classes
  • fixed file::open using O_CREAT without O_EXCL on windows

Friday, June 27, 2014

Retrocomputing with DB2 6.1 and SQL Relay

Keeping the Retro theme going, I recently got SQL Relay running against DB2 6.1 Personal Edition through a nearly equally contrived path as getting it running against Personal Oracle 7.

Again, I did it for fun, but the exercise demonstrates the versatility of both SQL Relay and Rudiments.

DB2 6.1 PE - 8. Access From Fedora 20 x64

In this case, I accessed DB2 6.1 running on Windows NT from Fedora 20 via an instance of SQL Relay running against an old instance of DB2 7.2 on Redhat 6.2 (that is, pre-Fedora Redhat 6.2).

Retrocomputing with DB2 6.1 Personal Edition


I'm a sucker for old-timey software. I especially like it when I can shoehorn it into some semi-modern environment somehow. That's always fun. I especially like old database systems, so whenever somebody puts something like DB2 6.1 Personal Edition on eBay for like $6.00, I'm not too slow to grab it up.

Such was the case recently.


What does DB2 6.1 PE even run on? The CD said Windows 95, 97 and NT. I have Personal Oracle 7.2.2 running on Windows 95, so I figured I'd try that first.

It's probably not impossible to get it working on Windows 95, but I couldn't figure out how. Everything installed but nothing ran. Rather than wrestle with it, I figured I'd just try it on NT and see if it went any better.

It did. Much better.

After one false start trying to install it as myself, I logged in as the Administrator, and the installation was straightforward.

I installed all product options.

DB2 6.1 PE - 1. Options

It asked for a user to run everything as. I accepted the default db2admin user. Oddly, some default password was given but it was also starred out. I guess I could have clicked Next and gotten whatever password it was, but I wouldn't have known it. So, just to be safe, I deleted the default passwords and typed in my own.

DB2 6.1 PE - 2. User

DB2 security is a little different from other DB's. When you create an instance, a system-level user is created and the instance runs as that user. If you're logged in as that user, you can access the DB without supplying credentials, but if you want to access it as another user, then you have to supply the system-level credentials.

It's funny though, when I was first learning DB2, none of the examples I ever saw showed how to supply credentials. For a long time, I thought you had to log in as the owner of the instance to be able to access that instance at all. It was all fairly confusing and it got even more confusing when I wanted access databases on remote machines, but that's another story...

The point is that the DB2 installation actually created an NT user named db2admin. This would be important later.

The rest of the install went smoothly.

DB2 6.1 PE - 3. Install

I eventually had to restart the system and when it came back up I was presented with some "First Steps".

DB2 6.1 PE - 4. First Steps

The Local Database

The first of the first steps was to create a sample database. I clicked that and it failed. Strange... I tried again (for some reason thinking it would work the second time) and got the same result. The Task Manager showed a bunch of db2-ish processes running. What the heck?

Ohhh... Heh. db2admin.

After reboot, I'd logged in as myself rather than the db2admin user that the installation process created.

The create-the-sample-db process was trying to run as me. I have no permissions. After logging out and back in as db2admin, creating the sample DB worked as expected.

The second of the first-steps led to the Command Center - basically a semi-graphical database shell.

DB2 6.1 PE - 5. Command Center

I used it to poke around in the sample database a bit.

The third led to the Control Center...

DB2 6.1 PE - 6. Control Center

...which allowed me to poke around a little bit more.

The fourth of the first-steps led to the Information Center (online documentation).

DB2 6.1 PE - 7. Information Center

The documentation viewer relied on Netscape to display the actual docs and the version of Netscape I had on there was a little flaky. It would complain about not being able to open a page, and then go ahead and open it. Fortunately the DB2 docs were about as old as Netscape itself and it didn't have any trouble rendering them.

Everything worked, but everything also ran terribly slowly. It turned out I'd configured the VM with 64m of ram and that just wasn't enough to run the DB. After bumping it up to 128m, everything ran really well.

Remote Access

So I had a working instance of DB2 in my NT VM and I could poke around in it with the provided tools. What kind of remote access could I get working though?

These days configuring remote access usually just means supplying a host name or address and port. That is, an IP address and TCP port. DB2 has been around for a while though, since long before TCP/IP emerged as the dominant protocol. Even Personal Edition supports TCP/IP, IPX/SPX, NetBIOS and APPC (whatever that is). As such, remote databases are abstracted out a bit beyond host and port.

DB2 achieves this via two catalogs: "nodes" and "databases". Nodes identify servers and databases identify instances of DB2 running on those servers. DB2 clients can be told what database to connect to, and they use the local catalog to figure out how to connect to it.

To add a node to the catalog, you have to specify the network protocol, name of the node, and protocol specific info. To add a database to the catalog, you have to tell specify the instance name (as it's known on the remote machine), a local alias for that instance, and the node it's running on.

I have several versions of DB2 running on various Linux VM's, including an instance of 7.2 running in a Redhat 6.2 VM and an instance of 10.5 in Fedora 19 VM.

I first tried connecting to 6.1 from the Linux VM's. This involved logging in as the db2inst1 user on Linux (the user the Linux instances are running under) and running commands like:

db2 "catalog tcpip node winnt remote winnt server 50000"
db2 "catalog database sample as sample61 at node winnt authentication server"
db2 "terminate"

The first command creates a node named "winnt" aimed at port 50000 on the host named "winnt". The second creates a database known locally as "sample61" aimed at "sample" on node "winnt". The third commits the configuration.

I could then try to access the 6.1 instance from Linux using the db2 shell interactively.

From DB2 7.2 on Redhat 6.2, it worked great.

db2 => connect to sample61 user db2admin using mypassword

Database Connection Information

Database server = DB2/NT 6.1.0
SQL authorization ID = DB2ADMIN
Local database alias = SAMPLE61

db2 => select * from employee where job='CLERK'

------ ------------ ------- --------------- -------- ------- ---------- -------- ------- --- ---------- ----------- ----------- -----------
000120 SEAN O'CONNELL A00 2167 12/05/1963 CLERK 14 M 10/18/1942 29250.00 600.00 2340.00
000230 JAMES J JEFFERSON D21 2094 11/21/1966 CLERK 14 M 05/30/1935 22180.00 400.00 1774.00
000240 SALVATORE M MARINO D21 3780 12/05/1979 CLERK 17 M 03/31/1954 28760.00 600.00 2301.00
000250 DANIEL S SMITH D21 0961 10/30/1969 CLERK 15 M 11/12/1939 19180.00 400.00 1534.00
000260 SYBIL P JOHNSON D21 8953 09/11/1975 CLERK 16 F 10/05/1936 17250.00 300.00 1380.00
000270 MARIA L PEREZ D21 9001 09/30/1980 CLERK 15 F 05/26/1953 27380.00 500.00 2190.00

6 record(s) selected.

db2 =>


It didn't work so well from DB2 10.5 on Fedora 19 though.

db2 => connect to sample61 user db2admin using mypassword
SQL30081N A communication error has been detected. Communication protocol
being used: "TCP/IP". Communication API being used: "SOCKETS". Location
where the error was detected: "". Communication function
detecting the error: "recv". Protocol specific error code(s): "*", "*", "0".
db2 =>

Not so well indeed.

I tried in in reverse too - accessing 7.2 and 10.5 on Linux from 6.1 on Windows NT. I ran analogous catalog commands using Command Center on Windows and got similar results. I could access 7.2 just fine but attempts to access 10.5 resulted in:

SQL5048N The release level of the client is not supported by the
release level of the database server.

Yeah, I figured as much. The two versions are separated by over a decade. Got to break back-compatibility at some point, I guess.

So I could connect back and forth between 6.1 on Windows NT and 7.2 on Redhat 6.2. It's a start, but I'd need another link in the chain to be able to get to 6.1 from a modern OS.

SQL Relay

It's been longer than I can remember since I tried running SQL Relay against DB2 7.2. I knew it would build, and there are even #ifdef's in the code for some 7.2 quirks, but would it work?

Turns out no. Not at first at least. DB2 7.2 has a "feature" where if you connect to the DB, then fork the process, the child loses the connection and all subsequent queries fail. At some point I had logic in SQL Relay to deal with that feature, but newer versions of DB2 don't have the problem and the logic had long been removed. It took me a while to figure out what was going on, but once I did I remembered the bug and it was easy to fix. I even fixed it in a more elegant manner than I had in the past.

So Relay could talk to 7.2. Could it talk to 6.1 via 7.2's catalog?

Turns out yes!

DB2 6.1 PE - 8. Access From Fedora 20 x64

Woohoo again!

I now had access to DB2 6.1 Personal Edition from Fedora 20 x64.

And what a contrived chain of software was involved:

SQL Relay client on Fedora 20 x64 ->
SQL Relay server on Redhat 6.2 x86 ->
DB2 7.2 on Redhat 6.2 x86 ->
DB2 6.1 on Windows NT

But hey, it worked.

After a few modifications, I was able to run my standard DB2 test script too.

success success success
success success success success
success success success
success success success success success success success success success success success success success success success success success success success success success success
success success success success success success success success success success success success success success success success success success success success success success

I did have to make a few tweaks to SQL Relay to get blobs and clobs working. There's still some issue with blob binds, but spending time on that is low on my list.

Endearing Features

Once I had everything running, I played with it for a while and began to discover some of the differences between 6.1 and newer versions.

Some of the more interesting ones...

The maximum size for a Blob or Clob column has to be specified during the create statement. For example:

create table test (col1 clob(1M), col2 blob(100K))

You can specify the maximum size in bytes, or use K, M or G.

Conversion errors are somewhat common and you often have to cast values, especially NULL values using CAST(X as CHAR(4)). I never did get a good feel for when this was necessary, but I ran into it a bit.

There's no obvious way to get the hostname from within the DB. In newer versions, you can run:

select host_name from table(sysproc.env_get_sys_info())

...but not in 6.1. I imagine that's another artifact of all the network protocols that it supports.

And my favorite... Stored procedures can only be written in C, Java or COBOL. There's no SQL PL. I guess it just hadn't been invented yet. Ha! I love it.

Ok, so that's it for DB2 6.1 PE, for now at least. Maybe I'll discover more weird stuff later and write more about it.