Tuesday, October 28, 2014

oracle-jvm error...

I'm not sure how long ago this happened, but there are now lots of different JVM's available for Debian and Ubuntu systems: openjdk-6, openjdk-7, openjdk-8, oracle-jvm-7 and oracle-jvm-8. Possibly more...


SQL Relay 0.57 has no trouble detecting the openjdk's, but doesn't properly detect oracle-jvm's and the build fails. I ran into this error trying to build on a Raspberry PI running Raspbian, but a few people have reported the same error on Debian and Ubuntu sytems.


There are 3 workarounds:


Disable Java

If you're not using SQL Relay with java then you can add --disable-java to the configure command and SQL Relay won't even try to build against oracle-jvm.


Replace Oracle JVM with OpenJDK

This may be tricky. I'm not sure which packages have a hard dependency on oracle-jvm as opposed to just needing some version of Java. Your mileage may vary.


Patch the Configure Script

Edit the configure script in the SQL Relay source distribution and look for a line like:


   for i in `ls -d /usr/java/jdk* /usr/java/j2sdk* /usr/local/jdk* 2> /dev/null` /usr/java 
/usr/local/java `ls -d /usr/local/openjdk* /usr/pkg/java/openjdk* 2> /dev/null` `ls -d /usr/lib64/jvm/java 2> /dev/null` 
`ls -d /usr/lib64/jvm/java-1.8* 2> /dev/null` `ls -d /usr/lib64/jvm/java-1.7* 2> /dev/null` `ls -d /usr/lib64
/jvm/java-1.6* 2> /dev/null` `ls -d /usr/lib/jvm/java 2> /dev/null` `ls -d /usr/lib/jvm/java-1.8* 2> /dev/null` `ls -d 
/usr/lib/jvm/java-1.7* 2> /dev/null` `ls -d /usr/lib/jvm/java-1.6* 2> /dev/null` /System/Library/Frameworks
/JavaVM.framework/Versions/Current /usr /usr/local

and replace it with these lines:


   for i in `ls -d /usr/java/jdk* /usr/java/j2sdk* /usr/local/jdk* 2> /dev/null` \
    /usr/java \
    /usr/local/java \
    `ls -d /usr/local/openjdk* /usr/pkg/java/openjdk* 2> /dev/null` \
    `ls -d /usr/lib64/jvm/java 2> /dev/null` \
    `ls -d /usr/lib64/jvm/java-1.8* 2> /dev/null` \
    `ls -d /usr/lib64/jvm/java-1.7* 2> /dev/null` \
    `ls -d /usr/lib64/jvm/java-1.6* 2> /dev/null` \
    `ls -d /usr/lib64/jvm/jdk-7-* 2> /dev/null` \
    `ls -d /usr/lib64/jvm/jdk-8-* 2> /dev/null` \
    `ls -d /usr/lib/jvm/java 2> /dev/null` \
    `ls -d /usr/lib/jvm/java-1.8* 2> /dev/null` \
    `ls -d /usr/lib/jvm/java-1.7* 2> /dev/null` \
    `ls -d /usr/lib/jvm/java-1.6* 2> /dev/null` \
    `ls -d /usr/lib/jvm/jdk-7-* 2> /dev/null` \
    `ls -d /usr/lib/jvm/jdk-8-* 2> /dev/null` \
    /System/Library/Frameworks/JavaVM.framework/Versions/Current \
    /usr \
    /usr/local

One of those three solutions should work.


The source has been updated and it's fixed in CVS. The next release will include the fix too.

Thursday, October 23, 2014

Retrocomputing with Solaris 2.6 x86

Introduction

If you search eBay for "Solaris" you'll find a lot of people selling old copies of 2.6, but all of these copies are for the SPARC platform. 2.6 for Intel was alleged to exist, but after leaving a fruitless search going for months and months, I started to wonder if it was a myth.

And then, one day, out of the blue, something turned up. It was too good to be true. I didn't believe it at first. But there it was. Solaris 2.6 for Intel.

Woohoo!

When it came in the mail, I was even happier. The packaging might have been reused from something else, but boy was it cute.

solaris 2.6 - 1. packaging

Also, the shipper included a little note on cat-themed stationery thanking me for my purchase, AND refunding me my change from shipping! kri-bri - best seller ever.

I was so wrapped up admiring the packaging that I almost forgot about what was inside. But there it was, the mythical Solaris 2.6 for Intel, shiny and still shrink-wrapped.

solaris 2.6 - 2. box

It almost felt wrong to open it. "Fortunately" I didn't have to right away. I was really busy at the time so all I could do for a week or more was put it on the shelf and wait for a break.

VM Configuration

(Two Weeks Later)

Not terribly long ago I'd managed to get the unexpectedly less-rare Solaris 2.5.1 x86 running in VMware 8.0.6. It had required jumping through a few hoops. Even longer ago I'd installed Solaris 7 x86, and in comparison, it had required jumping through flaming hoops while riding a unicycle. I was curious where 2.6 would fall on this spectrum and a little nervous that it might require some set of new gymnastics.

Turned out there were few surprises.

Through trial and error I discovered that 2.6 only supports about a 4G disk. Or, at least that a 4G disk image works but an 8G does not. You can install to an 8G disk, but you can't boot from it after installation. Or at least I couldn't get that to work. So, 4G disk.

128mb of memory seemed to be enough. I told VMware that the guest OS was Solaris 8. I removed the sound card, and aimed the virtual floppy at the Solaris Device Configuration Manager image.

Installation

Right away I got an odd error:


Loading driver blogic.bef
Failed to initialize host adapter

It looked like Solaris wasn't too happy with VMware's Buslogic implementation. My virtual drives were on the virtual IDE controller though, and it turned out I could just hit return to keep going.

The rest of the installation was straightforward-ish. It was a lot like other Solaris installations, with some of the same quirks. It didn't ask for DNS info or a default route. Several times it tried to get me to run kdmconfig and I had to either Bypass or Bypass and Suppress it.

The oddest thing was disk partitioning, but it was no odder than usual. In fact it was exactly the same as 2.5.1 and 7. You have to uncheck and re-check c0d0 (c-zero d-zero) or the installer will think that it has partitions defined even though it doesn't. Also, the default filesystem layout includes a small root and large /export/home and the editor for removing /export/home and resizing root isn't very intuitive.

But I was used to that, did that, and pretty soon it looked like this:

solaris 2.6 - 2. install

The main installer ran for a while, then it appeared to install some updates. I'd told it to not to auto-reboot so I'd be able to remove the floppy. When it was done it dropped me to a root prompt, I removed the floppy, halted the system and rebooted.

solaris 2.6 - 3. first boot

All right!

Post-Install Tweaks

Everything wasn't perfect yet though.

Even though I'd bypassed and suppressed configuration of the graphical environment it still tried to start anyway, so I disabled it for real.


dtconfig -d

Fortunately the emulated PCNet NIC was recognized (unlike in 2.5.1), but the installer didn't ask for a default route or DNS info so I configured that manually too.

Default route:


echo "192.168.123.254" > /etc/defaultrouter

/etc/resolv.conf


domainname firstworks.com
nameserver 192.168.123.37
search firstworks.com

/etc/nsswitch.conf


hosts: files dns

I rebooted after that step to make sure everything network-related came back up correctly on restart, and it did.

The default path didn't include some important stuff, so I updated /etc/profile


PATH=$PATH:/usr/ccs/bin:/usr/ucb:/usr/openwin/bin:/usr/openwin/demo:/usr/dt/bin
...
export LOGNAME PATH

I'd told the installer not to create a /export/home partition, but if you don't create the partition then the directory doesn't even get created. This is where home directories are created by default though, so I added it.


mkdir /export/home

I also added a user for myself.


useradd -m -d /export/home/dmuse dmuse
passwd dmuse

And I added /usr/local and /opt/sfw because I intended to install some software in those locations later.


mkdir /usr/local
chmod 777 /usr/local
mkdir /opt/sfw
chmod 777 /opt/sfw

As with Solaris 2.5.1 and 7, software was difficult to come by. There's plenty of source code out there but what was I going to compile it with?

Also as with 2.5.1 and 7, the solution was to build a compiler targeted for 2.6 on a different version of Solaris. This time I built gcc 2.95.3 on Solaris 9. I first tried building on 11 and 10 but the versions of gcc that I had installed on those systems appeared to be too new to build 2.95.3.

On Solaris 9, I ran:


./configure --prefix=/usr/local/gcc-2.95.3 --host=i386-pc-solaris2.6 --target=i386-pc-solaris2.6 --enable-shared
gmake
gmake install

Then I tarred up /usr/local/gcc-2.95.3, ftp'ed it over to the 2.6 VM, untarred it in /usr/local and added /usr/local/gcc-2.95.3/bin to my PATH.

As if by magic, I could now build software in my 2.6 VM.

Magic!

Software

Now that I had a working compiler, I rebuilt gcc natively and installed it under /opt/sfw.


CC=gcc ./configure --prefix=/opt/sfw --enable-shared
make
make install

I removed /usr/local/gcc-2.95.3, removed it from my path, added /opt/sfw/bin to my path (and /opt/sfw/lib to my LD_LIBRARY_PATH) and used the new gcc to build a bunch of other stuff: gzip-1.3.12, bash-1.14.7, sudo-1.6.9p23, perl-5.003_07, openssl-0.9.6h, zlib-1.2.8, openssh-3.1p1, GNU tar-1.13, apache-2.0.65, lynx-2.8.7, wget-1.5.3, cvs-1.11.23, GNU make-3.82, vim-7.4... Maybe some other stuff too.

I configured everything to install under /opt/sfw and just about everything compiled cleanly.

openssh required line 58 of openbsd-compat/bsd-snprintf.c to be commented out:


/*# undef HAVE_VSNPRINTF*/

I think that was all though.

Remote Access

It's fun to work on the console for a while, but it does get old. After building and installing openssh, I created an init script for it at /etc/init.d/ssh


#!/bin/sh

case "$1" in
start)
/opt/sfw/sbin/sshd
;;
stop)
killall sshd
;;
*)
echo $"Usage: $0 {start|stop}"
exit 1
esac

exit 0

...and set it up to run at boot:


chmod 755 /etc/init.d/sshd
cd /etc/rc3.d
ln -s ../init.d/sshd S90sshd

I tried enabling X-Forwarding in /opt/sfw/etc/sshd_config


X11Forwarding yes

...but for whatever reason it never worked.

But, after starting ssh


/etc/init.d/sshd start

I could log in remotely.

solaris 2.6 - 4. ssh

I could also scp files back and forth.

The Web

I'd built and installed lynx and wget earlier and they both worked as expected.

Netscape 4.78 was available and worked but struggled hilariously with modern websites.

solaris 2.6 - 5. netscape

It did support PNG's though, which I didn't expect.

The hotjava web browser was also available but it struggled even more. Side note though... I think Solaris 2.6 was the first version to support java (version 1.1.3 no less), or at least to ship with support for it.

On the server side, I was able to get Apache working but I had to do the same Group trick that I did under Solaris 7. By default, Apache is configured to run as Group #-1. On most systems, group id's are 16-bit, -1 translates to 2^16-2, and the "nobody" group has that group id. Apparently, on Solaris, group id's are 32-bit, "nobody" has an id like 60001, and it couldn't make any sense of #-1. This fixed it though:


#Group #-1
Group nobody

I added an init script for it too, at /etc/init.d/httpd


#!/bin/sh

case "$1" in
start)
/opt/sfw/bin/apachectl start
;;
stop)
/opt/sfw/bin/apachectl stop
;;
*)
echo $"Usage: $0 {start|stop}"
exit 1
esac

exit 0

...and configured it to run at boot.


chmod 755 /etc/init.d/httpd
cd /etc/rc3.d
ln -s ../init.d/httpd S90httpd
X Windows

Solaris 2.6 comes with Sun's X server but it doesn't support VESA framebuffers so under VMware it's limited to 16-color VGA mode at 640x480.

But!

Binaries of XFree86 4.7.0 are available for Solaris 2.6 at http://ftp.xfree86.org/pub/XFree86/4.7.0/binaries/Solaris. And they work!

They do take a little coercion though.

To get it working, I downloaded the following files:


Xbin.tgz
Xdoc.tgz
Xetc.tgz
Xf100.tgz
Xfcyr.tgz
Xfenc.tgz
Xfnts.tgz
Xfscl.tgz
Xfsrv.tgz
Xhtml.tgz
Xinstall.sh
Xlib.tgz
Xman.tgz
Xmod.tgz
Xnest.tgz
Xpdf.tgz
Xprog.tgz
Xprt.tgz
Xps.tgz
Xrc.tgz
Xvfb.tgz
Xxserv.tgz
extract

Then I ran:


sudo sh Xinstall.sh

The prompts were intuitive and I installed all optional components but I told it not to create links for opengl or rstart.

I also added /usr/X11R6/bin ahead of other paths in the PATH variable and added /usr/X11R6/lib ahead of other paths in the LD_LIBRARY_PATH variable.

Configuring XFree86 (xf86config) took some trial-and-error but I eventually got it. The important settings are:

  • mouse protocol: 1. Auto
  • monitor: 2 31.5 - 35.1; Super VGA, 800x600 @ 56 Hz
  • vertical sync range: 4 40-150
  • yes, look at a card database
  • card: 0 * Generic VESA compatible
  • video memory: 3 1024K
  • color depth by default: 4 16 bits (65536 colors)

The color depth appears to have to be the same as the depth of the host display.

After configuration the mouse didn't work, but editing /etc/X11/XF86Config manually and commenting out the following line seemed to fix it:


#Option "Device" "/dev/mouse"

After all that I was able to get a simple X session running with startx.

Running openwin gave me an OPENLOOK session.

solaris 2.6 - 7. openwin

I was able to get a CDE session by creating an .xinitrc with:


/usr/dt/bin/Xsession

...and running startx.

solaris 2.6 - 8. cde

A graphical login was a little more difficult.

It turns out that the dtlogin program runs /usr/openwin/bin/Xsun directly, and passes it an option that the XFree86 server doesn't understand, so I replaced it with a script that removes the option:


#!/bin/sh

ARGS=`echo $@ | sed -e "s|-nobanner||g"`

/usr/X11R6/bin/X $ARGS

I'd had the same issue with Solaris 7 though, so the solution was familiar.

Enabling dtlogin was simple.


dtconfig -e

It didn't work perfectly though. As on Solaris 7, I could login and run a CDE session, but I had to use ctrl-backspace to exit the session and get back to the login.

Development

I had previously built and installed all the tools I usually use to build my software, so I gave it a try.

Rudiments required one modification. I'd added a hack at some point for FSU pthreads on SCO something-or-other, but then later disabled FSU pthreads support but forgot to remove the hack. The hack caused a problem on Solaris 2.6 and removing it solved it.

Everything else built cleanly and I was able to run the SQL Relay client software.

solaris 2.6 - 9. sqlrsh

I'd really like to get the SQL Relay server running on Solaris though.

I'm still looking for a version of Oracle or Sybase or DB2 or something that'll run on older Solaris x86. I remember seeing Oracle 8.0.5 on eBay once, but only once. I guess I should have jumped on it then.

I probably should try to build MySQL, PostgreSQL, Firebird and SQLite. I'm not sure how much work that would be but if I could get it going then it'd at least be something.

Eh, something to work on later.

Endearing Features

Solaris 2.6 has the standard set of Solaris quirks: you have to use delete for backspace on the console, home directories are created in /export/home, the default shell doesn't understand ~/, filesytem performance is not so good under VMware (untarring and rm -rf'ing take a while) and setting the domain in /etc/resolv.conf requires the "domainname" keyword rather than "domain".

2.6 also only supports 4g disks, which make sense, given its vintage - 1998. Unfortunately it also makes host cpu run at 100%. I can't say that was unexpected either though, as both 2.5.1 and 7 do the same.

So there you have it. Solaris 2.6 x86. About what I expected. Not too surprising. Not too disappointing.

I'll have to try to get more software running on it though - databases, maybe some old java. Yeah, that would be fun.

Saturday, October 11, 2014

Oops...

It appears that I forgot to run "make distclean" before building that last round of tar.gz and zip files for both Rudiments and SQL Relay. So, if you downloaded last week (the week of October 6th, 2014), or the week before, then you may have had trouble building. Configure probably ran, but make might not have made anything.


Who knows what platform the binaries that were in there ran on? Fedora 20 x64 maybe... Maybe not though.


Anyway...


I just updated the tar.gz and zip files with clean versions. I also downloaded and verified them this time. They ought to be correct.


Thanks Florin... If you hadn't pointed it out, I might never have noticed.

Saturday, October 4, 2014

SQL Relay - 0.57 is out

SQL Relay 0.57 is now available!


Windows Support

Support for Windows is much improved in this release. The command line clients all work now - sqlrsh, sqlr-export, sqlr-import, etc. The C/C++, C# and ADO.NET API's have worked for a while, but are now joined by native API's for Perl, Python, PHP and Java as well as drivers for Perl DBI, Python DB and PHP PDO.


It is possible to build all of this on Windows using MS Visual Studio or MS Visual Studio Express but it is currently quite difficult and requires specific versions of things to be installed in the right places. PHP is especially difficult to work with. In fact, I'm honestly not sure how 3rd party developers are expected to develop drivers for Windows. I had to shoehorn the headers in manually from the source code.


However... The SQL Relay Binary Distribution for Windows is now available (for a few bucks) and provides installers for binary versions of the above-mentioned software.


So... in addition to just accessing SQL Relay from command line clients, SQL Relay applications, including web-based applications, can now be developed for Windows in C, C++, C# (and other .NET languages), Perl, Python, PHP and Java.


Give it a try, let me know if you run into trouble.


Oracle "describe table" Improvements

In sqlrsh if you run "describe tablename" it returns info about the columns of the table - name, type, size, etc. This is accomplished by a call to the getColumnList() method of the C++ API. All SQL Relay API's support this method, and it has been improved.


For oracle tables, if a column is a primary, unique or foreign key, that information is now given. In previous releases it was not. This is a fairly expensive operation, as it turns out, so there is a config parameter to disable looking up keys. See disablekeylookup=yes/no in the oracle connection string in the SQL Relay Configuration Reference for more info.


Also, if you run "describe" on a synonym to a table in another schema, it will now return information about that table. In previous releases, only tables could be described.


Rudiments Improvements

The Rudiments library that SQL Relay depends on has been updated to randomize the list of hosts returned by DNS server when using round-robin DNS. If you are using round robin DNS to distribute SQL Relay clients over a pool of SQL Relay servers, then the behavior may different than in previous releases. In particular, if one host is down, then rather than slamming the next host in the list with all of the downed host's traffic, load will be distributed over the remaining hosts. This feature was added to work around an issue with many recent-ish implementations of getaddrinfo().


Rudiments has also been updated to support kqueue, epoll, port_create, /dev/poll, poll and select in its listener class. There should be no concerns now about large numbers of clients queuing up on the listener, or with listener performance with regard to using select internally.


See the rudiments release announcement for more information on these updates.


Bug Fixes and Minor Improvements

This release features a few bug fixes and minor improvements too.


The Python API's getRowDictionary method had a bug where None's were being returned as 0's when getNullsAsNones was set. That's fixed now.


Sensible errors are now returned when the format of a numeric bind variable is incorrect when using mysql, firebird or oracle. An error was fixed that could cause mysql connection to loop up if alphanumeric bind variable names were used instead of numbers. A bind-variable translation bug was fixed that could cause variables not to be translated when multiple formats are used in the same query and one of them is the correct format too.


The --with-system-libtool configure option was broken but it's fixed now.


Minix 3.3.0 is now supported. At least the client side is. Ie. you can access Oracle from Minix 3.3.0 using sqlrsh and SQL Relay running on another system. The server-side stuff doens't work yet. The Minix kernel doesn't support semaphores by default and Minix 3.3.0 ships without kernel source. I'm not sure if it supports semaphores at all though, so maybe that's moot.


The drop-in replacement library for MySQL had bugs with its implementations of mysql_row_seek() and mysql_row_tell(). Those are fixed now.


The configure script properly detects Maria DB on Ubuntu 14.04.1 now.



Give it a try and report any bugs that you find!

Friday, October 3, 2014

Rudiments - 0.48 is out

Rudiments 0.48 is now available!


This release has a few fairly important features.


Random Number Generation

The randomnumber class has been updated to support CryptGenRandom (on Windows), arc4random, random_r, rand_r, lrand48_r, random, rand and lrand48. It will use whichever of those it can find, in that order. The class has static and non-static members now. When the non-static members are used, the class can be seeded once and numbers (or scaled numbers) can be generated over and over without explicitly re-seeding.


Inet Socket Clients

Several improvements have been made to the inetsocketclient class.


Internally, inetsocketclient uses either getaddrinfo() or some variant of gethostbyname(), depending on what's available.


First, a bug was fixed that caused timeouts not to work on systems that don't have getaddrinfo().


Second, it seems that the behavior of getaddrinfo() has changed over the years and a workaround has been introduced to deal with some issues that the new behavior causes...


When round-robin DNS is used, gethostbyname() and getaddrinfo() fetch the entire list of IP addresses that the host name resolves to. The inetsocketclient's connect() method gets this list and tries to connect to each of these IP's, in the order that they were returned by gethostbyname()/getaddrinfo(). When round-robin DNS is used, the order of the IP's in the list is rotated with each DNS request. This can be used to implement a simple load-balancing scheme.


"Recent" (I'm not sure how recent) implementations of getaddrinfo() have begun sorting the IP's, defeating round-robin DNS entirely. Some implementations have a /etc/gai.conf file that allow a sortv4=no directive, but many do not. Some implementations randomize the list of IP's. Some implementations return them in the order that the DNS server returned them in.


Rudiments has no way of knowing what the behavior of getaddrinfo() is, so now, by default, the list of IP addresses is randomized. For the sake of consistency, it's randomized whether it uses getaddrinfo() internally or some variant of gethostbyname(). Methods have been added to the class to enable or disable this behavior too.


Randomization provides a benefit over round-robin DNS as well. With round-robin DNS, when a host goes down, the next host in the list will receive all of the downed host's traffic. With randomization, traffic is distributed evenly over the hosts that are still up.


listner class

The listener class has been refactored entirely to match the paradigms established by kqueue, epoll, /dev/poll, etc. It now supports kqueue, epoll, port_create, /dev/poll, poll and select internally as well, and will use whichever is available, in that order.


In the previous release, support was added for kqueue, epoll and poll but the implementation still did some inefficient things like rebuilding the list of file descriptors to listen on before each iteration. The list is only rebuilt if a file descriptor is added or removed now.


Minix 3.3.0

Minix 3.3.0 came out since the last release and it's now supported.


General Bug Fixes and Improvements

Some general bug fixes and improvements were implemented as well. Cygwin builds ought to work correctly now. Setting file permissions ought to work natively on Windows now too.