Leopard 64bit OS and Firebird

For anybody who may even be remotely interested MacOS 10.5 (Leopard) is a 64 bit OS and allows you to build and run 64 bit binaries. The current Firebird build for MacOS 10.4 and MacOS 10.5 is 32bit. I now have a 64bit build for Leopard. When I get the chance I will explain exactly what was done. I think we will start offering 64bit builds for MacOS with Firebird 2.1 for those who want them.

O_DIRECT and MacOS

Mac's don't support the O_DIRECT flag for files but F_NOCACHE should do the trick....
For more details look at jrd/os/posix/unix.cpp and forced writes in the 2.1 cvs tree.

Preprocesser Macros

The following are the C preprocessor macros used for making various compile-time checks against MacOSX architectures - Firebird currently checks for the 32bit macros... I hope to try a 64bit intel build on Leopard in the not too distant future. This will mean defining yet another "new" platform.

__ppc__
__ppc64__
__i386__
__x86_64__

MACOSX_DEPLOYMENT_TARGET

I have read in a number of places that I should be using

gcc -mmacosx-version-min=10.4

rather than the

MACOSX_DEPLOYMENT_TARGET=10.4

environment variable because the latter has now been deprecated.

NO! Any attempt to compile Firebird on MacOS 10.5 and run it on MacOS 10.4 will create bus errors. You must set the MACOSX_DEPLOYMENT_TARGET variable!

(The Firebird build environment now uses both - just to make 100% certain)

MacOS Architectures Compile Flags

It is possible to define what architecture you wish to compile against on MacOS using the following C/C++ flag(s):

-arch ppc (ppc 32bit)
-arch ppc64 (ppc 64bit)
-arch i386 (intel 32bit)
-arch x86_64 (intel 64bit)

Its even possible to combine them to create "Universal Binaries" however the endian-ness of Firebird on ppc v's intel and the way that Firebird is currently built makes this somewhat impossible. Something for the future I think.

Trying To Debug Semaphores

We are trying to debug semaphore creation and usage on MacOS to get Signal Safe Semaphores implemented in SuperServer. But we are having some issues... (as per previous posts)
We have a gds__log print in semaphore.h to see whether the semaphore is created.

If we try and start the server automatically via
the StartupItem (Mac's method for starting Services)
we see:
sem=0x0
If we try and run ./fbserver from gdb or the command
line we see
sem=0x3
In the firebird.log file.

It looks like for darwin semaphore's handle is really a file descriptor - nothing more. When statring firebird as a service, there are no opened files passed to it, because it's supposed to be a daemon. Therefore the first opened file (or semaphore) is 0. When starting from the command line, stdin(0), stdout(1) and stderr(2) are already opened. Therefore - we get 3 which is the next free handle.

DTrace

MacOS 10.5 (Leopard) has replaced the kernel debugging tool KTrace with a new tool called DTrace.
KTrace was relatively simple to use, and although not probably the best tool around for looking at what the kernel is doing - it did give you information that you needed simply and easily.
For those of you who might even be marginally interested in DTrace.
Here's a link on MacTech.
And thats just the tip of the iceberg.

Great - something completely new to try and understand. Why couldn't Apple have left KTrace and its friends on the 10.5 system, so we could use it, while we got to grips with DTrace?

Looks like DTruss will do the trick....

Whats That Number?

For those who might not know, the number that you see after the table name in gstat for example (130) is the relation id. For those of you who might not have 100+ tables, you might ask why do you always see 3 digits. Well the answer is as follows:

Numbers 0-127 are reserved for system table relation ids. So 128, 129+, etc., in creation order are used for normal table/relation ids.

Analyzing database pages ...
ACC_CHARGES (130)
Primary pointer page: 264, Index root page: 265
Data pages: 1, data page slots: 1, average fill: 16%
Fill distribution:
0 - 19% = 1
20 - 39% = 0
40 - 59% = 0
60 - 79% = 0
80 - 99% = 0

Leopard V's Tiger

Looks like we are going to need separate download binaries for Firebird on Leopard and pre-Leopard (Tiger). Firebird 2.0.3 compiled on Leopard works perfectly on Leopard, but when you try to run and install Firebird that was built on Leopard on Tiger, all the utilities give a bus error, and you get the following type of debug trace from gdb:

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
0x302188f0 in Firebird::MemoryPool::allocate_nothrow ()
(gdb) bt
#0 0x302188f0 in Firebird::MemoryPool::allocate_nothrow ()
#1 0x3000e5d6 in gds__alloc_debug ()
#2 0x00012c8f in parse_arg ()
#3 0x00018d51 in ISQL_main ()
#4 0x000190bf in main ()

Supposedly a EXC_BAD_ACCESS/KERN_PROTECTION_FAILURE — is caused by the thread trying to write to read-only memory. This is always caused by a data access.

So why does Firebird 2.03 compiled on Tiger work perfectly? For some strange reason we smell a rat with either the Operating System or the compiler. But what the problem is - is a very good question. I am open to suggestions.

Using the link command:

-Wl,-macosx_version_min -Wl,10.4 has no effect.

Building Firebird 2.0x MacOS - semaphore.h

If you download the Firebird Source Code and want to build your own version of Firebird 2.0x on MacOS 10.4 (Tiger) you will probably have to make a minor change to the code. Seems there are changes in /usr/include/sys/semaphore.h between 10.4 and 10.5 to make semaphore.h more compliant.
We use (for Mac purposes) the following in src/common/classes/semaphore.h at line 106

if (sem == SEM_FAILED) {

Which seems obvious right?

Well on 10.4 the build will fail with the following error:

make -f ../gen/Makefile.libfbclient libfbclient
g++ -DSUPERCLIENT -D_THREAD_SAFE -I../src/include/gen -I../src/include -I../src/vulcan -I../extern/icu/source/common -I../extern/icu/source/i18n -DNAMESPACE=Vulcan -ggdb -DDARWIN -pipe -p -MMD -fPIC -fno-common -Wall -arch i386 -DDEV_BUILD -c ../src/jrd/alt.cpp -o ../temp/superclient/jrd/alt.o
../src/include/../jrd/../common/classes/semaphore.h: In constructor 'Firebird::SignalSafeSemaphore::SignalSafeSemaphore()':
../src/include/../jrd/../common/classes/semaphore.h:106: error: ISO C++ forbids comparison between pointer and integer
../src/jrd/../jrd/common.h: At global scope:

A quick check of /usr/include/sys/semaphore.h on 10.4 and 10.5 shows:

10.4 #define SEM_FAILED -1
10.5 #define SEM_FAILED ((sem_t *)-1)

Hmm - 10.5 looks more like what we would expect (same as linux for example)
A problem with types? Looks like it.

Change line 106 in src/common/classes/semaphore.h

from:
if (sem == SEM_FAILED) {
to:
if (sem == (sem_t*)SEM_FAILED) {

Or upgrade your /usr/include/sys/semaphore.h to something thats correct!

Semaphores With Timeouts

Firebird normally makes use of semaphores with timeouts, and I read on an Apple mailing list the following question posted in May 2004:

"I have a project that depends on semaphores with timeouts (sometimes
referred to as realtime extensions to semaphores). I use autoconf to
select either SysV semaphores or POSIX semaphores, but MacOS X does not
have a complete set of either one. Neither sem_timedwait nor semtimedop
are implemented. On top of that, the POSIX semaphore implementation
doesn't even have sem_init (though I did see the recent discussion about
it on this list, so I'm moderately hopeful for 10.4).
Are semtimedop or sem_timedwait implementations planned anytime soon?"

Well he answer is no. Leopard MacOS 10.5 doesn't solve the problem either. Hence a number of changes we have had to make to the Firebird code and build process to handle this. Look at the latest (being committed) configure.in and semaphore.h to see how we have had to try and work around what should not be a problem by using a poor man's semaphore based on a mutex and conditional variable.

Hint to the Apple developers - You really should fix this soon.

32bit or 64bit binaries on MacOS

I have just built a version of Firebird 2.03 Classic for MacOS using a new MacOS 10.5 (Leopard) system I got recently. My basic tests show everything working OK. So a copy of the build was sent to Philippe Makowski for QA. However after install he gets a bus error trying to run isql. He is running MacOS 10.4 (Tiger).

First thought as to what the problem might be was that Leopard is a 64bit OS whilst Tiger is 32bit and that gcc/ld might have built incompatible binaries for the two platforms by default.

Well - Hmm - heres the problem, how do you find out whether a binary is 64bit or not?
MacOS has no ldd. But it does have something called otool
file ./isql gives the following answer: ./isql: Mach-O executable i386
Without going into the ins and outs of how MacOS loads binaries etc there is no simple and easy way to find out for sure - even using otool...

The solution:
otool -h ./isql
./isql
mach header
magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
0xfeedface 7 3 0x00 2 16 1868 0x00000085

The key is the magic number, looking at /usr/include/mach-o/loader.h will give you the magic number for 32bit and 64bit binaries...
0xfeedface = 32bit
0xfeedfacf = 64bit

So we can see from the information above that we do have a 32 bit build. So thats not the problem.