Thursday, September 20, 2012

Firebird Embedded on MacOSX


Based on a document written by Fulvio, a while ago, I finally spent time writing a make file that will automatically create an embedded version of Firebird that will run on MacOSX as a bundle. All you do is make a normal build of Firebird Classic and then use this make file to create the Firebird.app folder which is set up in a way that allows you to access a database via isql for example without the need for a full framework. The make file detailed below was written for Firebird 2.5.x

 (updated 15th April 2014)

file embed.darwin - committed to svn B2_5_Release/builds/install/arch-specific/darwin and also to trunk
Code was added to config_root.cpp to simulate binreloc (posix) on MacOSX. Find the name of the excutable in the directory below firebird, strip the executable name, strip the lowest directory, now use the remainder to find the configuration file, firebird.conf, now read the conf file to get the actual RootDirectory for Firebird.

embed.darwin updated below, to fix a couple of other issues.

To make a Classic version of Firebird for MacOSX:
./configure
make
cd gen
make -B -f Makefile.install

Then you can use the following make file to create an embedded version.
copy embed.darwin from builds/install/arch-specific/darwin

make -B -f embed.darwin

# Makefile script to generate an embedded Firebird bundle from an existing Framework

    FBE=../gen/firebird/Firebird.app
    BINLOC=../gen/firebird/Firebird.app/Contents/MacOS/firebird/bin
    LIBLOC=../gen/firebird/Firebird.app/Contents/MacOS/firebird
    INTLOC=../gen/firebird/Firebird.app/Contents/MacOS/firebird/intl
    OLDPATH=/Library/Frameworks/Firebird.framework/Versions/A/Libraries

all:
    -$(RM) -rf $(FBE) ../gen/firebird/Firebird.app
    mkdir -p $(FBE)/Contents
    mkdir -p $(FBE)/Contents/MacOS
    mkdir -p $(FBE)/Contents/MacOS/firebird
    mkdir -p $(FBE)/Contents/Resources
    mkdir -p $(FBE)/Contents/Frameworks
    mkdir -p $(FBE)/Contents/Plugins
    mkdir -p $(FBE)/Contents/SharedSupport
    mkdir -p $(FBE)/Contents/MacOS/firebird/bin
    mkdir -p $(FBE)/Contents/MacOS/firebird/intl

    cp ../gen/install/misc/firebird.conf $(FBE)/Contents/MacOS/firebird/firebird.conf
    cp ../gen/firebird/firebird.msg $(FBE)/Contents/MacOS/firebird/firebird.msg
    cp ../gen/firebird/lib/libfbembed.dylib $(FBE)/Contents/MacOS/firebird/libfbembed.dylib
    cp ../gen/firebird/lib/libicudata.dylib $(FBE)/Contents/MacOS/firebird/libicudata.dylib
    cp ../gen/firebird/lib/libicui18n.dylib $(FBE)/Contents/MacOS/firebird/libicui18n.dylib
    cp ../gen/firebird/lib/libicuuc.dylib $(FBE)/Contents/MacOS/firebird/libicuuc.dylib
    cp ../gen/firebird/lib/libib_util.dylib $(FBE)/Contents/MacOS/firebird/libib_util.dylib
    cp ../gen/firebird/security2.fdb $(FBE)/Contents/MacOS/firebird/security2.fdb
    cp ../gen/firebird/bin/gbak $(FBE)/Contents/MacOS/firebird/bin/gbak
    cp ../gen/firebird/bin/isql $(FBE)/Contents/MacOS/firebird/bin/isql
    cp ../builds/install/misc/fbintl.conf $(FBE)/Contents/MacOS/firebird/intl/fbintl.conf
    cp ../gen/firebird/intl/libfbintl.dylib $(FBE)/Contents/MacOS/firebird/intl/fbintl.dylib
    cp ../builds/install/arch-specific/darwin/embed.Info.plist $(FBE)/Contents/Info.plist

    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Firebird \
     ../libfbembed.dylib $(BINLOC)/isql
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicuuc.dylib \
    ../libicuuc.dylib $(BINLOC)/isql
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicudata.dylib \
    ../libicudata.dylib $(BINLOC)/isql
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicui18n.dylib \
    ../libicui18n.dylib $(BINLOC)/isql
  
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Firebird \
     ../libfbembed.dylib $(BINLOC)/gbak
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicuuc.dylib \
    ../libicuuc.dylib $(BINLOC)/gbak
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicudata.dylib \
    ../libicudata.dylib $(BINLOC)/gbak
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicui18n.dylib \
    ../libicui18n.dylib $(BINLOC)/gbak
  
    install_name_tool -change $(OLDPATH)/libicuuc.dylib @loader_path/libicuuc.dylib \
    $(LIBLOC)/libfbembed.dylib
    install_name_tool -change $(OLDPATH)/libicudata.dylib @loader_path/libicudata.dylib \
    $(LIBLOC)/libfbembed.dylib
    install_name_tool -change $(OLDPATH)/libicui18n.dylib @loader_path/libicui18n.dylib \
    $(LIBLOC)/libfbembed.dylib
    install_name_tool -change $(OLDPATH)/libicudata.dylib @loader_path/libicudata.dylib \
    $(LIBLOC)/libicuuc.dylib
    install_name_tool -change $(OLDPATH)/libicuuc.dylib @loader_path/libicuuc.dylib \
    $(LIBLOC)/libicui18n.dylib
    install_name_tool -change $(OLDPATH)/libicudata.dylib @loader_path/libicudata.dylib \
    $(LIBLOC)/libicui18n.dylib

    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicuuc.dylib \
        @loader_path/../libicuuc.dylib $(INTLOC)/fbintl.dylib
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicudata.dylib \
        @loader_path/../libicudata.dylib $(INTLOC)/fbintl.dylib
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicui18n.dylib \
        @loader_path/../libicui18n.dylib $(INTLOC)/fbintl.dylib

    install_name_tool -id @rpath/libfbembed.dylib $(LIBLOC)/libfbembed.dylib
    install_name_tool -id @rpath/libicudata.dylib $(LIBLOC)/libicudata.dylib
    install_name_tool -id @rpath/libicui18n.dylib $(LIBLOC)/libicui18n.dylib
    install_name_tool -id @rpath/libicudata.dylib $(LIBLOC)/libicudata.dylib
    install_name_tool -id @rpath/libicuuc.dylib $(LIBLOC)/libicuuc.dylib
    install_name_tool -id @rpath/libicudata.dylib $(LIBLOC)/libicudata.dylib
    install_name_tool -id @rpath/libib_util.dylib $(LIBLOC)/libib_util.dylib


You can now tar a copy of the Firebird.app directory (and underlying), place anywhere on your system,
and set the RootDirectory in the firebird.conf (17th July 2013 - this now works, previously it didn't), or set the FIREBIRD environment variable to the relevant location of the firebird directory in Firebird.app, If your application is under launchctl, you can set the FIREBIRD variable in the Info.plist file provided and your application should run if placed in the same directory. Unfortunately OS X Mavericks does not use the environment.plist, so you need to set the environment variable using launchctl a simple script like the following should do the trick.

 (setup.command)
DIR=$(cd $(dirname "$0"); pwd)
launchctl setenv FIREBIRD $DIR/Firebird.app/Contents/MacOS/firebird
echo setenv FIREBIRD $DIR/Firebird.app/Contents/MacOS/firebird | sudo tee
/etc/launchd.conf

Should we produce a dedicated MacOSX embedded build along with the others (32bit/64bit/lipo)?

The (Firebird.app) can be downloaded from www.ibphoenix.com/downloads/firebirdApp.zip should anybody want it.

Friday, September 7, 2012

CORE-3740 and Firebird V2.5.2


CORE-3740 - SELECT using IN list with 153 or more elements causes crash.
This problem only occurs on MacOS, and cannot be reproduced on Linux or Windows. Initial analysis showed that adjusting the optimisation level of the code as it links (O1 instead of O3) would increase the number of INS that could be supported but you would still get a crash eventually if the number of INS was large enough.
The consensus of opinion was that this was a stack issue, so we increased the stack from the default 8mb in launchctl to 64mb, our test still crashed.

We then tried embedded (local) firebird and increased the cache there using ulimit, strangely enough this worked. So it definitely was a stack issue but where? Some debugging code and some careful googling showed the problem. Even Classic launches a forked inet_server via a new thread.

From an Apple Technical Q&A
"Each Mac OS X process is launched with a default stack size of 8 Megabytes. This allocation is used exclusively for the main thread's stack needs. Each subsequent thread created is allocated its own default stack, the size of which differs depending on the threading API used. For example, the Mac OS X implementation of Pthreads defines a default stack size of 512 Kilobytes, while Carbon MPTasks are created with a 4 Kilobyte stack."

Patch applied to src/jrd/ThreadStart.cpp

-#ifdef _AIX
-// adjust stack size for AIX
+#if defined(_AIX) || defined(DARWIN)
+// adjust stack size

 // For AIX 32-bit compiled applications, the default stacksize is 96 KB,
 // see . For 64-bit compiled applications, the default stacksize
 // is 192 KB. This is too small - see HP-UX note above

+// For MacOS default stack is 512 KB, which is also too small in 2012.
+
     size_t stack_size;
     state = pthread_attr_getstacksize(&pattr, &stack_size);
     if (state)
         Firebird::system_call_failed::raise("pthread_attr_getstacksize");

-    if (stack_size < 0x40000L)
+    if (stack_size < 0x400000L)
     {
-        state = pthread_attr_setstacksize(&pattr, 0x40000L);
+        state = pthread_attr_setstacksize(&pattr, 0x400000L);
         if (state)
             Firebird::system_call_failed::raise("pthread_attr_setstacksize", state);

Monday, June 18, 2012

An ODBC driver for Firebird on MacOSX


I was recently contacted by a Firebird user on MacOSX who was trying to get the Firebird ODBC driver to build on MacOSX. I thought I would try and step in to help.

For those of you who follow the CVS checkins for OdbcJdbc you might have noticed the addition of a number of a new directory in the Builds directory called Gcc.darwin containing a makefile, a readme and a .sh file to create a lipo'ed dylib. The driver sucessfully builds and passes some simple tests. Feel free to contact me for a copy of the dylib to test further.

The readme contains the following information:

These instructions should allow a user to get a working user DSN for ODBC to connect
to Firebird on MacOS. Comments please to Paul Beach (pbeach at ibphoenix.com.

To build the library, edit makefile.darwin, and select the ARCH (i386 or X86_64).
then make -B -f makefile.darwin all

The Firebird ODBC library can be built in both 64bit or 32bit format.
lipo.sh creates a fat libary that can be used on either version of Firebird.

1. Download the MacOSX ODBC Administrator from support.apple.com
http://support.apple.com/downloads/ODBC_Administrator_Tool_for_Mac_OS_X
and install.

2. Once installed it can be accessed via Applications/Utilities/ODBC Administrator

3. Place the libOdbcFB.dylib in $HOME/odbc for example then add the Firebird driver
(dylib) name and location to the Drivers tab in the Administrator.
e.g
Description: Firebird ODBC Driver
Driver file: /Users/username/odbc/libOdbcFB.dylib
Define as: User
This will create an odbcinst.ini file in $HOME/Library/ODBC

4. Now you need to create a User DSN via an odbc.ini file.
Use the text below as an example, copy and paste into an odbc.ini
file placed in $HOME/Library/ODBC
Make sure that you modify the text so it points to your database
and uses your username and password.


[ODBC Data Sources]
Test = Firebird

[Test]
Driver               = /Users/username/odbc/libOdbcFb.dylib
Description          = Test Firebird ODBC
Dbname               = localhost:/Users/databases/test.fdb
Client               =
User                 = SYSDBA
Password             = masterkey
Role                 =
CharacterSet         = NONE
ReadOnly             = No
NoWait               = No
Dialect              = 3
QuotedIdentifier     = Yes
SensitiveIdentifier  = No
AutoQuotedIdentifier = No

[ODBC]
Trace         = 0
TraceAutoStop = 0
TraceFile     =
TraceLibrary  =

This User DSN should appear in the User DSN tab the next time you load the
ODBC Administrator.

5. You can test whether it works using iodbctest and then using the dsn
dsn=Test, if all is well it should connect and you can issue SQL statements.

To create a System wide version of the ODBC driver, copy the libOdbcFB.dylib
to /usr/lib make sure the Administrators Drivers tab now points to this file.

Copy the DSN above to /Library/ODBC and modify.

Note: (13th June 2012)
The ODBC library is linked to libfbclient.dylib found in the Firebird framework
Libraries directory. Not all SuperServer builds of Firebird have this library installed
by default. If this is the case get a copy of the Firebird Classic build and extract
the libfbclient library and place it in
/Library/Frameworks/Firebird.framework/Versions/A/Libraries

Note: (30th Nov 2015)
Since Mavericks (10.9) Apple no longer ship the default odbc header files as part of their SDK, so you need to get the files from an older SDK e.g. /Developer/SDKs/MacOSX10.7.sdk/usr/include or download the files you need from http://www.iodbc.org/

Wednesday, February 22, 2012

Firebird V2.0.7


We are currently preparing to release Firebird 2.0.7, since I take responsibility for the Mac builds, I did a 2.0.7 build on MacOSX 10.7 using the development tools installed by XCode 4.1 (gcc 4.2.1 etc). I set up the relevant environment variables for this older 32bit only build CFLAGS, CXXFLAGS, LD_FLAGS and also set the MACOSX_DEPLOYMENT_TARGET=10.4. The builds completed without any problems, some simple tests on MacOSX 10.7 showed no problems.

Now - Imagine my surprise when Philippe told me that when he tried to QA the builds on MacOSX 10.5 we got this error on SuperServer startup.

Process: fbserver [623]
Path:
/Library/Frameworks/Firebird.framework/Resources/English.lproj/var/bin/fbserver
Identifier: fbserver
Version: ??? (???)
Code Type: X86 (Native)
Parent Process: fbguard [310]

Date/Time: 2012-02-09 09:42:13.939 +0100
OS Version: Mac OS X 10.5.8 (9L31a)
Report Version: 6
Anonymous UUID: FA7F8C0C-581B-4153-ADBE-2BCB59C5F823

Exception Type: EXC_BREAKPOINT (SIGTRAP)
Exception Codes: 0x0000000000000002, 0x0000000000000000
Crashed Thread: 0

Dyld Error Message:
Symbol not found: ___moddi3
Referenced from:
/Library/Frameworks/Firebird.framework/Resources/English.lproj/var/bin/fbserver
Expected in: /usr/lib/libSystem.B.dylib

A simple nm and grep shows the following for the 10.5 libSystem.B.dylib

nm /usr/lib/libSystem.B.dylib | grep moddi3
00083490 t ___moddi3
0003df70 t ___umoddi3

On 10.5 ___moddi3 is defined in libgcc_s.1.dylib which links to libsystem.B.dylib

From what I can gather these *moddi3 symbols are routines for doing 64bit maths on 32bit systems.

However the same on 10.7 returns nothing, however the symbols can be found in libgcc_s.10.5.dylib which does not link to libgcc_s.1.dylib hence the missing symbols message.

Supposedly this can be fixed by linking explicity to to libgcc_s.10.5.dylib
using -lgcc_s.10.5. I got round it by booting up my copy of Snow Leopard (10.6) and
building Firebird 2.0.7 there instead of on MacOSX 10.7

So basically if you want to buld an application that is deployable on 10.5 or less, you can't - unless you know how to get around the above problem. Its as if although you can build 32bit applications for 32bit versions of MacOSX, they won't run, because they are incompatible with any version of the OS less than 10.6.