Thursday, September 20, 2012

Firebird Embedded on MacOSX


Based on a document written by Fulvio, a while ago, I finally spent time writing a make file that will automatically create an embedded version of Firebird that will run on MacOSX as a bundle. All you do is make a normal build of Firebird Classic and then use this make file to create the Firebird.app folder which is set up in a way that allows you to access a database via isql for example without the need for a full framework. The make file detailed below was written for Firebird 2.5.x

 (updated 15th April 2014)

file embed.darwin - committed to svn B2_5_Release/builds/install/arch-specific/darwin and also to trunk
Code was added to config_root.cpp to simulate binreloc (posix) on MacOSX. Find the name of the excutable in the directory below firebird, strip the executable name, strip the lowest directory, now use the remainder to find the configuration file, firebird.conf, now read the conf file to get the actual RootDirectory for Firebird.

embed.darwin updated below, to fix a couple of other issues.

To make a Classic version of Firebird for MacOSX:
./configure
make
cd gen
make -B -f Makefile.install

Then you can use the following make file to create an embedded version.
copy embed.darwin from builds/install/arch-specific/darwin

make -B -f embed.darwin

# Makefile script to generate an embedded Firebird bundle from an existing Framework

    FBE=../gen/firebird/Firebird.app
    BINLOC=../gen/firebird/Firebird.app/Contents/MacOS/firebird/bin
    LIBLOC=../gen/firebird/Firebird.app/Contents/MacOS/firebird
    INTLOC=../gen/firebird/Firebird.app/Contents/MacOS/firebird/intl
    OLDPATH=/Library/Frameworks/Firebird.framework/Versions/A/Libraries

all:
    -$(RM) -rf $(FBE) ../gen/firebird/Firebird.app
    mkdir -p $(FBE)/Contents
    mkdir -p $(FBE)/Contents/MacOS
    mkdir -p $(FBE)/Contents/MacOS/firebird
    mkdir -p $(FBE)/Contents/Resources
    mkdir -p $(FBE)/Contents/Frameworks
    mkdir -p $(FBE)/Contents/Plugins
    mkdir -p $(FBE)/Contents/SharedSupport
    mkdir -p $(FBE)/Contents/MacOS/firebird/bin
    mkdir -p $(FBE)/Contents/MacOS/firebird/intl

    cp ../gen/install/misc/firebird.conf $(FBE)/Contents/MacOS/firebird/firebird.conf
    cp ../gen/firebird/firebird.msg $(FBE)/Contents/MacOS/firebird/firebird.msg
    cp ../gen/firebird/lib/libfbembed.dylib $(FBE)/Contents/MacOS/firebird/libfbembed.dylib
    cp ../gen/firebird/lib/libicudata.dylib $(FBE)/Contents/MacOS/firebird/libicudata.dylib
    cp ../gen/firebird/lib/libicui18n.dylib $(FBE)/Contents/MacOS/firebird/libicui18n.dylib
    cp ../gen/firebird/lib/libicuuc.dylib $(FBE)/Contents/MacOS/firebird/libicuuc.dylib
    cp ../gen/firebird/lib/libib_util.dylib $(FBE)/Contents/MacOS/firebird/libib_util.dylib
    cp ../gen/firebird/security2.fdb $(FBE)/Contents/MacOS/firebird/security2.fdb
    cp ../gen/firebird/bin/gbak $(FBE)/Contents/MacOS/firebird/bin/gbak
    cp ../gen/firebird/bin/isql $(FBE)/Contents/MacOS/firebird/bin/isql
    cp ../builds/install/misc/fbintl.conf $(FBE)/Contents/MacOS/firebird/intl/fbintl.conf
    cp ../gen/firebird/intl/libfbintl.dylib $(FBE)/Contents/MacOS/firebird/intl/fbintl.dylib
    cp ../builds/install/arch-specific/darwin/embed.Info.plist $(FBE)/Contents/Info.plist

    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Firebird \
     ../libfbembed.dylib $(BINLOC)/isql
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicuuc.dylib \
    ../libicuuc.dylib $(BINLOC)/isql
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicudata.dylib \
    ../libicudata.dylib $(BINLOC)/isql
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicui18n.dylib \
    ../libicui18n.dylib $(BINLOC)/isql
  
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Firebird \
     ../libfbembed.dylib $(BINLOC)/gbak
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicuuc.dylib \
    ../libicuuc.dylib $(BINLOC)/gbak
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicudata.dylib \
    ../libicudata.dylib $(BINLOC)/gbak
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicui18n.dylib \
    ../libicui18n.dylib $(BINLOC)/gbak
  
    install_name_tool -change $(OLDPATH)/libicuuc.dylib @loader_path/libicuuc.dylib \
    $(LIBLOC)/libfbembed.dylib
    install_name_tool -change $(OLDPATH)/libicudata.dylib @loader_path/libicudata.dylib \
    $(LIBLOC)/libfbembed.dylib
    install_name_tool -change $(OLDPATH)/libicui18n.dylib @loader_path/libicui18n.dylib \
    $(LIBLOC)/libfbembed.dylib
    install_name_tool -change $(OLDPATH)/libicudata.dylib @loader_path/libicudata.dylib \
    $(LIBLOC)/libicuuc.dylib
    install_name_tool -change $(OLDPATH)/libicuuc.dylib @loader_path/libicuuc.dylib \
    $(LIBLOC)/libicui18n.dylib
    install_name_tool -change $(OLDPATH)/libicudata.dylib @loader_path/libicudata.dylib \
    $(LIBLOC)/libicui18n.dylib

    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicuuc.dylib \
        @loader_path/../libicuuc.dylib $(INTLOC)/fbintl.dylib
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicudata.dylib \
        @loader_path/../libicudata.dylib $(INTLOC)/fbintl.dylib
    install_name_tool -change /Library/Frameworks/Firebird.framework/Versions/A/Libraries/libicui18n.dylib \
        @loader_path/../libicui18n.dylib $(INTLOC)/fbintl.dylib

    install_name_tool -id @rpath/libfbembed.dylib $(LIBLOC)/libfbembed.dylib
    install_name_tool -id @rpath/libicudata.dylib $(LIBLOC)/libicudata.dylib
    install_name_tool -id @rpath/libicui18n.dylib $(LIBLOC)/libicui18n.dylib
    install_name_tool -id @rpath/libicudata.dylib $(LIBLOC)/libicudata.dylib
    install_name_tool -id @rpath/libicuuc.dylib $(LIBLOC)/libicuuc.dylib
    install_name_tool -id @rpath/libicudata.dylib $(LIBLOC)/libicudata.dylib
    install_name_tool -id @rpath/libib_util.dylib $(LIBLOC)/libib_util.dylib


You can now tar a copy of the Firebird.app directory (and underlying), place anywhere on your system,
and set the RootDirectory in the firebird.conf (17th July 2013 - this now works, previously it didn't), or set the FIREBIRD environment variable to the relevant location of the firebird directory in Firebird.app, If your application is under launchctl, you can set the FIREBIRD variable in the Info.plist file provided and your application should run if placed in the same directory. Unfortunately OS X Mavericks does not use the environment.plist, so you need to set the environment variable using launchctl a simple script like the following should do the trick.

 (setup.command)
DIR=$(cd $(dirname "$0"); pwd)
launchctl setenv FIREBIRD $DIR/Firebird.app/Contents/MacOS/firebird
echo setenv FIREBIRD $DIR/Firebird.app/Contents/MacOS/firebird | sudo tee
/etc/launchd.conf

Should we produce a dedicated MacOSX embedded build along with the others (32bit/64bit/lipo)?

The (Firebird.app) can be downloaded from www.ibphoenix.com/downloads/firebirdApp.zip should anybody want it.

Friday, September 7, 2012

CORE-3740 and Firebird V2.5.2


CORE-3740 - SELECT using IN list with 153 or more elements causes crash.
This problem only occurs on MacOS, and cannot be reproduced on Linux or Windows. Initial analysis showed that adjusting the optimisation level of the code as it links (O1 instead of O3) would increase the number of INS that could be supported but you would still get a crash eventually if the number of INS was large enough.
The consensus of opinion was that this was a stack issue, so we increased the stack from the default 8mb in launchctl to 64mb, our test still crashed.

We then tried embedded (local) firebird and increased the cache there using ulimit, strangely enough this worked. So it definitely was a stack issue but where? Some debugging code and some careful googling showed the problem. Even Classic launches a forked inet_server via a new thread.

From an Apple Technical Q&A
"Each Mac OS X process is launched with a default stack size of 8 Megabytes. This allocation is used exclusively for the main thread's stack needs. Each subsequent thread created is allocated its own default stack, the size of which differs depending on the threading API used. For example, the Mac OS X implementation of Pthreads defines a default stack size of 512 Kilobytes, while Carbon MPTasks are created with a 4 Kilobyte stack."

Patch applied to src/jrd/ThreadStart.cpp

-#ifdef _AIX
-// adjust stack size for AIX
+#if defined(_AIX) || defined(DARWIN)
+// adjust stack size

 // For AIX 32-bit compiled applications, the default stacksize is 96 KB,
 // see . For 64-bit compiled applications, the default stacksize
 // is 192 KB. This is too small - see HP-UX note above

+// For MacOS default stack is 512 KB, which is also too small in 2012.
+
     size_t stack_size;
     state = pthread_attr_getstacksize(&pattr, &stack_size);
     if (state)
         Firebird::system_call_failed::raise("pthread_attr_getstacksize");

-    if (stack_size < 0x40000L)
+    if (stack_size < 0x400000L)
     {
-        state = pthread_attr_setstacksize(&pattr, 0x40000L);
+        state = pthread_attr_setstacksize(&pattr, 0x400000L);
         if (state)
             Firebird::system_call_failed::raise("pthread_attr_setstacksize", state);