Thursday, March 10, 2016

DYLD_LIBRARY_PATH and El Capitan


With the release of Firebird 3 RC2 it is time to concentrate properly on the port of Firebird on MacOSX, Firebird 3 was buildable but we don't have a proper installer, ICU is no longer directly included and there
are a number of other bits and pieces that need to be addressed.

The last time I built Firebird 3 was before I upgraded to El Capitan. I did the upgrade to address some issues with the Firebird installer.

Lets just say when I started ny first build I wasn't expecting too many problems....

When we build Firebird on OSX we arbitrarily point to the ICU location (build_environment/firebird/lib), after copying over the files from the initial ICU build, the Firebird build then finds the ICU libraries dynamically (when needed) because DYLD_LIBRARY_PATH is sey to point to the location via prefix.darwin (make.platform).

While the ICU libraries are being built we set the final actual framework install names and locations of the ICU libraries (using -install_name) and they then placed in the appropriate directory of the framework
when the installer is packaged and built.

Every time I tried to build Firebird it would fall over when trying to build msg.fdb with ISQL.
The error was:
 "Could not find acceptable ICU library"
which seemed very strange as I know it was placed exactly where it should be and where DYLD_LIBRARY_PATH should be picking it up.

Setting DYLD_LIBRARY_PATH in my default terminal worked successfully, checking DYLD_PRINT_ENV seeemed to show the path being set etc., after much head scratching I eventually put the following line of code into unicode_util.cpp at the function:
UnicodeUtil::ConversionICU& UnicodeUtil::getConversionICU()

printf("DYLD_LIBRARY_PATH=%s\n", getenv("DYLD_LIBRARY_PATH"));

When I next ran the build, this is what was returned..

DYLD_LIBRARY_PATH=(null)

Something very strange was definitely going on. Some further research eventually turned up this thread on the
PostgreSQL forums

http://www.postgresql.org/message-id/561E73AB.8060800@gmx.net

"Apparently the behaviour is that DYLD_LIBRARY_PATH is prevented from being inherited into any system binaries. E.g. a shell. But specifying it inside a shell script will allow it to be inherited to children of
that shell, unless the child is a protected process in turn."

My reaction was the same as Tom Lane's but expressed in much stronger terms.

The simplest solution to the problem is to turn off System Integrity Protection

via Recovery Mode (Command-R), and then set csrutil disable in a terminal and reboot.

Note that this is a development only issue and will not affect anybody using the built
binaries when the port is complete.

Wednesday, October 7, 2015

How to install/upgrade Firebird manually on El Capitan


Background: The current Firebird automatic installer on El Capitan is no longer working, its format has been
finally deprecated. The installer is currently undergoing a complete re-write to comply with the Flat Package Format. Until this work is completed, you can manually install Firebird using the following method.

Classic

Download FirebirdCS-2.5.4-26856-x86_64.pkg.zip
Unzip it
cd FirebirdCS-2.5.4-26856-x86_64.pkg
cd Contents
gunzip Archive.pax
pax -r -f Archive.pax (or unpack the file using Finder)
as sudo
cp -r Firebird.framework /Library/Frameworks/Firebird.framework
cd Resources
./postinstall

If you are upgrading from a previous version use:
./preupgrade
and then
./postupgrade

SuperServer

Download FirebirdSS-2.5.4-26856-x86_64.pkg.zip
Then follow the same steps as above, and ignore any errror messages that reference
a, /Resources/doc/doc: No such file or directory
b, chmod /Library/StartupItems/Firebird/Firebird: No such file or directory
c, /Library/LaunchDemons/org.firebird.gds.plist: service is already loaded.

a ps -eaf should show both fbserver and fbguard running.

Monday, July 27, 2015

gbak -stat

From the Bug Tracker:

Vlad Khorsun commented on CORE-1999:
------------------------------------

gbak now has a new command-line switch

    -ST(ATISTICS) TDRW    show statistics:
        T                  time from start
        D                  delta time
        R                  page reads
        W                  page writes

Sample output:

firebird>gbak -v -stat tdrw -r o a.fbk a.fdb
gbak:opened file a.fbk
gbak: time     delta  reads  writes
gbak:    0.173  0.173      0      0 transportable backup -- data in XDR format
gbak:    0.175  0.002      0      0             backup file is compressed
gbak:    0.177  0.001      0      0 backup version is 10
gbak:    0.270  0.092      0    650 created database s:\Temp\A+.FDB, page_size 8192 bytes
gbak:    0.273  0.002      0      2 started transaction
gbak:    0.274  0.001      0      0 restoring domain RDB$29
gbak:    0.275  0.001      0      0 restoring domain RDB$12
...
gbak:   18.661  0.002      0      0 restoring data for table TEST1
gbak:   18.698  0.036      0      0    10000 records restored
gbak:   18.735  0.036      0      0    20000 records restored
...
gbak:   25.177  0.036      0      0    1770000 records restored
gbak:   25.220  0.042      0   1633    1780000 records restored
gbak:   25.256  0.036      0      0    1790000 records restored
...
gbak:   38.702  0.002      0      0     restoring privilege for user SYSDBA
gbak:   38.707  0.004     22      0 creating indexes
gbak:   45.015  6.308     82  38394     activating and creating deferred index T2_VAL
gbak:   45.132  0.116      3      9     activating and creating deferred index TEST_S_UNQ
gbak:   46.486  1.354      0  10775     activating and creating deferred index RDB$PRIMARY1
gbak:   46.566  0.079      0      9     activating and creating deferred index T1_IDX
gbak:   46.661  0.095      5     15 committing metadata
gbak:   46.665  0.003      8     10 fixing views dbkey length
gbak:   46.666  0.001      1     18 updating ownership of packages, procedures and tables
gbak:   46.671  0.005      0      0 adding missing privileges
gbak:   46.673  0.001      0      0 fixing system generators
gbak:   46.682  0.008      4     13 finishing, closing, and going home
gbak:   46.684  0.002    171  82442 total statistics
gbak:adjusting the ONLINE and FORCED WRITES flags

Monday, June 1, 2015

The semantics of isc_tpb_autocommit

From Vlad Horsun:

A simplified overview of the autocommit code:

When a transaction, marked as TRA_autocommit performs any of following actions, it is marked also as TRA_perform_autocommit

  • insert
  • update
  • delete
  • select with lock
  • post event
The TRA_perform_autocommit flag is checked when
  • the engine receives input message
  • the engine sends an output message
  • the engine starts to execute a request
  • the engine finishes executing a DDL request
When the TRA_perform_autocommit flag is detected, the engine runs on-commit triggers (not for DDL, that looks like a bug) and performs commit retaining. A new transaction will have TRA_autocommit flag set and TRA_perform_autocommit not set.

Wednesday, February 18, 2015

Yosemite, SuperServer, and StartupItems


There seems to be a small problem with Firebird on Yosemite (OSX 10.10), it looks like Apple have finally deprecated StartupItems, and currently SuperServer uses this to start itself on reboot.

Apple state:
"Startup Items
Deprecation Note: Startup items are a deprecated technology. Launching of daemons through this process may be removed or eliminated in a future release of OS X."

Which is obviously what has happened.

This is followed by

"Unless your software requires compatibility with OS X v10.3 or earlier, use the launchd facility instead."

Which is currently what Classic does.


If you install Firebird after upgrading to Yosemite, you can start the server manually by simply using

/Library/StartupItems/Firebird/Firebird start

However if Firebird was already installed, it looks as if the StartupItems folder is removed during the upgrade, but you should be able to start the server using the following....

/Library/Frameworks/Firebird.framework/Resources/bin/fbguard -daemon -forever

But if you want Firebird SuperServer to start automatically on reboot you could use the following org.firebird.gds.plist file in /Library/LaunchDaemons



and the command

launchctl load org.firebird.gds.plist

should now start Firebird immediately and also on reboot.

Update 2nd July 2015

I got an email from David Nock suggesting that the following plist would do a better job at managing SuperServer on Yosemite (10.10). Some brief tests suggest that he is right. I will be committing the following for 2.5.5, but I am documenting this here in case anybody wants to use this.



Wednesday, November 12, 2014

Services API extension - Nbackup Support

From docs/README.services_extension.

Services API extension - Nbackup support.
(Alex Peshkov, peshkoff@mail.ru, 2008)

Nbackup performs two logical groups of operations - locking and unlocking a database and backup and restoring it. It doesn't make much sense duplicating locking and unlocking in using services, because that functionality is present remotely in via the SQL language interface (ALTER DATABASE). But backup and restore must be run on localhost and the only way to access them is via nbackup utility.

Therefore expanding the services API to support this functionalty is useful.

The following actions were added:

isc_action_svc_nbak - incremental nbackup,
isc_action_svc_nrest - incremental database restore.

The following parameters were added:
isc_spb_nbk_level - backup level (integer),
isc_spb_nbk_file - backup file name (string),
isc_spb_nbk_no_triggers - do not run DB triggers (option).

Samples of use of new parameters in fbsvcmgr utility (supposing login and
password have been set using some other method):

 Create backup level 0:
  fbsvcmgr service_mgr action_nbak dbname employee nbk_file e.nb0 nbk_level 0
 Create backup level 1:
  fbsvcmgr service_mgr action_nbak dbname employee nbk_file e.nb1 nbk_level 1
 Restore database from this files:
  fbsvcmgr service_mgr action_nrest dbname e.fdb nbk_file e.nb0 nbk_file e.nb1

Thursday, September 26, 2013

An update about Firebird on iOS


Based on a comment I made in response to someone asking about the availability of Firebird on IOS (after someone mentioned that the recent release of Delphi XE5 has support for an embedded version of InterBase for iOS and Android), I thought I would write a few more details of the hows and wherefores here.

1. I can confirm that yes, I have managed to cross-compile Firebird for iOS on my Mac. But note, cross-compiling in Firebird 2.5 is difficult and not for the faint hearted (especially ICU). The ability to cross-compile is only really available in the Firebird 3.0 tree.

2. The current build I have created is a normal Mac framework, and it needs to be finessed into an embedded bundle using this script.

Firebird embedded on MacOSX

The script is available in svn in the B2_5_Release tree, located in the builds/install/arch-specific/darwin directory. This will take an existing new build of Firebird 2.5 and convert it into an appropriate bundle format. However the current code still has a few issues. The current (2.5.2) code expects Firebird to be created as a framework, so the config_root module for darwin doesn't have the necessary "smarts" to be able to work out where things are automatically depending on whether the install is a framework, or if its embedded. However new code committed recently to B2_5_Release does. The build I currently have doesn't use this new code and as such requires that the environment variable FIREBIRD must me set.

3. I had planned to rebuild the code using the new embedded support for locating various important files that embedded needs, when my Mac died. The Mac is now currently being repaired and having a new motherboard fitted. Cost 600.00 Euros. It seems this is common fault on Macs of this age. So any further work will have to wait.

4. To test the build currently, it seems you will need an iOS device (iPhone or iPad) running iOS 6.1+ as the code was compiled with the following switch -miphonos-version-min=6.1. You also need to be need to be registered as a developer at Apple, and you will need to get the relevant certificate ids from them for the devices that you will test or use. Firebird like InterBase will never be availble via the App Store as it relies on dynamically linked libraries. Only statically linked applications are allowed in the App Store (security issues).

5. I now have an iPad I can use to start testing (thanks to a very generous donation), but haven't yet signed up as an official Apple developer and logged my devices.

If anyone wants to help/assist in anyway, please feel free to contact me.

18th December 2013


An updated build of embedded Firebird for IOS can now be downloaded from

http://www.ibphoenix.com/downloads/FirebirdIOSembed.tar

There were some issues with loading dylibs, these have been corrected using the install_name_tool -change
command. The default embedded script I used to build the embedded package forgets that I had to cross compile ICU 5.1 rather than build ICU with Firebird.

isql/gbak (or any other firebired utility) can now find the relevant libs they need by searching the directory above them i.e. MacOS. There was however an issue if you used

symbolic links, symbolic links, extra ``/'' characters, and references to /./ and /../ in the file_name.

This has now been corrected using the realpath function.
I haven't tested this myself yet on IOS, so once again feedback would be appreciated.