Firebird Embedded in a sandboxed MacOS App

For those who might not be aware, Firebird on MacOS is now relocatable, in that you don't necessarily have to install it as a Framework, this also means that you can create an embedded version out of the current installer.

A typical structure would look like this

|-- firebird.conf
|-- firebird.msg
|-- intl
| |-- fbintl.conf
| `-- libfbintl.dylib
|-- lib
| |-- libfbclient.dylib
| |-- libib_util.dylib
| |-- libicudata.dylib
| |-- libicui18n.dylib
| |-- libicuuc.dylib
|`-- plugins
    `-- libEngine12.dylib

And the firebird.conf file would then typically be amended for the following

Providers = Engine12
ServerMode = Classic
 
For the last few weeks Alex and I (along with a Firebird user) have been working on getting Firebird embedded to work properly in a sandboxed app that can then be deployed on the App Store...
 
To do this we had to solve issues with temp files, the use of inter process communications by the
Firebird lock manager and the location of the Firebird log file.

Note: adding LSEnvironment to the plist file defining new locations for FIREBIRD_TEMP and FIREBIRD_LOCK does not work.

<key>LSEnvironment</key>
<dict>
    <key>ENV_VAR</key>
    <string>value</string>
</dict>
 
So the first issue to be addressed was the following

macosx    Wed Dec  2 15:33:57 2020
    ConfigStorage: Cannot initialize the shared memory region
    Can not access lock files directory /tmp/firebird.tmp.YDzrhQ

It seems thats sandboxed Apps cannot to write into /tmp at all, they have to use their own /tmp-folder. We can detect whether we are running in a sandboxed environment using the following calls to the Mac security subsystem

task = SecTaskCreateFromSelf(nil),        
value = SecTaskCopyValueForEntitlement(task, "com.apple.security.app-sandbox" as CFString, nil),

Now if we now know if we are sandboxed we can use  NSTemporaryDirectory() for the location of the relevant Firebird temporary files.

Having fixed that issue, it was time to move onto the lock manager.
 
[FireDAC][Phys][FB]lock manager error.)
 
macosx    Thu Dec 17 17:37:00 2020
    event_init()
     operating system directive semctl failed
     Operation not permitted
 
According to the Apple sandbox guide restricting IPC (Inter Process Communication) is also part of MacOS sandbox implementation. After some serious head scratching we tried the following test
 
#include <stdio.h>
#include <pthread.h>
 
#define ER(x) { int tmpState = (x); if (tmpState) { printf("Failed %s error %d\n", #x, tmpState); } }
 
int main() {
 
    pthread_mutex_t mutex;
     pthread_mutexattr_t mutexattr;
 
     ER(pthread_mutexattr_init(&mutexattr));
     ER(pthread_mutexattr_setpshared(&mutexattr, PTHREAD_PROCESS_SHARED));
     ER(pthread_mutex_init(&mutex, &mutexattr));
     ER(pthread_mutex_lock(&mutex));
     ER(pthread_mutex_unlock(&mutex));
     ER(pthread_mutex_destroy(&mutex));
 
c++ test.cpp -pthread
 
In case of failure it will print error messages, on success we should get nothing. We got nothing.

For many years Firebird worked with system V IPC on MacOS which is different to many other *nixes where mutexes and conditional variables in shared memory were used instead. The use of system V IPC was caused by a lack of  for shared mutexes on MacOS when Firebird was originally ported to it. Currently (as was proved by test above) MacOS now supports such mutexes. Based on this it we decided to stop using system V IPC in Firebird and a perform a cleanup of the lock manager code as part of the effort to provide sandbox support. As a side effect Firebird should now run faster using shared mutexes.
 
At the same time as we worked on the lock manager issue we also had an issue with the firebird log file, we were not allowed to write to its normal default location by the sandbox. Because we now know we are running in a sandbox we can relocate the placement of the log file within utils.cpp
 
Supposedly using something like ~/Library/Application Support/Firebird/ should work
 
case Firebird::IConfigManager::DIR_LOG:
             s = "~/Library/Application Support/Firebird/";
             s += name;
             return s;
 
Unfortunately it seems this workaround was removed relatively recently forcing us to resort to putting the log file within the App itself ~/LibraryContainers/Company.App/Data/Library/Application Support/Company/
 
Changing the location of the log file is relatively easy but  although this allowed the App to run and resolved the issue, it was not an ideal solution since the location of the log file is now hard coded within Firebird and is App dependent and this would mean that every time somebody wanted an embedded Firebird for a sandboxed App they would have to change the path to the log file and recompile Firebird.
 
The solution was to do away with the Firebird log file and use the OSLog framework instead, and use this to capture any messages from the database engine, so if you are running embedded Firebird in a sandboxed App you can access the Firebird log messages using the Console or a terminal command like the following
 
log show --predicate 'eventMessage contains "macpro.home"' --start '2021-01-06 14:00:00' --end '2021-01-06 14:30:00' --info
 
where macpro.home is the name of the computer. You don't need the start and end, but it does help reduce the number of messages.
 
There is an added plus from using this. The OSLog framework is also supported on IOS 10+ and now that the App Store will accept dynamic libraries, this means that we should be able to compile an embedded version of Firebird for IOS that can also run sandboxed Apps. If anyone is interested in sponsoring this work please contact me.  

This work was sponsored by kiC Gesellschaft für Softwareentwicklung mbH.