Firebird /Java Plugin Milestone One

Posted by Adriano in Firebird Development:

"I'm pleased to announce availability [1] of the first test version of
FB/Java plugin.

FB/Java is a plugin for FB external engines feature, to be presented in
v3.0, that allows execution of Java code at the server side. The plugin
allows classes/resources in the file system or stored in the database.
File system classes/resources are global and the code run with all
permissions. Database classes/resources are per-database and code runs
accordingly with per-engine configured security policy.

The plugin has been made with the assumption that anything could be done
using existing FB infrastructure. You don't need to setup FTP server or
shared directory in the server to update your code or edit security police.

Installation/update of database JARs works from client command line
utility or from SQLJ package procedures. Actually, the same Java class
that run in the client is the one that runs in the server. You can put
JARs in the database using a Java URL, so lookup of them is not
restricted to client or server.

Security handling is configured in a central database, with a single
PERMISSION table. Permissions are granted to users that executes the
Java code. This way to configure permissions is not adequate for every
(or most) cases, and roles are not supported at the moment. I suppose
and desire we can have a better design in the future, before a final
release.

It's welcome any attempt to bypass the security system. But please,
report it. :-)

This version also lacks some notification mechanism for when the code
are going to be unloaded. You may note the problem in the example, right
after disconnect all attachments from the database the log file remains
locked, as it wasn't been garbaged collected. And no kind of "hot
redeployment" support had been implemented or tested. However, I have
ideas of how these problems can be solved in very good ways, based on
existing models. For unloading (or any other type of notification),
JPA-like listeners. For maintain state after redeployments, serializable
sessions.

In the file plugins\java-engine\examples\fbjava\README, you found
instructions for a "quick start" with it. In
plugins\java-engine\examples\tests there are functional tests for many
of its feature (i.e., how type mappings works, etc).

The plugin is based on Jaybird 2.2 (HEAD). I've put it in a maven
repository at [2], with its version defined as 2.2-jrt-SNAPSHOT.

The server includes my v3.0 changes, supporting packages, DDL triggers,
over clause and some internal changes. It may have others instabilities.

If you have a JAVA_HOME environment variable defined, it should just
works without any other configuration. The quick start assumes you had
defined FIREBIRD, ISC_USER and ISC_PASSWORD. In [1], I've zipped plugin
and engine together, but plugin installation consists in just unpacking
it in FB directory.

Documentation has room for improvements, and is at [3]. I've done it
with FB docbook tools.

Source code is in [4].

Happy testing and thanks.
Adriano"

[1] http://www.firebirdsql.org/downloads/rabbits/asfernandes/fbjava/fbjava-m1.zip
[win32]
[2] http://www.firebirdsql.org/downloads/rabbits/asfernandes/maven2
[3] http://www.firebirdsql.org/downloads/rabbits/asfernandes/fbjava/fbjava.pdf
(included in [1])
[4] https://firebird.svn.sourceforge.net/svnroot/firebird/java-plugin/tags/2.2-jrt-M1

IBPhoenix Web Site

I need to post a note on my blog about the IBPhoenix Web site, it seems we are having problems with the T1 line. It looks like there is a problem in the central phone office, where our T1 line originates. It's been a problem now for several days. The issue has been escalated, so hopefully things will improve shortly.

../src/include/types_pub.h:88: error: conflicting declaration 'typedef int ISC_LONG'

Building Firebird 64bit on Solaris (sparc or intel) can sometimes produce the following error:
-c ../src/jrd/dsc.cpp -o ../temp/boot/jrd/dsc.o
../src/include/types_pub.h:88: error: conflicting declaration 'typedef int ISC_LONG'
../src/include/fb_types.h:69: error: 'ISC_LONG' has a previous declaration as 'typedef SLONG ISC_LONG'
../src/include/types_pub.h:89: error: conflicting declaration 'typedef unsigned int ISC_ULONG'
../src/include/fb_types.h:70: error: 'ISC_ULONG' has a previous declaration as 'typedef ULONG ISC_ULONG

If you see this - then you made a mistake in the build process. Configure.in when you invoked it, assumed that the size of long was 4, and you are attempting to build Firebird for a 32bit environment. As per the prefix.solaris files, you must set the following before running configure:

export CXXFLAGS='-m64'
export CFLAGS='-m64'
export LDFLAGS='-m64'

Configure will now automtically realise that you want to build in a 64 bit environment and set the size of long to 8.

Some Positive Feedback for Firebird

This message appeared on the Firebird Development list, from Jens Zurawski

"I think from time to time, one should really send some positive feedback
to the people who are involved into the development of the firebird
DBMS. Thank you very much for your efforts to make this software this good!

Just before a week or so we eventually upgraded our production server
from linux kernel 2.4 to 2.6.x and the firebird engine from 2.0.x to
2.1.2. As some of you might remember, long time ago we had some big
troubles with kernel 2.6.x and firebird, because our system is under
very heavy contention and we stumbled over many freezes, so we were
stuck to the 2.4.x kernel. (In average there are 75-150 simultaneous
connections to one database, in peak times up to 300. The database is
more than 20GBs in size and is serving for a pool of approx. 7 different
applications, starting from web server (php/java) over some internal
logistics processings (java) up to the CRM front end (Win32 native), and
additionally there are frequent update tasks with long running
transactions (about 10-30 minutes) and massive changes to the database
records in them).

Now our old database server broke down (hardware problem) and we had to
buy a new one and this time we needed to go for a 2.6.x kernel because
the new hardware was not supported from the old kernel. To my pleasant
surprise, the actual FB 2.1.2 is working very well with a 2.6 kernel and
the heavy contention we still have on our database. Only thing one has
to do is to dramatically increase the semaphores. In the first place we
increased the LockSemCount to 250 (which was the limit of the default
kernel setup) but that wasn't enough. After half of the day firebird
crashed down with a "Fatal lock manager error: semop failed (acquire),
errno: 22", accepted no more new connections and also didn't serve the
existing connections any more. Now after we increased the kernel limit
and set the LockSemCount up to 1024, everything is working very well. No
more freezes, no more crashes. (BTW: ironically this hint I was getting
from a guy who just has collected some experiences with an Oracle
installation on Linux ;-)

So, what I want to say: Thanks a lot for all the good work! firebird is
really my most favourite open source project.

Ah, and the new monitoring tables in FB 2.1 are a gift from heaven! Now
I'm able to track down problems from applications which are not written
by me, because now I can simply monitor hanging transactions or long
running "sub-optimal" queries and send the developer a hint ;) Also
thanks a lot for this feature!"

64 bit Firebird 2.5, and Solaris Sparc

Looks like the job is already done... I read the following post on Firebird Development from Bill Oliver (SAS)

"I confirm that the issues are fixed in HEAD. I confirmed this on 64-bit
Solaris Sparc, built with the Sun Studio native compiler - using pthreads!!!

"How cool is that! Thanks very much guys for the quick turn-around.

Also, big thanks to Alex for his assitance with this port!!"

Excellent work!

GCC 4.01 MacOSX & Weak Symbols

During the QA process for Firebird V2.1.2 we found that there was a problem with creating a database remotely. In effect the inet server would crash.

On further analysis we found that we have a problem where the function free()(standard C dynamic memory deallocator) was being invoked with the incorrect parameter, an address which had not been allocated using malloc().

After setting up the code to make the inet server call abort we got the following stack trace

(fragment)

#0 0x90005efc in free ()
#1 0x010440b8 in Firebird::AbstractString::~AbstractString (this=0xbfffcb04)
at ../src/include/../common/classes/fb_string.h:375
#2 0x010440f8 Firebird::StringBase::~StringBase
(this=0xbfffcb04) at ../src/include/../common/classes/fb_string.h:391
#3 0x0101593c in LD_setup_attributes (textTypeName=0xbfffcd60 "ES_ES",
charSetName=0xbfffccdc "ISO8859_1",
configInfo=0x6a1204 "filename=$(this)/fbintl;icu_versions=default",
srcLen=39, src=0x69a898 "DISABLE-COMPRESSIONS=1;SPECIALS-FIRST=1",
dstLen=512, dst=0xbfffcdb8 "DISABLE-COMPRESSIONS=1;SPECIALS-FIRST=1,")
at ../src/intl/ld.cpp:457
#4 0x3011b1b4 in Jrd::IntlManager::setupCollationAttributes
(collationName=@0xbfffd0e4, charSetName=@0xbfffd0b8,
specificAttributes=@0xbfffd14c, newSpecificAttributes=@0xbfffd178)
at ../src/jrd/IntlManager.cpp:317

From this back trace you can see that free() is called from the operator delete[], where we would have expected MemoryPool::deallocate() to be used. For some
reason the default operator delete[] is used.

Strange, how did this code arrive at address 0x010440b8?

After an analysis of nm output for fbintl and the linker's map of symbols Alex found that at this address is in fact located the redefined operator delete[], which was compiled in via ld.cpp. However in the assembly version of the file (ld.cpp), it was correctly redefined (i.e. it was calling MemoryPool::deallocate()) operator delete[].

Houston it looks as if we have a buggy toolchain.

But we also need to take into account that AbstractString dtor() and operator delete[] are also WEAK symbols. Weak meaning that such symbols may be present in many object files, and the linker at link-time will take it from the relevant one. But in this case it looks as if it was taken from another file...

But after we compiled into assembler all of the files that are used in the fbintl
library we found that they all contained the correctly redefined operator
delete[]. As such there is no use of operator delete[], calling std free() function in our code, but anyway we get such use in the resulting binary.

In effect it looks like weak symbols are broken on MacOSX GCC 4.01 (Apple build 5747). And the only way to fix a problem is to avoid them.

We are currently compiling Firebird on PPC 10.4 (Tiger) using a new switch -fno-weak

fno-weak:
"Do not use weak symbol support, even if it is provided by the linker. By default, G++ will use weak symbols if they are available. This option exists only for testing, and should not be used by end-users; it will result in inferior code and has no benefits."

MallocLogFile, MacOSX & LaunchDaemons

I have been debugging the fb_inet_server trying to find a problem in Firebird 2.1.2 Classic for PowerPC. That takes a little bit of setting up, but I will try and explain that in another blog post. Part of one of the things that I wanted to check was what malloc was doing. However by default any "error" messages get sent to standard out, however you can set the environment variable MallocLogFile to capture messages to a specified file. But setting that in my normal environment is going to work because the fb_inet_server is actually launched by the LaunchDaemon and is then also forked. So we need to set the environment variable in the LaunchDaemon script directly.

The solution:

You need to edit the xml script that "launches" the fb_inet_server. This file can be found in /Library/LaunchDaemons and is called org.firebird.gds.plist

First of all unload the current launch definition

launchctl unload /Library/LaunchDaemons/org.firebird.gds.plist

Then edit the file as follows
After

<string>org.firebird.gds</string>

Add

<Key>EnvironmentVariables</Key>
<dict>
<Key>MallocLogFile</Key>
<String>/Library/Frameworks/Firebird.framework/Malloc.log</String>
</dict>

Save and

launchctl load /Library/LaunchDaemons/org.firebird.gds.plist