Обсуждение: Is the linking with -lodbc necessary? (--with-odbc)

Hello all!

Long story short: Is there a need to link psqlodbcw.so plugin against
libodbc.so?  Principal problem: That library provides ABI for applications
- not for plugins; at least it seem to be like that.

I tried to remove this linking by following tweak:

 | diff --git a/configure.ac b/configure.ac
 | index 5fb401e..5d75db3 100644
 | --- a/configure.ac
 | +++ b/configure.ac
 | @@ -65,7 +65,7 @@ fi
 |  # ODBC include and library
 |  #
 |
 | -if test "$ODBC_CONFIG" != ""; then
 | +if false; then
 |         if test "$with_iodbc" != no; then
 |                 ODBC_INCLUDE=`${ODBC_CONFIG} --cflags`
 |                 CPPFLAGS="$CPPFLAGS ${ODBC_INCLUDE}"

And both unixODBC and psqlodbcw.so communicates with each other correctly
at the first sight.

---------------------------------------------------------------------------

And now the long story.

The real problem comes now for 'isql' from unixODBC (and most probably
also for others) on s390x architecture in Linux.

As you know, unixODBC (== libodbc.so) uses lt_dlopen() (which just wraps
dlopen() in described scenario) for opening plugins.  And also, the 'isql'
tool is linked against libodbc.so library.

And now.  Lets look at one example API function now, e.g.
SQLAllocConnect() (but the same problem may be with any other API
function).  This function is called directly from 'isql' somewhere
- and on that place, there desired to call the SQLAllocConnect() function
provided by libodbc.so.

The 'isql' loads plugins through libodbc.so.
The psqlodbcw.so plugin does not provide SQLAllocConnect().  But during
loading of any plugin, libodbc.so tries to search for possible existence
of that function inside the plugin.  According to documented dlsym()
behavior - if the SQLAllocConntect() function is not defined in the
plugin, it tries to look more deeply in plugin's dependencies..  and it is
successful ~> because one of the dependencies is: "libodbc.so".  But
because libodbc.so don't wan't to use that "internal" method - only if the
method was defined in plugin directly, unixODBC guys prepared very ugly
workaround for that situation - they try to "blacklist" addresses of
internal methods into template_func[] array for later comparison with
return of lt_dlsym() output.

Now we have the problem: That template_func array is initialized (at
least) on s390x differently than on other platforms - the
template_func[0].dm_func is *CORRECTLY* initialized to the "internal"
SQLAllocConntect() function address.  But note!  It is different address
than also *CORRECT* address  returned from lt_dlsym() call later - so
detecting based on template_func[] is not successful (actually, the
function is located in memory at least twice).  You can look into unixODBC
into the file DriverManager/SQLConnect.c:

 582 /*
 583  * structure to contain the loaded lib entry points
 584  */
 585
 586 static struct driver_func  template_func[] =
 587 {
 588     /* 00 */ { SQL_API_SQLALLOCCONNECT,      "SQLAllocConnect", (void*)SQLAllocConnect },

That part of template_func[0] is initialized to the same SQLAllocConnect
address as it is in 'isql'.  Actually, if there was _no_ direct call to
SQLAllocConnect in 'isql' code, linker on s390 would assign to that place
the expected address (the same which would be later returned by dlsym()).

----

I don't know yet whether the linker behavior is correct or not (whether
such linker behavioral expectations are ok), I'll observe it in next few
days and file a bug against s390x linker.  Also, I'll try to ask for
better unixODBC api later, probably.

But firstly asking here:  is that linkage against libodbc.so needed?  I
can see that the blacklisting ugly hack in unixODBC is there just because
of plugins which are linked back against unixODBC.

Pavel

Hi Pavel,

(2013/11/12 2:50), Pavel Raiskup wrote:
> Hello all!
>
> Long story short: Is there a need to link psqlodbcw.so plugin against
> libodbc.so?  Principal problem: That library provides ABI for applications
> - not for plugins; at least it seem to be like that.
>
> I tried to remove this linking by following tweak:
>
>   | diff --git a/configure.ac b/configure.ac
>   | index 5fb401e..5d75db3 100644
>   | --- a/configure.ac
>   | +++ b/configure.ac
>   | @@ -65,7 +65,7 @@ fi
>   |  # ODBC include and library
>   |  #
>   |
>   | -if test "$ODBC_CONFIG" != ""; then
>   | +if false; then
>   |         if test "$with_iodbc" != no; then
>   |                 ODBC_INCLUDE=`${ODBC_CONFIG} --cflags`
>   |                 CPPFLAGS="$CPPFLAGS ${ODBC_INCLUDE}"
>
> And both unixODBC and psqlodbcw.so communicates with each other correctly
> at the first sight.

Seems -lodbcinst is really necessary not -lodbc.
Could you please try the attached patch for the current git?

regards,
Hiroshi Inoue

> Seems -lodbcinst is really necessary not -lodbc.
> Could you please try the attached patch for the current git?

Hi, thanks a lot for looking at it!  The patch works (I have lost access
to s390x for a moment, but testing on that arch is not needed iiuic).
That means hand testing via 'isql' works OK on x86_64.  Also ldd is fine:

   $ ldd ./.libs/psqlodbcw.so | grep libodbc
   libodbcinst.so.2 => /lib64/libodbcinst.so.2 (0x00007fbf039b3000)

But looking at configure.ac - we should probably fix the iodbc case..  but
I'm not sure about that.

Just a little update to this issue:  I tried other architectures I am able
to test and the problematic architectures seem to be at least armv7hl,
s390x, s390.

Thanks, Pavel

> But looking at configure.ac - we should probably fix the iodbc case..  but
> I'm not sure about that.

Please scratch that ^^ sentence.  I have looked badly at the code - your
patch should fix also iodbc case - so it is OK from my POV, thanks.
(Tested also on s390x).

[+cc back psqlodbc]
[+cc unixODBC]

On Tuesday, November 12, 2013 11:46:39 Nick Gorham wrote:
> On 11/11/13 17:50, Pavel Raiskup wrote:
> > Hello all!
> >
> > Long story short: Is there a need to link psqlodbcw.so plugin against
> > libodbc.so?  Principal problem: That library provides ABI for applications
> > - not for plugins; at least it seem to be like that.
> >
> > I tried to remove this linking by following tweak:
>
> Hi,
>
> As you have noticed, there is code in the DM to try and avoid what you
> describe. But AFAIK, a driver would normally link against -lodbcinst to
> gain access to the ini functions SQLGetPrivateProfileString and so on,
> no need to link to the driver manager (-lodbc) that is for application
> land use.

Hello Nick, thanks for looking at the problem from unixODBC perspective!

Would not there be better rather block loading badly linked plugins rather
than just try to expect that the linking is done the same way on all arches?
I attached possible solution.

Pavel

> I attached possible solution.

I removed too much code in the previous patch.  Attached once again.

On 13/11/13 10:41, Pavel Raiskup wrote:
> [+cc back psqlodbc]
> [+cc unixODBC]
>
> On Tuesday, November 12, 2013 11:46:39 Nick Gorham wrote:
>> On 11/11/13 17:50, Pavel Raiskup wrote:
>>> Hello all!
>>>
>>> Long story short: Is there a need to link psqlodbcw.so plugin against
>>> libodbc.so?  Principal problem: That library provides ABI for applications
>>> - not for plugins; at least it seem to be like that.
>>>
>>> I tried to remove this linking by following tweak:
>> Hi,
>>
>> As you have noticed, there is code in the DM to try and avoid what you
>> describe. But AFAIK, a driver would normally link against -lodbcinst to
>> gain access to the ini functions SQLGetPrivateProfileString and so on,
>> no need to link to the driver manager (-lodbc) that is for application
>> land use.
> Hello Nick, thanks for looking at the problem from unixODBC perspective!
>
> Would not there be better rather block loading badly linked plugins rather
> than just try to expect that the linking is done the same way on all arches?
> I attached possible solution.
>
> Pavel
Yes, I see your solution. Possible though that the driver is linked
against libiodbc.so or other non unixODBC version of ODBC driver manager
and so has entry point for SQLAllocStmt (for example) that is not in the
driver (as in this case).

There are also situations I have seen where a driver decides to call
itself (say SQLAllocStmt() in the driver makes a call to
SQLAllocHandle() expecting the driver version) but finds the driver
manager entry point.

Anyway you have the good solution for your driver in linking against
libodbcinst

--
Nick Gorham

On 13/11/13 10:41, Pavel Raiskup wrote:
> [+cc back psqlodbc]
> [+cc unixODBC]
>
> On Tuesday, November 12, 2013 11:46:39 Nick Gorham wrote:
>> On 11/11/13 17:50, Pavel Raiskup wrote:
>>> Hello all!
>>>
>>> Long story short: Is there a need to link psqlodbcw.so plugin against
>>> libodbc.so?  Principal problem: That library provides ABI for applications
>>> - not for plugins; at least it seem to be like that.
>>>
>>> I tried to remove this linking by following tweak:
>> Hi,
>>
>> As you have noticed, there is code in the DM to try and avoid what you
>> describe. But AFAIK, a driver would normally link against -lodbcinst to
>> gain access to the ini functions SQLGetPrivateProfileString and so on,
>> no need to link to the driver manager (-lodbc) that is for application
>> land use.
> Hello Nick, thanks for looking at the problem from unixODBC perspective!
>
> Would not there be better rather block loading badly linked plugins rather
> than just try to expect that the linking is done the same way on all arches?
> I attached possible solution.
>
> Pavel
Yes, I see your solution. Possible though that the driver is linked
against libiodbc.so or other non unixODBC version of ODBC driver manager
and so has entry point for SQLAllocStmt (for example) that is not in the
driver (as in this case).

There are also situations I have seen where a driver decides to call
itself (say SQLAllocStmt() in the driver makes a call to
SQLAllocHandle() expecting the driver version) but finds the driver
manager entry point.

Anyway you have the good solution for your driver in linking against
libodbcinst

--
Nick Gorham