Обсуждение: Integrating C++ singletons into postgresql extensions???

Hi,

I've been writing C++ code that needs to get wrapped into a postgresql
extension and it has a few singleton classes like:

1. Config object that holds some general configuration information.
2. Stats object for global stats collection.
3. Logger object for enabling/disabling debug logging to a file.
4. curlpp class for making outbound calls to an OSRM web service from
the c++ code

I can probably do without 1-3 if I had to when the code is integrated,
but they are handy to keep around in the code for when we are developing
and working outside the postgresql database. I could #ifdef then all,
but I would like to avoid cluttering the code with tons of #ifdef.

Item 4 is a little more problematic and needs to be kept around or
something else implemented to take its place. I have looked at the curl
extension and actual built another module in C that uses those ideas.,
but the curlpp is a nice OO class that encapsules and hides all the curl
stuff from the C++ code.

So my question(s) are:

1. Any insights into singletons working/not working within the
postgresql server extensions? My understanding is that these are
implemented as global static classes with a lifetime of the process and
they have no destructor, so I'm a little worried about memory leaks or
multiple queries sharing the singleton.

2. Is there another way implement the equivalent of a singleton for C++
code that has a lifetime of the query in the server?

3. What do for logging classes when integrating C++ code into postgresql?

Any help or ideas would be appreciated.

Thanks,
   -Steve

On 2014-10-17 19:59:54 -0400, Stephen Woodbridge wrote:
> Hi,
>
> I've been writing C++ code that needs to get wrapped into a postgresql
> extension and it has a few singleton classes like:
>
> 1. Config object that holds some general configuration information.
> 2. Stats object for global stats collection.
> 3. Logger object for enabling/disabling debug logging to a file.
> 4. curlpp class for making outbound calls to an OSRM web service from the
> c++ code
>
> I can probably do without 1-3 if I had to when the code is integrated, but
> they are handy to keep around in the code for when we are developing and
> working outside the postgresql database. I could #ifdef then all, but I
> would like to avoid cluttering the code with tons of #ifdef.
>
> Item 4 is a little more problematic and needs to be kept around or something
> else implemented to take its place. I have looked at the curl extension and
> actual built another module in C that uses those ideas., but the curlpp is a
> nice OO class that encapsules and hides all the curl stuff from the C++
> code.
>
> So my question(s) are:
>
> 1. Any insights into singletons working/not working within the postgresql
> server extensions? My understanding is that these are implemented as global
> static classes with a lifetime of the process and they have no destructor,
> so I'm a little worried about memory leaks or multiple queries sharing the
> singleton.

There's no general problem that I can see. Postgres uses processes, not
threads, for individual connections. So any process level leak doesn't
persist for longer than the connection. And if you create the singleton
inside the individual backend, instead of in the postmaster (the parent
server) they won't be shared either.

> 2. Is there another way implement the equivalent of a singleton for C++ code
> that has a lifetime of the query in the server?

Hm. Now it needs to be query lifetime, not connection lifetime? That's
slightly harder. If that's what you need probably the best way to go
about it would be to register a callback with
RegisterResourceReleaseCallback().

> 3. What do for logging classes when integrating C++ code into postgresql?

I don't really know what you want to do here. My guess is that it'd be
most appropriate to map the logging onto postgres' internal
logging. That should easily be possible.


The most important thing to be aware of here is that you need to take
*great* care that exception, *NEVER* permeate through postgres
code. Doing so will cause bad things.

Greetings,

Andres Freund

--
 Andres Freund                       http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

On 10/18/2014 5:38 AM, Andres Freund wrote:
> On 2014-10-17 19:59:54 -0400, Stephen Woodbridge wrote:
>> Hi,
>>
>> I've been writing C++ code that needs to get wrapped into a postgresql
>> extension and it has a few singleton classes like:
>>
>> 1. Config object that holds some general configuration information.
>> 2. Stats object for global stats collection.
>> 3. Logger object for enabling/disabling debug logging to a file.
>> 4. curlpp class for making outbound calls to an OSRM web service from the
>> c++ code
>>
>> I can probably do without 1-3 if I had to when the code is integrated, but
>> they are handy to keep around in the code for when we are developing and
>> working outside the postgresql database. I could #ifdef then all, but I
>> would like to avoid cluttering the code with tons of #ifdef.
>>
>> Item 4 is a little more problematic and needs to be kept around or something
>> else implemented to take its place. I have looked at the curl extension and
>> actual built another module in C that uses those ideas., but the curlpp is a
>> nice OO class that encapsules and hides all the curl stuff from the C++
>> code.
>>
>> So my question(s) are:
>>
>> 1. Any insights into singletons working/not working within the postgresql
>> server extensions? My understanding is that these are implemented as global
>> static classes with a lifetime of the process and they have no destructor,
>> so I'm a little worried about memory leaks or multiple queries sharing the
>> singleton.
>
> There's no general problem that I can see. Postgres uses processes, not
> threads, for individual connections. So any process level leak doesn't
> persist for longer than the connection. And if you create the singleton
> inside the individual backend, instead of in the postmaster (the parent
> server) they won't be shared either.
>
>> 2. Is there another way implement the equivalent of a singleton for C++ code
>> that has a lifetime of the query in the server?
>
> Hm. Now it needs to be query lifetime, not connection lifetime? That's
> slightly harder. If that's what you need probably the best way to go
> about it would be to register a callback with
> RegisterResourceReleaseCallback().

First, thank you for the very helpful answers. I'll think about this
some more, I might be able to get away with connection level handling,
but it is good to know what my options are.

>> 3. What do for logging classes when integrating C++ code into postgresql?
>
> I don't really know what you want to do here. My guess is that it'd be
> most appropriate to map the logging onto postgres' internal
> logging. That should easily be possible.

Ahh, sorry this is just debug info that we write to stdout when we are
testing and debug the code from the command line. There are some
packages like log4cpp and log4cxx that provide objects that can be
easily configure to change the output stream from stdout to a log file
or turn it off completely. The benefit of a logging class like this is
that you don't have to #ifdef all these statements in the code and you
can move your code from one environment to another with nothing more
than changing a parameter or two.

> The most important thing to be aware of here is that you need to take
> *great* care that exception, *NEVER* permeate through postgres
> code. Doing so will cause bad things.

Yes, our C code does all the SPI and interface with postgresql and then
calls a C++ wrapper that has a try {} catch() block to guard against
this and that has been working well.

Thanks!

Best regards,
   -Steve

> Greetings,
>
> Andres Freund
>

Stephen Woodbridge <woodbri@swoodbridge.com> writes:
> On 10/18/2014 5:38 AM, Andres Freund wrote:
>>> 3. What do for logging classes when integrating C++ code into postgresql?

>> I don't really know what you want to do here. My guess is that it'd be
>> most appropriate to map the logging onto postgres' internal
>> logging. That should easily be possible.

> Ahh, sorry this is just debug info that we write to stdout when we are
> testing and debug the code from the command line. There are some
> packages like log4cpp and log4cxx that provide objects that can be
> easily configure to change the output stream from stdout to a log file
> or turn it off completely.

FWIW, if you are using the logging collector (highly recommended), output
to a backend process's stdout or stderr will be caught and included in the
log, though it won't have a log_line_prefix.  This might be a usable
substitute for adapting your code.

            regards, tom lane

Tom Lane-2 wrote
> FWIW, if you are using the logging collector (highly recommended), output
> to a backend process's stdout or stderr will be caught and included in the
> log, though it won't have a log_line_prefix.  This might be a usable
> substitute for adapting your code.
>
>             regards, tom lane
>
>
> --
> Sent via pgsql-general mailing list (

> pgsql-general@

> )
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general

How do you use the logging collector on c++ code?

#include "utils/elog.h"
at linking time:
undefined reference to `elog_start(char const*, int, char const*)'

Vicky




--
View this message in context:
http://postgresql.nabble.com/Integrating-C-singletons-into-postgresql-extensions-tp5823506p5830010.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.