Обсуждение: conn = PQconnectdb(conninfo);

Peter Kroon <plakroon@gmail.com> writes:
> Every now and then my program will abort.
> IAnd this is because: conn = PQconnectdb(conninfo);
> The error given:
> *** Error in `./server_prog': malloc(): smallbin double linked list
> corrupted: 0x092c10a0 ***

This looks like malloc() complaining because something has corrupted its
bookkeeping data, which generally means that something previously wrote
past the end of a malloc'd data chunk, or tried to write into an
already-freed chunk.  The odds are very high that the bug is in your
program rather than libpq, though, because no such problems have been
found within libpq recently.

valgrind is a fairly useful tool for tracking down such issues.

            regards, tom lane

Peter Kroon wrote:
> I've found perhaps a bug.
> I've narrowed down my code and the problem is indeed at: conn = PQconnectdb(conninfo);
> 
> My connection string: host=192.168.178.12 dbname=DATABASE user=foo password=bar
> 
> When I remove key/value host=xxx then everything is OK. Valgrind mentions: no leaks are possible.
> 
> When key/value host=xxx is added, not everything is freed and there are tons of bytes still reachable.
> 
> 
> ==9195==
> ==9195== HEAP SUMMARY:
> ==9195==     in use at exit: 450,080 bytes in 2,829 blocks
> ==9195==   total heap usage: 9,476 allocs, 6,647 frees, 7,810,733 bytes allocated
> ==9195==
> ==9195== LEAK SUMMARY:
> ==9195==    definitely lost: 0 bytes in 0 blocks
> ==9195==    indirectly lost: 0 bytes in 0 blocks
> ==9195==      possibly lost: 0 bytes in 0 blocks
> ==9195==    still reachable: 450,080 bytes in 2,829 blocks
> ==9195==         suppressed: 0 bytes in 0 blocks
> ==9195== Rerun with --leak-check=full to see details of leaked memory
> ==9195==
> ==9195== For counts of detected and suppressed errors, rerun with: -v
> ==9195== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 14 from 6)
> 
> 
> 
> The network address is the machine's address where I was testing on. So I could also have used
> localhost or 127.0.0.1 but this gave me the same result when using the network address.
> 
> 
> Played with hostaddr as well and gave me the same result.
> 
> 
> http://www.postgresql.org/docs/9.4/static/libpq-connect.html#LIBPQ-PQCONNECTDB
> http://www.postgresql.org/docs/9.4/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS
> "PostgreSQL 9.4.4 on x86_64-unknown-linux-gnu, compiled by gcc (Debian 4.7.2-5) 4.7.2, 64-bit"
> 
> 
> The attachment is the program I've used for testing.

You should run valgrind with --leak-check=full to see details.

I tried your program and I get "still reachable" only when SSL is enabled; all the memory is in
OpenSSL.  Dou you use SSL?

Without SSL (sslmode=disable) I get no "still reachable" memory.

I don't know of reachable memory is a problem, I'd suspect not.

Yours,
Laurenz Albe

Peter Kroon wrote:
>>> I've found perhaps a bug.
>>> I've narrowed down my code and the problem is indeed at: conn = PQconnectdb(conninfo);
>>>
>>> My connection string: host=192.168.178.12 dbname=DATABASE user=foo password=bar
>>>
>>> When I remove key/value host=xxx then everything is OK. Valgrind mentions: no leaks are possible.
>>>
>>> When key/value host=xxx is added, not everything is freed and there are tons of bytes still reachable.
>>>
>>>
>>> ==9195==
>>> ==9195== HEAP SUMMARY:
>>> ==9195==     in use at exit: 450,080 bytes in 2,829 blocks
>>> ==9195==   total heap usage: 9,476 allocs, 6,647 frees, 7,810,733 bytes allocated
>>> ==9195==
>>> ==9195== LEAK SUMMARY:
>>> ==9195==    definitely lost: 0 bytes in 0 blocks
>>> ==9195==    indirectly lost: 0 bytes in 0 blocks
>>> ==9195==      possibly lost: 0 bytes in 0 blocks
>>> ==9195==    still reachable: 450,080 bytes in 2,829 blocks
>>> ==9195==         suppressed: 0 bytes in 0 blocks
>>> ==9195== Rerun with --leak-check=full to see details of leaked memory
>>> ==9195==
>>> ==9195== For counts of detected and suppressed errors, rerun with: -v
>>> ==9195== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 14 from 6)

>> I tried your program and I get "still reachable" only when SSL is enabled; all the memory is in
>> OpenSSL.  Dou you use SSL?
>> 
>> Without SSL (sslmode=disable) I get no "still reachable" memory.
>> 
>> I don't know of reachable memory is a problem, I'd suspect not.

> I'm not using ssl at the moment.
> 
> The thing is when the reachable leak is there is just grows. It is not constant. And at a given point
> the program will break.
> 
> I've ran the program with: valgrind --leak-check=full --show-reachable=yes --log-
> file="_pg_test_debug.log" ./_pg_test_debug
> 
> View logfile here: http://pastebin.com/7rjBRbkD
> SSL is mentioned in combination with pg objects
> 
> Your suggestion: sslmode=disable seems to have fix my issue..

I looked into this some more, an this is expected behaviour.

If you do not use sslmode=disable or sslmode=allow, PostgreSQL will first try to
establish an SSL connection.  This requires that the SSL library be initialized
(a call to OPENSSL_config()).

This is done only once and will allocate some memory that will never be deallocated.
That should not be a problem, and the memory leak should not increase if more
than one connection is opened.
Since your code starts several threads, I believe than there is a slim chance that
due to race conditions, the memory is allocated more than once.

If you want more control over that, you can explicitly initialize and destroy
this memory, see http://www.postgresql.org/docs/9.4/static/libpq-ssl.html#LIBPQ-SSL-INITIALIZE

Sample code (untested) would look like that:

#include <openssl/ssl.h>

[...]

/* initialize SSL library */
OPENSSL_config();

/* tell PostgreSQL about it */
PQinitOpenSSL(0, 1);

[start threads, open database connections, do some database work, close connections]

/* free SSL memory */
CONF_modules_free();

Yours,
Laurenz Albe