Обсуждение: Leaky Perl / DBIx / Postgres 9.0.1 Trio
I have a Perl daemon handling some events on a system which inserts or updates rows in Postgres, and something is causing memory usage to grow. I see the RSS memory size allocated to the Postgres connection continual going up (slowly) and the RSS size allocated to the Perl daemon also continually going up (slowly). The Perl daemon is using DBIx to make insert and update queries to the Postgres DB.
There is only one connection open between the Perl daemon and the Postgres DB. That connection stays open until the daemon exits or is killed. When the daemon is killed, the Postgres back end is terminated and all of its memory is released.
Any suggestions on how to go about troubleshooting this would be useful. Most specifically, I would like to know how to find out what the Postgres process for that connection is doing with all that memory. Is there a way I can see what the memory is allocated for? Would running GDB on the Postgres process help? Is there any logging I can turn on which might help shed some light on what's using the memory? Anything in particular anyone knows about which I should look for which might cause both Perl and Postgres memory usage to climb continually?
Thanks in advance for any suggestions.
Eliot Gable
"We do not inherit the Earth from our ancestors: we borrow it from our children." ~David Brower
"I decided the words were too conservative for me. We're not borrowing from our children, we're stealing from them--and it's not even considered to be a crime." ~David Brower
"Esse oportet ut vivas, non vivere ut edas." (Thou shouldst eat to live; not live to eat.) ~Marcus Tullius Cicero
Hi, On 04/04/12 19:21, Eliot Gable wrote: > I have a Perl daemon handling some events on a system which inserts or > updates rows in Postgres, and something is causing memory usage to > grow. I see the RSS memory size allocated to the Postgres connection > continual going up (slowly) and the RSS size allocated to the Perl > daemon also continually going up (slowly). The Perl daemon is using > DBIx to make insert and update queries to the Postgres DB. I believe DBIx is more of a namespace than a specific module - do you mean DBIx::Class or DBI perhaps? If the former, the IRC channel #dbix-class on irc.perl.org tends to be very helpful in tracing problems like this, particularly if you can provide a reproducible test case. > There is only one connection open between the Perl daemon and the > Postgres DB. That connection stays open until the daemon exits or is > killed. When the daemon is killed, the Postgres back end is terminated > and all of its memory is released. > > Any suggestions on how to go about troubleshooting this would be > useful. Most specifically, I would like to know how to find out what > the Postgres process for that connection is doing with all that > memory. Is there a way I can see what the memory is allocated for? > Would running GDB on the Postgres process help? Is there any logging I > can turn on which might help shed some light on what's using the > memory? Anything in particular anyone knows about which I should look > for which might cause both Perl and Postgres memory usage to climb > continually? On the Perl side, Devel::Size and possible Test::MemoryGrowth may be of some use in tracing where the memory is going. Does the memory usage hit a ceiling at any point, or does it carry on until all virtual memory is exhausted? You could try running the postgres and perl daemons with lower ulimit -m / ulimit -v settings if the increase in memory is too slow to test this in a reasonable timeframe. Other than that, maybe try looking for statement handles that aren't closed or long-running transactions? best regards, Tom
On Wed, Apr 4, 2012 at 2:29 PM, Tom Molesworth <tom@audioboundary.com> wrote:
-- I believe DBIx is more of a namespace than a specific module - do you mean DBIx::Class or DBI perhaps? If the former, the IRC channel #dbix-class on irc.perl.org tends to be very helpful in tracing problems like this, particularly if you can provide a reproducible test case.
It is using DBIx::Class. I will check on the IRC channel, too; thanks.
On the Perl side, Devel::Size and possible Test::MemoryGrowth may be of some use in tracing where the memory is going. Does the memory usage hit a ceiling at any point, or does it carry on until all virtual memory is exhausted? You could try running the postgres and perl daemons with lower ulimit -m / ulimit -v settings if the increase in memory is too slow to test this in a reasonable timeframe.
The memory usage eventually exhausts all virtual memory on the system.
Other than that, maybe try looking for statement handles that aren't closed or long-running transactions?
I checked for long running transactions, but did not find any. Is there a way in Postgres to check for unclosed statement handles? If such a method exists, it would be far easier than looking through the DBIx::Class code for unclosed statement handles.
Thanks for your suggestions.
Eliot Gable
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 > I have a Perl daemon handling some events on a system > which inserts or updates rows in Postgres, and something > is causing memory usage to grow. The most likely culprit is the part of the chain you did not list in the subject, DBD::Pg. Do you know what version of it you are running? There have been various memory leak problems fixed over the years. The latest is 2.19.2. Your first solution should be to upgrade that if needed. - -- Greg Sabino Mullane greg@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201204041558 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iEYEAREDAAYFAk98qdYACgkQvJuQZxSWSshRZwCg9Nk045WbJuYv6hGeHXHQq+ZG haoAoMml2Nkn5GL1sMhMD1ofklVKYBC6 =UmNz -----END PGP SIGNATURE-----