We had a crash this morning of our server ... not the machine, just the
postmaster processes ... all three of them spread across three seperate
ports ...
Just looking through logs now, I'm finding:
FATAL: s_lock(20048065) at spin.c:127, stuck spinlock. Aborting.
In one of them ...
I've got a load of core files all from about the same time, from the
various processes running on teh different ports:
714266 9936 -rw------- 1 pgsql pgsql 5074944 Aug 23 10:32
./data/base/petpostings/postgres.core
944391 9984 -rw------- 1 pgsql pgsql 5099520 Aug 23 10:32
./data/base/trends_acctng/postgres.core
1015889 9984 -rw------- 1 pgsql pgsql 5099520 Aug 23 10:33
./data/base/pg_banners/postgres.core
1055605 10768 -rw------- 1 pgsql pgsql 5505024 Aug 23 10:34
./data/base/rockwell/postgres.core
904800 10032 -rw------- 1 pgsql pgsql 5124096 Aug 23 10:32 ./data/base/area902/postgres.core
619085 74720 -rw------- 1 pgsql pgsql 38219776 Jun 7 21:09
./data/base/thtphone/postgres.core
1944360 9936 -rw------- 1 pgsql pgsql 5074944 Aug 23 10:32 ./data/base/counter/postgres.core
896891 9808 -rw------- 1 pgsql pgsql 5009408 Aug 23 10:32
./data/base/hub_traf_stats/postgres.core
1849088 20656 -rw------- 1 pgsql pgsql 10567680 Aug 23 09:56 ./data/base/horde/postgres.core
849311 7136 -rw------- 1 pgsql pgsql 3645440 Aug 23 12:36
./special/mukesh/base/archies/postgres.core
857377 7104 -rw------- 1 pgsql pgsql 3629056 Aug 23 12:36
./special/mukesh/base/water/postgres.core8009 44176 -rw------- 1 pgsql pgsql 22589440 Aug 23
10:39./data2/udmsearch/postgres.core
if any of those help any?
looking in /var/log/messages, I'm seeing the following just before the
crashes:
Aug 23 12:33:47 pgsql syslogd: /dev/console: Too many open files in system: Too many open files in system
Aug 23 12:33:47 pgsql syslogd: /var/run/utmp: Too many open files in system
Aug 23 12:33:47 pgsql /kernel: file: table is full
would this be the cause? if so, I have to raise some limits, no problem
there, but just want to confirm ...
thanks ...
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org