Обсуждение: Postgresql Crasching
Hi all, We are experiencing the follow error in last few weeks: Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited on signal 11 (core dumped) Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-1] 2019-01-25 16:03:51.135 -02 [73403] WARNING: terminating connection because of crash of another server process Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-2] 2019-01-25 16:03:51.135 -02 [73403] DETAIL: The postmaster has commanded this server process to roll back the current transacti on and exit, because another server process exited abnormally and possibly corrupted shared memory. Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-3] 2019-01-25 16:03:51.135 -02 [73403] HINT: In a moment you should be able to reconnect to the database and repeat your command. This is our test machine and some days ago we try to increase memory and vcpu, but the problem persist. Some information: root@FreeBSD11:~ # uname -a FreeBSD FreeBSD11.2 11.2-RELEASE-p4 FreeBSD 11.2-RELEASE-p4 #0: Thu Sep 27 08:16:24 UTC 2018 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 root@FreeBSD11:~ # swapinfo -h Device 1K-blocks Used Avail Capacity /dev/da0s1b 4194268 0B 4.0G 0% root@FreeBSD11:~ # df -h Filesystem Size Used Avail Capacity Mounted on /dev/da0s1a 141G 13G 117G 10% / devfs 1.0K 1.0K 0B 100% /dev # select version(); PostgreSQL 11.0 on amd64-portbld-freebsd11.2, compiled by FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0), 64-bit Any suggestion? Tks a lot -- Att Márcio A. Sepp
On 1/25/19 10:21 AM, Márcio Antônio Sepp wrote: > > Hi all, > > > We are experiencing the follow error in last few weeks: > > Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited on > signal 11 (core dumped) > Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-1] 2019-01-25 16:03:51.135 -02 > [73403] WARNING: terminating connection because of crash of another server > process > Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-2] 2019-01-25 16:03:51.135 -02 > [73403] DETAIL: The postmaster has commanded this server process to roll > back the current transacti > on and exit, because another server process exited abnormally and possibly > corrupted shared memory. > Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-3] 2019-01-25 16:03:51.135 -02 > [73403] HINT: In a moment you should be able to reconnect to the database > and repeat your command. What is the logs just before the above? > > > This is our test machine and some days ago we try to increase memory and > vcpu, but the problem persist. > > > Some information: > root@FreeBSD11:~ # uname -a > FreeBSD FreeBSD11.2 11.2-RELEASE-p4 FreeBSD 11.2-RELEASE-p4 #0: Thu Sep 27 > 08:16:24 UTC 2018 > root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 > > > root@FreeBSD11:~ # swapinfo -h > Device 1K-blocks Used Avail Capacity > /dev/da0s1b 4194268 0B 4.0G 0% > > > root@FreeBSD11:~ # df -h > Filesystem Size Used Avail Capacity Mounted on > /dev/da0s1a 141G 13G 117G 10% / > devfs 1.0K 1.0K 0B 100% /dev > > > # select version(); > PostgreSQL 11.0 on amd64-portbld-freebsd11.2, compiled by FreeBSD clang > version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0), 64-bit > > > Any suggestion? > > Tks a lot > > > -- > Att > Márcio A. Sepp > > > -- Adrian Klaver adrian.klaver@aklaver.com
=?iso-8859-1?Q?M=E1rcio_Ant=F4nio_Sepp?= <marcio@zyontecnologia.com.br> writes: > We are experiencing the follow error in last few weeks: > Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited on > signal 11 (core dumped) > # select version(); > PostgreSQL 11.0 on amd64-portbld-freebsd11.2, compiled by FreeBSD clang > version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0), 64-bit First thing to try is updating to 11.1, just to see if problem is already fixed. If not, find out what query is crashing (see postmaster log), and if you can get a stack trace out of one of the core files, that'd be pretty helpful too. https://wiki.postgresql.org/wiki/Generating_a_stack_trace_of_a_PostgreSQL_backend We might need to ask for a self-contained test case, if the stack trace doesn't make it clear what's failing. regards, tom lane
> > On 1/25/19 10:21 AM, Márcio Antônio Sepp wrote: > > > > Hi all, > > > > > > We are experiencing the follow error in last few weeks: > > > > Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited > on > > signal 11 (core dumped) > > Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-1] 2019-01-25 16:03:51.135 > -02 > > [73403] WARNING: terminating connection because of crash of another > server > > process > > Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-2] 2019-01-25 16:03:51.135 > -02 > > [73403] DETAIL: The postmaster has commanded this server process to roll > > back the current transacti > > on and exit, because another server process exited abnormally and > possibly > > corrupted shared memory. > > Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-3] 2019-01-25 16:03:51.135 > -02 > > [73403] HINT: In a moment you should be able to reconnect to the > database > > and repeat your command. > > What is the logs just before the above? Jan 25 16:01:55 FreeBSD11 postgres[71605]: [6-2] 2019-01-25 16:01:55.733 -02 [71605] DETAIL: Key (tdoemp, codemp, codfil,origem, sercai, numdoc, tdopes, codpes, datpar)=(1, 59 84305, 1, 55, 1, 160, 1, 5984305000107, 201901) is not present in table "e026pag_p2019". Jan 25 16:01:55 FreeBSD11 postgres[71605]: [6-3] 2019-01-25 16:01:55.733 -02 [71605] STATEMENT: COMMIT Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-1] 2019-01-25 16:02:35.184 -02 [73403] ERROR: insert or update on table "e026par_p2019"violates foreign key constraint "e026par_p 2019_tdoemp_fkey" Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-2] 2019-01-25 16:02:35.184 -02 [73403] DETAIL: Key (tdoemp, codemp, codfil,origem, sercai, numdoc, tdopes, codpes, datpar)=(1, 59 84305, 1, 55, 1, 160, 1, 5984305000107, 201901) is not present in table "e026pag_p2019". Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-3] 2019-01-25 16:02:35.184 -02 [73403] STATEMENT: COMMIT Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited on signal 11 (core dumped) > > > > This is our test machine and some days ago we try to increase memory and > > vcpu, but the problem persist. > > > > > > Some information: > > root@FreeBSD11:~ # uname -a > > FreeBSD FreeBSD11.2 11.2-RELEASE-p4 FreeBSD 11.2-RELEASE-p4 #0: Thu Sep > 27 > > 08:16:24 UTC 2018 > > root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 > > > > > > root@FreeBSD11:~ # swapinfo -h > > Device 1K-blocks Used Avail Capacity > > /dev/da0s1b 4194268 0B 4.0G 0% > > > > > > root@FreeBSD11:~ # df -h > > Filesystem Size Used Avail Capacity Mounted on > > /dev/da0s1a 141G 13G 117G 10% / > > devfs 1.0K 1.0K 0B 100% /dev > > > > > > # select version(); > > PostgreSQL 11.0 on amd64-portbld-freebsd11.2, compiled by FreeBSD clang > > version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0), 64- > bit > >
On 1/25/19 10:50 AM, Márcio Antônio Sepp wrote: >> >> On 1/25/19 10:21 AM, Márcio Antônio Sepp wrote: >>> >>> Hi all, >>> >>> >>> We are experiencing the follow error in last few weeks: >>> >>> Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited >> on >>> signal 11 (core dumped) >>> Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-1] 2019-01-25 16:03:51.135 >> -02 >>> [73403] WARNING: terminating connection because of crash of another >> server >>> process >>> Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-2] 2019-01-25 16:03:51.135 >> -02 >>> [73403] DETAIL: The postmaster has commanded this server process to roll >>> back the current transacti >>> on and exit, because another server process exited abnormally and >> possibly >>> corrupted shared memory. >>> Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-3] 2019-01-25 16:03:51.135 >> -02 >>> [73403] HINT: In a moment you should be able to reconnect to the >> database >>> and repeat your command. >> >> What is the logs just before the above? > > > Jan 25 16:01:55 FreeBSD11 postgres[71605]: [6-2] 2019-01-25 16:01:55.733 -02 [71605] DETAIL: Key (tdoemp, codemp, codfil,origem, sercai, numdoc, tdopes, codpes, datpar)=(1, 59 > 84305, 1, 55, 1, 160, 1, 5984305000107, 201901) is not present in table "e026pag_p2019". > Jan 25 16:01:55 FreeBSD11 postgres[71605]: [6-3] 2019-01-25 16:01:55.733 -02 [71605] STATEMENT: COMMIT > Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-1] 2019-01-25 16:02:35.184 -02 [73403] ERROR: insert or update on table"e026par_p2019" violates foreign key constraint "e026par_p > 2019_tdoemp_fkey" > Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-2] 2019-01-25 16:02:35.184 -02 [73403] DETAIL: Key (tdoemp, codemp, codfil,origem, sercai, numdoc, tdopes, codpes, datpar)=(1, 59 > 84305, 1, 55, 1, 160, 1, 5984305000107, 201901) is not present in table "e026pag_p2019". > Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-3] 2019-01-25 16:02:35.184 -02 [73403] STATEMENT: COMMIT > Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited on signal 11 (core dumped) > Hmm, a 1:16 between the query failing and the crash. Hard to say if there is a cause and effect. I would follow Tom Lane's advice and see if you can get a stack trace. -- Adrian Klaver adrian.klaver@aklaver.com