Обсуждение: FATAL: semctl(999999, 6, SETVAL, 0) failed: Invalid argument

Поиск
Список
Период
Сортировка

FATAL: semctl(999999, 6, SETVAL, 0) failed: Invalid argument

От
DrakoRod
Дата:
Hi everybody

I have a problem with to Postgres 9.5 with the semaphores, mainly in the
Standby node, but in the mater node has been happening too.

The implementation are in RedHat 7.2 with the NAS Storage, the data
directory of PostgreSQL is in the NAS storage. I was read that PostgreSQL
doesn't have problem with this type storage but now I have this problem and
I'm not source that the problem are the storage. The error is following:

2016-12-05 03:02:31 CST FATAL:  connection to client lost
2016-12-05 03:04:10 CST FATAL:  semctl(5832710, 6, SETVAL, 0) failed:
Invalid argument
2016-12-05 03:04:10 CST LOG:  server process (PID 4852) exited with exit
code 1
2016-12-05 03:04:10 CST LOG:  terminating any other active server processes
2016-12-05 03:04:10 CST WARNING:  terminating connection because of crash of
another server process
2016-12-05 03:04:10 CST DETAIL:  The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2016-12-05 03:04:10 CST HINT:  In a moment you should be able to reconnect
to the database and repeat your command.
2016-12-05 03:04:10 CST LOG:  archiver process (PID 15360) exited with exit
code 1
2016-12-05 03:04:10 CST WARNING:  terminating connection because of crash of
another server process
2016-12-05 03:04:10 CST DETAIL:  The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2016-12-05 03:04:10 CST HINT:  In a moment you should be able to reconnect
to the database and repeat your command.
2016-12-05 03:04:10 CST WARNING:  terminating connection because of crash of
another server process
2016-12-05 03:04:10 CST DETAIL:  The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2016-12-05 03:04:10 CST HINT:  In a moment you should be able to reconnect
to the database and repeat your command.
2016-12-05 03:04:11 CST LOG:  all server processes terminated;
reinitializing
2016-12-05 03:04:11 CST LOG:  could not remove shared memory segment
"/PostgreSQL.1804289383": No such file or directory
2016-12-05 03:04:11 CST LOG:  semctl(5636096, 0, IPC_RMID, ...) failed:
Invalid argument
2016-12-05 03:04:11 CST LOG:  semctl(5668865, 0, IPC_RMID, ...) failed:
Invalid argument
2016-12-05 03:04:11 CST LOG:  semctl(5701634, 0, IPC_RMID, ...) failed:
Invalid argument
2016-12-05 03:04:11 CST LOG:  semctl(5734403, 0, IPC_RMID, ...) failed:
Invalid argument
2016-12-05 03:04:11 CST LOG:  semctl(5767172, 0, IPC_RMID, ...) failed:
Invalid argument
2016-12-05 03:04:11 CST LOG:  semctl(5799941, 0, IPC_RMID, ...) failed:
Invalid argument
2016-12-05 03:04:11 CST LOG:  semctl(5832710, 0, IPC_RMID, ...) failed:
Invalid argument
2016-12-05 03:04:11 CST LOG:  semctl(5865479, 0, IPC_RMID, ...) failed:
Invalid argument
2016-12-05 03:04:11 CST LOG:  database system was interrupted; last known up
at 2016-12-05 03:03:26 CST
2016-12-05 03:04:12 CST LOG:  database system was not properly shut down;
automatic recovery in progress
2016-12-05 03:04:12 CST LOG:  redo starts at 6/984DDFD0
2016-12-05 03:04:12 CST LOG:  invalid record length at 6/984DE100
2016-12-05 03:04:12 CST LOG:  redo done at 6/984DE0C8
2016-12-05 03:04:12 CST LOG:  MultiXact member wraparound protections are
now enabled
2016-12-05 03:04:12 CST LOG:  database system is ready to accept connections

I read this in web and forums:

http://serverfault.com/questions/723636/connecting-to-postgresql-fails-with-semctl-invalid-argument
http://postgresql.nabble.com/GENERAL-postgreslog-semctl-7438339-4-SETVAL-0-failed-td1860547.html
https://groups.google.com/forum/#!topic/pgbarman/Hx4U9F24vnw

I found this maybe some bug or something?


http://stackoverflow.com/questions/36077235/all-auxiliary-procs-are-in-use-at-streaming-replication-server-in-postgresql-9-5

I have calculated and modificated kernel resources with the information this
link:
https://www.postgresql.org/docs/9.5/static/kernel-resources.html

But the problem does not fix. I have the same implmentation but with SAN
storage that is the production enviroment. I'm really worried that this not
fix it.

It is a virtual machine in VMWare. I have read many post talk about  freeBSD
jails, but I do not understand why freeBSD jails. What do you have to do
with this?

Any suggestion?

Best Regards.






-----
Dame un poco de fe, eso me bastará.
Rozvo Ware Solutions
--
View this message in context:
http://postgresql.nabble.com/FATAL-semctl-999999-6-SETVAL-0-failed-Invalid-argument-tp5933635.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.


Re: FATAL: semctl(999999, 6, SETVAL, 0) failed: Invalid argument

От
Tom Lane
Дата:
DrakoRod <drakoflames@hotmail.com> writes:
> I have a problem with to Postgres 9.5 with the semaphores, mainly in the
> Standby node, but in the mater node has been happening too.

> The implementation are in RedHat 7.2 with the NAS Storage, the data
> directory of PostgreSQL is in the NAS storage. I was read that PostgreSQL
> doesn't have problem with this type storage but now I have this problem and
> I'm not source that the problem are the storage. The error is following:

> 2016-12-05 03:02:31 CST FATAL:  connection to client lost
> 2016-12-05 03:04:10 CST FATAL:  semctl(5832710, 6, SETVAL, 0) failed:
> Invalid argument

I doubt this has anything much to do with NAS storage.  What it sounds
like to me is systemd's "helpful" idea of removing SysV semaphores when
it thinks you've logged out.  There are at least two ways to stop it
from doing that; see
https://wiki.postgresql.org/wiki/Systemd
or the longer discussion at
https://www.postgresql.org/message-id/flat/57828C31.5060409%40gmail.com
or a couple of other discussions you can find by searching the PG mail
archives for "systemd semaphores".

            regards, tom lane


Re: FATAL: semctl(999999, 6, SETVAL, 0) failed: Invalid argument

От
DrakoRod
Дата:
Hi Tom!

Thanks for your quick response, in fact, it is not a problem with NAS
storage. It is the systemd configuration. I had detected that the semaphores
were removed every 20-30 minutes.

After that configuration (the links that you share), I monitored during 1
hour and everything is going well.

https://wiki.postgresql.org/wiki/Systemd
https://www.postgresql.org/message-id/CAK7tEys9-O4BTERbs3Xuk2BfFNNd55u2sM9j5R2Fi7v6BHjrQw@mail.gmail.com

Thanks!



-----
Dame un poco de fe, eso me bastará.
Rozvo Ware Solutions
--
View this message in context:
http://postgresql.nabble.com/FATAL-semctl-999999-6-SETVAL-0-failed-Invalid-argument-tp5933635p5933649.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.