Обсуждение: [HACKERS] max_files_per_processes vs others uses of file descriptors
Hi, currently the default max_files_per_process is 1000. fd.c uses close to that many (- NUM_RESERVED_FDS/10). count_usable_fds() makes sure that at server start there's at most that many fds available, but that doesn't mean that much for runtime. These days there's a number of other consumers of fds. E.g. postgres_fdw, epoll, ... All these aren't accounted for by fd.c. Given how close max_files_per_process is to the default linux limit of 1024 fds, I wonder if we shouldn't increase NUM_RESERVED_FDS by quite a bit? Greetings, Andres Freund
Andres Freund <andres@anarazel.de> writes:
> These days there's a number of other consumers of
> fds. E.g. postgres_fdw, epoll, ... All these aren't accounted for by
> fd.c.
> Given how close max_files_per_process is to the default linux limit of
> 1024 fds, I wonder if we shouldn't increase NUM_RESERVED_FDS by quite a
> bit?
No, I don't think so. If you're depending on the NUM_RESERVED_FDS
headroom for anything meaningful, *you're doing it wrong*. You should be
getting an FD via fd.c, so that there is an opportunity to free up an FD
(by closing a VFD) if you're up against system limits. Relying on
NUM_RESERVED_FDS headroom can only protect against EMFILE not ENFILE.
What this means is that the epoll stuff needs to be tied into fd.c more
than it is now, but that's likely a good thing anyway; it would for
example provide a more robust way of ensuring we don't leak epoll FDs at
transaction abort.
regards, tom lane
On 2017-08-07 16:52:42 -0400, Tom Lane wrote: > Andres Freund <andres@anarazel.de> writes: > > These days there's a number of other consumers of > > fds. E.g. postgres_fdw, epoll, ... All these aren't accounted for by > > fd.c. > > > Given how close max_files_per_process is to the default linux limit of > > 1024 fds, I wonder if we shouldn't increase NUM_RESERVED_FDS by quite a > > bit? > > No, I don't think so. If you're depending on the NUM_RESERVED_FDS > headroom for anything meaningful, *you're doing it wrong*. You should be > getting an FD via fd.c, so that there is an opportunity to free up an FD > (by closing a VFD) if you're up against system limits. Relying on > NUM_RESERVED_FDS headroom can only protect against EMFILE not ENFILE. How would this work for libpq based stuff like postgres fdw? Or some random PL doing something with files? There's very little headroom here. > What this means is that the epoll stuff needs to be tied into fd.c more > than it is now, but that's likely a good thing anyway; it would for > example provide a more robust way of ensuring we don't leak epoll FDs at > transaction abort. Not arguing against that. Greetings, Andres Freund
Andres Freund <andres@anarazel.de> writes:
> On 2017-08-07 16:52:42 -0400, Tom Lane wrote:
>> No, I don't think so. If you're depending on the NUM_RESERVED_FDS
>> headroom for anything meaningful, *you're doing it wrong*. You should be
>> getting an FD via fd.c, so that there is an opportunity to free up an FD
>> (by closing a VFD) if you're up against system limits. Relying on
>> NUM_RESERVED_FDS headroom can only protect against EMFILE not ENFILE.
> How would this work for libpq based stuff like postgres fdw? Or some
> random PL doing something with files? There's very little headroom here.
Probably the best we can hope for there is to have fd.c provide a function
"close an FD please", which postgres_fdw could call if libpq fails because
of ENFILE/EMFILE, and then retry. (Though I'm unsure how reliably
postgres_fdw can detect that failure reason right now --- I don't know
that we preserve errno on the way out of PQconnect.)
regards, tom lane
On 2017-08-07 17:05:06 -0400, Tom Lane wrote: > Andres Freund <andres@anarazel.de> writes: > > On 2017-08-07 16:52:42 -0400, Tom Lane wrote: > >> No, I don't think so. If you're depending on the NUM_RESERVED_FDS > >> headroom for anything meaningful, *you're doing it wrong*. You should be > >> getting an FD via fd.c, so that there is an opportunity to free up an FD > >> (by closing a VFD) if you're up against system limits. Relying on > >> NUM_RESERVED_FDS headroom can only protect against EMFILE not ENFILE. > > > How would this work for libpq based stuff like postgres fdw? Or some > > random PL doing something with files? There's very little headroom here. > > Probably the best we can hope for there is to have fd.c provide a function > "close an FD please", which postgres_fdw could call if libpq fails because > of ENFILE/EMFILE, and then retry. Unless that takes up a slot in fd.c while in use, that'll still leave us open to failures to open files in some critical parts, unless I miss something. And then we'd have to teach similar things to PLs etc. I agree that having some more slop isn't a proper solution, but only having ~30 fds as slop on the most common systems seems mightily small. > (Though I'm unsure how reliably postgres_fdw can detect that failure > reason right now --- I don't know that we preserve errno on the way > out of PQconnect.) Yea, probably not really... Regards, Andres
On Mon, Aug 7, 2017 at 1:40 PM, Andres Freund <andres@anarazel.de> wrote: > Given how close max_files_per_process is to the default linux limit of > 1024 fds, I wonder if we shouldn't increase NUM_RESERVED_FDS by quite a > bit? Personally, any time I've seen a problem with this it was because an extension leaked FDs, which is always going to fail in the end. The extension leaked FDs because it didn't fully buy into using Postgres resource managers, perhaps only in a subtle way. I find it hard to imagine an extension author explicitly relying on any particular amount of slop for FDs. Is this specifically about postgres_fdw, or is there some other specific problem you have in mind, that this would help solve? -- Peter Geoghegan
Andres Freund <andres@anarazel.de> writes:
> On 2017-08-07 17:05:06 -0400, Tom Lane wrote:
>> Probably the best we can hope for there is to have fd.c provide a function
>> "close an FD please", which postgres_fdw could call if libpq fails because
>> of ENFILE/EMFILE, and then retry.
> Unless that takes up a slot in fd.c while in use, that'll still leave us
> open to failures to open files in some critical parts, unless I miss
> something.
Well, there's always a race condition there, in that someone else can
eat the kernel FD as soon as you free it. That's why we do this in a
retry loop.
> And then we'd have to teach similar things to PLs etc. I agree that
> having some more slop isn't a proper solution, but only having ~30 fds
> as slop on the most common systems seems mightily small.
Meh. The lack of field complaints about this doesn't indicate to me that
we have a huge problem, and in any case, just increasing NUM_RESERVED_FDS
would do nothing for the system-wide limits.
regards, tom lane
Hi, On 2017-08-07 17:30:13 -0400, Tom Lane wrote: > Meh. The lack of field complaints about this doesn't indicate to me that > we have a huge problem, and in any case, just increasing NUM_RESERVED_FDS > would do nothing for the system-wide limits. Howso? Via count_usable_fds() we test for max_files_per_process / RLIMIT_NOFILE fds, and *then* subtract NUM_RESERVED_FDS. - Andres
Andres Freund <andres@anarazel.de> writes:
> On 2017-08-07 17:30:13 -0400, Tom Lane wrote:
>> Meh. The lack of field complaints about this doesn't indicate to me that
>> we have a huge problem, and in any case, just increasing NUM_RESERVED_FDS
>> would do nothing for the system-wide limits.
> Howso? Via count_usable_fds() we test for max_files_per_process /
> RLIMIT_NOFILE fds, and *then* subtract NUM_RESERVED_FDS.
The limit I'm worried about is the kernel's overall FD table size limit
(ENFILE failures), not the per-process limit. PG has a well-known
propensity for eating the entire kernel table under heavy load. We
wouldn't ever have bothered with those retry loops otherwise.
regards, tom lane