Обсуждение: Compiling on Termux
Folks, I'm trying to compile master (c952eae52a33069e2e92d34f217b43d0eca3d7de) on Termux, using the supplied settings, as follows. pg_config --configure | xargs ./configure > configure.out 2>configure.err make -j 4 > make.out 2> make.err There appears to be some confusion somewhere about sync_file_range, namely that it's found by ./configure and then not found in make. What should I be poking at to make this work? Best, David. -- David Fetter <david(at)fetter(dot)org> http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
Вложения
On Sat, Dec 22, 2018 at 5:56 AM David Fetter <david@fetter.org> wrote: > > Folks, > > I'm trying to compile master (c952eae52a33069e2e92d34f217b43d0eca3d7de) > on Termux, using the supplied settings, as follows. > > pg_config --configure | xargs ./configure > configure.out 2>configure.err > make -j 4 > make.out 2> make.err > > There appears to be some confusion somewhere about sync_file_range, > namely that it's found by ./configure and then not found in make. > > What should I be poking at to make this work? Apparently your libc (or something else) defines the function so the configure test passes, but your <fcntl.h> doesn't declare it so we can't use it. I guess Termux supplies the headers but your Android supplies the libraries, so there may be sync issues. I'd try hunting around for declarations with something like find /usr/include -name '*.h' | xargs grep sync_file_range. Here's an interesting similar case: https://github.com/termux/termux-packages/issues/899 That talks about using -D__ANDROID_API__=23 (or presumably higher) to make sure that sigtimedwait is exposed by signal.h. Something similar may be afoot here. -- Thomas Munro http://www.enterprisedb.com
On Sat, Dec 22, 2018 at 07:03:32AM +1100, Thomas Munro wrote: > On Sat, Dec 22, 2018 at 5:56 AM David Fetter <david@fetter.org> wrote: > > > > Folks, > > > > I'm trying to compile master (c952eae52a33069e2e92d34f217b43d0eca3d7de) > > on Termux, using the supplied settings, as follows. > > > > pg_config --configure | xargs ./configure > configure.out 2>configure.err > > make -j 4 > make.out 2> make.err > > > > There appears to be some confusion somewhere about sync_file_range, > > namely that it's found by ./configure and then not found in make. > > > > What should I be poking at to make this work? > > Apparently your libc (or something else) defines the function so the > configure test passes, but your <fcntl.h> doesn't declare it so we > can't use it. I guess Termux supplies the headers but your Android > supplies the libraries, so there may be sync issues. I'd try hunting > around for declarations with something like find /usr/include -name > '*.h' | xargs grep sync_file_range. Here's an interesting similar > case: > > https://github.com/termux/termux-packages/issues/899 > > That talks about using -D__ANDROID_API__=23 (or presumably higher) to > make sure that sigtimedwait is exposed by signal.h. Something similar > may be afoot here. That worked perfectly...to expose the next issue, namely that at least part of the System-V IPC mechanisms have been left out. Do we support other systems where this is true? Best, David. -- David Fetter <david(at)fetter(dot)org> http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
On Sat, Dec 22, 2018 at 7:32 AM David Fetter <david@fetter.org> wrote: > On Sat, Dec 22, 2018 at 07:03:32AM +1100, Thomas Munro wrote: > > That talks about using -D__ANDROID_API__=23 (or presumably higher) to > > make sure that sigtimedwait is exposed by signal.h. Something similar > > may be afoot here. > > That worked perfectly...to expose the next issue, namely that at least > part of the System-V IPC mechanisms have been left out. Do we support > other systems where this is true? Interesting, they left that stuff out deliberately out basically because it all sucks: https://android.googlesource.com/platform/ndk/+/4e159d95ebf23b5f72bb707b0cb1518ef96b3d03/docs/system/libc/SYSV-IPC.TXT PostgreSQL currently needs SysV shm just for the small segment that's used for preventing multiple copies running in the same pgdata. You'd need to find a new way to do that. We don't use SysV semaphores (though we can) or message queues. -- Thomas Munro http://www.enterprisedb.com
On 12/21/18 4:04 PM, Thomas Munro wrote: > On Sat, Dec 22, 2018 at 7:32 AM David Fetter <david@fetter.org> wrote: >> On Sat, Dec 22, 2018 at 07:03:32AM +1100, Thomas Munro wrote: >>> That talks about using -D__ANDROID_API__=23 (or presumably higher) to >>> make sure that sigtimedwait is exposed by signal.h. Something similar >>> may be afoot here. >> That worked perfectly...to expose the next issue, namely that at least >> part of the System-V IPC mechanisms have been left out. Do we support >> other systems where this is true? > Interesting, they left that stuff out deliberately out basically > because it all sucks: > > https://android.googlesource.com/platform/ndk/+/4e159d95ebf23b5f72bb707b0cb1518ef96b3d03/docs/system/libc/SYSV-IPC.TXT > > PostgreSQL currently needs SysV shm just for the small segment that's > used for preventing multiple copies running in the same pgdata. You'd > need to find a new way to do that. We don't use SysV semaphores > (though we can) or message queues. That doc says: Killing processes automatically to make room for new ones is an important part of Android's application lifecycle implementation. How's that going to play with Postgres? Sounds like the OOM killer on steroids. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Fri, Dec 21, 2018 at 04:24:20PM -0500, Andrew Dunstan wrote: > On 12/21/18 4:04 PM, Thomas Munro wrote: > > On Sat, Dec 22, 2018 at 7:32 AM David Fetter <david@fetter.org> wrote: > >> On Sat, Dec 22, 2018 at 07:03:32AM +1100, Thomas Munro wrote: > >>> That talks about using -D__ANDROID_API__=23 (or presumably higher) to > >>> make sure that sigtimedwait is exposed by signal.h. Something similar > >>> may be afoot here. > >> That worked perfectly...to expose the next issue, namely that at least > >> part of the System-V IPC mechanisms have been left out. Do we support > >> other systems where this is true? > > Interesting, they left that stuff out deliberately out basically > > because it all sucks: > > > > https://android.googlesource.com/platform/ndk/+/4e159d95ebf23b5f72bb707b0cb1518ef96b3d03/docs/system/libc/SYSV-IPC.TXT > > > > PostgreSQL currently needs SysV shm just for the small segment that's > > used for preventing multiple copies running in the same pgdata. You'd > > need to find a new way to do that. We don't use SysV semaphores > > (though we can) or message queues. > > That doc says: > > Killing processes automatically to make room for new ones is an > important part of Android's application lifecycle implementation. > > How's that going to play with Postgres? Sounds like the OOM killer on > steroids. I don't know precisely how it's going to play with Postgres, but Termux does supply a Postgres in its native packages. That package appears to work, at least in the single-connection case, so they're doing something somehow to get it up and running. It could be that Android won't be a platform we recommend for high workloads, at least as long as these things remain true. Best, David. -- David Fetter <david(at)fetter(dot)org> http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
On Sat, Dec 22, 2018 at 9:19 AM David Fetter <david@fetter.org> wrote: > On Fri, Dec 21, 2018 at 04:24:20PM -0500, Andrew Dunstan wrote: > > On 12/21/18 4:04 PM, Thomas Munro wrote: > > > On Sat, Dec 22, 2018 at 7:32 AM David Fetter <david@fetter.org> wrote: > > >> On Sat, Dec 22, 2018 at 07:03:32AM +1100, Thomas Munro wrote: > > >>> That talks about using -D__ANDROID_API__=23 (or presumably higher) to > > >>> make sure that sigtimedwait is exposed by signal.h. Something similar > > >>> may be afoot here. > > >> That worked perfectly...to expose the next issue, namely that at least > > >> part of the System-V IPC mechanisms have been left out. Do we support > > >> other systems where this is true? > > > Interesting, they left that stuff out deliberately out basically > > > because it all sucks: > > > > > > https://android.googlesource.com/platform/ndk/+/4e159d95ebf23b5f72bb707b0cb1518ef96b3d03/docs/system/libc/SYSV-IPC.TXT > > > > > > PostgreSQL currently needs SysV shm just for the small segment that's > > > used for preventing multiple copies running in the same pgdata. You'd > > > need to find a new way to do that. We don't use SysV semaphores > > > (though we can) or message queues. > > > > That doc says: > > > > Killing processes automatically to make room for new ones is an > > important part of Android's application lifecycle implementation. > > > > How's that going to play with Postgres? Sounds like the OOM killer on > > steroids. > > I don't know precisely how it's going to play with Postgres, but > Termux does supply a Postgres in its native packages. That package > appears to work, at least in the single-connection case, so they're > doing something somehow to get it up and running. > > It could be that Android won't be a platform we recommend for high > workloads, at least as long as these things remain true. They use libandroid-shmem which emulates SysV shmem. https://github.com/termux/termux-packages/blob/master/packages/postgresql/src-backend-Makefile.patch https://github.com/termux/libandroid-shmem -- Thomas Munro http://www.enterprisedb.com
Thomas Munro <thomas.munro@enterprisedb.com> writes:
> On Sat, Dec 22, 2018 at 9:19 AM David Fetter <david@fetter.org> wrote:
>> I don't know precisely how it's going to play with Postgres, but
>> Termux does supply a Postgres in its native packages. That package
>> appears to work, at least in the single-connection case, so they're
>> doing something somehow to get it up and running.
> They use libandroid-shmem which emulates SysV shmem.
Interesting. I wonder how well it emulates the aspect we actually
care about, ie counting the number of attached processes correctly ...
regards, tom lane
On Sat, Dec 22, 2018 at 11:51 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > Thomas Munro <thomas.munro@enterprisedb.com> writes: > > On Sat, Dec 22, 2018 at 9:19 AM David Fetter <david@fetter.org> wrote: > >> I don't know precisely how it's going to play with Postgres, but > >> Termux does supply a Postgres in its native packages. That package > >> appears to work, at least in the single-connection case, so they're > >> doing something somehow to get it up and running. > > > They use libandroid-shmem which emulates SysV shmem. > > Interesting. I wonder how well it emulates the aspect we actually > care about, ie counting the number of attached processes correctly ... I hadn't paid much attention to Android before, but it's quite impressive that it can run PostgreSQL. One interesting thing (to me at least) is that although it's some kind of Linux kernel, it has a new libc called Bionic that is a mixture of cleanroom bits and *BSD bits, probably because they liked the licence better[1]. The allocator is jemalloc so I wonder if it eats 40k for every 32k hash join chunk. I wonder if we could mmap a small file created by initdb under pgdata and do a new kind of interlocking in there, so you don't need SysV shm at all. That would work for Android (which is obviously just a curiosity for now, but kinda neat), and maybe also some other interesting new platforms like CloudABI[2] that get rid of global stuff like the System V facilities, so that you can trap software entirely inside a root directory identified by an inherited file descriptor (not that I have seriously studied what else would be required to run in that environment). [1] https://en.wikipedia.org/wiki/Bionic_(software) [2] https://archive.fosdem.org/2017/schedule/event/cloudabi/ -- Thomas Munro http://www.enterprisedb.com