Обсуждение: make -C libpq check fails obscurely if tap tests are disabled
make -C ./src/interfaces/libpq check
PATH=... && @echo "TAP tests not enabled. Try configuring with --enable-tap-tests"
/bin/sh: 1: @echo: not found
make is telling the shell to run "@echo" , rather than running "echo" silently.
Since:
commit 6b04abdfc5e0653542ac5d586e639185a8c61a39
Author: Andres Freund <andres@anarazel.de>
Date: Sat Feb 26 16:51:47 2022 -0800
Run tap tests in src/interfaces/libpq.
--
Justin
On 2022-07-20 We 13:23, Justin Pryzby wrote: > make -C ./src/interfaces/libpq check > PATH=... && @echo "TAP tests not enabled. Try configuring with --enable-tap-tests" > /bin/sh: 1: @echo: not found > > make is telling the shell to run "@echo" , rather than running "echo" silently. > > Since: > > commit 6b04abdfc5e0653542ac5d586e639185a8c61a39 > Author: Andres Freund <andres@anarazel.de> > Date: Sat Feb 26 16:51:47 2022 -0800 > > Run tap tests in src/interfaces/libpq. Yeah. It's a bit ugly but I think the attached would fix it. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
Вложения
On 2022-Jul-20, Andrew Dunstan wrote: > On 2022-07-20 We 13:23, Justin Pryzby wrote: > > PATH=... && @echo "TAP tests not enabled. Try configuring with --enable-tap-tests" > > /bin/sh: 1: @echo: not found > > > > make is telling the shell to run "@echo" , rather than running "echo" silently. > Yeah. It's a bit ugly but I think the attached would fix it. Here's a different take. Just assign the variable separately. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "No renuncies a nada. No te aferres a nada."
Вложения
On 2022-07-21 Th 04:53, Alvaro Herrera wrote: > On 2022-Jul-20, Andrew Dunstan wrote: > >> On 2022-07-20 We 13:23, Justin Pryzby wrote: >>> PATH=... && @echo "TAP tests not enabled. Try configuring with --enable-tap-tests" >>> /bin/sh: 1: @echo: not found >>> >>> make is telling the shell to run "@echo" , rather than running "echo" silently. >> Yeah. It's a bit ugly but I think the attached would fix it. > Here's a different take. Just assign the variable separately. Nice, I didn't know you could do that. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
On 2022-Jul-21, Andrew Dunstan wrote: > On 2022-07-21 Th 04:53, Alvaro Herrera wrote: > > Here's a different take. Just assign the variable separately. > > Nice, I didn't know you could do that. It's not very common -- we do have some target-specific variable assignments, but none of them use 'export'. I saw somewhere that this works from Make 3.77 onwards, and we require 3.80, so it should be okay. The buildfarm will tell us ... -- Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/ "Industry suffers from the managerial dogma that for the sake of stability and continuity, the company should be independent of the competence of individual employees." (E. Dijkstra)
On 2022-Jul-22, Alvaro Herrera wrote: > It's not very common -- we do have some target-specific variable > assignments, but none of them use 'export'. I saw somewhere that this > works from Make 3.77 onwards, and we require 3.80, so it should be okay. > The buildfarm will tell us ... Hm, so prairiedog didn't like this: make -C libpq all Makefile:146: *** multiple target patterns. Stop. but I don't understand which part it is upset about. The rules are: check installcheck: export PATH := $(CURDIR)/test:$(PATH) check: test-build all $(prove_check) installcheck: test-build all $(prove_installcheck) I think "multiple target patterns" means it doesn't like the fact that there are two colons in the first line. But if I use a recursive assignment (PATH = ...), that of course doesn't work because PATH appears on both sides of the assignment: Makefile:146: *** Recursive variable 'PATH' references itself (eventually). Stop. Now, maybe that colon is not the issue and perhaps the problem can be solved by splitting the rule: check: export PATH := $(CURDIR)/test:$(PATH) installcheck: export PATH := $(CURDIR)/test:$(PATH) According to 32568.1536241083@sss.pgh.pa.us, prairiedog is on Make 3.80. Hmmm. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ Voy a acabar con todos los humanos / con los humanos yo acabaré voy a acabar con todos (bis) / con todos los humanos acabaré ¡acabaré! (Bender)
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> On 2022-Jul-22, Alvaro Herrera wrote:
>> It's not very common -- we do have some target-specific variable
>> assignments, but none of them use 'export'. I saw somewhere that this
>> works from Make 3.77 onwards, and we require 3.80, so it should be okay.
>> The buildfarm will tell us ...
> Hm, so prairiedog didn't like this:
> According to 32568.1536241083@sss.pgh.pa.us, prairiedog is on Make 3.80.
Yeah, it is. I looked at the gmake manual on that machine, and its
description of "export" seems about the same as what I see in a
modern version. So it should work ... but we've found bugs in 3.80
before.
Let me poke at it and see if there's a variant that works.
The wording of the message suggests that maybe breaking it into
two separate rules would help.
regards, tom lane
I wrote:
> Yeah, it is. I looked at the gmake manual on that machine, and its
> description of "export" seems about the same as what I see in a
> modern version.
Um ... I was not looking in the right place. The description of
"target-specific variables" does not say you can use "export",
whereas the modern manual mentions that specifically. I found
a relevant entry in their changelog:
2002-10-13 Paul D. Smith <psmith@gnu.org>
...
* read.c (eval): Fix Bug #1391: allow "export" keyword in
target-specific variable definitions. Check for it and set an
"exported" flag.
* doc/make.texi (Target-specific): Document the ability to use
"export".
So it'll work in 3.81 (released 2006) and later, but not 3.80.
TBH my inclination here is to move our goalposts to say "we support
gmake 3.81 and later". It's possible that prairiedog's copy of 3.80 is
the last one left in the wild, and nearly certain that it's the last
one left that anyone would try to build PG with. (I see gmake 3.81 in
the next macOS version, 10.5.) I doubt it'd take long to install a newer
version on prairiedog.
Alternatively, we could use Andrew's hacky solution from upthread.
regards, tom lane
I wrote:
> So it'll work in 3.81 (released 2006) and later, but not 3.80.
Confirmed that things are fine with 3.81.
> TBH my inclination here is to move our goalposts to say "we support
> gmake 3.81 and later".
Barring objections, I'll push the attached patch. I suppose we
could undo whatever dumbing-down was done in _create_recursive_target,
but is it worth troubling with?
regards, tom lane
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index 32e0d3fd9d..70d188e2bc 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -77,7 +77,7 @@ su - postgres
<primary>make</primary>
</indexterm>
- <acronym>GNU</acronym> <application>make</application> version 3.80 or newer is required; other
+ <acronym>GNU</acronym> <application>make</application> version 3.81 or newer is required; other
<application>make</application> programs or older <acronym>GNU</acronym> <application>make</application>
versionswill <emphasis>not</emphasis> work.
(<acronym>GNU</acronym> <application>make</application> is sometimes installed under
the name <filename>gmake</filename>.) To test for <acronym>GNU</acronym>
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index bb177a8162..0d766cfd15 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -872,11 +872,11 @@ install-strip:
# allows parallel make across directories and lets make -k and -q work
# correctly.
-# We need the $(eval) function and order-only prerequisites, which are
-# available in GNU make 3.80. That also happens to be the version
-# where the .VARIABLES variable was introduced, so this is a simple check.
-ifndef .VARIABLES
-$(error GNU make 3.80 or newer is required. You are using version $(MAKE_VERSION))
+# We need the ability to export target-specific variables, which was
+# added in GNU make 3.81. That also happens to be the version
+# where the .FEATURES variable was introduced, so this is a simple check.
+ifndef .FEATURES
+$(error GNU make 3.81 or newer is required. You are using version $(MAKE_VERSION))
endif
# This function is only for internal use below. It should be called
@@ -884,7 +884,7 @@ endif
# given subdirectory. For the tree-wide all/install/check/installcheck cases,
# ensure we do our one-time tasks before recursing (see targets above).
# Note that to avoid a nasty bug in make 3.80,
-# this function has to avoid using any complicated constructs (like
+# this function was written to not use any complicated constructs (like
# multiple targets on a line) and also not contain any lines that expand
# to more than about 200 bytes. This is why we make it apply to just one
# subdirectory at a time, rather than to a list of subdirectories.
On 2022-Jul-22, Tom Lane wrote: > Barring objections, I'll push the attached patch. I suppose we > could undo whatever dumbing-down was done in _create_recursive_target, > but is it worth troubling with? Excellent, many thanks. I tried to get Make 3.80 built here, to no avail. I have updated that to 3.81, but still haven't found a way past the automake phase. Anyway, I tried a revert of 1bd201214965 -- I ended up with the attached. However, while a serial compile fails, parallel ones fail randomly, and apparently because two submakes compete in building libpq.a and each deletes the other's file. What I think this is saying is that the 3.80-induced wording of that function limits concurrency of the generated recursive rules, which prevents the problem from occurring; and if we were to fix that bug we would probably end up with more concurrency. Here's the bottom of the 'make -j8' log: rm -f libpq.a ranlib libpq.a ranlib: 'libpq.a': No such file make[5]: *** [/pgsql/source/master/src/Makefile.shlib:261: libpq.a] Error 1 make[5]: *** Waiting for unfinished jobs.... ar crs libpq.a fe-auth-scram.o fe-connect.o fe-exec.o fe-lobj.o fe-misc.o fe-print.o fe-protocol3.o fe-secure.o fe-trace.olegacy-pqsignal.o libpq-events.o pqexpbuffer.o fe-auth.o fe-secure-common.o fe-secure-openssl.o ranlib libpq.a touch libpq.a rm -f libpq.so.5 ln -s libpq.so.5.16 libpq.so.5 rm -f libpq.so ln -s libpq.so.5.16 libpq.so touch libpq-refs-stamp rm -f libpq.so.5 ln -s libpq.so.5.16 libpq.so.5 rm -f libpq.so ln -s libpq.so.5.16 libpq.so rm -f libpq.so.5 ln -s libpq.so.5.16 libpq.so.5 touch libpq-refs-stamp rm -f libpq.so gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute-Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard-Wno-format-truncation -Wno-stringop-truncation -g -O2 -pthread -D_REENTRANT -D_THREAD_SAFE -fPIC-shared -Wl,-soname,libecpg.so.6 -Wl,--version-script=exports.list -o libecpg.so.6.16 connect.o data.o descriptor.oerror.o execute.o memory.o misc.o prepare.o sqlda.o typename.o -L../../../../src/port -L../../../../src/common-L../pgtypeslib -lpgtypes -L../../../../src/common -lpgcommon_shlib -L../../../../src/port -lpgport_shlib-L../../../../src/interfaces/libpq -lpq -L/usr/lib/llvm-11/lib -Wl,--as-needed -Wl,-rpath,'/pgsql/install/master/lib',--enable-new-dtags -lm ln -s libpq.so.5.16 libpq.so rm -f libecpg.a make[4]: *** [../../../../src/Makefile.global:618: submake-libpq] Error 2 ar crs libecpg.a connect.o data.o descriptor.o error.o execute.o memory.o misc.o prepare.o sqlda.o typename.o make[3]: *** [Makefile:17: all-ecpglib-recursive] Error 2 make[3]: *** Waiting for unfinished jobs.... ranlib libecpg.a touch libecpg.a rm -f libecpg.so.6 ln -s libecpg.so.6.16 libecpg.so.6 rm -f libecpg.so ln -s libecpg.so.6.16 libecpg.so gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute-Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard-Wno-format-truncation -Wno-stringop-truncation -g -O2 -pthread -D_REENTRANT -D_THREAD_SAFE -fPIC-shared -Wl,-soname,libecpg_compat.so.3 -Wl,--version-script=exports.list -o libecpg_compat.so.3.16 informix.o -L../../../../src/port-L../../../../src/common -L../ecpglib -lecpg -L../pgtypeslib -lpgtypes -L../../../../src/common -lpgcommon_shlib-L../../../../src/port -lpgport_shlib -L../../../../src/interfaces/libpq -lpq -L/usr/lib/llvm-11/lib -Wl,--as-needed-Wl,-rpath,'/pgsql/install/master/lib',--enable-new-dtags -lm rm -f libecpg_compat.a ar crs libecpg_compat.a informix.o ranlib libecpg_compat.a touch libecpg_compat.a rm -f libecpg_compat.so.3 ln -s libecpg_compat.so.3.16 libecpg_compat.so.3 rm -f libecpg_compat.so ln -s libecpg_compat.so.3.16 libecpg_compat.so make[2]: *** [Makefile:17: all-ecpg-recursive] Error 2 make[1]: *** [Makefile:42: all-interfaces-recursive] Error 2 make: *** [GNUmakefile:11: all-src-recursive] Error 2 -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "Los dioses no protegen a los insensatos. Éstos reciben protección de otros insensatos mejor dotados" (Luis Wu, Mundo Anillo)
On 2022-Jul-25, Alvaro Herrera wrote: > Anyway, I tried a revert of 1bd201214965 -- I ended up with the > attached. -- Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Вложения
Ah, I found what the actual problem is: we have sprinkled a few dependencies on "...-recurse" throught the tree, but the patch I posted yesterday changes the manufactured target as "-recursive", as it was prior to 1bd201214965; so essentially these manually added dependencies all became silent no-ops. With this version I keep the target name as -recurse, and at least the ecpg<->libpq problem is no more AFAICT. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Вложения
Hi, On 2022-07-22 19:50:02 -0400, Tom Lane wrote: > I wrote: > > So it'll work in 3.81 (released 2006) and later, but not 3.80. > > Confirmed that things are fine with 3.81. Thanks for looking into this Alvaro, Andrew, Justin, Tom - I was on vacation... Greetings, Andres Freund
On 2022-Jul-26, Alvaro Herrera wrote: > With this version I keep the target name as -recurse, and at least the > ecpg<->libpq problem is no more AFAICT. ... but I think we're missing some more dependencies, because if I remove everything (beyond make clean), then a "make -j8 world-bin" fails, per Cirrus. https://cirrus-ci.com/build/6305635338813440 With that, I'm going to set this aside for the time being. If somebody wants to play with these Makefile rules, be my guest. It sounds like there's some compile time gains to be had, but it may require some fiddling and it's not clear to me if the move to Meson is going to make this moot. Running it locally, I get this: [...] gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute-Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard-Wno-format-truncation -Wno-stringop-truncation -g -O2 -DFRONTEND -I. -I/pgsql/source/master/src/common-I../../src/include -I/pgsql/source/master/src/include -D_GNU_SOURCE -DVAL_CC="\"gcc\""-DVAL_CPPFLAGS="\"-D_GNU_SOURCE\"" -DVAL_CFLAGS="\"-Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement-Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation-g -O2\"" -DVAL_CFLAGS_SL="\"-fPIC\"" -DVAL_LDFLAGS="\"-L/usr/lib/llvm-11/lib -Wl,--as-needed -Wl,-rpath,'/pgsql/install/master/lib',--enable-new-dtags\""-DVAL_LDFLAGS_EX="\"\"" -DVAL_LDFLAGS_SL="\"\"" -DVAL_LIBS="\"-lpgcommon-lpgport -llz4 -lssl -lcrypto -lz -lreadline -lpthread -lrt -ldl -lm \"" -c -o hashfn.o /pgsql/source/master/src/common/hashfn.c-MMD -MP -MF .deps/hashfn.Po [...] gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute-Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard-Wno-format-truncation -Wno-stringop-truncation -g -O2 -DFRONTEND -I. -I/pgsql/source/master/src/common-I../../src/include -I/pgsql/source/master/src/include -D_GNU_SOURCE -DVAL_CC="\"gcc\""-DVAL_CPPFLAGS="\"-D_GNU_SOURCE\"" -DVAL_CFLAGS="\"-Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement-Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation-g -O2\"" -DVAL_CFLAGS_SL="\"-fPIC\"" -DVAL_LDFLAGS="\"-L/usr/lib/llvm-11/lib -Wl,--as-needed -Wl,-rpath,'/pgsql/install/master/lib',--enable-new-dtags\""-DVAL_LDFLAGS_EX="\"\"" -DVAL_LDFLAGS_SL="\"\"" -DVAL_LIBS="\"-lpgcommon-lpgport -llz4 -lssl -lcrypto -lz -lreadline -lpthread -lrt -ldl -lm \"" -c -o relpath.o /pgsql/source/master/src/common/relpath.c-MMD -MP -MF .deps/relpath.Po [...] In file included from /pgsql/source/master/src/include/postgres.h:47, from /pgsql/source/master/src/common/hashfn.c:24: /pgsql/source/master/src/include/utils/elog.h:75:10: fatal error: utils/errcodes.h: No existe el fichero o el directorio 75 | #include "utils/errcodes.h" | ^~~~~~~~~~~~~~~~~~ compilation terminated. [...] make[2]: *** [../../src/Makefile.global:945: hashfn.o] Error 1 make[2]: *** Se espera a que terminen otras tareas.... /pgsql/source/master/src/common/relpath.c:21:10: fatal error: catalog/pg_tablespace_d.h: No existe el fichero o el directorio 21 | #include "catalog/pg_tablespace_d.h" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. make[2]: *** [../../src/Makefile.global:945: relpath.o] Error 1 make[2]: se sale del directorio '/home/alvherre/Code/pgsql-build/master/src/common' make[1]: *** [Makefile:42: all-common-recurse] Error 2 make[1]: se sale del directorio '/home/alvherre/Code/pgsql-build/master/src' make: *** [GNUmakefile:21: world-bin-src-recurse] Error 2 -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "I must say, I am absolutely impressed with what pgsql's implementation of VALUES allows me to do. It's kind of ridiculous how much "work" goes away in my code. Too bad I can't do this at work (Oracle 8/9)." (Tom Allison) http://archives.postgresql.org/pgsql-general/2007-06/msg00016.php