Обсуждение: Intermittent ECPG test failures on Windows buildfarm machines
Observe the following buildfarm failures: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=whelk&dt=2018-03-29%2013%3A41%3A13 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=woodlouse&dt=2018-04-18%2016%3A42%3A03 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dory&dt=2018-04-27%2016%3A15%3A25 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dory&dt=2018-04-30%2000%3A45%3A25 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dory&dt=2018-04-30%2002%3A00%3A25 The common feature is that a single ECPG test case emitted an empty stdout file. There is no other indication of a problem: the corresponding stderr output files are correct (and no, the "correct" contents of those aren't empty), the test process exited with status zero, and there's no sign of an issue in the postmaster log. And it's a different test case each time. I trawled the buildfarm logs back to the beginning of 2017 and can't find any similar failures before these. So it seems like we broke something fairly recently, probably during March; but what? The woodlouse case is on 9.6 not HEAD, suggesting that whatever we did wrong got back-patched, and for sure I see no recent commits in relevant-seeming code in 9.6. Baffled ... any ideas? regards, tom lane
Am Montag, den 30.04.2018, 00:22 -0400 schrieb Tom Lane: > Observe the following buildfarm failures: > ... > The common feature is that a single ECPG test case emitted an empty > stdout > file. There is no other indication of a problem: the corresponding > stderr > output files are correct (and no, the "correct" contents of those > aren't > empty), the test process exited with status zero, and there's no sign > of > an issue in the postmaster log. And it's a different test case each > time. > > Baffled ... any ideas? AFAICT there were like 4 commits to ecpg in March that were also backported to 9.6. And while some included changes to the test suite I have no idea which, if any, might result in this kind of problem. Also there was at least one change to the Windows build system that impacted ecpg. Is there anyone out there with a Windows system who could bisect the tree and find which commit is the culprit? Or did we have any changes to the buildfarm scripts that may be causing this? Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Meskes at (Debian|Postgresql) dot Org Jabber: michael at xmpp dot meskes dot org VfL Borussia! Força Barça! SF 49ers! Use Debian GNU/Linux, PostgreSQL
On 04/30/2018 12:22 AM, Tom Lane wrote: > Observe the following buildfarm failures: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=whelk&dt=2018-03-29%2013%3A41%3A13 > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=woodlouse&dt=2018-04-18%2016%3A42%3A03 > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dory&dt=2018-04-27%2016%3A15%3A25 > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dory&dt=2018-04-30%2000%3A45%3A25 > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dory&dt=2018-04-30%2002%3A00%3A25 > > The common feature is that a single ECPG test case emitted an empty stdout > file. There is no other indication of a problem: the corresponding stderr > output files are correct (and no, the "correct" contents of those aren't > empty), the test process exited with status zero, and there's no sign of > an issue in the postmaster log. And it's a different test case each time. > > I trawled the buildfarm logs back to the beginning of 2017 and can't find > any similar failures before these. So it seems like we broke something > fairly recently, probably during March; but what? The woodlouse case is > on 9.6 not HEAD, suggesting that whatever we did wrong got back-patched, > and for sure I see no recent commits in relevant-seeming code in 9.6. > > Baffled ... any ideas? > > I have long had problems along these lines on Cygwin, and occasionally worse, where the ecpg tests would hang on Cygwin. I have never been able to discover any rhyme or reason for it, so on lorikeet and the retiring brolga the ecpg tests are just disabled. If Cygwin were a more important platform I would have put more effort into diagnosing the problem. But I don't recall having similar issues on other Windows configurations. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services