On 5/8/21 9:39 AM, David Rowley wrote:
> On Sat, 8 May 2021 at 19:03, Yura Sokolov <y.sokolov@postgrespro.ru> wrote:
>> Perhaps there is need for flag "heavy asserts". Or option "--enable-cassert=heavy". Then USE_ASSERT_CHECKING could
bedefined to integer 1 or 2 depending on "heaviness" of enabled checks.
>
> I'd rather explore all other options before we went and did something
> like that. The reason I feel this way is; a couple of jobs ago, I
> worked on a project that had quite a large test suite. In order to
> run the tests on a typical dev machine took about 16 hours. When
> writing new tests, nobody really ever gave much thought to how long
> tests would take to run. The time it took had already grown to
> something much longer than was practical to run before a commit.
> Instead, we had a test farm. We'd commit to master (yuck) and the test
> farm would run the tests. Unfortunately, the farm was not big enough
> to run every test on every commit, so if it encountered a test failure
> (as it commonly did) it tried to bisect and find the first breaking
> commit for that test then alert the person (probably the next day).
> bisecting works out pretty badly when someone commits a build breaking
> commit.
>
> Now that I work on PostgreSQL, I very much like the fact that I can
> run make check-world -j and see very quickly that no tests fail. I'd
> rather not have something like heavy asserts as I believe it would
> just become a dumping ground for badly thought through Asserts that
> most likely could be done in some better way that didn't require so
> much effort. (I'll put mine in that category) It might not take long
> before most committers opted to run make check-world without the heavy
> asserts because it became just too slow. When that happens, it means
> the buildfarm is left to catch these errors and most likely the
> feedback will take much longer to get back to the committer due to the
> animals running these flags also having to churn through the slow
> tests.
>
> So, I'm -1 for "heavy asserts".
>
Yeah, -1 from me to.
But I think we should differentiate between asserts that (a)
significantly affect "our" regression tests with limited amounts of
data, and (b) asserts that are an issue for large data sets. The asserts
discussed in this thread are in (b), but not (a).
I don't know if there's a better way to do these tests, but if there's
not I'd not worry about it too much for now.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company