Re: [HACKERS] Current sources?
От | Thomas G. Lockhart |
---|---|
Тема | Re: [HACKERS] Current sources? |
Дата | |
Msg-id | 356B78CD.49C83719@alumni.caltech.edu обсуждение исходный текст |
Ответ на | Re: [HACKERS] Current sources? (dg@illustra.com (David Gould)) |
Список | pgsql-hackers |
> > Perhaps the real lesson to be learned is that a little more effort > > should be expended on the regression tests. I have a couple of > > suggestions: > > 1. As far as I've seen there is no documentation on how to create > > regression tests. This should be documented and made as easy as > > possible, to encourage people to create tests for missing cases. Hmm. It ain't hard, but afaik the only people who have pushed on the regression tests are scrappy and myself. We went for years with no updates to the regression tests at all, and now have a somewhat stable set of tests which actually measure many features of a s/w build. > Excellent idea. If everyone making a new feature could also make a > test case for it this would help us keep the system stable. This would seem to be a truism. Any takers?? > > 2. System variations (roundoff error differences, etc) create > > spurious test complaints that make it hard to interpret the results > > properly. Can anything be done to clean this up? > Hmmm, perhaps we could modify the tests to display results through a > function that rounded to the expected precision eg: > select display(floatcol, 8), display(doublecol, 16) from testtab; Gee, maybe we should go back to the original v4.2/v1.0.x behavior of rounding _all_ double precision floating point results to 6 digits :( We've worked hard to get all of the regression tests to match at least one platform (at the moment, Linux/i686) and scrappy has extended the test mechanism to allow for platform-specific differences. But we don't have access to all of the supported platforms, so others will need to help (and they have been, at least some). > > 3. The TODO list should maintain a section on missing regression > > tests; any failure that gets by the regression tests should cause an > > entry to get made here. This list would have a side benefit of > > warning developers about areas that are not getting tested, so that > > they know they have to do some hand testing if they change relevant > > code. imho it will take more effort to maintain a todo list than to just submit a patch for testing. I would be happy to maintain the "expected" output if people would suggest new tests (and better, submit patches for the test). > I will argue once again for a clean snapshot that is known to pass > regression. I snapshot the system all the time, and then do development for a week or two or more on that revlocked snapshot. afaik the failures in regression testing at the time I submitted my last "typing" patches were due to differences in type conversion behavior; I didn't want the changed behavior to become formalized until others had had a chance to test and comment. (btw, no one has, and anyway I'm changing the results for most of those back to what it was before). It's pretty clear that many patches are submitted without full regression testing; either way it would be helpful to post a comment with patches saying how the patches affect the regression tests, or that no testing was done. I'd like to see another person test patches before committing to the source tree, but others might like to see where the patches/changes are heading even before that so I can see arguments both ways. As has been suggested by yourself and others, regression test contributions would be very helpful; so far the discussion amounts to asking scrappy and myself to do _more_ work on the regression tests. I'd like to see someone offering specific help at some point :) Anyway, if Marc or I led this discussion you will probably just get more ideas similar to what is already there; more brainstorming on the hackers list from y'all will lead to some good new ideas so I'll shut up now... - Tom
В списке pgsql-hackers по дате отправления: