Обсуждение: BUG #18246: pgstathashindex() attempts to read invalid file for hash index attached to partitioned table

The following bug has been logged on the website:

Bug reference:      18246
Logged by:          Alexander Lakhin
Email address:      exclusion@gmail.com
PostgreSQL version: 16.1
Operating system:   Ubuntu 22.04
Description:

The following script:
CREATE EXTENSION pgstattuple;
CREATE TABLE t (a int, b int[], c int) PARTITION BY RANGE (a);

CREATE INDEX a_idx ON t USING btree(a);
CREATE INDEX b_idx ON t USING gin(b);
CREATE INDEX c_idx ON t USING hash(c);

SELECT pgstatindex('a_idx');
SELECT pgstatginindex('b_idx');
SELECT pgstathashindex('c_idx');

gives an unexpected error for the hash index:
ERROR:  could not open file "pg_tblspc/0/PG_16_202307071/0/0": No such file
or directory

while for btree and hash indexes errors are more adequate:
ERROR:  relation "a_idx" is not a btree index
ERROR:  relation "b_idx" is not a GIN index

Reproduced on REL_12_STABLE .. master.
(On master, thanks to commit 049ef3398, which added an Assert in smgr.c,
that assertion fails.)

This anomaly can be observed since 8b08f7d48 from 2018-01-19, but IMO the
culprit is e759854a0 from 2017-02-03, which introduced the following
asymmetry in pgstatindex.c:
if (!IS_INDEX(rel) || !IS_BTREE(rel))
if (!IS_INDEX(rel) || !IS_GIN(rel))
But:
if (!IS_HASH(rel))

On Wed, Dec 13, 2023 at 09:00:01AM +0000, PG Bug reporting form wrote:
> This anomaly can be observed since 8b08f7d48 from 2018-01-19, but IMO the
> culprit is e759854a0 from 2017-02-03, which introduced the following
> asymmetry in pgstatindex.c:
> if (!IS_INDEX(rel) || !IS_BTREE(rel))
> if (!IS_INDEX(rel) || !IS_GIN(rel))
> But:
> if (!IS_HASH(rel))

Fun, let's fix that.  Would you like to write a patch?
--
Michael

Hello Michael,

13.12.2023 17:18, Michael Paquier wrote:
> On Wed, Dec 13, 2023 at 09:00:01AM +0000, PG Bug reporting form wrote:
>> This anomaly can be observed since 8b08f7d48 from 2018-01-19, but IMO the
>> culprit is e759854a0 from 2017-02-03, which introduced the following
>> asymmetry in pgstatindex.c:
>> if (!IS_INDEX(rel) || !IS_BTREE(rel))
>> if (!IS_INDEX(rel) || !IS_GIN(rel))
>> But:
>> if (!IS_HASH(rel))
> Fun, let's fix that.  Would you like to write a patch?

Yes, please look at the attached,
I've decided to change also index_open() -> relation_open() because
index_open() only adds a check, that is not suitable for this concrete
case, so there is no much sense to use the different function.
(In fact, that check was good enough at the time of e759854a0, but
8b08f7d48 changed it.)
As a result, error messages for hash indexes are becoming consistent with
other indexes. Also I've added a test case for the issue, but maybe it's
superfluous...

Best regards,
Alexander

13.12.2023 21:00, Alexander Lakhin wrote:
> Hello Michael,
>
> 13.12.2023 17:18, Michael Paquier wrote:
>> On Wed, Dec 13, 2023 at 09:00:01AM +0000, PG Bug reporting form wrote:
>>> This anomaly can be observed since 8b08f7d48 from 2018-01-19, but IMO the
>>> culprit is e759854a0 from 2017-02-03, which introduced the following
>>> asymmetry in pgstatindex.c:
>>> if (!IS_INDEX(rel) || !IS_BTREE(rel))
>>> if (!IS_INDEX(rel) || !IS_GIN(rel))
>>> But:
>>> if (!IS_HASH(rel))
>> Fun, let's fix that.  Would you like to write a patch?
>
> Yes, please look at the attached,

Looking around, I found the similar index_open() usage within pageinspect,
function hash_bitmap_info():
CREATE EXTENSION pageinspect;
CREATE TABLE t (a int, b int[], c int) PARTITION BY RANGE (a);
CREATE INDEX c_idx ON t USING hash(c);
SELECT hash_bitmap_info('c_idx'::regclass, 1);

Leads to:
TRAP: failed Assert("false"), File: "bufmgr.c", Line: 3606, PID: 3984550

I'm inclined to leave index_open() there for the internal consistency
inside pageinspect, and to use the same condition as in
bt_index_block_validate(), which called by functions bt_page_stats(...),
bt_multi_page_stats(...), bt_page_items(relname text, blkno bigint).

Functions for other index types, brin/gin/gist, don't read the index,
instead they receive a page as input.

Please look at the additional patch.

Best regards,
Alexander

On Fri, Dec 15, 2023 at 04:00:01PM +0300, Alexander Lakhin wrote:
> 13.12.2023 21:00, Alexander Lakhin wrote:
>> 13.12.2023 17:18, Michael Paquier wrote:
>> > On Wed, Dec 13, 2023 at 09:00:01AM +0000, PG Bug reporting form wrote:
>> > > This anomaly can be observed since 8b08f7d48 from 2018-01-19, but IMO the
>> > > culprit is e759854a0 from 2017-02-03, which introduced the following
>> > > asymmetry in pgstatindex.c:
>> > > if (!IS_INDEX(rel) || !IS_BTREE(rel))
>> > > if (!IS_INDEX(rel) || !IS_GIN(rel))
>> > > But:
>> > > if (!IS_HASH(rel))
>> > Fun, let's fix that.  Would you like to write a patch?
>>
>> Yes, please look at the attached,

Thanks.  The one for pgstattuple has been applied down to 12 as of
a8dd62ef4959.

> Looking around, I found the similar index_open() usage within pageinspect,
> function hash_bitmap_info():
> CREATE EXTENSION pageinspect;
> CREATE TABLE t (a int, b int[], c int) PARTITION BY RANGE (a);
> CREATE INDEX c_idx ON t USING hash(c);
> SELECT hash_bitmap_info('c_idx'::regclass, 1);
>
> Leads to:
> TRAP: failed Assert("false"), File: "bufmgr.c", Line: 3606, PID: 3984550

Right.

> I'm inclined to leave index_open() there for the internal consistency
> inside pageinspect, and to use the same condition as in
> bt_index_block_validate(), which called by functions bt_page_stats(...),
> bt_multi_page_stats(...), bt_page_items(relname text, blkno bigint).
>
> Functions for other index types, brin/gin/gist, don't read the index,
> instead they receive a page as input.

Not sure to agree on that.  It seems a bit strange to bump on a
different error when using a partitioned index or a relkind not
allowed by index_open().  And this is after looking at the pgstattuple
code.

At the end, I've switched to relation_open() for this one, added a
test and applied the second patch as of 208470136421.  I've checked
the rest and yes, it looks like there are no other holes like the
ones you have reported for the contrib modules when it comes to index
manipulations.
--
Michael