Обсуждение: validating foreign tables
The API for FDW validators doesn't appear to have any way that the validator function can check that the defined foreign table shape matches the FDW options sanely. Maybe it's a chicken and egg problem, but there seems to be something missing, unless I'm mistaken. We'll have the info when we come to make plan estimates, but that seems like the wrong place to be doing this sort of validation. Can we extend the validator API somehow to make this possible? cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> The API for FDW validators doesn't appear to have any way that the
> validator function can check that the defined foreign table shape
> matches the FDW options sanely.
Huh? The options ought to be orthogonal to the table column info.
If they're not, maybe you need to rethink your option definitions.
regards, tom lane
On 02/21/2011 06:56 PM, Tom Lane wrote: > Andrew Dunstan<andrew@dunslane.net> writes: >> The API for FDW validators doesn't appear to have any way that the >> validator function can check that the defined foreign table shape >> matches the FDW options sanely. > Huh? The options ought to be orthogonal to the table column info. > If they're not, maybe you need to rethink your option definitions. > > Well, let's take a couple of cases. 1. My old favorite, file as a text array. 2. A hypothetical RSS feed, where the options specify which RSS fields we want. Of course, we could just let these break or give odd results when we run a SELECT if the foreign table doesn't match what's expected. file_fdw will presumably break if the input file has rows with the wrong number of columns, just as COPY does. But there will be cases, like the two above, where a sanity check on the table shape could usefully be done at validation time as opposed to run time, and it would be nice to be able to do such a check. cheers andrew
On Tue, Feb 22, 2011 at 10:12, Andrew Dunstan <andrew@dunslane.net> wrote: >>> The API for FDW validators doesn't appear to have any way that the >>> validator function can check that the defined foreign table shape >>> matches the FDW options sanely. >> >> Huh? The options ought to be orthogonal to the table column info. >> If they're not, maybe you need to rethink your option definitions. > > Well, let's take a couple of cases. > > 1. My old favorite, file as a text array. > 2. A hypothetical RSS feed, where the options specify which RSS fields we > want. I think we need to overhaul validators in 9.2 listening to FDW developers' opinions anyway. The text array is an example, but there should be many other requirements. Personally, I'd like to have a method to list available options from SQL. We should also consider column-level options for foreign tables then. -- Itagaki Takahiro
On 02/21/2011 08:59 PM, Itagaki Takahiro wrote: > I think we need to overhaul validators in 9.2 listening to FDW developers' > opinions anyway. The text array is an example, but there should be many > other requirements. Personally, I'd like to have a method to list available > options from SQL. We should also consider column-level options for foreign > tables then. > Ok, I guess. It just seems to me like it will be harder to extend the API later than now, so if we can reasonably foresee a likely need we should try to provide for it. cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> On 02/21/2011 06:56 PM, Tom Lane wrote:
>> Huh? The options ought to be orthogonal to the table column info.
>> If they're not, maybe you need to rethink your option definitions.
> Well, let's take a couple of cases.
> 1. My old favorite, file as a text array.
> 2. A hypothetical RSS feed, where the options specify which RSS fields
> we want.
As above, I claim that an FDW that has such options is badly designed to
begin with. Why can't you generate the RSS command on-the-fly from the
table rowtype?
> Of course, we could just let these break or give odd results when we run
> a SELECT if the foreign table doesn't match what's expected. file_fdw
> will presumably break if the input file has rows with the wrong number
> of columns, just as COPY does. But there will be cases, like the two
> above, where a sanity check on the table shape could usefully be done at
> validation time as opposed to run time, and it would be nice to be able
> to do such a check.
I can't get excited about this. For one thing, you'd then need to worry
about involving the validator in random ALTER TABLE situations, not just
when changing the options it's supposed to be checking.
regards, tom lane
Andrew Dunstan <andrew@dunslane.net> writes:
> On 02/21/2011 08:59 PM, Itagaki Takahiro wrote:
>> I think we need to overhaul validators in 9.2 listening to FDW developers'
>> opinions anyway.
> Ok, I guess. It just seems to me like it will be harder to extend the
> API later than now, so if we can reasonably foresee a likely need we
> should try to provide for it.
Perhaps we should put a large friendly "EXPERIMENTAL, SUBJECT TO CHANGE"
notice on all the FDW API stuff? Just tell people up front that we're
not prepared to promise any API stability yet. There's stuff we *know*
is lacking (it's read-only, the optimization support sucks) in addition
to whatever we may later realize is misdesigned.
regards, tom lane
2011/2/22 Tom Lane <tgl@sss.pgh.pa.us>: > Andrew Dunstan <andrew@dunslane.net> writes: >> On 02/21/2011 08:59 PM, Itagaki Takahiro wrote: >>> I think we need to overhaul validators in 9.2 listening to FDW developers' >>> opinions anyway. > >> Ok, I guess. It just seems to me like it will be harder to extend the >> API later than now, so if we can reasonably foresee a likely need we >> should try to provide for it. > > Perhaps we should put a large friendly "EXPERIMENTAL, SUBJECT TO CHANGE" > notice on all the FDW API stuff? Just tell people up front that we're > not prepared to promise any API stability yet. There's stuff we *know* > is lacking (it's read-only, the optimization support sucks) in addition > to whatever we may later realize is misdesigned. > > regards, tom lane +1 regards Pavel Stehule > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >