On Thu, Oct 8, 2009 at 11:50 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Another possible approach, which isn't perfect either, is the idea of
>> allowing COPY to generate a single column of output of type text[].
>> That greatly reduces the number of possible error cases, and at least
>> gets the data into the DB where you can hack on it. But it's still
>> going to be painful for some use cases.
>
> Yeah, that connects to the previous discussion about refactoring COPY
> into a series of steps that the user can control.
>
> Ultimately, there's always going to be a tradeoff between speed and
> flexibility. It may be that we should just say "if you want to import
> dirty data, it's gonna cost ya" and not worry about the speed penalty
> of subtransaction-per-row. But that still leaves us with the 2^32
> limit. I wonder whether we could break down COPY into sub-sub
> transactions to work around that...
How would that work? Don't you still need to increment the command counter?
...Robert