Обсуждение: Status during copy [patch]
Hi, The following (link to a) patch is a first attempt to add updates during a \copy. This is so that one has some idea of progress. The output looks as follows: kleptog=# \copy website to /tmp/g COPY OUT 0:00:00 28 lines completed. kleptog=# \copy a from /tmp/h ERROR: copy: line 186645, Invalid UNICODE character sequence found (0xe05f6c) lost synchronization with server, resetting connection kleptog=# \copy a from /tmp/i COPY IN 0:00:42 100000 lines completed. \. http://svana.org/kleptog/pgsql/psql-copy.patch File: src/bin/psql/copy.c +64 -3 The output is updated every 1000 rows. A total is listed at the end of the copy. When there's an error, no summary is printed. The output is sent to stderr, because it needs to be unbuffered and you generally don't want the progress to be redirected to a file. However, if stderr is redirected, should we just not print anything? What about non-interactive? Should it be a command-line option? Note it's not against current CVS since the anoncvs server is not working for me (error below). However, the sources don't appear to have changed too much. cvs [update aborted]: connect to anoncvs.postgresql.org(64.49.215.9):2401 failed: Connection refused Comments appreciated. -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > "the West won the world not by the superiority of its ideas or values or > religion but rather by its superiority in applying organized violence. > Westerners often forget this fact, non-Westerners never do." > - Samuel P. Huntington
How do people feel about this patch? Currently COPY doesn't even return a line count of the number of lines processed, while this patch would make psql \copy produce date/time and count every 1000 rows, then print a similar completion message. --------------------------------------------------------------------------- Martijn van Oosterhout wrote: -- Start of PGP signed section. > Hi, > > The following (link to a) patch is a first attempt to add updates during a > \copy. This is so that one has some idea of progress. The output looks as > follows: > > kleptog=# \copy website to /tmp/g > COPY OUT 0:00:00 28 lines completed. > kleptog=# \copy a from /tmp/h > ERROR: copy: line 186645, Invalid UNICODE character sequence found (0xe05f6c) > lost synchronization with server, resetting connection > kleptog=# \copy a from /tmp/i > COPY IN 0:00:42 100000 lines completed. > \. > > http://svana.org/kleptog/pgsql/psql-copy.patch > > File: src/bin/psql/copy.c +64 -3 > > The output is updated every 1000 rows. A total is listed at the end of the > copy. When there's an error, no summary is printed. > > The output is sent to stderr, because it needs to be unbuffered and you > generally don't want the progress to be redirected to a file. However, if > stderr is redirected, should we just not print anything? What about > non-interactive? Should it be a command-line option? > > Note it's not against current CVS since the anoncvs server is not working > for me (error below). However, the sources don't appear to have changed too > much. > > cvs [update aborted]: connect to anoncvs.postgresql.org(64.49.215.9):2401 > failed: Connection refused > > Comments appreciated. > -- > Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > > "the West won the world not by the superiority of its ideas or values or > > religion but rather by its superiority in applying organized violence. > > Westerners often forget this fact, non-Westerners never do." > > - Samuel P. Huntington -- End of PGP section, PGP failed! -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Yes, completely agree. Added to TODO: > o Return the number of rows loaded/unloaded --------------------------------------------------------------------------- Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > How do people feel about this patch? Currently COPY doesn't even return > > a line count of the number of lines processed, while this patch would > > make psql \copy produce date/time and count every 1000 rows, then print > > a similar completion message. > > Seems much too noisy for me. That would be appropriate behavior in a > GUI, but psql is not and never will be a GUI. Martijn's concern about > hacking the behavior depending on where stderr points demonstrates > exactly why we don't want to do this. > > I would not object to adding a line count to the COPY command completion > report (on the server side), though. That's arguably comparable to the > number-of-rows-processed values for INSERT/UPDATE/DELETE. > > regards, tom lane > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> How do people feel about this patch? Currently COPY doesn't even return
> a line count of the number of lines processed, while this patch would
> make psql \copy produce date/time and count every 1000 rows, then print
> a similar completion message.
Seems much too noisy for me. That would be appropriate behavior in a
GUI, but psql is not and never will be a GUI. Martijn's concern about
hacking the behavior depending on where stderr points demonstrates
exactly why we don't want to do this.
I would not object to adding a line count to the COPY command completion
report (on the server side), though. That's arguably comparable to the
number-of-rows-processed values for INSERT/UPDATE/DELETE.
regards, tom lane
Added to TODO:
o Have COPY return number of rows loaded/unloaded
This is a backend change to return the number of rows affected as part
of the command status string returned, like we do with UPDATE/INSERT/DELETE.
---------------------------------------------------------------------------
Jim C. Nasby wrote:
> On Mon, Jun 02, 2003 at 03:23:26PM -0400, Tom Lane wrote:
> > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > > How do people feel about this patch? Currently COPY doesn't even return
> > > a line count of the number of lines processed, while this patch would
> > > make psql \copy produce date/time and count every 1000 rows, then print
> > > a similar completion message.
> >
> > Seems much too noisy for me. That would be appropriate behavior in a
> > GUI, but psql is not and never will be a GUI. Martijn's concern about
> > hacking the behavior depending on where stderr points demonstrates
> > exactly why we don't want to do this.
>
> What about an option to enable it? Or a command to find out how many
> rows a running copy command has imported? It would be very handy to be
> able to know how far along a large copy operation is.
> --
> Jim C. Nasby (aka Decibel!) jim@nasby.net
> Member: Triangle Fraternity, Sports Car Club of America
> Give your computer some brain candy! www.distributed.net Team #1828
>
> Windows: "Where do you want to go today?"
> Linux: "Where do you want to go tomorrow?"
> FreeBSD: "Are you guys coming, or what?"
>
-- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610)
359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square,
Pennsylvania19073