Обсуждение: Re: [External] Multiple COPY on the same table
> What is the goal you are trying to achieve here.
> To make pgdump/restore faster?
> To make replication faster?
> To make backup faster ?
None of the above.
We got csv files from external vendor which are 880GB in total size, in 44 files. Some of the large tables had COPY running for several hours. I was just thinking of a faster way to load.
On Mon, 20 Aug 2018 at 12:53, Ravi Krishna <sravikrishna@aol.com> wrote: > > What is the goal you are trying to achieve here. > > To make pgdump/restore faster? > > To make replication faster? > > To make backup faster ? > > None of the above. > > We got csv files from external vendor which are 880GB in total size, in 44 files. Some of the large tables had COPY runningfor several hours. I was just thinking of a faster way to load. Seems like #4... #4 - To Make Recovery faster Using COPY pretty much *is* the "faster way to load"... The main thing you should consider doing to make it faster is to drop indexes and foreign keys from the tables, and recreate them afterwards. -- When confronted by a difficult problem, solve it by reducing it to the question, "How would the Lone Ranger handle this?"
I guess this should help you, Ravi. https://www.postgresql.org/docs/10/static/populate.html On 8/20/18, 10:30 PM, "Christopher Browne" <cbbrowne@gmail.com> wrote: On Mon, 20 Aug 2018 at 12:53, Ravi Krishna <sravikrishna@aol.com> wrote: > > What is the goal you are trying to achieve here. > > To make pgdump/restore faster? > > To make replication faster? > > To make backup faster ? > > None of the above. > > We got csv files from external vendor which are 880GB in total size, in 44 files. Some of the large tables had COPYrunning for several hours. I was just thinking of a faster way to load. Seems like #4... #4 - To Make Recovery faster Using COPY pretty much *is* the "faster way to load"... The main thing you should consider doing to make it faster is to drop indexes and foreign keys from the tables, and recreate them afterwards. -- When confronted by a difficult problem, solve it by reducing it to the question, "How would the Lone Ranger handle this?"