Обсуждение: Database-wide vacuum can take a long time, during which tables are not being analyzed
Database-wide vacuum can take a long time, during which tables are not being analyzed
От
"Steven Flatt"
Дата:
Here is a potential problem with the auto-vacuum daemon, and I'm wondering if anyone has considered this. To avoid transaction ID wraparound, the auto-vacuum daemon will periodically determine that it needs to do a DB-wide vacuum, which takes a long time. On our system, it is on the order of a couple of weeks. (The system is very busy and there is a lot of I/O going on pretty much 24/7). During this period of time, there is nothing to automatically analyze any of the tables, leading to further performance problems. What are your thoughts on having the DB-wide vacuum running on a separate thread so that the daemon can concurrently wake up and take care of analyzing tables?
Steven Flatt wrote: > Here is a potential problem with the auto-vacuum daemon, and I'm > wondering if anyone has considered this. To avoid transaction ID > wraparound, the auto-vacuum daemon will periodically determine that it > needs to do a DB-wide vacuum, which takes a long time. On our system, > it is on the order of a couple of weeks. (The system is very busy and > there is a lot of I/O going on pretty much 24/7). During this period of > time, there is nothing to automatically analyze any of the tables, > leading to further performance problems. What are your thoughts on > having the DB-wide vacuum running on a separate thread so that the > daemon can concurrently wake up and take care of analyzing tables? Two issues here: 1)XID Wraparound: There has been work done on this already, and in 8.2 I believe there will no longer be a requirement that a database wide vacuum be issued, rather, XID wraparound will be managed on a per table basis rather than per database, so that will solve this problem. 2)Concurrent Vacuuming: There has been a lot of talk about multiple-concurrent vacuums and I believe that this is required in the long run, but it's not here yet, and won't be in 8.2, hopefully it will get done for 8.3. Matt
Re: Database-wide vacuum can take a long time, duringwhich tables are not being analyzed
От
"Simon Riggs"
Дата:
On Wed, 2006-11-01 at 14:15 -0500, Steven Flatt wrote: > Here is a potential problem with the auto-vacuum daemon, and I'm > wondering if anyone has considered this. To avoid transaction ID > wraparound, the auto-vacuum daemon will periodically determine that it > needs to do a DB-wide vacuum, which takes a long time. On our system, > it is on the order of a couple of weeks. (The system is very busy and > there is a lot of I/O going on pretty much 24/7). During this period > of time, there is nothing to automatically analyze any of the tables, > leading to further performance problems. What are your thoughts on > having the DB-wide vacuum running on a separate thread so that the > daemon can concurrently wake up and take care of analyzing tables? Yes, do it. Every couple of weeks implies a transaction rate of ~~500tps, so I'd be interested to hear more about your system. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com
Re: Database-wide vacuum can take a long time, duringwhich tables are not being analyzed
От
"Steven Flatt"
Дата:
Sorry, I think there's a misunderstanding here. Our system is not doing near that number of transactions per second. I meant that the duration of a single DB-wide vacuum takes on the order of a couple of weeks. The time between DB-wide vacuums is a little over a year, I believe.
Every couple of weeks implies a transaction rate of ~~500tps, so I'd be
interested to hear more about your system.
--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com
Re: Database-wide vacuum can take a long time, duringwhich tables are not being analyzed
От
Alvaro Herrera
Дата:
Steven Flatt wrote: > Sorry, I think there's a misunderstanding here. Our system is not doing > near that number of transactions per second. I meant that the duration of a > single DB-wide vacuum takes on the order of a couple of weeks. The time > between DB-wide vacuums is a little over a year, I believe. I wonder if this is using some vacuum delay setting? If that's the case, I think you could manually run a database-wide vacuum with a zero vacuum delay setting, so that said vacuum takes less time to finish (say, once every 8 months). (8.2 pretty much solves this issue BTW, by not requiring database-wide vacuums). -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.