Обсуждение: Detecting schema changes via WAL logs
Hello everyone,
I’m building an open-source data integration tool (SQLpipe) and am now trying to incorporate CDC into the product.
Is it possible to detect when schema changes happen via the WAL? The WAL doesn’t have to tell me what was changed, it just needs to tell me when something changed.
For example, anytime I run create table or some other DDL command, there is a begin and commit message in the log, with nothing in between. If there aren’t any other operations that create such “empty” logs, it seems possible to just re-scan the tables that are being synced for schema changes whenever one of these logs is encountered.
Is it possible to detect when schema changes happen via the WAL? The WAL doesn’t have to tell me what was changed, it just needs to tell me when something changed.
For example, anytime I run create table or some other DDL command, there is a begin and commit message in the log, with nothing in between. If there aren’t any other operations that create such “empty” logs, it seems possible to just re-scan the tables that are being synced for schema changes whenever one of these logs is encountered.
On 19.02.22 17:42, Cal Mitchell wrote: > Hello everyone, > > I’m building an open-source data integration tool (SQLpipe) and am now > trying to incorporate CDC into the product. > > Is it possible to detect when schema changes happen via the WAL? The WAL > doesn’t have to tell me /what/ was changed, it just needs to tell me > /when/ something changed. > > For example, anytime I run create table or some other DDL command, there > is a begin and commit message in the log, with nothing in between. If > there aren’t any other operations that create such “empty” logs, it > seems possible to just re-scan the tables that are being synced for > schema changes whenever one of these logs is encountered. I remember having seen a similar question recently here - that is detecting schema changes. Also I think having seen WAL to SQL scripts somewhere. I might be wrong on both accounts, but maybe you want to review the last few months of list archives if you haven't done so already and maybe check postgres' Debian package repository if there aren't WAL to SQL scripts somewhere. As I said, this is what my fuzzy in-brain SQL returns so please don't SIGKILL me if my query was too fuzzy indeen. *t