Hello all,
I have some needs that seem to support changing Postgres slightly to
give user programs a lot more power over how to process logging output
that neither the log collector nor the syslog output can well-satisfy
as-is.
I am approaching this from the angle of increasing power by exposing
the log collector ("syslogger") pipe protocol.
Included is a patch whose general aesthetic is to touch as little of
the logging code as possible while achieving maximum power in a
contrib and stand-alone sample program. However, a more satisfying
treatment that may be slightly more invasive to the logging system is
very welcome.
The general idea here is that the logging collector pipe protocol is a
pretty reasonable one for non-Postgres programs to consume as-is
pretty easily. The proof of concept consists of three things:
* A hook into the log collector. It is small but a little weird
compared to the other hooks in the system.
* A contrib, pg_logcollectdup. This contrib lets one forward logs to
a named pipe specified in postgresql.conf.
* A stand-alone program, pg_logcollect_sample, that renders protocol
traffic newline separated and with its binary fields converted to
readable text.
The patch can also be seen from https://github.com/fdr/postgres.git in
the branch 'logdup'. I do rebase this at-will for the time being.
I have a few detailed dissatisfactions with this approach, but I'd
rather hear your own dissatisfactions. I also don't like the name.
A demo of configuring all of the above and seeing some output follows.
I can paste this in one go on my machine. It creates /tmp/pg-logdup
and /tmp/pgdata-logdup. All spawned processes can be listed
afterwards with 'jobs'.
# Install postgres and pg_logcollectdup contrib
./configure --prefix=/tmp/pg-logdup
make -sj16 install
cd contrib/pg_logcollectdup/
make -s install
/tmp/pg-logdup/bin/initdb -D /tmp/pgdata-logdup
mkfifo /tmp/pgdata-logdup/log-pipe
# Configure postgresql.conf
echo "logging_collector = on" >> \
/tmp/pgdata-logdup/postgresql.conf
echo "shared_preload_libraries = 'pg_logcollectdup'" >> \
/tmp/pgdata-logdup/postgresql.conf
echo "logcollectdup.destination = '/tmp/pgdata-logdup/log-pipe'" >> \
/tmp/pgdata-logdup/postgresql.conf
# Set up destination pipe
mkfifo /tmp/pgdata-logdup/log-pipe
# Build sample pipe formatter
make pg_logcollect_sample
# Run it in the background
./pg_logcollect_sample \
< /tmp/pgdata-logdup/log-pipe \
> /tmp/pgdata-logdup/duplogs.txt &
# Run Postgres with a non-default port for convenience
/tmp/pg-logdup/bin/postgres -D /tmp/pgdata-logdup --port=2345 &
# Prevent race against file creation, then look at the logs
sleep 1
cat /tmp/pgdata-logdup/duplogs.txt
--
fdr