Re: free(): invalid pointer , when to dumping data to a new server

>

>From: Tom Lane <tgl@sss.pgh.pa.us>Sent: Tuesday, October 27, 2020 5:04 PMTo: Lars Aksel Opsahl <Lars.Opsahl@nibio.no>Cc: Laurenz Albe <laurenz.albe@cybertec.at>; pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>Subject: Re: free(): invalid pointer , when to dumping data to a new server

>

>Lars Aksel Opsahl <Lars.Opsahl@nibio.no> writes:

>> Here is the new stackstrace not too much info that helps mee thou.

>

>This is probably the reason why "bt" is producing garbage:

>

>> BFD: warning: /var/lib/systemd/coredump/core.postmaster.26.6f7d98107f544085a41c6f8b876418d0.298120.1603810048000000 is truncated: expected core file size >= 2326896640, found: 2147483648

>

>So the core file is being truncated at 2GB, but the process image was a

>little bit larger than that.  You need to get a complete core file.

>

>First thing I'd check is the "ulimit -c" setting that the postmaster is

>being started under.  I see that systemd is capturing the core dump,

>so it could also be that there's some systemd configuration you need

>to adjust ("man systemd-coredump" might help here).

>

>                        regards, tom lane

>

>

Hi

Thanks that fixed core dump output . The ulimit -c was unlimited.

We now have a stacktrace. Right now I test with Postgis 2.5, but it also failed with Postgis 3 also, but I may go back to Postgis 3 because that's what we plan to use in production on this server and then I can remove proj-* .

Here is the output from bt

Program terminated with signal SIGABRT, Aborted.

#0  0x00007fa9c241e70f in raise () from /lib64/libc.so.6

Missing separate debuginfos, use: yum debuginfo-install CGAL-4.14-1.rhel8.x86_64 SuperLU-5.2.0-7.el8.x86_64 armadillo-9.700.2-1.el8.x86_64 arpack-3.7.0-1.el8.x86_64 atlas-3.10.3-7.el8.x86_64 audit-libs-3.0-0.17.20191104git1c2f876.el8.x86_64 blas-3.8.0-8.el8.x86_64 boost-atomic-1.66.0-7.el8.x86_64 boost-chrono-1.66.0-7.el8.x86_64 boost-date-time-1.66.0-7.el8.x86_64 boost-serialization-1.66.0-7.el8.x86_64 boost-system-1.66.0-7.el8.x86_64 boost-thread-1.66.0-7.el8.x86_64 brotli-1.0.6-1.el8.x86_64 bzip2-libs-1.0.6-26.el8.x86_64 cfitsio-3.47-1.el8.x86_64 cyrus-sasl-lib-2.1.27-1.el8.x86_64 expat-2.2.5-3.el8.x86_64 fontconfig-2.13.1-3.el8.x86_64 freetype-2.9.1-4.el8.x86_64 freexl-1.0.6-1.el8.x86_64 gdal31-libs-3.1.3-2.rhel8.x86_64 geos38-3.8.1-2.rhel8.x86_64 giflib-5.1.4-3.el8.x86_64 glibc-2.28-101.el8.x86_64 gmp-6.1.2-10.el8.x86_64 hdf5-1.10.5-4.el8.x86_64 json-c-0.13.1-0.2.el8.x86_64 keyutils-libs-1.5.10-6.el8.x86_64 krb5-libs-1.17-18.el8.x86_64 lapack-3.8.0-8.el8.x86_64 libblkid-2.32.1-22.el8.x86_64 libcap-2.26-3.el8.x86_64 libcap-ng-0.7.9-5.el8.x86_64 libcom_err-1.45.4-3.el8.x86_64 libcurl-7.61.1-12.el8.x86_64 libgcc-8.3.1-5.el8.0.2.x86_64 libgcrypt-1.8.3-4.el8.x86_64 libgfortran-8.3.1-5.el8.0.2.x86_64 libgomp-8.3.1-5.el8.0.2.x86_64 libgpg-error-1.31-1.el8.x86_64 libgta-1.2.1-1.el8.x86_64 libicu-60.3-2.el8_1.x86_64 libidn2-2.2.0-1.el8.x86_64 libjpeg-turbo-1.5.3-10.el8.x86_64 libmount-2.32.1-22.el8.x86_64 libnghttp2-1.33.0-3.el8_2.1.x86_64 libpng-1.6.34-5.el8.x86_64 libpsl-0.20.2-5.el8.x86_64 libselinux-2.9-3.el8.x86_64 libspatialite-5.0.0-beta0_1.el8.x86_64 libstdc++-8.3.1-5.el8.0.2.x86_64 libtiff-4.0.9-17.el8.x86_64 libtirpc-1.1.4-4.el8.x86_64 libuuid-2.32.1-22.el8.x86_64 libwebp-1.0.0-1.el8.x86_64 libxcrypt-4.1.1-4.el8.x86_64 libxml2-2.9.7-7.el8.x86_64 libzstd-1.4.2-2.el8.x86_64 lz4-libs-1.8.1.2-4.el8.x86_64 mpfr-3.1.6-1.el8.x86_64 netcdf-4.7.0-2.el8.x86_64 nspr-4.25.0-2.el8_2.x86_64 openblas-openmp-0.3.3-5.el8.x86_64 openblas-threads-0.3.3-5.el8.x86_64 openldap-2.4.46-11.el8_1.x86_64 openssl-libs-1.1.1c-15.el8.x86_64 pam-1.3.1-8.el8.x86_64 pcre-8.42-4.el8.x86_64 pgrouting_12-3.1.0-2.rhel8.x86_64 poppler-0.66.0-26.el8.x86_64 postgis25_12-2.5.5-1.rhel8.x86_64 proj-6.3.2-4.el8.x86_64 protobuf-c-1.3.0-4.el8.x86_64 sqlite-libs-3.26.0-6.el8.x86_64 sssd-client-2.2.3-20.el8.x86_64 systemd-libs-239-31.el8_2.2.x86_64 xz-libs-5.2.4-3.el8.x86_64 zlib-1.2.11-16.el8_2.x86_64

(gdb) bt

#0  0x00007fa9c241e70f in raise () from /lib64/libc.so.6

#1  0x00007fa9c2408b25 in abort () from /lib64/libc.so.6

#2  0x00007fa9c2461897 in __libc_message () from /lib64/libc.so.6

#3  0x00007fa9c2467fdc in malloc_printerr () from /lib64/libc.so.6

#4  0x00007fa9c24698dc in _int_free () from /lib64/libc.so.6

#5  0x00007f992fe099f9 in osgeo::proj::common::UnitOfMeasure::~UnitOfMeasure() () from /lib64/libproj.so.15

#6  0x00007fa9c2420e9c in __run_exit_handlers () from /lib64/libc.so.6

#7  0x00007fa9c2420fd0 in exit () from /lib64/libc.so.6

#8  0x000000000075beb0 in proc_exit (code=code@entry=0) at ipc.c:152

#9  0x000000000077fd21 in PostgresMain (argc=<optimized out>, argv=argv@entry=0x182b6d8, dbname=<optimized out>, username=<optimized out>) at postgres.c:4455

#10 0x00000000007085a1 in BackendRun (port=0x1819b90, port=0x1819b90) at postmaster.c:4448

#11 BackendStartup (port=0x1819b90) at postmaster.c:4139

#12 ServerLoop () at postmaster.c:1704

#13 0x00000000007094d0 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x17e7800) at postmaster.c:1377

#14 0x0000000000482c7e in main (argc=3, argv=0x17e7800) at main.c:228