Обсуждение: Detoasting and memory usage
Hi, I discovered a slightly misleading (or at least non-intuitive to me) behaviour of the pg_detoast_datum call while processing variable length objects. I was reading a large number of text fields on fetched results and noticed some strange memory behavior. I was seeing that in some situations, memory usage would shoot up dramatically, while in others it would not. Puzzled, I tracked it down my usage of DatumGetTextP(), which eventually reduces to pg_detoast_datum. As it turns out, most of the text fields I was processing were small enough not to be toasted and everything was OK. For the larger ones, pg_detoast_datum would give me a palloced copy, and since I was making all these calls in the same memory context and not pfreeing the results, I had a memory leak. Just pfreeing the result would obviously not work since pg_detoast_datum doesn't return a palloced result in most cases (i.e. smaller varlength objects). I ended up using DatumGetTextPCopy (to ensure that the result was palloced) with a pfree and all problems were solved. I guess my situation is an odd fringe case that probably doesn't happen too much, but I thought that knowledge of it might be useful to someone somewhere. -Aaron
apb18@cornell.edu wrote: > I guess my situation is an odd fringe case that probably doesn't happen > too much, but I thought that knowledge of it might be useful to someone > somewhere. See PG_FREE_IF_COPY in fmgr.h /* * Support for cleaning up detoasted copies of inputs. This must only * be used for pass-by-ref datatypes, and normally would only be used * for toastable types. If the given pointer is different from the * original argument, assume it's a palloc'd detoasted copy, and pfree * it. * * NOTE: most functions on toastable types do not have to worry about * this, but we currently require that support functions for indexes * not leak memory. */ #define PG_FREE_IF_COPY(ptr,n) \ do { \ if ((Pointer) (ptr) != PG_GETARG_POINTER(n)) \ pfree(ptr); \ } while (0) Maybe you can use that or do something similar? HTH, Joe
Hmm.. I can't believe I missed that. While that exact macro wouldn't apply in my situation (because I fetch the value from a field in a tuple and not the argument of a function), the underlying concept is the same as what I would need. Now I do not need to feel the wrath of unnecessary memory allocation/deallocation for the non-toasted case. Excellent. Thanks very much! -Aaron > #define PG_FREE_IF_COPY(ptr,n) \ > do { \ > if ((Pointer) (ptr) != PG_GETARG_POINTER(n)) \ > pfree(ptr); \ > } while (0) > > Maybe you can use that or do something similar? > > HTH, > > Joe