On Wed, Mar 6, 2024 at 1:07 PM Amit Langote <amitlangote09@gmail.com> wrote:
> Hi Tomas,
>
> On Wed, Mar 6, 2024 at 6:30 AM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
> > I'd say 2.5GB in ExecutorState seems a bit excessive ... Seems there's
> > some memory management issue? My guess is we're not releasing memory
> > allocated while parsing the JSON or building JSON output.
> >
> > I'm not attaching the data, but I can provide that if needed - it's
> > about 600MB compressed. The structure is not particularly complex, it's
> > movie info from [1] combined into a JSON document (one per movie).
>
> Thanks for the report.
>
> Yeah, I'd like to see the data to try to drill down into what's piling
> up in ExecutorState. I want to be sure of if the 1st, query functions
> patch, is not implicated in this, because I'd like to get that one out
> of the way sooner than later.
I tracked this memory-hogging down to a bug in the query functions
patch (0001) after all. The problem was with a query-lifetime cache
variable that was never set to point to the allocated memory. So a
struct was allocated and then not freed for every row where it should
have only been allocated once.
I've fixed that bug in the attached. I've also addressed some of
Jian's comments and made quite a few cleanups of my own.
Now I'll go look if Himanshu's concerns are a blocker for committing 0001. ;)
--
Thanks, Amit Langote