Обсуждение: Dynamic execution returning large result sets

Поиск
Список
Период
Сортировка

Dynamic execution returning large result sets

От
Emrul
Дата:
Hi,

I have a function that returns an SQL string as follows:
CREATE OR REPLACE FUNCTION t1() RETURNS text AS
$$
BEGIN
RETURN 'SELECT * FROM mytable';
END
$$ LANGUAGE plpgsql;

and I want to create a second function (t2) that will execute the string
returned by t1() and return the results.  I thought about using RETURN QUERY
EXECUTE as documented here:
https://www.postgresql.org/docs/9.6/static/plpgsql-control-structures.html
but there's a note towards the end that says '/if a PL/pgSQL function
produces a very large result set, performance might be poor: data will be
written to disk to avoid memory exhaustion, but the function itself will not
return until the entire result set has been generated./'

Is there any other way I can achieve execution of the dynamic SQL from t1()
without having the whole result set retrieved inside the function itself?

note: My example above is simplified, my real t1() function takes some
parameters and generates SQL depending upon those parameters.


Thanks!



--
View this message in context: http://postgresql.nabble.com/Dynamic-execution-returning-large-result-sets-tp5929177.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.


Re: Dynamic execution returning large result sets

От
Adrian Klaver
Дата:
On 11/06/2016 05:12 PM, Emrul wrote:
> Hi,
>
> I have a function that returns an SQL string as follows:
> CREATE OR REPLACE FUNCTION t1() RETURNS text AS
> $$
> BEGIN
> RETURN 'SELECT * FROM mytable';
> END
> $$ LANGUAGE plpgsql;
>
> and I want to create a second function (t2) that will execute the string
> returned by t1() and return the results.  I thought about using RETURN QUERY
> EXECUTE as documented here:
> https://www.postgresql.org/docs/9.6/static/plpgsql-control-structures.html
> but there's a note towards the end that says '/if a PL/pgSQL function
> produces a very large result set, performance might be poor: data will be
> written to disk to avoid memory exhaustion, but the function itself will not
> return until the entire result set has been generated./'
>
> Is there any other way I can achieve execution of the dynamic SQL from t1()
> without having the whole result set retrieved inside the function itself?

https://www.postgresql.org/docs/9.5/static/plpgsql-cursors.html

"Rather than executing a whole query at once, it is possible to set up a
cursor that encapsulates the query, and then read the query result a few
rows at a time. One reason for doing this is to avoid memory overrun
when the result contains a large number of rows. (However, PL/pgSQL
users do not normally need to worry about that, since FOR loops
automatically use a cursor internally to avoid memory problems.) A more
interesting usage is to return a reference to a cursor that a function
has created, allowing the caller to read the rows. This provides an
efficient way to return large row sets from functions."

>
> note: My example above is simplified, my real t1() function takes some
> parameters and generates SQL depending upon those parameters.
>
>
> Thanks!
>
>
>
> --
> View this message in context:
http://postgresql.nabble.com/Dynamic-execution-returning-large-result-sets-tp5929177.html
> Sent from the PostgreSQL - general mailing list archive at Nabble.com.
>
>


--
Adrian Klaver
adrian.klaver@aklaver.com


Re: Dynamic execution returning large result sets

От
Emrul
Дата:
Read up on refcursors - exactly what I wanted, thank you Adrian!



--
View this message in context:
http://postgresql.nabble.com/Dynamic-execution-returning-large-result-sets-tp5929177p5929211.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.