Обсуждение: how to make query
Hi Team,
--
Best Regards,
Sachin Kumar
we are using Python Django and Postgresql. Our requirement is to search 10 Million Card data and bring specific records as a result after checking the record every time.
We have to find data from a 2nd table that is related to the first one which we are checking using the loop.
For Ex:-
10000 data - loop 10000 times
to find data from the 2nd table.
This makes our query too slower and it gets time out.
Best Regards,
Sachin Kumar
Hi Philip,
I am using Python Django and PostgreSQL to build our report app.
I am using below mention query to find some details. the query mention below is running fine in PostgreSQL command line and giving result in 2 sc for 2 million cards but when i i am try the same in Django application it is taking too much of time and the web page is expired while running this query.
can anyone help how to make query faster in Django? Or help me in correcting this query to be used in Django
rows = M_Assign.objects.raw(''' select C.id,M."merchant_name",C."MERCHANT_ID",S."store_id" Owning_Store_ID ,S."store_name" Owning_Store_Name,F."franchisee_name" Owning_Group,C."ACCOUNT_NUMBER",C."STORE_ID",C."GIFT_LIST_REF" FROM vdaccount_card_assign C INNER JOIN vd_merchant_master M ON C."MERCHANT_ID"=M."merchant_id" AND C."MERCHANT_ID"='003561002966107' INNER JOIN vd_store_master S ON S."store_id"=C."GIFT_LIST_REF" OR C."GIFT_LIST_REF"='' INNER JOIN vd_franchisee F ON S."franchisee_id"=F."franchisee_id" '''
I am using below mention query to find some details. the query mention below is running fine in PostgreSQL command line and giving result in 2 sc for 2 million cards but when i i am try the same in Django application it is taking too much of time and the web page is expired while running this query.
can anyone help how to make query faster in Django? Or help me in correcting this query to be used in Django
rows = M_Assign.objects.raw(''' select C.id,M."merchant_name",C."MERCHANT_ID",S."store_id" Owning_Store_ID ,S."store_name" Owning_Store_Name,F."franchisee_name" Owning_Group,C."ACCOUNT_NUMBER",C."STORE_ID",C."GIFT_LIST_REF" FROM vdaccount_card_assign C INNER JOIN vd_merchant_master M ON C."MERCHANT_ID"=M."merchant_id" AND C."MERCHANT_ID"='003561002966107' INNER JOIN vd_store_master S ON S."store_id"=C."GIFT_LIST_REF" OR C."GIFT_LIST_REF"='' INNER JOIN vd_franchisee F ON S."franchisee_id"=F."franchisee_id" '''
On Thu, Jan 7, 2021 at 7:28 PM Philip Semanchuk <philip@americanefficient.com> wrote:
> On Jan 7, 2021, at 5:15 AM, Sachin Kumar <sachinkumaras@gmail.com> wrote:
>
> Hi Team,
>
> we are using Python Django and Postgresql. Our requirement is to search 10 Million Card data and bring specific records as a result after checking the record every time.
>
> We have to find data from a 2nd table that is related to the first one which we are checking using the loop.
>
> For Ex:-
> 10000 data - loop 10000 times
> to find data from the 2nd table.
>
> This makes our query too slower and it gets time out.
Hi Sachin,
I’m not sure based on your description, but it sounds like you need to tell Django to select related objects all in one query rather than one at a time. If that’s the case, then your problem is a Django issue, not a Postgres one. Check out select_related() --
https://docs.djangoproject.com/en/3.1/ref/models/querysets/#django.db.models.query.QuerySet.select_related
Cheers
Philip
Best Regards,
Sachin Kumar
On Thu, Jan 7, 2021 at 6:26 AM Sachin Kumar <sachinkumaras@gmail.com> wrote:
Hi Philip,I am using Python Django and PostgreSQL to build our report app.
I am using below mention query to find some details. the query mention below is running fine in PostgreSQL command line and giving result in 2 sc for 2 million cards but when i i am try the same in Django application it is taking too much of time and the web page is expired while running this query.
can anyone help how to make query faster in Django? Or help me in correcting this query to be used in Django
rows = M_Assign.objects.raw(''' select C.id,M."merchant_name",C."MERCHANT_ID",S."store_id" Owning_Store_ID ,S."store_name" Owning_Store_Name,F."franchisee_name" Owning_Group,C."ACCOUNT_NUMBER",C."STORE_ID",C."GIFT_LIST_REF" FROM vdaccount_card_assign C INNER JOIN vd_merchant_master M ON C."MERCHANT_ID"=M."merchant_id" AND C."MERCHANT_ID"='003561002966107' INNER JOIN vd_store_master S ON S."store_id"=C."GIFT_LIST_REF" OR C."GIFT_LIST_REF"='' INNER JOIN vd_franchisee F ON S."franchisee_id"=F."franchisee_id" '''On Thu, Jan 7, 2021 at 7:28 PM Philip Semanchuk <philip@americanefficient.com> wrote:
> On Jan 7, 2021, at 5:15 AM, Sachin Kumar <sachinkumaras@gmail.com> wrote:
>
> Hi Team,
>
> we are using Python Django and Postgresql. Our requirement is to search 10 Million Card data and bring specific records as a result after checking the record every time.
>
> We have to find data from a 2nd table that is related to the first one which we are checking using the loop.
>
> For Ex:-
> 10000 data - loop 10000 times
> to find data from the 2nd table.
>
> This makes our query too slower and it gets time out.
Hi Sachin,
I’m not sure based on your description, but it sounds like you need to tell Django to select related objects all in one query rather than one at a time. If that’s the case, then your problem is a Django issue, not a Postgres one. Check out select_related() --
https://docs.djangoproject.com/en/3.1/ref/models/querysets/#django.db.models.query.QuerySet.select_related
Cheers
Philip
Hi Sachin,
As a general matter, people on this list will be able to help you more effectively if you provide some DDL, ideally some sample data, and an "explain analyze" report from your current setup, along with the query SQL. If you can simplify your table structures in the DDL to focus only on the place where you are having performance problems, that's even better.
Best,
Steve