2 million queries against a table

Поиск
Список
Период
Сортировка
От Adam Sanchez
Тема 2 million queries against a table
Дата
Msg-id CAEP=qeOSNniLWhCKkyU774xFH9feQ=SKsuB-zoXthxdMj5GtKw@mail.gmail.com
обсуждение исходный текст
Ответы Re: 2 million queries against a table  (Ron <ronljohnsonjr@gmail.com>)
Re: 2 million queries against a table  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-general
Hi

I need to run 2 million queries against a three columns table t
(s,p,o) which size is 10 billions rows. The data type of each column
is string.  The server has 512G RAM, 32 cores and 14T SSD (RAID 0)

Only two types of queries:

select s p o from t where s = param
select s p o from t where o = param

If I store the table in a Postgresql database takes 6 hours using a
Java ThreadPoolExecutor.

Do you think Postgresql itself can speed up the queries processing
even more? What would be the best strategy?

These are my ideas:

1. Use Spark to launch queries against the table (extracted from
Postgresql) loaded in a dataframe
2. Use PG-Strom, an extension module of PostgreSQL with GPU support
and launch the queries against the table.


Any suggestion will be appreciated

Best,



В списке pgsql-general по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: single table - fighting a seq scan
Следующее
От: Ron
Дата:
Сообщение: Re: 2 million queries against a table