Hi, again,
I just had this wacky idea, and wanted to share it:
what do you think of having the dataset divided among several servers,
and sending the query to all of them, and then just have the
application "unify" the results from all the servers?
Would that work for this kind of *one table* search? (there are no
joins, and will never be). I think it should, but: what do you think?
Ildefonso.
On Tue, Aug 17, 2010 at 9:51 PM, Jose Ildefonso Camargo Tolosa
<ildefonso.camargo@gmail.com> wrote:
> Hi!
>
> I'm analyzing the possibility of using PostgreSQL to store a huge
> amount of data (around 1000M records, or so....), and these, even
> though are short (each record just have a timestamp, and a string that
> is less than 128 characters in length), the strings will be matched
> against POSIX Regular Expressions (different regexps, and maybe
> complex).
>
> Because I don't have a system large enough to test this here, I have
> to ask you (I may borrow a medium-size server, but it would take a
> week or more, so I decided to ask here first). How is the performance
> of Regexp matching in PostgreSQL? Can it use indexes? My guess is:
> no, because I don't see a way of generally indexing to match regexp :(
> , so, tablescans for this huge dataset.....
>
> What do you think of this?
>
> Sincerely,
>
> Ildefonso Camargo
>