On Tue, 2 Sep 2003, Marc G. Fournier wrote:
>
> okay, first off ... what URL? I've been working on, and testing, archives
> all week, and altho getting things down below a 2min search appears to be
> near impossible (anyone know something *better* then mnogosearch that will
> handle >250k URLs and growing ... ? we tried ht/Dig before, and the
> resources it suck'd were outrageous) ...
Hi Marc, I use htdig at work to index a rather large web site and it
doesn't use resources all that bad. My htdig db directory is only 250
megs and it screams. Are you sure it was that bad?
Would it be ok if I pointed a test instance of htdig at postgresql.org at
night and indexed it to get an idea of how much space it'll take to index
it? I'm a huge fan of htdig/ warts and all, so I'd kinda like to see if
it can be made to work.