Обсуждение: Re: PostgreSQL search engine (Perl)
I can recommend using the Lucene search-engine, it's in Java and it's got nothing to do with PostgreSQL, but it performs excellently. -----Original Message----- From: Kaare Rasmussen [mailto:kar@kakidata.dk] Sent: 31. maj 2002 12:57 To: pgsql-general@postgresql.org Subject: [GENERAL] PostgreSQL search engine (Perl) > Search the documentation does not work right now: What I was trying to find was information about full text indexing. It's supposed to be used in a search engine. But maybe someone knows a search engine in Perl optimized for PostgreSQL? One that works ? ;-) -- Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582 Kaki Data tshirts, merchandize Fax: 3816 2501 Howitzvej 75 Åben 14.00-18.00 Web: www.suse.dk 2000 Frederiksberg Lørdag 11.00-17.00 Email: kar@kakidata.dk ---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
On Fri, 31 May 2002, Anders Nielsen wrote: > I can recommend using the Lucene search-engine, it's in Java and it's got > nothing to do with PostgreSQL, but it performs excellently. main drawback of not-db-affiliated search engines is problem to access metadata stored in db. Modern portals require a lot of such metadata, think about authorization layer, versioning etc. Besides that Lucene and many others like namazu (www.namazu.org) are good enough. OpenFTS os our attempt to overcome metadata barrier. > > -----Original Message----- > From: Kaare Rasmussen [mailto:kar@kakidata.dk] > Sent: 31. maj 2002 12:57 > To: pgsql-general@postgresql.org > Subject: [GENERAL] PostgreSQL search engine (Perl) > > > > Search the documentation does not work right now: > > What I was trying to find was information about full text indexing. > > It's supposed to be used in a search engine. But maybe someone knows a > search engine in Perl optimized for PostgreSQL? > > One that works ? ;-) > > -- > Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582 > Kaki Data tshirts, merchandize Fax: 3816 2501 > Howitzvej 75 еben 14.00-18.00 Web: www.suse.dk > 2000 Frederiksberg LЬrdag 11.00-17.00 Email: kar@kakidata.dk > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
The problem with in-database search-engines are that they're usually tagged on as a add-on feature and doesn't recieve the concentrated programming effort that is required to create a high-performance full-text indexing solution. I'd be interested in comparing performance stats with people who use OpenFTS, as I have a 8.5 gig fulltext Lucene index (including the indexed text) and I am more than happy with the response times I'm seeing. regards, Anders Nielsen -----Original Message----- From: Oleg Bartunov [mailto:oleg@sai.msu.su] Sent: 31. maj 2002 13:05 To: Anders Nielsen Cc: 'Kaare Rasmussen'; pgsql-general@postgresql.org Subject: Re: [GENERAL] PostgreSQL search engine (Perl) On Fri, 31 May 2002, Anders Nielsen wrote: > I can recommend using the Lucene search-engine, it's in Java and it's got > nothing to do with PostgreSQL, but it performs excellently. main drawback of not-db-affiliated search engines is problem to access metadata stored in db. Modern portals require a lot of such metadata, think about authorization layer, versioning etc. Besides that Lucene and many others like namazu (www.namazu.org) are good enough. OpenFTS os our attempt to overcome metadata barrier. > > -----Original Message----- > From: Kaare Rasmussen [mailto:kar@kakidata.dk] > Sent: 31. maj 2002 12:57 > To: pgsql-general@postgresql.org > Subject: [GENERAL] PostgreSQL search engine (Perl) > > > > Search the documentation does not work right now: > > What I was trying to find was information about full text indexing. > > It's supposed to be used in a search engine. But maybe someone knows a > search engine in Perl optimized for PostgreSQL? > > One that works ? ;-) > > -- > Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582 > Kaki Data tshirts, merchandize Fax: 3816 2501 > Howitzvej 75 еben 14.00-18.00 Web: www.suse.dk > 2000 Frederiksberg LЬrdag 11.00-17.00 Email: kar@kakidata.dk > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
What kind of hardware resources do you have it sitting on? CPUs, memory, disks? On Fri, 31 May 2002, Anders Nielsen wrote: > The problem with in-database search-engines are that they're usually tagged > on as a add-on feature and doesn't recieve the concentrated programming > effort that is required to create a high-performance full-text indexing > solution. > > I'd be interested in comparing performance stats with people who use > OpenFTS, as I have a 8.5 gig fulltext Lucene index (including the indexed > text) and I am more than happy with the response times I'm seeing. > > regards, > Anders Nielsen > > -----Original Message----- > From: Oleg Bartunov [mailto:oleg@sai.msu.su] > Sent: 31. maj 2002 13:05 > To: Anders Nielsen > Cc: 'Kaare Rasmussen'; pgsql-general@postgresql.org > Subject: Re: [GENERAL] PostgreSQL search engine (Perl) > > > On Fri, 31 May 2002, Anders Nielsen wrote: > > > I can recommend using the Lucene search-engine, it's in Java and it's got > > nothing to do with PostgreSQL, but it performs excellently. > > main drawback of not-db-affiliated search engines is problem to access > metadata stored in db. Modern portals require a lot of such metadata, > think about authorization layer, versioning etc. > Besides that Lucene and many others like namazu (www.namazu.org) > are good enough. OpenFTS os our attempt to overcome metadata barrier. > > > > > > -----Original Message----- > > From: Kaare Rasmussen [mailto:kar@kakidata.dk] > > Sent: 31. maj 2002 12:57 > > To: pgsql-general@postgresql.org > > Subject: [GENERAL] PostgreSQL search engine (Perl) > > > > > > > Search the documentation does not work right now: > > > > What I was trying to find was information about full text indexing. > > > > It's supposed to be used in a search engine. But maybe someone knows a > > search engine in Perl optimized for PostgreSQL? > > > > One that works ? ;-) > > > > -- > > Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582 > > Kaki Data tshirts, merchandize Fax: 3816 2501 > > Howitzvej 75 �ben 14.00-18.00 Web: www.suse.dk > > 2000 Frederiksberg L�rdag 11.00-17.00 Email: kar@kakidata.dk > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 5: Have you checked our extensive FAQ? > > > > http://www.postgresql.org/users-lounge/docs/faq.html > > > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > Sternberg Astronomical Institute, Moscow University (Russia) > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(095)939-16-83, +007(095)939-23-83 > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org >
On Fri, 31 May 2002, Anders Nielsen wrote: > The problem with in-database search-engines are that they're usually tagged > on as a add-on feature and doesn't recieve the concentrated programming > effort that is required to create a high-performance full-text indexing > solution. You're right. RDBMS isn't good for high-performance full-text indexing, im my opinion. I dont' know commercial databases which have built-in fts support. I mean 'fair' built-in search engine with transactions support and simultaneous indexing. We did all the best we could at the moment and for small and medium sites openfts is very good. There is room for improving and we hope we'll keep our hands on it. > > I'd be interested in comparing performance stats with people who use > OpenFTS, as I have a 8.5 gig fulltext Lucene index (including the indexed > text) and I am more than happy with the response times I'm seeing. I bet Lucene will be a winner. There is no magick in Lucene performance. Inverted indices are fast for searching but very-very-very slow for updating. Probably, your application doesnt require so much from database. It's another story. We've built many search engines like Lucene and we do know all pro's and contra's, but there are applicatios where transactional integrity and access to metadata are much more important than performance. > > regards, > Anders Nielsen > > -----Original Message----- > From: Oleg Bartunov [mailto:oleg@sai.msu.su] > Sent: 31. maj 2002 13:05 > To: Anders Nielsen > Cc: 'Kaare Rasmussen'; pgsql-general@postgresql.org > Subject: Re: [GENERAL] PostgreSQL search engine (Perl) > > > On Fri, 31 May 2002, Anders Nielsen wrote: > > > I can recommend using the Lucene search-engine, it's in Java and it's got > > nothing to do with PostgreSQL, but it performs excellently. > > main drawback of not-db-affiliated search engines is problem to access > metadata stored in db. Modern portals require a lot of such metadata, > think about authorization layer, versioning etc. > Besides that Lucene and many others like namazu (www.namazu.org) > are good enough. OpenFTS os our attempt to overcome metadata barrier. > > > > > > -----Original Message----- > > From: Kaare Rasmussen [mailto:kar@kakidata.dk] > > Sent: 31. maj 2002 12:57 > > To: pgsql-general@postgresql.org > > Subject: [GENERAL] PostgreSQL search engine (Perl) > > > > > > > Search the documentation does not work right now: > > > > What I was trying to find was information about full text indexing. > > > > It's supposed to be used in a search engine. But maybe someone knows a > > search engine in Perl optimized for PostgreSQL? > > > > One that works ? ;-) > > > > -- > > Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582 > > Kaki Data tshirts, merchandize Fax: 3816 2501 > > Howitzvej 75 еben 14.00-18.00 Web: www.suse.dk > > 2000 Frederiksberg LЬrdag 11.00-17.00 Email: kar@kakidata.dk > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 5: Have you checked our extensive FAQ? > > > > http://www.postgresql.org/users-lounge/docs/faq.html > > > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > Sternberg Astronomical Institute, Moscow University (Russia) > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(095)939-16-83, +007(095)939-23-83 > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Single CPU Athlon Palomino 1800 2 IDE HD RAID 1 1.5 gigabytes DDR RAM -----Original Message----- From: Marc G. Fournier To: Anders Nielsen Cc: 'Oleg Bartunov'; 'Kaare Rasmussen'; pgsql-general@postgresql.org Sent: 31-05-2002 16:25 Subject: Re: [GENERAL] PostgreSQL search engine (Perl) This message uses a character set that is not supported by the Internet Service. To view the original message content, open the attached message. If the text doesn't display correctly, save the attachment to disk, and then open it using a viewer that can display the original character set. <<message.txt>>