Обсуждение: tsearch stop words
How do I get a list of what tsearch considers a stop word? eg. 'and', 'or', 'the', 'up', 'down', etc. There seem to be heaps of them...! Chris
Christopher, current implementation is ugly, we still didn't move functionality from OpenFTS to tsearch. Look at makedict subdirectory to create your custom dictionary. Default list is in engstoptree[] defined in dic/porter_english.dct On Mon, 2 Sep 2002, Christopher Kings-Lynne wrote: > How do I get a list of what tsearch considers a stop word? > > eg. 'and', 'or', 'the', 'up', 'down', etc. There seem to be heaps of > them...! > > Chris > > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Ummm...I totally don't understand the format of that array at all! static ESWNODE engstoptree[] = { {'m',L,9,126}, {'d',L,4,71}, {'b',L,2,40}, {'a',F,0,14}, {'c',0,0,62}, {'f',L,2,79}, {'e',0,0,75}, {'h',0,1,90}, {'i',F,0,108}, {'t',L,4,177}, {'o',L,2,135}, {'n',0,0,131}, {'s',0,0,156}, How do I figure out the actual word it's matching? Chris > -----Original Message----- > From: Oleg Bartunov [mailto:oleg@sai.msu.su] > Sent: Monday, 2 September 2002 5:33 PM > To: Christopher Kings-Lynne > Cc: Hackers > Subject: Re: [HACKERS] tsearch stop words > > > Christopher, > > current implementation is ugly, we still didn't move functionality > from OpenFTS to tsearch. Look at makedict subdirectory to create your > custom dictionary. Default list is in engstoptree[] defined > in dic/porter_english.dct > > On Mon, 2 Sep 2002, Christopher Kings-Lynne wrote: > > > How do I get a list of what tsearch considers a stop word? > > > > eg. 'and', 'or', 'the', 'up', 'down', etc. There seem to be heaps of > > them...! > > > > Chris > > > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 2: you can get off all lists at once with the unregister command > > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > > > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > Sternberg Astronomical Institute, Moscow University (Russia) > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(095)939-16-83, +007(095)939-23-83 >
On Mon, 2 Sep 2002, Christopher Kings-Lynne wrote: > Ummm...I totally don't understand the format of that array at all! stop words in suffix tree :) I've told it's default list of stop words and it's certainly far from perfect. Please find attached file with stop words we used (I just found on my notebook). But again, it's better to build your own dictionary specific for you domain, using makedict script. OpenFTS is much more flexible in this respect and we hope we'd be able to implement most features of OpenFTS in tsearch, so OpenFTS would be just a perl wrapper. Oleg > > static ESWNODE engstoptree[] = { > {'m',L,9,126}, > {'d',L,4,71}, > {'b',L,2,40}, > {'a',F,0,14}, > {'c',0,0,62}, > {'f',L,2,79}, > {'e',0,0,75}, > {'h',0,1,90}, > {'i',F,0,108}, > {'t',L,4,177}, > {'o',L,2,135}, > {'n',0,0,131}, > {'s',0,0,156}, > > How do I figure out the actual word it's matching? > > Chris > > > > -----Original Message----- > > From: Oleg Bartunov [mailto:oleg@sai.msu.su] > > Sent: Monday, 2 September 2002 5:33 PM > > To: Christopher Kings-Lynne > > Cc: Hackers > > Subject: Re: [HACKERS] tsearch stop words > > > > > > Christopher, > > > > current implementation is ugly, we still didn't move functionality > > from OpenFTS to tsearch. Look at makedict subdirectory to create your > > custom dictionary. Default list is in engstoptree[] defined > > in dic/porter_english.dct > > > > On Mon, 2 Sep 2002, Christopher Kings-Lynne wrote: > > > > > How do I get a list of what tsearch considers a stop word? > > > > > > eg. 'and', 'or', 'the', 'up', 'down', etc. There seem to be heaps of > > > them...! > > > > > > Chris > > > > > > > > > ---------------------------(end of broadcast)--------------------------- > > > TIP 2: you can get off all lists at once with the unregister command > > > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > > > > > > > Regards, > > Oleg > > _____________________________________________________________ > > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > > Sternberg Astronomical Institute, Moscow University (Russia) > > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > > phone: +007(095)939-16-83, +007(095)939-23-83 > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83