martin:
couple o things:
use something like sed to pre process the file taking out the erronous
data, if you know what the bad data is
one thing i use sed for is html escape single quotes and the like,
it should work well for you
the regex for substituting like that is:
s/old/new/g
the sed command operates regexes with an -e switch :
sed -e s/error/correction/g errorfile>correctedfile
takes errorfile in and changes any occurence of 'error' to 'correction'
and
prints all data to correctedfile
also if its a flat file you could just use the copy command in psql
instead of inserts:
\copy tablein from filename