Обсуждение: Converting from LATIN1 to UNICODE encoding?

I have a database that was created using the LATIN1 encoding.

Is it necessary for me to convert the database to some other encoding
(e.g. UNICODE) before I can store non-LATIN1 characters or does
PostgreSQL catch these and encode them somehow?

I have actually been attempting to convert a database by doing a pg_dump
(from the LATIN1 database) followed by a pg_restore (to one created with
the UNICODE encoding).  Seemed to work with a sparsely populated 8.0.3
database, but I am running into all sorts of problems with 7.3.10 (e.g.
corrupted database followed by corrupted pg_clog).

What about JDBC, how does it know the encoding of the data I throw at
it?  I guess I should ask this last question on the JDBC list.

Thanks in advance for any comments.

Scott

On Wed, Sep 21, 2005 at 06:16:31PM +1000, Scott Eade wrote:
> I have a database that was created using the LATIN1 encoding.
>
> Is it necessary for me to convert the database to some other encoding
> (e.g. UNICODE) before I can store non-LATIN1 characters or does
> PostgreSQL catch these and encode them somehow?
>
> I have actually been attempting to convert a database by doing a pg_dump
> (from the LATIN1 database) followed by a pg_restore (to one created with
> the UNICODE encoding).  Seemed to work with a sparsely populated 8.0.3
> database, but I am running into all sorts of problems with 7.3.10 (e.g.
> corrupted database followed by corrupted pg_clog).

Hmm.  Do tell, what database did you use as template?  Was it correctly
frozen?  If not, has it suffered from Xid wraparound?

Try dumping the template database.  If it succeeds, then it's OK.  If it
complains in the same way, with a missing pg_clog file, there lies the
problem.

--
Alvaro Herrera                                http://www.planetpostgresql.org
"En las profundidades de nuestro inconsciente hay una obsesiva necesidad
de un universo lógico y coherente. Pero el universo real se halla siempre
un paso más allá de la lógica" (Irulan)

Am Mittwoch, 21. September 2005 10:16 schrieb Scott Eade:
> Is it necessary for me to convert the database to some other encoding
> (e.g. UNICODE) before I can store non-LATIN1 characters or does
> PostgreSQL catch these and encode them somehow?

Yes and no.

> I have actually been attempting to convert a database by doing a pg_dump
> (from the LATIN1 database) followed by a pg_restore (to one created with
> the UNICODE encoding).

That is the right method.

> Seemed to work with a sparsely populated 8.0.3
> database, but I am running into all sorts of problems with 7.3.10 (e.g.
> corrupted database followed by corrupted pg_clog).

That is the symptom of a different, much bigger problem.  Encoding problems
certainly never corrupt the clog.

> What about JDBC, how does it know the encoding of the data I throw at
> it?

That depends on where your throws come from.  As you surely know, Java mostly
uses Unicode internally, so the data sent by the JDBC driver to the database
is in Unicode, but as to what encoding the data that you input into your Java
program has, you need to sort that out with the Java library functions that
you use to read that data.