Обсуждение: [NOVICE] Regression test : pg_conversion validation
In PostgreSQL9.6, the test opr_sanity from server regression suite has added following validations for the System Catalog pg_conversion.
SELECT p1.oid, p1.conname
FROM pg_conversion as p1
WHERE condefault AND
convert('ABC'::bytea, pg_encoding_to_char(conforencoding),
pg_encoding_to_char(contoencoding)) != 'ABC';
It looks like this validation might not reflect the correct behaviour for a well defined conproc for certain encodings.
e.g. Consider LATIN1 as src_encoding and EBCDIC as dst_encoding in the above query. Now the convert() result for this should be:
select convert('ABC'::bytea,'LATIN1','EBCDIC');
convert
----------
\xc1c2c3
The resultant hex value above(which is the actual EBCDIC code for 'ABC') is not same as ASCII hex value for 'ABC'. Hence the validation query from opr_sanity would show this conproc in the output indicating that the conproc is not well defined. This does not seem correct.
Please let me know if this understanding is correct. If it is, then the validation test should indicate that the specific conversion is a valid conversion.
Regards,
Neha
Neha
neha khatri <nehakhatri5@gmail.com> writes:
> It looks like this validation might not reflect the correct behaviour for a
> well defined conproc for certain encodings.
> e.g. Consider LATIN1 as src_encoding and EBCDIC as dst_encoding in the
> above query.
Um ... well, that's already a bridge too far. Postgres does not support
any encodings that aren't ASCII supersets, and the possibility that we
would do so in future isn't measurably different from zero. There's too
much code that depends on that assumption, and too little benefit to
getting rid of it.
If you can show a problem case that doesn't involve EBCDIC, then I'm
all ears.
regards, tom lane
On Mon, Feb 13, 2017 at 6:27 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
neha khatri <nehakhatri5@gmail.com> writes:
> It looks like this validation might not reflect the correct behaviour for a
> well defined conproc for certain encodings.
> e.g. Consider LATIN1 as src_encoding and EBCDIC as dst_encoding in the
> above query.
Um ... well, that's already a bridge too far. Postgres does not support
any encodings that aren't ASCII supersets, and the possibility that we
would do so in future isn't measurably different from zero. There's too
much code that depends on that assumption, and too little benefit to
getting rid of it.
If you can show a problem case that doesn't involve EBCDIC, then I'm
all ears.
Can't find another encoding, that isn't superset of ASCII.
But how PostgreSQL deals with data movement from EBCDIC machine to
ASCII machine. Would it be the applications responsibility to appropriately
convert before such transfer happens?
Regards,
Neha
I'm not sure why this would be a problem. Postgres isn't going to run on the EBCDIC machine so the data will just have to be run through a code inverter going back an forth.
There are many of them available for this kind of application.
There are many of them available for this kind of application.
Carol
Sent from my iPad
On Mon, Feb 13, 2017 at 6:27 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:neha khatri <nehakhatri5@gmail.com> writes:
> It looks like this validation might not reflect the correct behaviour for a
> well defined conproc for certain encodings.
> e.g. Consider LATIN1 as src_encoding and EBCDIC as dst_encoding in the
> above query.
Um ... well, that's already a bridge too far. Postgres does not support
any encodings that aren't ASCII supersets, and the possibility that we
would do so in future isn't measurably different from zero. There's too
much code that depends on that assumption, and too little benefit to
getting rid of it.
If you can show a problem case that doesn't involve EBCDIC, then I'm
all ears.Can't find another encoding, that isn't superset of ASCII.But how PostgreSQL deals with data movement from EBCDIC machine toASCII machine. Would it be the applications responsibility to appropriatelyconvert before such transfer happens?Regards,Neha
On Mon, Feb 13, 2017 at 11:57 PM, Carol Walter <carol.walter@sbcglobal.net> wrote:
I'm not sure why this would be a problem. Postgres isn't going to run on the EBCDIC machine so the data will just have to be run through a code inverter going back an forth.
There are many of them available for this kind of application.Carol
It is not a problem as such. Just that this kind of conversion in Postgres
might simplify the data transfer.
Regards,
Neha
Neha
neha khatri <nehakhatri5@gmail.com> writes:
> On Mon, Feb 13, 2017 at 11:57 PM, Carol Walter <carol.walter@sbcglobal.net>
> wrote:
>> I'm not sure why this would be a problem. Postgres isn't going to run on
>> the EBCDIC machine so the data will just have to be run through a code
>> inverter going back an forth.
>> There are many of them available for this kind of application.
> It is not a problem as such. Just that this kind of conversion in Postgres
> might simplify the data transfer.
It just doesn't come up often enough to justify our investing the work to
create and maintain such a facility within Postgres. I just did a quick
search of the PG list archives, and cannot find another request for this
feature within the last ten years. (In the same interval I find just one
request to support EBCDIC as a native encoding, which is even less likely
to happen.) So basically there's no interest in this problem among our
user community, and we have lots of higher-priority things to do.
regards, tom lane
On Tue, Feb 14, 2017 at 2:16 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
neha khatri <nehakhatri5@gmail.com> writes:
> On Mon, Feb 13, 2017 at 11:57 PM, Carol Walter <carol.walter@sbcglobal.net>
> wrote:
>> I'm not sure why this would be a problem. Postgres isn't going to run on
>> the EBCDIC machine so the data will just have to be run through a code
>> inverter going back an forth.
>> There are many of them available for this kind of application.
> It is not a problem as such. Just that this kind of conversion in Postgres
> might simplify the data transfer.
It just doesn't come up often enough to justify our investing the work to
create and maintain such a facility within Postgres. I just did a quick
search of the PG list archives, and cannot find another request for this
feature within the last ten years. (In the same interval I find just one
request to support EBCDIC as a native encoding, which is even less likely
to happen.) So basically there's no interest in this problem among our
user community, and we have lots of higher-priority things to do.
The readily available code inverter could be one reason for almost NIL demand
for this conversion. Probably when in need, the users are making use of code
inverter.
Now I understand Postgres' ASCII affinity and reason for not supporting Non-ASCII
encoding.
Thanks,
Neha