Re: Same double precision operations, different results

Поиск
Список
Период
Сортировка
От Tom Lane
Тема Re: Same double precision operations, different results
Дата
Msg-id 6262.1392312408@sss.pgh.pa.us
обсуждение исходный текст
Ответ на Same double precision operations, different results  (Daniel Vázquez <daniel2d2art@gmail.com>)
Ответы Re: Same double precision operations, different results  (Daniel Vázquez <daniel2d2art@gmail.com>)
Список pgsql-hackers
Daniel Vázquez <daniel2d2art@gmail.com> writes:
> Please can someone tell me why this behavior? Thx!!

You're confusing numeric and double precision; in particular this
calculation is *not* being done in float8, but numeric:

> # select (0.766238989559398 * 0.766238989559398 * 1 + 0.642555686986733 *
> 0.642555686986733) calc;

> (*) Why this calculation produces 1 and not 0.999999999999999633651488135693

Actually, it's not producing 1, but a smidgen more:

regression=# set extra_float_digits TO 3;
SET
regression=# select ( cast (  cos(radians(39.9826557))  * cos(radians(39.9826557))  * cos(radians(-0.04773120000004383)
-radians(-0.04773120000004383))  + sin(radians(39.9826557))  * sin(radians(39.9826557)) as double precision ));
float8       
 
---------------------1.00000000000000022
(1 row)

You've got roundoff error either way, but this way happens to be in the
direction that makes acos() complain.
        regards, tom lane



В списке pgsql-hackers по дате отправления:

Предыдущее
От: Daniel Vázquez
Дата:
Сообщение: Same double precision operations, different results
Следующее
От: Alvaro Herrera
Дата:
Сообщение: Re: truncating pg_multixact/members