Google
 

Monday, January 10, 2005

Anyone still thinks we don't need 38-digit arithmetics?

it started as an problem description on firebird-devel


"Marco Kregar" wrote:
>

> CREATE TABLE TEST (
> V1 DECIMAL(18,4),
> V2 DECIMAL(18,4),
> V3 DECIMAL(18,4)
> );
>
> INSERT INTO TEST (V1, V2, V3) VALUES (10000000, 1, 1);
>
> SELECT V1*V2*V3 FROM TEST
>
> The select statement generate an unespected INTEGER OVERFLOW


This is by design. The resulting scale will be 4 + 4 + 4 = 12, to avoid any
precision loss. Since the maximum precision for NUMERIC is 18, it means that
only 6 decimal digits are allowed for the integral part of the result. Your
10M value overflows this limit.

Anyone still thinks we don't need 38-digit arithmetics? ;-)


Dmitry

No comments: