Bug report #4955: Wrong results, when calculated results are too long for DBF field - QGIS Application - QGIS Issue Tracking

Bug report #4955

Wrong results, when calculated results are too long for DBF field

Added by zirneklitis - almost 13 years ago. Updated about 10 years ago.

Status:	Closed
Priority:	Normal
Assignee:	-
Category:	Vectors
Affected QGIS version:	master	Regression?:	No
Operating System:		Easy fix?:	No
Pull Request or Patch supplied:	No	Resolution:	duplicate
Crashes QGIS or corrupts data:	No	Copied to github as #:	14750

Description

This was tested both on 1.7.3 and 1.9.90.

A new field in a shape file was created with the field calculator (within 'Attribute table' window). An area of a polygon was calculated with $area. The returned value was absolutely wrong. It turns out that the field is too short for the result. But there were no warnings!!! Further experiments return following results:

Field type: N(10,0) Result: 727869376 — wrong result (N. B. – there are only 9 numbers)

Field type: N(11,0) Result: -2147483648 — wrong result (N. B. – the integer part of the result is 11 numbers)

Field type: N(13,1) Result: 65152378815,7 — correct result

Additional verification of field type and number should be introduce before writing out to shape file. I suppose this is similar problem to Bug #3500.

Latvia.7z - Shape file (compresed with 7z) used for the experiment. (917 Bytes) zirneklitis -, 2012-02-07 01:37 AM

Related issues

History

#1 Updated by Giovanni Manghi over 12 years ago

I can confirm this issue, it can indeed lead to further errors.

#2 Updated by alobo - over 12 years ago

Actually, it's a problem of the field type:
If Precision = 0, even if field type has been set to Float
the result is wrong, Precision must be > 0,
I think that if Precision = 0, then the type is set to integer
(even if the user has selected float) and the results are wrong.
Field width=20, real and Precision=3 gives a correct result
Field width=20, real and Precision=0 gives a wrong result
Agus

#3 Updated by Giovanni Manghi over 12 years ago

Nathan, I added you as watcher to this ticket as you worked in the field calculator. Not sure if you want/can do something about this issue, eventually just remove your name from the list. Thanks.