Bug report #4955
Wrong results, when calculated results are too long for DBF field
| Status: | Closed | ||
|---|---|---|---|
| Priority: | Normal | ||
| Assignee: | - | ||
| Category: | Vectors | ||
| Affected QGIS version: | master | Regression?: | No |
| Operating System: | Easy fix?: | No | |
| Pull Request or Patch supplied: | No | Resolution: | duplicate |
| Crashes QGIS or corrupts data: | No | Copied to github as #: | 14750 |
Description
This was tested both on 1.7.3 and 1.9.90.
A new field in a shape file was created with the field calculator (within 'Attribute table' window). An area of a polygon was calculated with $area. The returned value was absolutely wrong. It turns out that the field is too short for the result. But there were no warnings!!! Further experiments return following results:
Field type: N(10,0) Result: 727869376 — wrong result (N. B. – there are only 9 numbers)
Field type: N(11,0) Result: -2147483648 — wrong result (N. B. – the integer part of the result is 11 numbers)
Field type: N(13,1) Result: 65152378815,7 — correct result
Additional verification of field type and number should be introduce before writing out to shape file. I suppose this is similar problem to Bug #3500.
Related issues
History
#1
Updated by Giovanni Manghi over 12 years ago
I can confirm this issue, it can indeed lead to further errors.
#2
Updated by alobo - over 12 years ago
Actually, it's a problem of the field type:
If Precision = 0, even if field type has been set to Float
the result is wrong, Precision must be > 0,
I think that if Precision = 0, then the type is set to integer
(even if the user has selected float) and the results are wrong.
Field width=20, real and Precision=3 gives a correct result
Field width=20, real and Precision=0 gives a wrong result
Agus
#3
Updated by Giovanni Manghi over 12 years ago
Nathan, I added you as watcher to this ticket as you worked in the field calculator. Not sure if you want/can do something about this issue, eventually just remove your name from the list. Thanks.
#4
Updated by Paolo Cavallini about 12 years ago
- Target version set to Version 2.0.0
#5
Updated by Giovanni Manghi almost 12 years ago
- Priority changed from High to Normal
#6
Updated by Giovanni Manghi over 10 years ago
see also #10186
#7
Updated by Jürgen Fischer over 10 years ago
- Target version changed from Version 2.0.0 to Future Release - Lower Priority
#8
Updated by Giovanni Manghi about 10 years ago
- Resolution set to duplicate
- Status changed from Open to Closed
duplicate of #4766