Bug report #20053
decimal separator in csv files
Status: | Closed | ||
---|---|---|---|
Priority: | Normal | ||
Assignee: | Alessandro Pasotti | ||
Category: | Unknown | ||
Affected QGIS version: | 3.5(master) | Regression?: | No |
Operating System: | Mac OSX 10.14.3 | Easy fix?: | No |
Pull Request or Patch supplied: | Yes | Resolution: | no timely feedback |
Crashes QGIS or corrupts data: | No | Copied to github as #: | 27875 |
Description
When opening cvs files with data that contains decimals separator with "." it change after opening in QGIS to "," the problem then is that when saving the csv file to GeoJson it will become a "string" instead of numbers. This habit has come with version 3, it does not do this in 2.18 (never had).
Associated revisions
Fixes #20053 decimal separator in csv files
Actually it had nothing to do with CSV being the
source, but it was the json exporter passing
the values through all field formatters except for
the fallback.
This resulted in all fields using a 'Range' formatter
(which is the default for all numeric types) passing
through the formatter and being returned as strings
in the json. Worse, if the locale was not a "dot"
locale and decimal separator was on, the resulting
string could not be easily converted into its original
numeric type.
Now, instead of checking for the fallback formatter
only, there is a white list of formatters that
can be applied when we want a json.
This is a temporary solution because the "right" way
to do it would be either a flag in the formatter to
tell if it can be applied when converting to json
and/or other "data" formats (csv etc.) or a different
new method similar to representValue.
History
#1 Updated by Alessandro Pasotti about 6 years ago
- Assignee set to Alessandro Pasotti
Just to be sure: if you change your locale number formatting options to "English" or "C" in QGIS options general tab, does that fixes the problem (override current locale must be checked) ?
#2 Updated by Giovanni Manghi about 6 years ago
- Regression? changed from No to Yes
- Status changed from Open to Feedback
- Priority changed from Normal to High
was the csv added with the "add delimited text layer" tool?
#3 Updated by Alessandro Pasotti about 6 years ago
- Pull Request or Patch supplied changed from No to Yes
- Status changed from Feedback to In Progress
- Priority changed from High to Normal
- Regression? changed from Yes to No
#4 Updated by Alessandro Pasotti about 6 years ago
Giovanni, this had actually nothing to do with CSV, it was a bug in the json formatter.
#5 Updated by Soren Lindqvist about 6 years ago
Alessandro Pasotti wrote:
Just to be sure: if you change your locale number formatting options to "English" or "C" in QGIS options general tab, does that fixes the problem (override current locale must be checked) ?
Yes, I did right now change the locale formatting options to "C", the import does not change the "." to "," and exporting to GeoJson is now correct. It seems like the locale settings override "." and also export it as such and turning it to a string instead of number.
Great if you have find the solutions.
Thank you
#6 Updated by Anonymous about 6 years ago
- % Done changed from 0 to 100
- Status changed from In Progress to Closed
Applied in changeset qgis|859b39a63438e426f227114747a49ddcfd49bc09.
#7 Updated by Alessandro Pasotti about 6 years ago
- Resolution set to fixed/implemented
Thanks for checking, actually I think that even with C locale the output was not entirely correct: you did not get the ',' decimal separator and you did not get the thousand group separator (like "'" in italian locale), but you still did get the numer as a string (surrounded by quotes).
#8 Updated by Soren Lindqvist almost 6 years ago
- Status changed from Closed to Reopened
Problem back again....
#9 Updated by Giovanni Manghi almost 6 years ago
- Status changed from Reopened to Feedback
- Resolution deleted (
fixed/implemented)
Soren Lindqvist wrote:
Problem back again....
on 3.4.4, correct?
#10 Updated by Giovanni Manghi almost 6 years ago
Soren Lindqvist wrote:
Problem back again....
same OS (and version)?
#11 Updated by Soren Lindqvist almost 6 years ago
Yes
went back and installed v 2.18.28, export to geojson was correct!
Problem is now back in v 3.4.4 and in the master 3.5
#12 Updated by Soren Lindqvist almost 6 years ago
Mac OSX 10.14.3
#13 Updated by Giovanni Manghi almost 6 years ago
- Operating System changed from OSX version 10.11.6 to Mac OSX 10.14.3
- Status changed from Feedback to Open
- Affected QGIS version changed from 3.2.3 to 3.5(master)
#14 Updated by Alessandro Pasotti almost 6 years ago
- Status changed from Open to Feedback
Can you please provide some more detail and a step-by-step procedure and sample data to reproduce?
The original issue has been identified and fixed and it is protected by a test which is not failing, hence it highly likely that you have found a completely new issue.
#15 Updated by Soren Lindqvist almost 6 years ago
I will, at the moment on the road traveling...
#16 Updated by Giovanni Manghi almost 6 years ago
Soren Lindqvist wrote:
I will, at the moment on the road traveling...
please leave further feedback.
#17 Updated by Alessandro Pasotti over 5 years ago
- Status changed from Feedback to Closed
- Resolution set to no timely feedback