Bug report #20760

Edits in GeoJson datasources are not saved anymore

Added by Ehsan Aliverdi almost 6 years ago. Updated over 5 years ago.

Status:Open
Priority:High
Assignee:-
Category:Data Provider
Affected QGIS version:3.4.4 Regression?:Yes
Operating System:Windows Easy fix?:No
Pull Request or Patch supplied:No Resolution:
Crashes QGIS or corrupts data:No Copied to github as #:28580

Description

I tested this issue in Qgis 3.4.0, 3.4.1, 3.4.2 in all the same issue
in our company we work with Geojson files in Qgis. When editing the attributes of an existing feature in Geojson layer, QGIS creates a temp layer and applies the changes on that new temp layer and never applies the changes on original layer.
this problem wasn't there in version 3.2.3

equipment.geojson (3.96 KB) Ehsan Aliverdi, 2018-12-17 11:10 PM

2018-12-18_9-15-07.png - QGIS about (30.5 KB) Ehsan Aliverdi, 2018-12-17 11:14 PM

edit-geojson.gif (386 KB) Ben Hur Pintor, 2018-12-18 01:35 PM

geoJsonProblem.gif (1.68 MB) Ehsan Aliverdi, 2018-12-20 06:00 AM

History

#1 Updated by Giovanni Manghi almost 6 years ago

  • Subject changed from QGIS does not update GeoJson Files after creation to Edits in GeoJson datasources are not saved anymore
  • Category changed from Attribute table to Data Provider
  • Affected QGIS version changed from 3.4.0 to 3.4.2
  • Operating System deleted (Windows 10)
  • Crashes QGIS or corrupts data changed from Yes to No

#2 Updated by Ben Hur Pintor almost 6 years ago

Hi Ehsan,

Can you provide a sample dataset to test this? I just tested editing a GeoJSON layer on 3.4.2 (Ubuntu 18.04) and everything works as expected. I can edit the fields and even add/delete fields. No temp layer is created and the original layer is edited.

Thanks!

Sincerely,
Ben Hur

#3 Updated by Ehsan Aliverdi almost 6 years ago

Ben Hur Pintor wrote:

Hi Ehsan,

Can you provide a sample dataset to test this? I just tested editing a GeoJSON layer on 3.4.2 (Ubuntu 18.04) and everything works as expected. I can edit the fields and even add/delete fields. No temp layer is created and the original layer is edited.

Thanks!

Sincerely,
Ben Hur

Hi Ben
Thanks for reply
Please find a sample dataset enclosed. I have tested this in more than 10 different pc with windows. I just did another test with the following qgis(see the image enclosed)
step to reproduce the issue:
-Put equipment layer on editable mode
-use identify tool to id a feature(anyone will work)
-use edit feature form to edit an attribute of a feature
-save the changes
-toggle edit
-use identify tool to id the feature.
You will see nothing is changed.

Kindly Regards
Ehsan

#4 Updated by Ben Hur Pintor almost 6 years ago

Ehsan Aliverdi wrote:

Hi Ben
Thanks for reply
Please find a sample dataset enclosed. I have tested this in more than 10 different pc with windows. I just did another test with the following qgis(see the image enclosed)
step to reproduce the issue:
-Put equipment layer on editable mode
-use identify tool to id a feature(anyone will work)
-use edit feature form to edit an attribute of a feature
-save the changes
-toggle edit
-use identify tool to id the feature.
You will see nothing is changed.

Kindly Regards
Ehsan

Hi Ehsan,

For my setup (QGIS 3.4.2 on Ubuntu 18.04), I could edit the attributes of the GeoJSON. Kindly check the GIF attached if I followed your instructions to reproduce the bug. Maybe this is a Windows-specific problem. If possible, could you also send a video or GIF of the bug in action.

Thank you.

Sincerely,
Ben Hur

#5 Updated by Ehsan Aliverdi almost 6 years ago

Ben Hur Pintor wrote:

Ehsan Aliverdi wrote:

Hi Ben
Thanks for reply
Please find a sample dataset enclosed. I have tested this in more than 10 different pc with windows. I just did another test with the following qgis(see the image enclosed)
step to reproduce the issue:
-Put equipment layer on editable mode
-use identify tool to id a feature(anyone will work)
-use edit feature form to edit an attribute of a feature
-save the changes
-toggle edit
-use identify tool to id the feature.
You will see nothing is changed.

Kindly Regards
Ehsan

Hi Ehsan,

For my setup (QGIS 3.4.2 on Ubuntu 18.04), I could edit the attributes of the GeoJSON. Kindly check the GIF attached if I followed your instructions to reproduce the bug. Maybe this is a Windows-specific problem. If possible, could you also send a video or GIF of the bug in action.

Thank you.

Sincerely,
Ben Hur

Thanks alot for your reply. I am using windows. for your information I attached a gif file which shows the problem. we use qgis on 20 PCs in our company and all have the same issue because of this issue we downgraded every qgis to 2.3.2 which works perfect.

Regards
Ehsan

#6 Updated by Paul Blottiere over 5 years ago

FWIW, I didn't succeed in reproducing the issue either.

#7 Updated by Giovanni Manghi over 5 years ago

  • Affected QGIS version changed from 3.4.2 to 3.4.4

Paul Blottiere wrote:

FWIW, I didn't succeed in reproducing the issue either.

I can easily replicate on 3.4.4 (following the steps in the attached cast) on Windows (clean install of both QGIS and the OS).

#8 Updated by Paul Blottiere over 5 years ago

  • Operating System set to Windows

Hi @Giovanni,

I can easily replicate on 3.4.4 (following the steps in the attached cast) on Windows (clean install of both QGIS and the OS).

OK, so according to @Ben Hur comment and my own tests on Debian, it seems that it's pretty specific to Windows.

#9 Updated by Peter Petrik over 5 years ago

unable to replicate on MacOS too. So definitely windows-only issue.

#10 Updated by Adam Liddell over 5 years ago

Also replicated this bug on 3.6.0 with Windows 7. The edits are written to a *.geojson.tmp file, but never copied over the top of the *.geojson file.

#11 Updated by Adam Liddell over 5 years ago

Assuming this is the driver used for GeoJSON, could this be the offending section in GDAL OGR: https://github.com/OSGeo/gdal/blob/ab9e80368ef662712c963191fd86fde2c5b75600/gdal/ogr/ogrsf_frmts/geojson/ogrgeojsondatasource.cpp#L1056-L1098

Here's the steps it's taking:

  1. The new *.geojson.tmp file containing the changes is written
  2. The *.geojson is moved to *.geojson.bak
  3. The *.geojson.tmp file is renamed to *.geojson
  4. The *.geojson.bak file is removed

On Windows, this appears to be failing in step 2, as the tmp file is created but the bak file is not. Perhaps this is due to how Windows does file locking, meaning the rename is prohibited as the file appears to be open? Unfortunately I can't seem to get the log output from this section of code, so I don't see 'Cannot create backup copy' anywhere.

#12 Updated by Amine Aboufirass over 5 years ago

Has there been any progress on this? I also submitted a similar question on stackexchange... https://gis.stackexchange.com/questions/319450/qgis-values-in-attribute-table-become-null-after-saving

Pehaps there is a workaround?

#13 Updated by Evgeniy Lazarev over 5 years ago

I asked Oct-31-2018 on stackexcange about this problem: https://gis.stackexchange.com/questions/300860/why-does-qgis-3-4-and-higher-create-tmp-file-instead-of-refresh-editing-layer-a Since then I have to use 3.2.3 QGIS version.

#14 Updated by Edmond Lai over 5 years ago

I am having the same issue with QGIS version 3.6.2-Noosa on Windows 10.

#15 Updated by louis de clebs over 5 years ago

I am having the same issue on windows 7 (Unfortunately have to use windows at work).
Might also have something to do with line endings? When I commit a geojson file created by QGIS, git put a warning saying LF line ending in the geojson file are replaced by CRLF. Would it be something there https://github.com/qgis/QGIS/blob/master/src/core/qgsvectorfilewriter.cpp#L1060 ?
  • When adding a feature to the geojson file it is correctly handled
  • When deleting or updating a feature the issue is happening

Only workaround at this stage is to save the file, close qgis, delete old geojson, rename new file by removing ".tmp" extension and re-open qgis... Or is there an easier way?

Also available in: Atom PDF