Bug report #11965

Performance issues editing in 2.6.x/master compared to 2.4

Added by Matthew Yandell-Thomas almost 10 years ago. Updated over 9 years ago.

Status:Closed
Priority:Severe/Regression
Assignee:Martin Dobias
Category:Vectors
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:
Crashes QGIS or corrupts data:No Copied to github as #:20172

Description

Hello,

Our users have been complaining of performance bugs when editing a spatialite layer in 2.6 and 2.6.1 when compared to 2.4 which they were using previously.

Whenever the user makes an edit to the spatialite vector layer (eg split, merge, change an attribute) QGIS 2.6.x hangs and increases the length of time it takes to perform their work. These problems did not occur with the same database in 2.4.

The spatialite DB was created in FME and is indexed. It is 1.5GB in size and contains 629,201 features. A new database has been created using FME which also repaired geometry in the process but the problems still persist in 2.6.x

I have uploaded a video of the issue to illustrate what is going on:

https://www.youtube.com/watch?v=u-agfrnq6B8

Thanks,

Matthew

Associated revisions

Revision 255cbd2c
Added by Martin Dobias over 9 years ago

Fix #11965 - improve performance of selectedFeatures()

The slowness of merge / split features tools was caused by the change in the logic
in selectedFeatures(): instead of fetching individual features one by one by ID,
the whole layer is traversed. Such approach makes sense when many features are selected,
but with few features there is considerable delay when dealing with big layers.

The implementation is not ideal, but for some common cases the performance is much better.

Merging of features now does not request selected features when not necessary.

When rendering, avoid using layer's extent() method that may force expensive
calculation of layer's extent.

History

#1 Updated by Nathan Woodrow almost 10 years ago

Do you have any plugins that are enabled? We have had people report this before and it turned out to be a plugin.

#2 Updated by Matthew Yandell-Thomas almost 10 years ago

Hi Nathan, just about to test that. Disabling them would do the trick for trying this out or do they need to be completely uninstalled?

*Edit

After disabling all 'after market' plugins and restarting QGIS the problem still remains

Plugins used are: Go2Streetview, Dockable Mirror Map, Auto saver and a modified version of the Quick Multi Attribute Edit tool

#3 Updated by Nathan Woodrow almost 10 years ago

Run QGIS with --noplugins command line arg.

If this doesn't change anything you can use --optionspath "qgis" --configpath "qgis" to create new QGIS settings and run using those. This is good for isolating if it's settings/plugin issue or something else

#4 Updated by Matthew Yandell-Thomas almost 10 years ago

Just tried those suggestions and after doing both the slow response to edits still occurred.

#5 Updated by Giovanni Manghi almost 10 years ago

  • Category set to Vectors
  • Status changed from Open to Feedback

Matthew Yandell-Thomas wrote:

Just tried those suggestions and after doing both the slow response to edits still occurred.

I have tested with a SL database created within QGIS and I cannot see such issue, but now I cannot test a large dataset.

Do you see the same issue if creating the DB within QGIS?

Can you attach a minimal sample of your DB where the difference is still visible?

#6 Updated by Matthew Yandell-Thomas almost 10 years ago

Hi Giovanni,

Yea, the problem persists if I create the DB in both QGIS 2.4 and 2.6

Due to commercial reasons we aren't able to upload any data at this time as it's not supposed to be publicly available yet. It's due for release later in the year and after that point we would be allowed to upload it freely. We may be able to send it directly to someone to take a look at if that's a possible solution?

Thanks

#7 Updated by Giovanni Manghi almost 10 years ago

Matthew Yandell-Thomas wrote:

Hi Giovanni,

We may be able to send it directly to someone to take a look at if that's a possible solution?

if you want you can send it to me and I'll have a look: giovanni dot manghi at gmail dot com

#8 Updated by Jukka Rahkonen almost 10 years ago

It is simple and fast to create big Spatialite databases from OpenStreetMap data with GDAL. Try to repeat your issue for example with the German dataset http://download.geofabrik.de/europe/germany-latest.osm.pbf.

ogr2ogr -f SQLite -dsco spatialite=yes germany.sqlite germany-latest.osm.pbf -gt 20000 -progress --config OGR_SQLITE_SYNCHRONOUS OFF --config OSM_COMPRESS_NODES YES

#9 Updated by Matthew Yandell-Thomas almost 10 years ago

Hello Jukka,

We've just finished creating that DB using ogr and created a spatialite file that was nearly 14gig.

Loaded the polygon layer into 2.6 and it still hangs, but for even longer than before. And in 2.4 it performs all edits instantly.

Can I ask what OS you are running? We are getting these problems on all our PCs in the office which are Windows 7. We are about to try it on a linux box shortly

Thanks

#10 Updated by Giovanni Manghi almost 10 years ago

  • Affected QGIS version changed from 2.6.1 to master
  • Subject changed from Performance issues editing SpatiaLite DBs in 2.6.x compared to 2.4 to Performance issues editing in 2.6.x/master compared to 2.4
  • Status changed from Feedback to Open
  • Priority changed from Normal to Severe/Regression
  • Target version set to Version 2.8
  • Operating System deleted (Windows)
  • OS version deleted (7)

I confirm that there is a very clear decrease of speed in vector editing operation, and it affects also PostGIS (and maybe shapefiles, not tested yet).

To see clearly a user must use a very large vector.

It affects also master.

#11 Updated by Martin Dobias over 9 years ago

  • Assignee set to Martin Dobias

#12 Updated by Martin Dobias over 9 years ago

  • Status changed from Open to Closed

Also available in: Atom PDF