Bug report #14359

Primary key for topology face layer is fully scanned upfront

Added by Sandro Santilli over 8 years ago. Updated over 5 years ago.

Status:Closed
Priority:Normal
Assignee:-
Category:Data Provider/PostGIS
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:end of life
Crashes QGIS or corrupts data:No Copied to github as #:22343

Description

Followup to #14356.

The DBManager TopoViewr still triggers a full scan of face tables (dunno why only face table).
From the PostgreSQL logs:

2016-02-23 07:40:00.969 CET QGIS 13988 rt strk LOG:  duration: 0.370 ms  statement: DECLARE qgis_1 BINARY CURSOR FOR SELECT "face_id" FROM "rt09_wgs84_topo"."face" 
2016-02-23 07:40:00.971 CET QGIS 13988 rt strk LOG:  duration: 2.474 ms  statement: FETCH FORWARD 2000 FROM qgis_1
2016-02-23 07:40:00.977 CET QGIS 13988 rt strk LOG:  duration: 1.146 ms  statement: FETCH FORWARD 2000 FROM qgis_1
2016-02-23 07:40:00.982 CET QGIS 13988 rt strk LOG:  duration: 1.011 ms  statement: FETCH FORWARD 2000 FROM qgis_1
...
2016-02-23 07:40:03.758 CET QGIS 13988 rt strk LOG:  duration: 1.097 ms  statement: FETCH FORWARD 2000 FROM qgis_1
2016-02-23 07:40:03.763 CET QGIS 13988 rt strk LOG:  duration: 1.156 ms  statement: FETCH FORWARD 2000 FROM qgis_1
2016-02-23 07:40:03.769 CET QGIS 13988 rt strk LOG:  duration: 0.494 ms  statement: FETCH FORWARD 2000 FROM qgis_1
2016-02-23 07:40:03.771 CET QGIS 13988 rt strk LOG:  duration: 0.081 ms  statement: CLOSE qgis_1
2016-02-23 07:40:03.771 CET QGIS 13988 rt strk LOG:  duration: 0.133 ms  statement: COMMIT

I don't think the uniqueness of "face_id" should be checked, as it is a primary key.

History

#1 Updated by Sandro Santilli over 8 years ago

Same "face_id" scan happens if I just open the "face" table from "Add PostGIS Layer" dialog.
No key scan if I open the "edge_data" table.
I've no idea about what could be different between the two

#2 Updated by Sandro Santilli over 8 years ago

  • Assignee changed from Jürgen Fischer to Nyall Dawson
  • Tag changed from topology, dbmanager to performance, bigtable

I found the query comes from the StatisticalSummaryDockWidget. Dunno why would such widget be on by default.

#2  0x00007fff5e6dce7e in QgsPostgresConn::openCursor (this=0x7fff3c009aa0, cursorName=..., sql=...)
    at /usr/src/qgis/qgis-master/src/providers/postgres/qgspostgresconn.cpp:1024
#3  0x00007fff5e712d98 in QgsPostgresFeatureIterator::declareCursor (this=0x332aa90, whereClause=..., limit=-1, closeOnFail=false, orderBy=...)
    at /usr/src/qgis/qgis-master/src/providers/postgres/qgspostgresfeatureiterator.cpp:588
#4  0x00007fff5e70c1f8 in QgsPostgresFeatureIterator::QgsPostgresFeatureIterator (this=0x332aa90, source=0x32ec900, ownSource=false, request=...)
    at /usr/src/qgis/qgis-master/src/providers/postgres/qgspostgresfeatureiterator.cpp:161
#5  0x00007fff5e715d47 in QgsPostgresFeatureSource::getFeatures (this=0x32ec900, request=...)
    at /usr/src/qgis/qgis-master/src/providers/postgres/qgspostgresfeatureiterator.cpp:786
#6  0x00007ffff44b74f4 in QgsVectorLayerFeatureIterator::QgsVectorLayerFeatureIterator (this=0x323eab0, source=0x304a0d0, ownSource=true, 
    request=...) at /usr/src/qgis/qgis-master/src/core/qgsvectorlayerfeatureiterator.cpp:171
#7  0x00007ffff4460bb2 in QgsVectorLayer::getFeatures (this=0x2f72e40, request=...) at /usr/src/qgis/qgis-master/src/core/qgsvectorlayer.cpp:939
#8  0x00007ffff447db9d in QgsVectorLayer::getValues (this=0x2f72e40, fieldOrExpression=..., ok=@0x7fffffff9f4e: false, selectedOnly=false)
    at /usr/src/qgis/qgis-master/src/core/qgsvectorlayer.cpp:3144
#9  0x00007ffff447e42d in QgsVectorLayer::getDoubleValues (this=0x2f72e40, fieldOrExpression=..., ok=@0x7fffffff9f4e: false, selectedOnly=false, 
    nullCount=0x7fffffff9f60) at /usr/src/qgis/qgis-master/src/core/qgsvectorlayer.cpp:3176
#10 0x00007ffff71c50ec in QgsStatisticalSummaryDockWidget::refreshStatistics (this=0x1020b80)
    at /usr/src/qgis/qgis-master/src/app/qgsstatisticalsummarydockwidget.cpp:121
#11 0x00007ffff71c6995 in QgsStatisticalSummaryDockWidget::layerChanged (this=0x1020b80, layer=0x2f72e40)
    at /usr/src/qgis/qgis-master/src/app/qgsstatisticalsummarydockwidget.cpp:212
#12 0x00007ffff750feee in QgsStatisticalSummaryDockWidget::qt_static_metacall (_o=0x1020b80, _c=QMetaObject::InvokeMetaMethod, _id=1, 
    _a=0x7fffffffa270) at /usr/src/qgis/build/master/src/app/moc_qgsstatisticalsummarydockwidget.cxx:58
#13 0x00007ffff336e87a in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#14 0x00007ffff5fa700d in QgsMapLayerComboBox::layerChanged (this=0x1023800, _t1=0x2f72e40)
    at /usr/src/qgis/build/master/src/gui/moc_qgsmaplayercombobox.cxx:153
#15 0x00007ffff5e59cf8 in QgsMapLayerComboBox::rowsChanged (this=0x1023800) at /usr/src/qgis/qgis-master/src/gui/qgsmaplayercombobox.cpp:87
#16 0x00007ffff5fa6b29 in QgsMapLayerComboBox::qt_static_metacall (_o=0x1023800, _c=QMetaObject::InvokeMetaMethod, _id=3, _a=0x7fffffffa400)
    at /usr/src/qgis/build/master/src/gui/moc_qgsmaplayercombobox.cxx:67
#17 0x00007ffff336e87a in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#18 0x00007ffff33b8664 in QAbstractItemModel::rowsInserted(QModelIndex const&, int, int) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#19 0x00007ffff33545ce in QAbstractItemModel::endInsertRows() () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4

#3 Updated by Sandro Santilli over 8 years ago

I confirm QgsStatisticalSummaryDockWidget::refreshStatistics is not hit on loading the edge table (full of LineString, rather than Polygon, if that's what matters)

#4 Updated by Sandro Santilli over 8 years ago

Sorry I take it back, QgsStatisticalSummaryDockWidget::refreshStatistics do get called, but it returns immediately here:

void QgsStatisticalSummaryDockWidget::refreshStatistics()                       
{                                                                               
  if ( !mLayer || !mFieldExpressionWidget->isValidExpression() )                
  {                                                                             
    mStatisticsTable->setRowCount( 0 );                                         
    return;                                                                     
  }

#5 Updated by Nyall Dawson over 8 years ago

  • Assignee deleted (Nyall Dawson)

so nothing here related to the dock? In any case, I've checked and refreshStatistics is only called if the dock widget is visible.

#6 Updated by Sandro Santilli over 8 years ago

Where's the statistical summary dock widget ?

#7 Updated by Sandro Santilli over 8 years ago

I see refreshStatistics always called on the first layer being added to a map.
The "Statistics Panel" is NOT visible in my case, never.

#8 Updated by Sandro Santilli over 8 years ago

I just noticed that another call to refreshStatistics (triggering a full table scan) happens on closing qgis, which results in having to wait up to a minute before QGIS really closes..

I think this is definitely related to the statistics widget (but cannot find a Category field for it)

#9 Updated by Giovanni Manghi over 7 years ago

  • Regression? set to No
  • Easy fix? set to No

#10 Updated by Giovanni Manghi over 5 years ago

  • Resolution set to end of life
  • Status changed from Open to Closed

Also available in: Atom PDF