Bug report #15522

Qgis Server doesnt' respect the styling from Desktop

Added by Tudor Bărăscu about 8 years ago. Updated about 8 years ago.

Status:Closed
Priority:Severe/Regression
Assignee:René-Luc ReLuc
Category:QGIS Server
Affected QGIS version:2.14.5 Regression?:No
Operating System:Debian Stretch Easy fix?:No
Pull Request or Patch supplied:Yes Resolution:fixed/implemented
Crashes QGIS or corrupts data:No Copied to github as #:23446

Description

Basically, when doing a GetMap request, only the first GetMap response is rendered correctly.
The next ones are all rendered blank.
Unfortunately, there's no error in the logs.

Curiously, I just found out it only happens for a Postgis layer (if I export it to sqlite, everything is ok).

Tested with QGIS 2.14.6 and 2.16.2 on Debian.
This exact project worked for QGIS 2.14.2.

I've attached a test project:
- maps.qgs has all the layer sources sqlite
- maps_postgis.qgs has the pipe layer in postgis (So you need to restore the pipe.sql into postgres and edit project file so that it points to wherever you restored the db 

      <datasource>service='pg_qtibia' sslmode=disable key='id' srid=3844 type=LineString table="public"."pipe" (geom) sql=</datasource>
      <keywordList>
        <value></value>
      </keywordList>
      <layername>pipes</layername>

You can test these requests in the browser after you point them to where you downloaded the test project:
- This one works good every time as it points to the sqlite only project:

http://qgisltr.demo/cgi-bin/qgis_mapserv.fcgi?map=/home/web/Desktop/qgis_server_stilying_problem/maps.qgs&&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=399813.2452616940718,450176.0919703728869,400035.4952619207324,450428.8393977318774&CRS=EPSG:3844&WIDTH=663&HEIGHT=583&LAYERS=pipes&STYLES=&FORMAT=image/png&DPI=96&MAP_RESOLUTION=96&FORMAT_OPTIONS=dpi:96

This one points to the postgis layer project and it works good JUST FOR THE FIRST REQUEST:

http://qgisltr.demo/cgi-bin/qgis_mapserv.fcgi?map=/home/web/Desktop/qgis_server_stilying_problem/maps_postgis.qgs&&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=399813.2452616940718,450176.0919703728869,400035.4952619207324,450428.8393977318774&CRS=EPSG:3844&WIDTH=663&HEIGHT=583&LAYERS=pipes&STYLES=&FORMAT=image/png&DPI=96&MAP_RESOLUTION=96&FORMAT_OPTIONS=dpi:96

bug_in_action

If you drop the ELSE rule from the pipe styling you will get empty image from the second request on.
Any ideas?

styling_problem.gif - bug_in_action (217 KB) Tudor Bărăscu, 2016-09-02 05:02 AM

qgis_server_stilying_problem.zip - test_data (1.58 MB) Tudor Bărăscu, 2016-09-02 05:02 AM

qgis_bug_test_data.zip - Test data (1.54 MB) Tudor Bărăscu, 2016-09-03 01:03 AM

History

#1 Updated by Tudor Bărăscu about 8 years ago

I forgot to mention that at first I removed the ELSE rule from styling as I thought it was the culprit.
In absence of the ELSE rule, the image returned is blank.

#2 Updated by Tudor Bărăscu about 8 years ago

f6aad8b is the first bad commit

Thanks to Marco for pointing me to the git bisect option.

#3 Updated by Tudor Bărăscu about 8 years ago

I'm attaching simpler test data (project + sqlite source), please use this one.
You can replicate with the same request.

http://qgisltr.demo/cgi-bin/qgis_mapserv.fcgi?map=/home/projects/maps.qgs&&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=399813.2452616940718,450176.0919703728869,400035.4952619207324,450428.8393977318774&CRS=EPSG:3844&WIDTH=663&HEIGHT=583&LAYERS=pipes&STYLES=&FORMAT=image/png&DPI=96&MAP_RESOLUTION=96&FORMAT_OPTIONS=dpi:96

#4 Updated by Tudor Bărăscu about 8 years ago

More info: It seems that using a joined field column in the rule based styling filter is making qgis server rendering blank images for that specific layer. Same goes for the labels, if they have something like CASE WHEN.. based on joined layer columns the labels will not appear.

#5 Updated by René-Luc ReLuc about 8 years ago

Hi Tudor,

I'm investigating to fixed it.

#6 Updated by René-Luc ReLuc about 8 years ago

  • Assignee set to René-Luc ReLuc
  • Target version set to Version 2.14
  • Pull Request or Patch supplied changed from No to Yes

Hi Tudor,

I have probably fixed it:

Can you test it ?

#7 Updated by Tudor Bărăscu about 8 years ago

  • Status changed from Open to Closed

It works as it should now. Tested under LTR, 2.16, master and master_2.
Thanks a lot René!

#8 Updated by René-Luc ReLuc about 8 years ago

  • Status changed from Closed to Reopened

I reopened it because we found a memory leak in my previous fix.

Can you test my new fix ?

Thanks

#9 Updated by Tudor Bărăscu about 8 years ago

Tested on all branches and it works.
I'll let you close the issue.
Thanks a lot for your very nice work.

#10 Updated by René-Luc ReLuc about 8 years ago

  • Status changed from Reopened to Closed
  • Resolution set to fixed/implemented
Last commits:

#11 Updated by Tudor Bărăscu about 8 years ago

@Rene-Luc Although testing with the provided test data works good, my production project has problems.

I can replicate with a GetMap request and there's a pattern in that the first request works, the second doesn't, the third works, 4th not etc.

21cfb7d introduced this as with the previous commit all is working fine.

Is this helpful? I'll try and narrow it down further.
Thanks a lot!

#12 Updated by René-Luc ReLuc about 8 years ago

Hi Tudor,

Can I reproduced the problem with the data you provided ?
  • qgis_server_stilying_problem.zip
  • qgis_bug_test_data.zip

#13 Updated by Tudor Bărăscu about 8 years ago

Nope, that's just the thing, I didn't catch it with the test data.
I will try to isolate it tomorrow.

Thanks Rene!

It seems it's more to be a false positive.
The problem seems to be localized on my production server but on the GetProjectSettings request which throws Internal Server Error.

I've moved the project on my laptop and the request works without crashing although both servers are built from source and it's the same system.

Still, the fact that it's working if I rebuild the server with an earlier commit leads me to believe there's a catch/bug somewhere.

#14 Updated by Tudor Bărăscu about 8 years ago

@Rene

I just figured out why the server is crashing when doing a GetProjectSettings request.

The server cannot access one Postgis layer (detail: different user when connecting to the database to ensure some sensitive data doesn't get out - I also have exclude layers setup). Up until 21cfb7d the server simply discarded the layers that it couldn't connect to but now it crashes when doing the GetProjectSettings request.

This specific layer is involved in a 1-n Relation as a Referenced layer (child).

As soon as I gave SELECT privileges on that layer.. the problem is gone!

IMHO the server shouldn't crash in this situation but simply discard layers that it cannot access.

#15 Updated by Tudor Bărăscu about 8 years ago

You can replicate the problem by deleting the vl_material.. table from the offline.sqlite and restarting the apache server so that the cache is dropped.

Then do a http://qgisltr.demo/cgi-bin/qgis_mapserv.fcgi?map=/home/projects/maps.qgs&&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetProjectSettings

#16 Updated by Tudor Bărăscu about 8 years ago

@Rene

There is also a performance penalty regression. My production server seemed to be behaving less responsive and I timed a GetMap request on a more busy area:

Latest commit in release_2.14: 4.023 seconds

Before problems commit: 1.211 seconds (commit d708473d5d9ab8e0ad55a7113d7bf9d94a087b2f)

So, almost 4 times slower..on the same GetMap request.

I have 2 or 3 joined layers that are used in styling.

#17 Updated by René-Luc ReLuc about 8 years ago

Hi Tudor,

Thanks for founding issues, is the GetProjectSettings Request the only one to segfault QGIS Server ?
I apologized, but I never used this request.

About the time performance penalty, I think it's normal. Before my fix, no joins were used. So that mean that QGIS Server never get features in the joined layer to made the draw. With the bugfix, QGIS Server will used the 2 or 3 joined layers to render.

It will probably have some point to enhance the way QGIS (not only QGIS Server) uses joined layer.

#18 Updated by Tudor Bărăscu about 8 years ago

cHello again,

Yes, only the GetProjectSettings is buggy (when layers are missing), everything else seems to be ok.
The GetProjectSettings request is used by the QGIS Web Client (otherwise I wouldn't have noticed it).

The d708473d5d9ab8e0ad55a7113d7bf9d94a087b2f is right before (or two commits before) the commit that started this issue (f6aad8bad6db72d0b1e573998ac7dcb2405dce55).

So, in both tests the joined layers are used in the styling and the GetMap returned images are identical, which would be perfect if the performance penalty wouldn't have emerged.

#19 Updated by René-Luc ReLuc about 8 years ago

Hi Tudor,

Can you test my last commit on qgis-server ? I probably fixed your issue with getProjectSettings.

Regards,

#20 Updated by Tudor Bărăscu about 8 years ago

Hi Rene,
Indeed the problem has dissapeared.
I tested after and before your commit on the 2_14 branch.

Thanks a lot!

Regarding the performance hit, it's still present.
On the release-2_14 branch, the difference between d708473d5d9ab8e0ad55a7113d7bf9d94a087b2f (from june 13) and the latest commit is quite large. By huge I mean almost four times higher for a getmap request.

0.9 seconds versus 3.6 seconds.
Keep in mind that that everything works as it should from the user perspective (the joined layers work in both tests).

Also available in: Atom PDF