Bug report #11425

make check: 12 tests failed out of 113

Added by Sandro Santilli about 11 years ago. Updated over 10 years ago.

Status:Closed
Priority:Normal
Assignee:Larry Shaffer
Category:Tests Suite
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:invalid
Crashes QGIS or corrupts data:No Copied to github as #:19701

Description

I'm getting these failures on "make check" with current master (commit bb95a0f43a53602bd7ad1ea6783d251755c66019)

Application state:
QGIS_PREFIX_PATH env var:               /usr/src/qgis/build/master/output
Prefix:
Plugin Path:            /usr/src/qgis/build/master/output/lib/qgis/plugins
Package Data Path:      /usr/src/qgis/qgis-master
Active Theme Name:
Active Theme Path:      :/images/themes//
Default Theme Path:     :/images/themes/default/
SVG Search Paths:       /usr/src/qgis/qgis-master/images/svg/
                /home/strk/.qgis2//svg/
User DB Path:   /usr/src/qgis/qgis-master/resources/qgis.db

Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 50, in <module>
    MAPSERV = getLocalServer()
  File "/usr/src/qgis/qgis-master/tests/src/python/qgis_local_server.py", line 683, in getLocalServer
    assert os.path.exists(fcgi_path), msg
AssertionError: Could not locate qgis_mapserv.fcgi in build/bin directory

CMake Error at PyQgsPalLabelingServer.cmake:28 (MESSAGE):
  1

        Start 109: PyQgsVectorFileWriter
109/113 Test #109: PyQgsVectorFileWriter ................   Passed    0.23 sec
        Start 110: PyQgsSpatialiteProvider
110/113 Test #110: PyQgsSpatialiteProvider ..............   Passed    0.13 sec
        Start 111: PyQgsZonalStatistics
111/113 Test #111: PyQgsZonalStatistics .................   Passed    0.24 sec
        Start 112: PyQgsAppStartup
112/113 Test #112: PyQgsAppStartup ......................   Passed   13.23 sec
        Start 113: PyQgsDistanceArea  
113/113 Test #113: PyQgsDistanceArea ....................   Passed    0.25 sec

89% tests passed, 12 tests failed out of 113

Total Test time (real) = 111.23 sec

Are them known failures or is it just me ?
I'm building and running on an Ubuntu 14.04.1 LTS (aka "trusty") with GCC 4.8.2 and
Linux cdb 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

History

#1 Updated by Matthias Kuhn about 11 years ago

These are known failures.

There is an initiative going on to fix this (at least for this particular platform)
http://blog.vitu.ch/10102014-1046/crowdfunding-initiative-automated-testing

#2 Updated by Jürgen Fischer about 11 years ago

  • Assignee set to Matthias Kuhn

#3 Updated by Sandro Santilli almost 11 years ago

As of current master (26e06e6881760103c6d2f005dc15cb26fa221767) 3 tests failed out of 119:
The following tests FAILED:
111 - PyQgsPalLabelingCanvas (Failed)
112 - PyQgsPalLabelingComposer (Failed)
113 - PyQgsPalLabelingServer (Failed)

#4 Updated by Matthias Kuhn almost 11 years ago

  • Assignee changed from Matthias Kuhn to Larry Shaffer

Larry, I'm handing this over to you.
I think you have the best knowledge of the PAL test suite.

#5 Updated by Larry Shaffer almost 11 years ago

Thanks Matthias. I need to fix this. I just finished setting up whole new virtual build machines for Win 7 Pro (32 and 64), Ubuntu Trusty and Mac 10.6 thru 10.10.

Sandro, first off, in order to overcome:

AssertionError: Could not locate qgis_mapserv.fcgi in build/bin directory

you will need to ensure you have the CMake option "WITH_SERVER=ON" (master) or "WITH_MAPSERVER=ON" (<= 2.6) enabled, or the FCGI binary will not be built (the default) and the unit test local server will not find it.

The other labeling test suite issues I will work on once you report after testing with the FCGI binary.

#6 Updated by Sandro Santilli almost 11 years ago

After building with server active:

97% tests passed, 4 tests failed out of 120

Total Test time (real) = 112.37 sec

The following tests FAILED:
         44 - qgis_atlascompositiontest (Failed)
        108 - PyQgsAtlasComposition (Failed)
        112 - PyQgsPalLabelingCanvas (Failed)
        114 - PyQgsPalLabelingServer (Failed)

Details:

======================================================================
FAIL: test_background_rect (__main__.TestServerPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 87, in test_background_rect
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_background_rect" 

======================================================================
FAIL: test_background_rect_w_offset (__main__.TestServerPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 107, in test_background_rect_w_offset
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_background_rect_w_offset" 

======================================================================
FAIL: test_background_svg (__main__.TestServerPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 127, in test_background_svg
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_background_svg" 

======================================================================
FAIL: test_background_svg_w_offset (__main__.TestServerPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 150, in test_background_svg_w_offset
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_background_svg_w_offset" 

======================================================================
FAIL: test_default_label (__main__.TestServerPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 62, in test_default_label
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_default_label" 

======================================================================
FAIL: test_partials_labels_enabled (__main__.TestServerPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 162, in test_partials_labels_enabled
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_partials_labels_enabled" 

======================================================================
FAIL: test_text_color (__main__.TestServerPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 79, in test_text_color
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_text_color" 

======================================================================
FAIL: test_text_size_map_unit (__main__.TestServerPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 72, in test_text_size_map_unit
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_text_size_map_unit" 

======================================================================
FAIL: test_background_rect (__main__.TestServerVsCanvasPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 87, in test_background_rect
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_background_rect" 

======================================================================
FAIL: test_background_rect_w_offset (__main__.TestServerVsCanvasPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 107, in test_background_rect_w_offset
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_background_rect_w_offset" 

======================================================================
FAIL: test_background_svg (__main__.TestServerVsCanvasPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 127, in test_background_svg
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_background_svg" 

======================================================================
FAIL: test_background_svg_w_offset (__main__.TestServerVsCanvasPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 150, in test_background_svg_w_offset
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_background_svg_w_offset" 

======================================================================
FAIL: test_partials_labels_enabled (__main__.TestServerVsCanvasPoint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_tests.py", line 162, in test_partials_labels_enabled
    self.checkTest()
  File "/usr/src/qgis/qgis-master/tests/src/python/test_qgspallabeling_server.py", line 200, in checkTest
    imgpath=self._TestImage))
AssertionError: 
Render check failed for "sp_partials_labels_enabled" 

----------------------------------------------------------------------
Ran 18 tests in 1.666s

FAILED (failures=13)

CMake Error at PyQgsPalLabelingServer.cmake:28 (MESSAGE):
  1

#8 Updated by Sandro Santilli over 10 years ago

With current master, only PyQgsPalLabelingServer fails for me. Do I have to enable anything for it to pass ?

#9 Updated by Jürgen Fischer over 10 years ago

  • Category set to Tests Suite

#10 Updated by Nyall Dawson over 10 years ago

  • Resolution set to invalid
  • Status changed from Open to Closed

Closing this as Travis is a better solution for tracking failing tests

Also available in: Atom PDF