QGIS » QGIS Application

just trying to run this with gdb:

(gdb) bt
#0  0x00007ffff3abaa00 in QObject::thread() const () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#1  0x00007fffefcdcf4a in ?? () from /usr/lib/x86_64-linux-gnu/libqca.so.2
#2  0x00007fffefcdd949 in QCA::KeyStoreManager::scan() () from /usr/lib/x86_64-linux-gnu/libqca.so.2
#3  0x00007ffff4303065 in QgsAuthManager::init (this=0xa3c150, pluginPath=...)
    at /home/richard/dev/qgis/git/qgis/src/core/auth/qgsauthmanager.cpp:110
#4  0x00007ffff432fc9d in QgsApplication::initQgis () at /home/richard/dev/qgis/git/qgis/src/core/qgsapplication.cpp:737
#5  0x00007ffff5ac083b in meth_QgsApplication_initQgis (sipArgs=0x7ffff7f84050)
    at /home/richard/dev/qgis/git/qgis/build/python/core/sip_corepart3.cpp:141747
#6  0x00000000004ba21a in PyEval_EvalFrameEx ()
#7  0x00000000004b79e6 in PyEval_EvalCodeEx ()
#8  0x00000000004e845f in ?? ()
#9  0x0000000000447583 in PyRun_InteractiveOneFlags ()
#10 0x0000000000447276 in PyRun_InteractiveLoopFlags ()
#11 0x000000000042daaa in ?? ()
#12 0x0000000000491553 in Py_Main ()
#13 0x00007ffff6f14b45 in __libc_start_main (main=0x490fa0 <main>, argc=1, argv=0x7fffffffe0f8, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffffffe0e8) at libc-start.c:287
#14 0x0000000000490ec6 in _start ()

Oh, the error line is qgsauthmanager.cpp:110, which is: QCA::scanForPlugins();

So, with qcatool also try the following, which does the same scan for plugins:

qcatool plugins

Output of qcatool2:

$ qcatool2 plugins
Qt Library Paths:
  /usr/lib/x86_64-linux-gnu/qt4/plugins
  /usr/bin
Available Providers:
  qca-ossl
    This product includes cryptographic software written by Eric Young
    ([email protected])

$ qcatool2 keystore list-stores
Sys  f1da [System Trusted Certificates]
Unknown signature value:  668 
Unknown signature value:  669 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  795 
Unknown signature value:  668 
Unknown signature value:  7 
Unknown signature value:  795 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  795 
Unknown signature value:  668 
Unknown signature value:  795 
Unknown signature value:  669 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668 
Unknown signature value:  668

Hi dr,

And did the following cause a crash?

qcatool keystore list Sys

I do not need to see the output, if it does not cause a crash.

Mmm, larry when I run that here on Debian Testing, I get a system beep, an output and then my prompt contains some leftovers it seems, see below: the strange characters are as seen in my terminal too... Maybe I should run another terminal? I also start qgis from this one...

richard@kwik~$ qcatool keystore list Sys
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  7 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  794 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  7 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  795 
Unknown signature value:  794 
Unknown signature value:  795 
Unknown signature value:  795 
Cert 5ad4 [ACEDICOM Root]
Cert 4ada [AC Raíz Certicámara S.A.]
Cert 8e2a [Actalis Authentication Root CA]
Cert 5db7 [AddTrust External CA Root]
Cert e26e [AddTrust Class 1 CA Root]
Cert f3a7 [AddTrust Public CA Root]
Cert 9ee9 [AddTrust Qualified CA Root]
Cert 485d [AffirmTrust Commercial]
Cert f77e [AffirmTrust Networking]
Cert 51db [AffirmTrust Premium]
Cert 3f1a [AffirmTrust Premium ECC]
Cert f7fc [Japanese Government]
Cert 19c8 [A-Trust-nQual-03]
Cert a3e8 [Autoridad de Certificacion Firmaprofesional CIF A62634068]
Cert 9f80 [Baltimore CyberTrust Root]
Cert f72b [Buypass Class 2 CA 1]
Cert 785e [Buypass Class 2 Root CA]
Cert aeb8 [Buypass Class 3 CA 1]
Cert e97e [Buypass Class 3 Root CA]
Cert db64 [CA Disig]
Cert 6c24 [Chambers of Commerce Root]
Cert 3eea [Global Chambersign Root]
Cert b0ed [Certigna]
Cert 1cc8 [Certinomis - Autorité Racine]
Cert 40da [Class 2 Primary CA]
Cert 0ab5 [certSIGN]
Cert a1df [Certum CA]
Cert ba32 [Certum Trusted Network CA]
Cert 7a2d [Chambers of Commerce Root - 2008]
Cert bc05 [CNNIC ROOT]
Cert 1a7a [AAA Certificate Services]
Cert 6ba3 [COMODO Certification Authority]
Cert 320c [COMODO ECC Certification Authority]
Cert e18a [Secure Certificate Services]
Cert 7bda [Trusted Certificate Services]
Cert 32a6 [ComSign CA]
Cert 6420 [ComSign Secured CA]
Cert 668e [Cybertrust Global Root]
Cert b1fa [Deutsche Telekom Root CA 2]
Cert d623 [DigiCert Assured ID Root CA]
Cert 72d4 [DigiCert Global Root CA]
Cert c528 [DigiCert High Assurance EV Root CA]
Cert 9b87 [Digital Signature Trust Co. of DSTCA E1]
Cert 00ea [Digital Signature Trust Co. of DSTCA E2]
Cert 5fea [DST ACES CA X6]
Cert 2d3d [DST Root CA X3]
Cert 7a35 [EBG Elektronik Sertifika Hizmet Sağlayıcısı]
Cert 7015 [EC-ACC]
Cert e951 [EE Certification Centre Root CA]
Cert d884 [Entrust.net Certification Authority (2048)]
Cert 343b [Entrust Root Certification Authority]
Cert a714 [Chunghwa Telecom Co., Ltd.]
Cert 49c9 [Equifax]
Cert 5d4f [Equifax Secure eBusiness CA-1]
Cert 22bc [Equifax Secure Global eBusiness CA-1]
Cert bff3 [GeoTrust Global CA 2]
Cert 2bab [GeoTrust Global CA]
Cert fef2 [GeoTrust Primary Certification Authority]
Cert fee4 [GeoTrust Primary Certification Authority - G2]
Cert aec8 [GeoTrust Primary Certification Authority - G3]
Cert e651 [GeoTrust Universal CA 2]
Cert 6929 [GeoTrust Universal CA]
Cert e34f [Global Chambersign Root - 2008]
Cert fe72 [GlobalSign Root CA]
Cert 85d0 [GlobalSign of GlobalSign Root CA - R2]
Cert b67f [GlobalSign of GlobalSign Root CA - R3]
Cert 19a9 [The Go Daddy Group, Inc.]
Cert 8389 [Go Daddy Root Certificate Authority - G2]
Cert 0728 [Hellenic Academic and Research Institutions RootCA 2011]
Cert fae6 [Hongkong Post Root CA 1]
Cert d574 [IGC/A]
Cert 1099 [Izenpe.com]
Cert eeba [Juur-SK]
Cert e26a [Microsec e-Szigno Root CA 2009]
Cert cdf8 [Microsec e-Szigno Root CA]
Cert 6ea0 [NetLock Arany (Class Gold) Főtanúsítvány]
Cert c74c [NetLock Uzleti (Class B) Tanusitvanykiado]
Cert 0900 [NetLock Expressz (Class C) Tanusitvanykiado]
Cert 3446 [NetLock Kozjegyzoi (Class A) Tanusitvanykiado]
Cert 5952 [NetLock Minositett Kozjegyzoi (Class QA) Tanusitvanykiado]
Cert 242a [Network Solutions Certificate Authority]
Cert e954 [OISTE WISeKey Global Root GA CA]
Cert 0d19 [QuoVadis Root CA 2]
Cert bdde [QuoVadis Root CA 3]
Cert 541c [QuoVadis Root Certification Authority]
Cert 6b71 [Root CA Generalitat Valenciana]
Cert 4a6f [RSA Security Inc]
Cert b333 [Secure Global CA]
Cert d302 [SecureSign RootCA11]
Cert 68af [SecureTrust CA]
Cert 64a6 [SECOM Trust Systems CO.,LTD. of Security Communication EV RootCA1]
Cert 1fa7 [SECOM Trust Systems CO.,LTD. of Security Communication RootCA2]
Cert fa26 [SECOM Trust.net]
Cert 3ce6 [Sonera Class1 CA]
Cert 94b0 [Sonera Class2 CA]
Cert bd0d [Staat der Nederlanden Root CA]
Cert 06aa [Staat der Nederlanden Root CA - G2]
Cert cd06 [Starfield Technologies, Inc.]
Cert 6f63 [Starfield Root Certificate Authority - G2]
Cert 8927 [Starfield Services Root Certificate Authority - G2]
Cert ceac [StartCom Certification Authority for DigitalSignature]
Cert 6600 [StartCom Certification Authority G2]
Cert 149e [S-TRUST Authentication and Encryption Root CA 2005:PN]
Cert 73af [Swisscom Root CA 1]
Cert 18bc [SwissSign Gold CA - G2]
Cert 640a [SwissSign Platinum CA - G2]
Cert 8011 [SwissSign Silver CA - G2]
Cert 2a73 [Government Root Certification Authority]
Cert 29c0 [TC TrustCenter Class 2 CA II]
Cert dea4 [TC TrustCenter Class 3 CA II]
Cert 30fa [TC TrustCenter Universal CA I]
Cert 15f8 [thawte Primary Root CA]
Cert 4ec0 [thawte Primary Root CA - G2]
Cert c1e8 [thawte Primary Root CA - G3]
Cert 5cce [Trustis Limited]
Cert 4371 [T-TeleSec GlobalRoot Class 3]
Cert eaad [TÜBİTAK UEKAE Kök Sertifika Hizmet Sağlayıcısı - Sürüm 3]
Cert 5b04 [TÜRKTRUST Elektronik Sertifika Hizmet Sağlayıcısı of (c) 2005 TÜRKTRUST Bilgi İletişim ve Bilişim Güvenliği Hizmetleri A.Ş.]
Cert 200b [TÜRKTRUST Elektronik Sertifika Hizmet Sağlayıcısı of TÜRKTRUST Bilgi İletişim ve Bilişim Güvenliği Hizmetleri A.Ş. (c) Kasım 2005]
Cert a9c8 [TWCA Root Certification Authority]
Cert b1a7 [UTN - DATACorp SGC]
Cert 1006 [UTN-USERFirst-Client Authentication and Email]
Cert 91e5 [UTN-USERFirst-Hardware]
Cert 22cd [VeriSign, Inc. of Class 1 Public Primary Certification Authority]
Cert 5504 [VeriSign, Inc. of Class 1 Public Primary Certification Authority - G2]
Cert e03a [VeriSign Class 1 Public Primary Certification Authority - G3]
Cert f418 [VeriSign, Inc. of Class 2 Public Primary Certification Authority - G2]
Cert 2887 [VeriSign Class 2 Public Primary Certification Authority - G3]
Cert 7d3b [C=US, O=VeriSign, Inc., OU=Class 3 Public Primary Certification Authority]
Cert 14d9 [VeriSign, Inc. of Class 3 Public Primary Certification Authority - G2]
Cert 0b3f [VeriSign Class 3 Public Primary Certification Authority - G3]
Cert 2dce [VeriSign Class 3 Public Primary Certification Authority - G4]
Cert 1bbb [VeriSign Class 3 Public Primary Certification Authority - G5]
Cert 6b89 [VeriSign Class 4 Public Primary Certification Authority - G3]
Cert 5fa7 [VeriSign Universal Root Certification Authority]
Cert 8989 [Visa eCommerce Root]
Cert c2ce [WellsSecure Public Root Certificate Authority]
Cert 9e13 [XRamp Global Certification Authority]
Cert 0ff7 [Certificate Authority]
Cert 3cbe [ACCVRAIZ1]
Cert ba23 [Atos TrustedRoot 2011]
Cert 4cdb [CA Disig Root R1]
Cert a8f1 [CA Disig Root R2]
Cert 3904 [China Internet Network Information Center EV Certificates Root]
Cert 0a5d [D-TRUST Root Class 3 CA 2 2009]
Cert 9a18 [D-TRUST Root Class 3 CA 2 EV 2009]
Cert 7e60 [E-Tugra Certification Authority]
Cert e7cf [PSCProcert]
Cert 402a [SG TRUST SERVICES RACINE]
Cert 3221 [CN=StartCom Certification Authority, C=IL, O=StartCom Ltd., OU=Secure Digital Certificate Signing]
Cert b4b0 [Swisscom Root CA 2]
Cert 8d03 [Swisscom Root EV CA 2]
Cert 03f5 [TeliaSonera Root CA v1]
Cert 959a [T-TeleSec GlobalRoot Class 2]
Cert 6a43 [TÜRKTRUST Elektronik Sertifika Hizmet Sağlayıcısı of TÜRKTRUST Bilgi İletişim ve Bilişim Güvenliği Hizmetleri A.Ş. (c) Aralık 2007]
Cert c5ca [TWCA Global Root CA]
Cert 94a1 [C=US, O=VeriSign, Inc., OU=Class 3 Public Primary Certification Authority]
Cert 53cd [DigiCert Assured ID Root G2]
Cert 5774 [DigiCert Assured ID Root G3]
Cert 47ff [DigiCert Global Root G2]
Cert df30 [DigiCert Global Root G3]
Cert 00f3 [DigiCert Trusted Root G4]
Cert c79a [QuoVadis Root CA 1 G3]
Cert 29b2 [QuoVadis Root CA 2 G3]
Cert 0980 [QuoVadis Root CA 3 G3]
Cert b4c0 [CA 沃é€æ ¹è¯ä¹¦]
Cert 0397 [Certification Authority of WoSign]
Cert ec71 [CFCA EV ROOT]
Cert 1fa8 [COMODO RSA Certification Authority]
Cert 5556 [Entrust Root Certification Authority - EC1]
Cert 22bf [Entrust Root Certification Authority - G2]
Cert 6799 [GlobalSign of GlobalSign ECC Root CA - R4]
Cert bf89 [GlobalSign of GlobalSign ECC Root CA - R5]
Cert 7526 [IdenTrust Commercial Root CA 1]
Cert 4aa0 [IdenTrust Public Sector Root CA 1]
Cert 2e0e [Staat der Nederlanden EV Root CA]
Cert 2265 [Staat der Nederlanden Root CA - G3]
Cert 7f62 [S-TRUST Universal Root CA]
Cert 5347 [USERTrust ECC Certification Authority]
Cert 223f [USERTrust RSA Certification Authority]
^[[?62;crichard@kwik~$ 62;c

Hi Richard,

I'm gonna guess that is just your terminal not handling the non-ASCII (or unsupported) characters upon display (something Qt should handle fine). What happens if you pipe that to an output file instead?

qcatool keystore list Sys > ~/qcatool_Sys-keystore.txt

Do you still get any notification in terminal?

This looks more like a problem with the Python bindings and communication (or lack thereof) between threads.

OK, I am able to reproduce this on Mac as well:

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000008

VM Regions Near 0x8:
--> 
    __TEXT                 000000010dfa3000-000000010dfa5000 [    8K] r-x/rwx SM=COW  /usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   QtCore                            0x000000011013d816 QObject::thread() const + 4
1                                     0x0000000112cca60d QCA::KeyStoreManagerGlobal::KeyStoreManagerGlobal() + 51
2                                     0x0000000112cc0f2c QCA::ensure_init() + 112
3                                     0x0000000112cc12a4 QCA::KeyStoreManager::scan() + 16
4   org.qgis.qgis2_core               0x000000010f53ec86 QgsAuthManager::init(QString) + 246 (qstring.h:419)
5   org.qgis.qgis2_core               0x000000010f56ff16 QgsApplication::initQgis() + 102 (qstring.h:880)
6   _core.so                          0x000000010e8020f1 meth_QgsApplication_initQgis(_object*, _object*) + 65 (sip_corepart3.cpp:143847)
7   org.python.python                 0x000000010e03447e PyEval_EvalFrameEx + 14392
8   org.python.python                 0x000000010e030a12 PyEval_EvalCodeEx + 1409
9   org.python.python                 0x000000010e03048b PyEval_EvalCode + 54
10  org.python.python                 0x000000010e05067e run_mod + 53
11  org.python.python                 0x000000010e05049a PyRun_InteractiveOneFlags + 353
12  org.python.python                 0x000000010e04ffa9 PyRun_InteractiveLoopFlags + 192
13  org.python.python                 0x000000010e04fe53 PyRun_AnyFileExFlags + 60
14  org.python.python                 0x000000010e061a5f Py_Main + 3051
15  libdyld.dylib                     0x00007fff8eaf85fd start + 1

Yes, I think that is unrelated.

It seems that in order to create the QCA::Initializer object, it needs an available running QCoreApplication, otherwise it can't go back to the referenced current thread (because none exists).

For example, in the issue-reported code above:

>>> from qgis.core import QgsApplication
>>> QgsApplication.setPrefixPath('/usr', True)
>>> QgsApplication.initQgis()

this will NOT work because there is no available Qt application running. However if you do:

import sys
from PyQt4.QtCore import QCoreApplication
app = QCoreApplication(sys.argv)  #<-- this object needs to exist first, e.g. your own PyQt app needs instantiated first
from qgis.core import QgsApplication
QgsApplication.setPrefixPath('/usr', True)  #<-- optional, if you have set QGIS_PREFIX_PATH
QgsApplication.setAuthDbDirPath('/home/me/test')  #<-- need to define where the auth database is created (or exists)
QgsApplication.initQgis()
out = QgsApplication.showSettings()
print out

OR, something like this (as is done in the QGIS Python unit tests)

import sys
from qgis.core import QgsApplication
QgsApplication.setPrefixPath('/usr', True)  #<-- optional, if you have set QGIS_PREFIX_PATH
QgsApplication.setAuthDbDirPath('/home/me/test')  #<-- need to define where the auth database is created (or exists)
qgs = QgsApplication(sys.argv, False)
qgs.initQgis()
out = qgs.showSettings()
print out

You can also set some env variables: QGIS_PREFIX_PATH and QGIS_AUTH_DB_DIR_PATH

I'm not sure how to use the QgsApplication object without having a Qt app running, if it is to use the auth system. I suppose I could put a check in QgsAuthManager to disable the auth system if it finds no running app, or put in some type of QTimer::singleShot() to later set up the QCA::Initializer (though that still won't work without a running Qt app).

Yep... in qca/src/qca_keystore.cpp on line 650, when setting up a global manager, it references the current application thread:

KeyStoreManagerGlobal()
{
  thread = new KeyStoreThread;
  thread->moveToThread(QCoreApplication::instance()->thread());
  thread->start();
}

This is not in a QCA plugin, but in core, so I don't see any way to skip the requirement of having a running QCoreApplication before calling QgsApplication.initQgis(). Unless, a parameter is added like initQgis(bool useAuthSystem), which then tries to disable the auth system when set to FALSE or tries to verify a running Qt app before enabling the system if set to TRUE, with all that code inside the QgsAuthManager::init() member function.

This way of using QGIS in custom application picked from PyQGIS Developer Cookbook http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/intro.html#using-pyqgis-in-custom-application

>>> from qgis.core import QgsApplication
>>> QgsApplication.setPrefixPath('/usr', True)
>>> QgsApplication.initQgis()

As I can see this is not correct for recent QGIS and user have to use another approach:

>>> from qgis.core import QgsApplication
>>> QgsApplication.setPrefixPath('/usr', True)
>>> qgs = QgsApplication(sys.argv, False)
>>> qgs.initQgis()

In this case updating of PyQGIS Developer Cookbook may be enough for closing this ticket?

Other devs, do you think (as noted in comment 13) the Python access to the API should continue to be accessible without a running QCoreApplication? If so, it looks like the auth system would need to be disabled to accommodate that form of use.

I sent an email to the dev list, asking for input:

https://lists.osgeo.org/pipermail/qgis-developer/2015-October/040080.html

I thought all Qt code needed QApplication or QCoreApplication to work.

I have always done

@>>> qgs = QgsApplication(sys.argv, False)

qgs.initQgis()
#do stuff@

Nathan Woodrow wrote:

I thought all Qt code needed QApplication or QCoreApplication to work.

I have always done

qgs = QgsApplication(sys.argv, False)

qgs.initQgis()

#do stuff

This is what I thought was required to do most anything useful. So maybe the cookbook needs updated?

http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/intro.html#using-pyqgis-in-custom-application

Actually the QApplication/QCoreApplication are not necessary unless you want to do some stuff with painting, widgets, threads and some other stuff.

Larry - would it be possible to do lazy initialization of the auth subsystem, so the existing code still keeps working?

In general, the QgsApplication API is extremely confusing - and I hope we can fix it for QGIS 3.0. It derives from QApplication (IMHO bad idea), it also handles init/exit and paths. The init/exit handling should be ideally completely automatic with PyQGIS, hopefully avoiding different possible ways how people init PyQGIS in the future.

Any progress on this issue?

Sorry, I apparently forgot about this issue.

Martin Dobias wrote:

Larry - would it be possible to do lazy initialization of the auth subsystem, so the existing code still keeps working?

Can you give me a description of how you see this implemented, i.e. psuedo code or whatever?

Larry Shaffer wrote:

Can you give me a description of how you see this implemented, i.e. psuedo code or whatever?

I thought that QgsApplication::initQgis() would not call QgsAuthManager::init(), the call would be rather deferred to a later time, so other QgsAuthManager methods would start with something like "if (!mAuthInit) init();". However looking at QgsAuthManager class, there are plenty of methods in there, so I am not sure if that would be worth it. Maybe better to address that in 3.0 that would set a requirement that QGIS libraries can be only initialized after Q[Core]Application object has been created?

Martin Dobias wrote:

Maybe better to address that in 3.0 that would set a requirement that QGIS libraries can be only initialized after Q[Core]Application object has been created?

Gents, I think 3.0 is getting in place currently, that's the reason I try to attract some devs into this(in my view pretty important) issue :-)

Also see a recent discussion about the initing of QGIS:

https://lists.osgeo.org/pipermail/qgis-developer/2016-October/045104.html