QGIS » QGIS Application

I noticed that NVIZ7 is the only GRASS 7 tool that opening up a dialog.

I checked how QGIS had the variables set in the settings.

Some have double entries for grass 6:
GISBASE=/opt/grass64
PATH=/opt/grass64/bin:/opt/grass64/scripts:/usr/share/qgis/grass/scripts/:/opt/grass64/bin:/opt/grass64/scripts:/usr/share/qgis/grass/scripts/:$PATH
PYTHONPATH=/opt/grass64/etc/python:/opt/grass64/etc/python:$PYTHONPATH

I tried using the custom variables option and prepending the /opt/grass folders for GRASS 7 to those and it didn't make a difference.

Then I tried to set them at the command line before running qgis from there, didn't change the missing dependency error.

When installing QGIS 2.8 under Ubuntu from the http://qgis.org/ubuntugis-ltr with http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable , processing version 2.6 is shipped with it, which is working.
Updating to recommended processing 2.10.1 leads to the loss of all GRASS functionalities in the toolbox.

Manually downloading version 2.9.3 (http://plugins.qgis.org/plugins/processing/version/2.9.3/download/) and copying to the plugins folder restores the GRASS functions.

Failure also confirmed in Fedora 21 and Windows (https://gis.stackexchange.com/questions/156767/64-bit-win-8-1-grass-7-missing-dependency).

For msg improvement wish, see also bug #13188

Looks like there is some success in master version of QGIS¹ - hopefully those changes can be backported to the ltr and stable release.

[1] http://osgeo-org.1560.x6.nabble.com/Any-working-GRASS-in-QGIS-available-tp5218136p5218781.html

Also noticed that QGIS adds duplicate entries for some the environment variables that are only for grass64. For some reason, there isn't any entries for grass7, not sure if it's related or not so I tried updating them by prepending the related grass 7 locations (or exporting from command line prior to launch), but no dice.

PATH=/opt/grass64/bin:/opt/grass64/scripts:/usr/share/qgis/grass/scripts/:/opt/grass64/bin:/opt/grass64/scripts:/usr/share/qgis/grass/scripts/:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl

GRASS_PAGER=cat

GISBASE=/opt/grass64

PYTHONPATH=/opt/grass64/etc/python:/opt/grass64/etc/python:

GRASS_MESSAGE_FORMAT=gui

GRASS_BATCH_JOB=/home/saultdon/.qgis2//processing/grass_batch_job.sh

LD_LIBRARY_PATH=.::/jre/lib

But ld.conf.d files exist for both grass64 and grass (v7):

% cat /etc/ld.so.conf.d/grass64.conf
/opt/grass64/lib

% cat /etc/ld.so.conf.d/grass.conf
/opt/grass/lib

Processing wasn't executing anything for GRASS 7 algorithms. Do 5bcff73 (2.10) and 2a14ffd (master) fix this?

I tried locally, it doesn't fix it for me.

And the message is still obscure (no idea what Processing is looking for or expecting) so that I could rename( better: use a link) the GRASS binary as Processing wishes it to see.

Re-compiled QGIS from release-2_10 branch and looks like I'm still getting Missing Dependency error.

Sorry to bother. Any chance to get a fix? I wanted to show this functionality on the Geostat2015 in a few days...

I still don't know even the name of the GRASS "binary" which is expected by Processing. Thanks.

Markus Neteler - wrote:

Sorry to bother. Any chance to get a fix? I wanted to show this functionality on the Geostat2015 in a few days...

I still don't know even the name of the GRASS "binary" which is expected by Processing. Thanks.

Not sure how to reproduce. But did you try strace -f -e execve -p $(pidof qgis) (or qgis).

Trying r.info on an inserted tif here produced:

Process 30293 attached with 8 threads
Process 30519 attached
Process 30520 attached
Process 30523 attached
[pid 30523] execve("/bin/sh", ["/bin/sh", "-c", "grass70 /tmp/processing/grassdat"...], [/* 74 vars */]) = 0
Process 30524 attached
[pid 30524] execve("/usr/bin/grass70", ["grass70", "/tmp/processing/grassdata/temp_l"...], [/* 74 vars */]) = 0
[...]
[pid 30531] execve("/bin/sh", ["/bin/sh", "/home/fischer/.qgis2//processing"...], [/* 94 vars */]) = 0
Process 30532 attached
[pid 30532] execve("/usr/lib/grass70/bin/r.external", ["r.external", "input=/home/fischer/test/665735."..., "band=1", "output=tmp1439757780684", "--overwrite", "-o"], [/* 94 vars */]) = 0
[...]

Markus

The Grass7 provider calls grass70, previously setting a batch job in the GRASS_BATCH_JOB env variable.

In case of running linux (as I guess it's your case), there is no need to configure any path to GRASS in QGIS. Instead, you should just make sure that the grass executable is in your PATH env var.

Hope that helps

Good hint, I got this (Fedora):

qgis & sleep 2; strace -f -e execve -p $(pidof qgis)
...
[pid 19947] execve("/bin/sh", ["/bin/sh", "-c", "grass70 /tmp/processing/grassdat"...], [/* 67 vars /]) = 0
[pid 19947] execve("/usr/local/bin/grass70", ["grass70", "/tmp/processing/grassdata/temp_l"...], [/ 66 vars /]) = 0
[pid 19947] execve("/usr/lib64/grass/bin/python", ["python", "/usr/local/bin/grass70", "/tmp/processing/grassdata/temp_l"...], [/ 66 vars /]) = -1 ENOENT (No such file or directory)
[pid 19947] execve("/usr/lib64/grass/scripts/python", ["python", "/usr/local/bin/grass70", "/tmp/processing/grassdata/temp_l"...], [/ 66 vars */]) = -1 ENOENT (No such file or directory)

My local installation is:

ls -la /usr/local/bin/grass70
lrwxrwxrwx. 1 neteler gis 67 May 10 2014 /usr/local/bin/grass70 -> /home/neteler/software/grass70/bin.x86_64-unknown-linux-gnu/grass70*

ls -la /home/neteler/software/grass70/bin.x86_64-unknown-linux-gnu/grass70
-rwxrwxr-x 1 neteler neteler 49503 Aug 8 12:25 /home/neteler/software/grass70/bin.x86_64-unknown-linux-gnu/grass70*

"grass70" is installed and in the path:

[neteler@oboe ~]$ grass70 --config path
/home/neteler/software/grass70/dist.x86_64-unknown-linux-gnu

(the software does not reside in /usr/lib64/grass/ on my machine but it can be "anywhere" - the grass70 startup script knows where)

Why does it search in /usr/lib64/grass/bin/python? Is that hardcoded anywhere?

Markus Neteler - wrote:

Good hint, I got this (Fedora):

qgis & sleep 2; strace -f -e execve -p $(pidof qgis)
...
[pid 19947] execve("/bin/sh", ["/bin/sh", "-c", "grass70 /tmp/processing/grassdat"...], [/* 67 vars /]) = 0
[pid 19947] execve("/usr/local/bin/grass70", ["grass70", "/tmp/processing/grassdata/temp_l"...], [/ 66 vars /]) = 0
[pid 19947] execve("/usr/lib64/grass/bin/python", ["python", "/usr/local/bin/grass70", "/tmp/processing/grassdata/temp_l"...], [/ 66 vars /]) = -1 ENOENT (No such file or directory)
[pid 19947] execve("/usr/lib64/grass/scripts/python", ["python", "/usr/local/bin/grass70", "/tmp/processing/grassdata/temp_l"...], [/ 66 vars */]) = -1 ENOENT (No such file or directory)

Why does it search in /usr/lib64/grass/bin/python? Is that hardcoded anywhere?

I guess it's just traversing PATH looking for python to run grass70. Doesn't it find any? I suppose it does, so next thing I'd do would be to dump the environment in grass70 to a file and look for differences when run from the command line and from processing.

Here some code written by myself with support by Luca Delucchi for potential integration (...to speed up things):

        # find GISBASE folder, ask grass70 for this
        startcmd = 'grass70' + ' --config path'
        print startcmd
        p = subprocess.Popen(startcmd, shell=True,
                             stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        out, err = p.communicate()
        if p.returncode != 0:
            print >>sys.stderr, 'ERROR: %s' % err
            print >>sys.stderr, "ERROR: Cannot find GRASS GIS 70 start script (%s)" % startcmd
            sys.exit(-1)
        if sys.platform.startswith('linux'):
            gisbase = out.strip('\
')
        elif sys.platform.startswith('win'):
            if out.find("OSGEO4W home is") != -1:
                gisbase = out.strip().split('\
')[1]
            else:
                gisbase = out.strip('\
')
                os.environ['GRASS_SH'] = os.path.join(gisbase, 'msys', 'bin', 'sh.exe')
        # Set GISBASE environment variable
        os.environ['GISBASE'] = gisbase
        print gisbase
        # define GRASS-Python environment
        gpydir = os.path.join(gisbase, "etc", "python")
        sys.path.append(gpydir)   

I suppose that a change in git itself is not possible since you may have other ideas about style and integration. Otherwise let me know.

Markus Neteler - wrote:

Jürgen Fischer wrote:

I guess it's just traversing PATH looking for python to run grass70. Doesn't it find any?

There well is the system python which should be used:

[...]

Sure, you just quoted the strace output with the failing @execve@s - so it's was unclear whether it finds one.

I observe another issue:

[...]

Probably from some external plugin - Generic maybe (never heard of it)?

Printing the env shows that GISBASE is set wrongly to GRASS 6 which subsequently affects $PATH:

Maybe from the grass(6)plugin? processing doesn't seem to touch GISBASE (except on windows; at least from what I see in git grep GISBASE python/plugins/processing/) - maybe grass70 should.

As earlier indicated, Processing should ask grass70 where it is by running:
[...]

Shouldn't grass70 just execute the GRASS_BATCH_JOB in a GRASS7 environment (even if it's called from what seems to be a GRASS6 environment)?

Also these env vars are set wrongly by Processing:
[...]

Not sure that Processing does that. I imaging it's the grassplugin that does that. Is it activated in your setup?

PS: A pity to announce tomorrow in my Geostat2015 workshop that QGIS-Processing got broken in 2.10.x.

Maybe it's just a problem if you have both GRASS 6 & 7 and are using the grassplugin (which doesn't support GRASS7 yet).

Markus Neteler - wrote:

Here some code written by myself with support by Luca Delucchi for potential integration (...to speed up things):

[...]

I suppose that a change in git itself is not possible since you may have other ideas about style and integration. Otherwise let me know.

Does that break the grassplugin (if the above assumption is correct that grassplugin set GISBASE)? What would happen if you just call GISBASE= grass70 ..., would grass70 then setup GISBASE itself? Seems odd that you need to first fetch a value from a script, set an environment variable and then call it again.

Jürgen Fischer wrote:

Markus Neteler - wrote:
Shouldn't grass70 just execute the GRASS_BATCH_JOB in a GRASS7 environment (even if it's called from what seems to be a GRASS6 environment)?

Yes, I think so.

Also these env vars are set wrongly by Processing:
[...]

Not sure that Processing does that. I imaging it's the grassplugin that does that. Is it activated in your setup?

No, it is not even installed. I have a rather minimalistic QGIS installation.

PS: A pity to announce tomorrow in my Geostat2015 workshop that QGIS-Processing got broken in 2.10.x.

Maybe it's just a problem if you have both GRASS 6 & 7 and are using the grassplugin (which doesn't support GRASS7 yet).

I am not using the grassplugin. And Processing-GRASS worked in earlier versions (since I did most of the update to GRASS GIS 7 I know that :-).

Jürgen Fischer wrote:

Markus Neteler - wrote:

Here some code written by myself with support by Luca Delucchi for potential integration (...to speed up things):

[...]

I suppose that a change in git itself is not possible since you may have other ideas about style and integration. Otherwise let me know.

Does that break the grassplugin (if the above assumption is correct that grassplugin set GISBASE)?

No idea, I don't use the grassplugin since it is not ready for G7 yet.

What would happen if you just call GISBASE= grass70 ..., would grass70 then setup GISBASE itself? Seems odd that you need to first fetch a value from a script, set an environment variable and then call it again.

No need to fetch that when you use the batch job.
But I tried to understand and contribute to the way how it is done in Processing which I don't understand: why is GISBASE set at all if the batch job approach is used?

Markus Neteler - wrote:

What would happen if you just call GISBASE= grass70 ..., would grass70 then setup GISBASE itself? Seems odd that you need to first fetch a value from a script, set an environment variable and then call it again.

No need to fetch that when you use the batch job.
But I tried to understand and contribute to the way how it is done in Processing which I don't understand: why is GISBASE set at all if the batch job approach is used?

I removed libgrassplugin7, libgrassprovider7 and libgrassrasterprovider7 from the plugins directory and GISBASE disappeared (which previously was set to /usr/lib/grass70). So it's not Processing. Do you have gisbase set in the [GRASS] section of your ~/.config/QGIS/QGIS2.conf? That's where the plugin/provider gets GISBASE from, when it's not already set (after calling G_no_gisinit())

Jürgen Fischer wrote:

I removed libgrassplugin7, libgrassprovider7 and libgrassrasterprovider7 from the plugins directory and GISBASE disappeared (which previously was set to /usr/lib/grass70).

Again, please don't hardcode where it is. If QGIS needs to figure out, ask grass70 itself.

So it's not Processing. Do you have gisbase set in the [GRASS] section of your ~/.config/QGIS/QGIS2.conf? That's where the plugin/provider gets GISBASE from, when it's not already set (after calling G_no_gisinit())

This is what is (not) set there:

grep -i GISBASE ~/.config/QGIS/QGIS2.conf 
gisbase=

grep -i grass ~/.config/QGIS/QGIS2.conf 
GrassInstalled=true
recentProjectsList=/home/neteler/grassdata/conus_albers/qgis.qgs
text_path=/home/neteler/markus_repo/grass_software/globalsod/globalsod_italy
Configuration\\RECENT_ALGORITHMS="grass:v.clean;grass7:v.buffer.distance;grass7:v.random;grass7:r.watershed;grass7:r.aspect;grass7:v.generalize" 
Configuration\\ACTIVATE_GRASS70=true
Configuration\\GRASS7_LOG_COMMANDS=true
Configuration\\ACTIVATE_GRASS=false
Configuration\\GRASS_LOG_COMMANDS=false
Configuration\\GRASS7_LOG_CONSOLE=true
Configuration\\GRASS_LOG_CONSOLE=false
libgrassplugin=true
[GRASS]
lastGisdbase=/home/neteler/grassdata

So, GISBASE is empty.

Markus Neteler - wrote:

Jürgen Fischer wrote:

I removed libgrassplugin7, libgrassprovider7 and libgrassrasterprovider7 from the plugins directory and GISBASE disappeared (which previously was set to /usr/lib/grass70).

Again, please don't hardcode where it is. If QGIS needs to figure out, ask grass70 itself.

Hey, I don't to either processing or the grass plugin/provider. I don't even use them. I'm just trying to help.

I don't actually know where GISBASE comes from, I didn't set it, and it's apparently not hardcoded either (that would be a quick find). But it's set to an invalid directory on your machine, while it's apparently correct on mine - so there must be something different. Maybe there's cruft on your end, I don't know. What does:

import os
os.getenv("GISBASE")

give in the python console?

Looks like the grass plugin/provider either uses a preset GISBASE (inherited from where qgis was called), checks if it's valid (to some extent), if that fails tries the GRASS/gisbase setting, if that also fails, a local directory some packages apparently bundle grass in and if that also fails asks the user for one. The final result is written to /GRASS/gisbase to be used next time around (see qgsgrass.cpp, line 268).

Jürgen Fischer wrote:

Hey, I don't to either processing or the grass plugin/provider. I don't even use them. I'm just trying to help.

I appreciate that. I just spent hour and hours on this topic alias a formerly working integration...

...
This is what I get:

Python Console 
Use iface to access QGIS API interface or Type help(iface) for more info
import os
os.getenv("GISBASE")
'/usr/lib64/grass'

Something in QGIS is setting it - I didn't.

Looks like the grass plugin/provider either uses a preset GISBASE (inherited from where qgis was called),

I start QGIS from command line, no GISBASE is set.

So, now I just trash my local settings:

[neteler@oboe ~]$ rm -rf ~/.qgis2/ ~/.config/QGIS/
[neteler@oboe ~]$ echo $GISBASE

[neteler@oboe ~]$ qgis

import os
os.getenv("GISBASE")
'/usr/lib64/grass'

This indicates that something in QGIS is setting it.

checks if it's valid (to some extent), if that fails tries the GRASS/gisbase setting, if that also fails, a local directory some packages apparently bundle grass in and if that also fails asks the user for one. The final result is written to /GRASS/gisbase to be used next time around (see qgsgrass.cpp, line 268).

I cannot judge that but IMHO Processing should be clearly disentangled from the grassplugin since Processing just uses a batch job approach. It worked earlier this year...

FWIW - Nothing strange on my machine:

I just uninstalled QGIS 2.10 and installed back the version 2.8. The Processing-GRASS interface works in this older QGIS version.

Markus Neteler - wrote:

FWIW - Nothing strange on my machine:

I just uninstalled QGIS 2.10 and installed back the version 2.8. The Processing-GRASS interface works in this older QGIS version.

So you don't have a grassplugin there - and that's what (mis-)sets GISBASE. If you remove the grass plugin/provider from 2.10 GISBASE will disappear (like it did above) and processing will very likely work.

Jürgen Fischer wrote:

So you don't have a grassplugin there - and that's what (mis-)sets GISBASE. If you remove the grass plugin/provider from 2.10 GISBASE will disappear (like it did above) and processing will very likely work.

I'm afraid that I don't understand the suggestion.

What is the "grassplugin" compared to "grass plugin/provider"?

Markus Neteler - wrote:

Jürgen Fischer wrote:

So you don't have a grassplugin there - and that's what (mis-)sets GISBASE. If you remove the grass plugin/provider from 2.10 GISBASE will disappear (like it did above) and processing will very likely work.

I'm afraid that I don't understand the suggestion.

What is the "grassplugin" compared to "grass plugin/provider"?

well the plugin is libgrassplugin7.so and the providers are libgrassprovider7.so and libgrassrasterprovider7.so - all use libqgisgrass7.so.

Would anyone know the cmake options to compile without the grass plugin but with the providers so I can test processing?

-DWITH_GRASS7=OFF \\
-DGRASS_PREFIX7=/opt/grass \\
-DGRASS_INCLUDE_DIR7=/opt/grass/include/ \\
-DWITH_GRASS=OFF \\
-DGRASS_PREFIX=/opt/grass64 \\
-DGRASS_INCLUDE_DIR=/opt/grass64/include

I tried setting WITH_GRASS to OFF thinking that was for the plugin and that GRASS_PREFIX and GRASS_INCLUDE_DIR were for the provider, but setting WITH_GRASS=OFF disabled the other grass options in cmake.

Processing tools for GRASS 7 work after setting WITH_GRASS7=OFF and WITH_GRASS=OFF