Bug report #11447

Processing does not accept anymore CSV as input layers

Added by Giovanni Manghi about 10 years ago. Updated over 9 years ago.

Status:Closed
Priority:Severe/Regression
Assignee:Victor Olaya
Category:Processing/Core
Affected QGIS version:master Regression?:No
Operating System: Easy fix?:No
Pull Request or Patch supplied:No Resolution:fixed/implemented
Crashes QGIS or corrupts data:No Copied to github as #:19723

Description

On master it returns an error (here an example with a qgis tool, but happens also for other toolboxes)

Traceback (most recent call last):
  File "/usr/share/qgis/python/plugins/processing/gui/AlgorithmExecutionDialog.py", line 198, in accept
    if checkCRS and not self.alg.checkInputCRS():
  File "/usr/share/qgis/python/plugins/processing/core/GeoAlgorithm.py", line 382, in checkInputCRS
    crs = dataobjects.getObject(item).crs()
AttributeError: 'NoneType' object has no attribute 'crs'

while it works fine on 2.4 and older.

NET60_2013_O3.7z (3.37 MB) Olav Peeters, 2014-11-04 02:09 AM

History

#1 Updated by Salvatore Larosa about 10 years ago

  • Status changed from Open to Feedback

just tried and it has worked. Could you re-try it?

#2 Updated by Giovanni Manghi about 10 years ago

  • Status changed from Feedback to Closed
  • Resolution set to worksforme

right, I don't know why but now it works.

#3 Updated by Olav Peeters about 10 years ago

I'm having the same issue with a text (csv) file on both the 2.6 release and 2.7.0-Master Master (a68958f). I don't think the issue is solved..

#4 Updated by Giovanni Manghi about 10 years ago

Olav Peeters wrote:

I'm having the same issue with a text (csv) file on both the 2.6 release and 2.7.0-Master Master (a68958f). I don't think the issue is solved..

can you attach a sample of a csv file of yours causing this issue? thanks.

#5 Updated by Olav Peeters about 10 years ago

see attached
the geometry is in EPSG:31370
Thanks!

#6 Updated by Giovanni Manghi about 10 years ago

  • Resolution deleted (worksforme)
  • Target version changed from Version 2.6 to Version 2.8
  • Status changed from Closed to Reopened

Olav Peeters wrote:

see attached
the geometry is in EPSG:31370
Thanks!

right, it seems that the issue has not been fixed after all... (works fine until 2.4).

#7 Updated by Olav Peeters almost 10 years ago

I tested this on Windows, Linux (Fedora 20) and Mac OS X 10.9. Funny thing is that QGIS 2.6 on Windows and Linux have the same problem but on Mac OS X (the William Kyngesburye build) it does not. The python version in the three cases were the same.

It probably is related to one of the CRS checks changes here: https://github.com/qgis/QGIS/commits/master/python/plugins/processing/core/GeoAlgorithm.py
my guess would be this:
#b114085257d621c230db01d070b2e87b5db0c2fb

#8 Updated by Victor Olaya almost 10 years ago

hmm, there is some confusion here

If a layer is a csv, then it has no geometry. QGIS will load it as a simple geometry-less layer, which is just a table. Processing will not let you use such a "layer" as input when a vector layer is needed. You can convert your table to a real points layer using any of the algorithms in Processing that do it.

Now, to leave those csv layers out, a basic check is done, leaving files with csv extension out. Yours has txt extension, so it's passing that filter. I will change the check so it filters based on the presence/absence of geometries, so it is more robust.

This is what, at first sight, i guess is happening, but i will try to investigate with more detail

Hope this helps

#9 Updated by Victor Olaya almost 10 years ago

Sorry, my fault. The text file can have geometry if loaded as such using the "open delimited text layer"

By fixing the check as i explained, the issue should be fixed anyway, since it will be able to identify it as a valid vector layer when retrieving it to check the crs (now the method that should return the layer is returning None, since it cannot find the layer correctly)

it should be an easy fix

#10 Updated by Victor Olaya almost 10 years ago

it turns out that the problem is much more complex :-(

When opening a delimited text file, the data source uri contains semicolons, which are also used by Processing to separate multiple file inputs. Not so easy to fix...

#11 Updated by Victor Olaya almost 10 years ago

I have added the patch that fixes it, but only when the csv files are used in single inputs, not in multiple inputs

#12 Updated by Giovanni Manghi almost 10 years ago

Victor Olaya wrote:

it turns out that the problem is much more complex :-(

When opening a delimited text file, the data source uri contains semicolons, which are also used by Processing to separate multiple file inputs. Not so easy to fix...

this explain why with certain CSV works and others don't...

#13 Updated by Giovanni Manghi over 9 years ago

  • Resolution set to fixed/implemented
  • Status changed from Reopened to Closed

the attached file works ok on qgis master, in case of multiple inputs we should document the limitation when the separator is the semicolon, and suggest resave the files with another separator. Please reopen if necessary.

#14 Updated by Giovanni Manghi over 9 years ago

  • Category changed from 94 to Processing/Core

Also available in: Atom PDF