Developing a Split Layer by Attribute Tool using the QGIS Processing Framework

Background

It is somewhat traditional when extracting features from a layer to select features using a selection expression based upon a specific attribute value and save the selected features to a new layer. QGIS has a great Extract by attribute tool that combines these two actions into a single process. The Extract by attribute tool can be automated in Python scripting to further its use. This example will show one way to automate the Extract by attribute tool using the QGIS Processing Framework to develop a custom script that will repeatedly call the Extract by attribute tool to split a layer by the unique values of an attribute chosen by the user. The Processing Framework provides scripting access to the tools in the Processing Toolbox and a dialog interface for user inputs so custom scripts can be ran as easily as any other processing tool.

To get stated developing a Split Layer by Attribute tool, open QGIS and add a polygon layer that has a name attribute that uniquely identifies the features in the layer. In this example, a US States layer created from a subset of the 1:10 m scale States and Provinces layer downloaded from the Natural Earth website will be used to split the state features into new layers.

Attribute table from US States layer showing the name field.

Basic Script

Open the Extract by attribute tool from Processing Toolbox > QGIS algorithms > Vector selection tools. The documentation to use this tool in Python script is located in the Help tab of tool dialog.

processing.runalg('qgis:extractbyattribute', input, field, operator, value, output)

Running this statement from the Python console with meaningful parameters, will automate the Extract by attribute tool. Making meaningful parameters for the statement is as easy as declaring variables and populating them with values. However, instead of running this tool from the Python Console, we want a script that runs from the Scripting toolbox with a dialog interface.

Open Scripts > Tools > Create new script from the Processing Toolbox. This will start a new Script editor to write the Split Layer by Attribute script. In the Script editor enter the basic script to run the Extract by attribute tool.

import processing   # utilize the Processing Framework

input = 'c:/temp/US_States_NE.shp'  # input layer to split
field = 'name'                      # attribute with values to use for splitting
operator = 0                        # enumerator for equals (=)
value = 'Texas'                     # value to use for extracting to new layer
output = 'c:/temp/Texas.shp'        # output layer created by extract tool

# run the Extract by attribute tool
processing.runalg('qgis:extractbyattribute', input, field, operator, value, output)

Save the script with the name SplitLayerByAttribute.py in the scripts folder (in Windows - C:\Users\USER_NAME\.qgis2\processing\scripts) and close the Script editor to refresh the list of available scripts. It is worth noting that any syntax errors will cause the script to not be displayed in the Scripts Toolbox. Expand Scripts > User Scripts to show the new SplitLayerByAttribute script. Double click the script’s icon to open the default dialog and run the script. According to the variables defined in the script, it will run silently and create a Texas.shp layer in the designated location and close the dialog when finished. View Texas.shp layer in the current QGIS Project to ensure everything looks correct.

Unique Values

The above script works well for extracting a feature with a known value but is not very flexible because the parameters are hardcoded. To make the basic script more productive, the Extract by attribute call needs to create a new layer for each unique feature named in the US States layer. Essentially splitting US States layer into individual state layers.

The QGIS Toolbox also has a List unique values tool that returns a Python dictionary object that contains the list of unique values for a specified field. Viewing the Help tab for this tool shows many of the same parameters provided for the Extract by attribute tool can be used for the List unique values tool. The List unique values dictionary output contains a UNIQUE_VALUES key with a semicolon delimited list of unique values. Using the list of unique state names in a loop that repeatedly calls the Extract by attribute tool should meet the requirements of splitting a layer using attribute values.

Edit the basic script in the Script editor by right-clicking the SplitLayerByAttribute icon and selecting Edit script from the popup menu. Modify the script to include running the List unique values tool and looping through the returned list of unique values. This list of unique values is provided in a semicolon delimited string and will need to be split into an array before it can be used. The hardcoded output variable will need to be changed to point to an existing folder instead of a particular file name. To simplify the construction of final output file names, the updated script will use the os.path function to concatenate the output folder with each value in the unique names array to form a final output file name. The hardcoded value variable in the basic script can be removed because it will be replaced by the unique names array.

import processing   # utilize the Processing Framework
import os           # the standard Python os library

input = 'c:/temp/US_States_NE.shp'  # input layer to split
field = 'name'                      # attribute with values to use for splittting
operator = 0                        # enumerator for equals (=)
output = 'c:/temp/'                 # folder for outputs

# obtain dictionary object that contains unique values from specified field
unique = processing.runalg('qgis:listuniquevalues', input, field, None)
# get the list of unique values from the dictionary's UNIQUE_VALUES key
# unique values are in a semicolan delimeted string, so split them at them
# at the same time to a Python array
values = unique['UNIQUE_VALUES'].split(';')

# loop through the unique values array
for value in values:
    # build the output layer path
    layer_path = os.path.join(output , value + '.shp')
    # run the extract by attribute tool
    processing.runalg('qgis:extractbyattribute', input, field, operator, value, layer_path)

Save and run the SplitLayerByAttribute script again. It should take a while longer to complete because the script will be creating many more layers than before. Check the output folder to and verify that the new layers have been correctly created.

Dialog Interface

The script accomplishes its main purpose, splitting a layer by attribute values, but it would be more useful if the remaining hardcoded variables were available as user inputs on the dialog. This functionality can be easily implemented by editing the script again and adding dialog parameters at the beginning of the script file. The dialog parameters are denoted by using double Python comment (#) characters. For more information about using dialog parameters see the Processing algorithms as Python scripts documentation. Since the variables in this script have already been defined, we will simply assign the user inputs to the existing variables.

##Vector Scripts=group
##Split Layer by Attribute=name
##Input_layer=vector
##Field=field Input_layer
##Output_folder=folder

import processing   # utilize the Processing Framework
import os           # the standard Python os library

input = Input_layer        # input layer to split
field = Field              # attribute with values to use for splittting
operator = 0               # enumerator for equals (=)
output = Output_folder     # output layer created by the tool

# obtain dictionary object that contains unique values from specified field
unique = processing.runalg('qgis:listuniquevalues', input, field, None)
# get the list of unique values from the dictionary's UNIQUE_VALUES key
# unique values are in a semicolan delimeted string, so split them at them
# at the same time to a Python array
values = unique['UNIQUE_VALUES'].split(';')

# loop through the unique values array
for value in values:
    # build the output layer path
    layer_path = os.path.join(output , value + '.shp')
    # run the extract by attribute tool
    processing.runalg('qgis:extractbyattribute', input, field, operator, value, layer_path)

Save and run the script again. The script has been moved to the group Vector Scripts, has been given a user friendly name and has a dialog to get user inputs to run the script.

Split Layer by Attribute dialog.

Communicating with the User

The Split Layer by Attribute script now has a nice dialog but it does not communicate any information to the user. By adding a few additional lines of code, the script can inform the user of its progress and notify the user when the task is completed.

##Vector Scripts=group
##Split Layer by Attribute=name
##Input_layer=vector
##Field=field Input_layer
##Output_folder=folder

import processing   # utilize the Processing Framework
import os           # the standard Python os library
import qgis.utils   # get access to the QGIS message bar 

input = Input_layer       # input layer to split
field = Field             # attribute with values to use for splittting
operator = 0              # enumerator for equals (=)
output = Output_folder    # output layer created by the tool

# obtain dictionary object that contains unique values from specified field
unique = processing.runalg('qgis:listuniquevalues', input, field, None)

# get the list of unique values from the dictionary's UNIQUE_VALUES key
# unique values are in a semicolan delimeted string, so split them at them
# at the same time to a Python array
values = unique['UNIQUE_VALUES'].split(';')

# start an iteration count for showing progress
count = 0

# inform the user about the pending operation
progress.setInfo('\nSplitting ' + input + ' into layers using the ' + field + ' attribute.')

# loop through the unique values array
for value in values:
    # increment the counter
    count +=1
    
    # show the progress as percent complete
    pct = int(float(count/float(len(values))) * 100)
    progress.setPercentage(pct)
    
    # build the output layer path
    layer_path = os.path.join(output , value + '.shp')
    # run the extract by attribute tool
    processing.runalg('qgis:extractbyattribute', input, field, operator, value, layer_path)

# notify the user the operation is complet using the QGIS message bar
qgis.utils.iface.messageBar().pushMessage('Split layer complete:', 'Selected layer successfully split into ' + str(count) + ' layers.', level=0, duration=5)

To finalize script, provide users with help information about how to use the tool. Open the script again in the Script editor and click the Edit script help icon (dialog with a pencil) on the toolbar. Complete all the relevant elements in the Help dialog and click OK to save the information. Here’s how the Help tab looks when it is completed.

Split Layer by Attribute help.

Final Thoughts

Authoring custom scripts with the Script editor is a great way to begin automating your GIS workflows. QGIS offers additional ways of implementing custom scripting and graphically building models, all of which help to automate tasks. Check out the QGIS Documentation for more information about getting started with scripting in QGIS.