batch processing and raw input in python geomatica 2015 ...€¦ · batch processing and raw input...

Batch Processing and Raw Input in Python

Geomatica 2015 Tutorial

The first python tutorial, “Python Introduction” provides an introduction to python programming and performing simple Geomatica workflows. It is suggested that you complete the first tutorial before beginning this tutorial. This tutorial involves more complex scripts which introduce the concepts of batch file processing and raw input. The scripts created in this tutorial perform a permanent image enhancement on a number of images in a folder. In this tutorial the IDLE Python GUI is used to create the script. Notepad ++ can also be used, as shown in the Python Introduction tutorial.

Using IDLE 1. Open the IDLE Python GUI (Start Python 2.7 IDLE (Python GUI) 2. Open a new file (File New Window) 3. Save your new file as GeomaticaPython.py, making sure to add the .py at the end.

Batch File Processing There are many situations where you need to batch process many images. With python, a script can be written to loop through files within a folder and apply algorithms to all files. In this example an equalization enhancement will be applied to various files. The workflow to create channels which have permanent enhancements applied is as follows:

Create LUTs (FUN) Create 3 8bit channels (PCIMOD) Apply LUTs and save to new channels (LUT)

Import Algorithms Before you can begin using the algorithms in your script you must first import them.

1. Using the form of from ___ import ___, import the algorithms fun, lut and pcimod. 2. Add from pci.exception import * to ensure the exceptions are available. 3. For this tutorial you will also need to import fnmatch, sys and os. These are python packages that let

you access file structures. 4. Add the following code to your script:

import locale locale.setlocale(locale.LC_ALL, "") locale.setlocale(locale.LC_NUMERIC, "C") This section of code is added to ensure that Python is properly configured in that same was as PCI’s C/C++ code.

Page | 1

The information in this document is subject to change without notice and should not be construed as a commitment by PCI Geomatics.

PCI Geomatics assumes no responsibility for any errors that may appear in this document.

mailto:[email protected]


















































Setup the Data Paths 1. Find the path name for your input folder. This folder will hold all of the images that you wish to

enhance. 2. Set your first path variable: InFolder = r“your pathname”. Generally in python a backslash (\) is an

escape character which will move the rest of the string to the next line. Since we do not want this within our pathnames the r is added before the string. The r tells python to ignore the backslashes.

Create a List of All Files In order to navigate through all of the files within InFolder a list of all possible files needs to be generated. In this tutorial a filter will be added so only .pix files are used.

1. Add a line to specify the type of filter. file_filter = “*.pix”. Any file that has a .pix extension will be filtered.

2. Add another line to create an empty list. input_file_list =[] . This empty list will be populated with file names from the directory.

3. The next line of code will start a for loop which searches through a directory and its sub directory for any files. for r,d,f in os.walk(InFolder):

Page | 2





a. Three variables are extracted by os.walk: r, d and f. r is the main directory, d is any sub directories and f is the file names within r. During the first iteration of the loop r is equal to InFolder, d is all of the subfolders within InFolder and f is any files that are in InFolder (but not in the sub folders). On the next iteration the first subfolder is searched and f changes to become the files within the subfolder.

4. There is another for loop within the previous loop; for file in fnmatch.filter (f, file_filter): In this loop each file within f is iterated and filtered depending on the file extension (file_filter). In this case, only files that have the .pix extension are processed.

5. For each file in f, the name of the file is printed using the following line of code: print "found valid input file: ", file . The pathname and filename of each file is added to input_files_list using the append command; input_files_list.append(os.path.join(r,file)) . The command os.path.join(r, file) adds the filename (file) to the end of the pathname (r). As you will recall from above, r is the directory currently being searched by the first for loop.

6. These two for loops continue to search through the directories and add filenames to input_files_list until all files have been processed.

PCIMOD Algorithm Before applying enhancements you need to create new channels in which to save the LUTs. To perform this task the PCIMOD algorithm is used. Remember that you can find addition information about the algorithms, including example python scripts in the Geomatica Help. In the script a loop is created to process each image in input_files_list.

1. Start the for loop in a similar way as above. a. Type for pix in input_files_list: – pix represents the image that is currently being processed in

the input_file_list. 2. List the arguments that are used in the FUN algorithm. You can check FUN.py in C:\PCI

Geomatics\Geomatica 2015\Examples\Python or the Geomatica Help to determine the proper argument organization.

Page | 3





a. The first argument (file) will be set to pix. b. The argument pciop specifies whether channels are added, subtracted or deleted. Set this

variable to “ADD”. c. Pcival is the final variable, which will be set to [3,0,0,0]. This value will vary based on how

many bands are used in the enhancement. One channel is added for each band that will be enhanced. This variable specifies how many 8bit, 16bit signed, 16bit unsigned or 32bit real channels you want to add to pix. For this tutorial it is assumed that all three band of the input imagery will be enhanced.

3. Create a try and except clause. The purpose of this clause is to determine whether the algorithm is able to run properly. First the program tries to run the algorithm (try: pcimod(file, pciop, pcival)). If this fails one of two exceptions is produced; either a PCI exception (except PCIException, e: print e) or a python exception (except Exception, e: print e). The PCI exceptions are a result of an issue with the algorithm (ie. Incorrect arguments, wrong data type) or the software in general (ie. Licensing error). The python exceptions are produced because of errors in the code or python installation.

4. Add a print line after the loop which will print the string “PCIMOD Algorithm Complete. Channels added.” Ensure that the print statement is outside of the loop. This means that the print statement has the same indentation as the for statement. The print statement will not be executed until iteration of all files is complete and all new channels have been added. The purpose of this statement is to notify the user that this part of the process is complete.

FUN Algorithm The next algorithm that is run is the FUN algorithm, which creates look-up tables for each input band.

1. Using the same for loop organization with the try and except clauses write code for the FUN algorithm. The image below shows the code for this section.

2. The func variable specifies the enhancement type which will be “EQUA”. Dbic is the bands which the LUTs will be created for. The variable dbsn is the new LUT segment name (u"Equa LUT") and dbsd is the LUT segment description (“Equalization LUTs”). All other values are left as default ([]).

Page | 4





3. The lasc variable is of interest for the purpose of this script. Once fun has fully run lasc will be populated with all of the LUT segment numbers that were created. This variable will be useful for the next algorithm.

4. Add the print line at the end of the try and except clause to tell the user that the FUN algorithm has completed (print "FUN Algorithm Complete. LUTs created").

LUT Algorithm The final algorithm that is executed for this example is LUT. This algorithm applies the LUTs that were created by FUN to the three original images bands. The enhanced bands are then saved to the three new bands from PCIMOD.

1. Using the same structure as above write the for loop and the try and except clause. 2. This algorithm has three variables to define channels; dbic, dblut and dboc. These variables specify the

input image bands, LUTs (from FUN) and empty 8bit channels (from PCIMOD), respectively. In this tutorial dbic = [1,2,3], dblut = lasc and dboc = [4,5,6]. Dblut asks for a list of the LUTs that will be used in the enhancement, which are the LUTs created by FUN. Remember that lasc was a variable that was populated with all the generated LUTs when FUN ran.

3. Write two final print statements: "LUT Algorithm Complete. Enhancements applied" and “All Processing Complete”.

Page | 5





The full script is now complete. To run this script using the IDLE window navigate to Run Run Module. If it asks you to save your file click OK. You can find instructions on how to run scripts in Focus and Command Line in the first tutorial “Python Introduction”. If you are using IDLE the shell window should look similar to the following image after the script has completed.

Defining Variables as Raw Input A more advanced method of defining path names and algorithm input arguments is through raw input. Variables such as InFolder can be set to take in raw input from the user. The benefit of using raw input is that

Page | 6





users of the script will not have to open and edit the script. The script that you created above is only applicable to one specific project since all the inputs are defined in the script. Changing the algorithm and file inputs to raw input allows the script to be used for different projects.

Raw Input Data Paths The two folder variables from the Batch File Enhancement example can be changed to raw input.

1. Change InFolder definition to InFolder = raw_input (“Please state your input folder path”). The raw_input() function tells the program to define InFolder as the value that the user inputs. The string in the brackets is printed on the screen when raw_input is executed and tells the user what to do.

2. Save the python script and click run (Run Run Module). A line saying “Please state your input folder path: ” When this message appears you can enter the entire pathname

3. Depending on how generalized you want to make a script, you can continue to set any other arguments and inputs to raw_input. If you are making the script applicable for any project, you may need to add more print statements to provide the end user with more information. When setting algorithm arguments to raw_input keep in mind that the argument definitions currently reside within a loop. To avoid the unnecessary input on the user’s part you must move the raw_input definitions outside of the loop. The example below demonstrates how you can define the arguments of PCIMOD as raw_input. In the example script, any arguments that were left as default in our earlier example are not defined as raw_input.

Page | 7





Once you have changed all of the inputs to raw input run the script. The final result of the script completion in IDLE should look similar to the image below. The raw inputted information is shown in black. You can now open one of the images that you ran this script on to find the added LUTs and enhanced channels.

The scripts that you create in python can be adapted to suit your specific needs. This tutorial provided some basic knowledge to help you begin batch processing and adding raw inputs. The full scripts that were used in this tutorial can be found at https://github.com/PCIGeomatics. For a more advanced example of using python to integrate ArcGIS and Geomatica watch the video at https://www.youtube.com/watch?v=owrqMPWHjf8.

For any technical assistance you can always contact [email protected] or [email protected] accordingly. Please be sure to include all relevant screen captures, and to quote your customer number for easy reference.

Define the raw input variables outside of the loop

Define the algorithm arguments as the raw_input variables

Page | 8



https://github.com/PCIGeomatics

https://www.youtube.com/watch?v=owrqMPWHjf8



Page | 9



batch processing and raw input in python geomatica 2015 ...€¦ · batch processing and raw input...

Documents