Pythonian Adventures 3

Wednesday, 6 February, 2008

Over the last two blogs I’ve briefly covered why we may want to use a scripting language and how this fitted with ESRI’s view of how we should use ArcGIS. Since late last year I’ve been working on a small project that has required a not too complex, but quite fiddly, process to run iteratively over ~1000 files. This was definitely a job for scripting so I bit the bullet and started playing around with Python. My colleague James O’Brien does much more on the programming front than me and he set me on the right path with a nice sample script to work from (thanks James!). He also fielded more than a few calls with me cursing Python and ArcGIS. The ESRI-L list was also very helpful in getting responses to quite a few queries.

And in fact the best place to start is not with Python but with Model Builder. As I noted previously, Model Builder is an excellent method by which you can visually develop your process and save it. You can also run it on a single file to test all the stages of the model. Its also nice and easy to use which is a bonus!

I did actually think I would be able to do all my processing in Model Builder entirely which would have been nice. However it is painfully obvious that Model Builder was NOT designed to work iteratively and loop through the processing of multiple files. This functionality has actually been added in ArcGIS 9.2 however it is a real kludge. You need to set up a count variable at the beginning, setting the loop properties on it. The model properties then need to be changed to reflect the iterative status. You then need to spend ages working out how to add multiple files to the input variable; another kludge. This did actually work, EXCEPT the processing I needed to do required setting of ArcToolBox environment variables on EACH loop. You CANNOT do this in Model Builder. I would consider the looping functionality to be still in beta and it may serve some purposes if you can fathom ESRI’s logic. Is it too much too ask for a fully functional, well designed, product?

As an aside, when you are using Model Builder you’ll notice there is a “Save” and “Export” button. If you want to send your model to someone else you will find that there is no SaveAs button which means you’ve got no way to access the file itself. Or so I thought…. In true ESRI fashion they have totally overlooked this possibility and don’t tell you that, in actual fact, your model needs to reside in a toolbox you have created. The whole toolbox will be stored in a single file and will reside in your Windows profile under Application Data/ESRI/ArcToolBox/MyToolBoxes. All you need to do is copy the .TBX file related to your toolbox.

With your model complete it can be exported to Python (although note that Model Builder also supports JScript and VBScript) and then simply becomes a text file which can be edited in any text editor. If you look at the code generated it is pretty simple and the exported model actually comes pretty close to the final Python code you need. I used a FOR looping routine that James gave me which worked very well. As I noted above, I wanted to set an ArcToolBox environment variables on EACH loop. This is very easy to do in Python but it didn’t seem to work entirely properly. The solution that worked for me was to reset the variable at the beginning of each iteration and then set it again, later on, to the value I wanted.

With Python script complete it can be loaded back in to ArcToolBox as a “New Script”. If you go into “Edit” mode it will load the script into the Python IDE; hitting F5 will run the script and give you any console messages, although this crashed quite regularly when I run my script iteratively. It appears more stable when run directly from ArcGIS. Something that James noted was that ArcCatalog appears more stable than ArcMap for running scripts and that was my experience as well. Although note that the licensing of ESRI extensions in ArcCatalog is entirely separate from ArcMap. Which means you will need to go in to Tools->Extensions to enable them.

Add comment

Fill out the form below to add your own comments