Advanced Zope Scripting

Zope manages your presentation, logic and data with objects. So far, you've seen how Zope can manage presentation with DTML and Page Templates, and data with files and images. This chapter shows you how to add Script objects which allow you to write scripts in Python and Perl through your web browser.

What is logic and how does it differ from presentation? Logic provides those actions which change objects, send messages, test conditions and respond to events, whereas presentation formats and displays information and reports. Typically you will use DTML or Page Templates to handle presentation, and Zope scripting with Python and Perl to handle logic.

Zope Scripts

Zope Script objects are objects that encapsulate a small chunk of code written in a programming language. Script objects first appeared in Zope 2.3, and are now the preferred way to write programming logic in Zope. Currently, Zope comes with Python-based Scripts, which are written in the Python language. There are third-party extensions to Zope which allow you to write Perl-based Scripts in the Perl language.

So far in this book you have heavily used DTML Methods, DTML Documents, and Page Templates (ZPT) to create simple web applications in Zope. DTML and ZPT allow you to perform simple scripting operations such as string manipulation. For the most part, however, DTML and ZPT should be used for presentation. DTML Methods are explained in the chapter entitled Basic DTML, and the chapter entitled Advanced DTML. ZPT is explained in the chapter entitled Using Zope Page Templates, and the chapter entitled Advanced Page Templates. .

Here is an overview of Zope's scripts:

Python-based Scripts

You can use Python, a general purpose scripting language, to control Zope objects and perform other tasks. These Scripts give you general purpose programming facilities within Zope.

External Methods

These are also written in Python, but the code is stored on the filesystem. External Methods allow you to do many things that are restricted from Python-based Scripts for security reasons.

Perl-based Scripts

You can use Perl, a powerful text processing language, to script Zope objects and access Perl libraries. These scripts offer benefits similar to those of Python-based Scripts, but may be more appealing for folks who know Perl but not Python, or who want to use Perl libraries for which there are no Python equivalents. Currently you must download and install third-party extensions before you can use Perl scripts in Zope.

You can add these scripts to your Zope application just like any other object. Details about each type of script are provided below, in the sections "Using Python-based Scripts," "Using External Methods," and "Using Perl-based Scripts" respectively.

Calling Scripts

Any Zope script may be called "from the web" (for example, from a browser or another web-aware tool). In addition, any type of script may be called by any other type of object; you can call a Python-based Script from a DTML Method, or a built-in method from a Perl-based Script. In fact scripts can call scripts which call other scripts, and so on. As you saw in the chapter entitled Basic DTML, you can replace a script with a script implemented in another language transparently. For example, if you're using Perl to perform a task, but later decide that it would be better done in Python, you can usually replace the script with a Python-based Script with the same id.

Context

When you call a script, you usually want to single out some object that is central to the script's task, either because that object provides information that the script needs, or because the script will modify that object. In object-oriented terms, we want to call the script as a method of this particular object. But in conventional object-oriented programming, each object can perform the methods that are defined in (or inherited by) its class. How is it that one Zope script can be used as a method of potentially many objects, without the script being defined in the classes that define these objects?

Recall that in the chapter entitled Acquisition, we learned that Zope can find objects in different places by acquiring them from parent containers. Acquisition allows us to treat a script as a method that can be called in the context of any suitable object, just by constructing an appropriate URL. The object on which we call a script gives it a context in which to execute. It is simpler to just say that you are calling the script on the object. Or, to put it another way, the context is the environment in which the Script executes, from which the script may get information that it needs to do its job.

Another way to understand the context of a script is to think of the script as a function in a procedural programming language, and its context as an implicit argument to that function.

There are two general ways to call a script and provide it with a context: by visiting a URL, and by calling the script from another script or template.

Calling Scripts From the Web

You can call a script directly with a web browser by visiting its URL. You can call a single script on different objects by using different URLS. This works because Zope can determine the script's context by URL. This is a powerful feature that enables you to apply logic to objects like documents or folders without having to embed the actual code within the object.

To call a script on an object from the web, simply visit the URL of the object, followed by the name of the script. This places the script in the context of your object. For example, suppose you have a collection of objects and scripts as shown in the figure below.

Figure 14-1 A collection of objects and scripts

To call the feed script on the hippo object you would visit the URL Zoo/LargeAnimals/hippo/feed. To call the feed script on the kangarooMouse object you can visit the URL Zoo/SmallAnimals/kangarooMouse/feed. These URLs place the feed script in the context of the hippo and kangarooMouse objects, respectively.

Zope uses a URL as a map to find which object and which script you want to call.

URL Traversal and Acquisition

Zope breaks apart the URL and compares it to the object hierarchy, working backwards until it finds a match for each part. This process is called URL traversal. For example, when you give Zope the URL Zoo/LargeAnimals/hippo/feed, it starts at the root folder and looks for an object named Zoo. It then moves to the Zoo folder and looks for an object named LargeAnimals. It moves to the LargeAnimals folder and looks for an object named hippo. It moves to the hippo object and looks for an object named feed. The feed script cannot be found in the hippo object and is located in the Zoo folder by using acquisition. Zope always starts looking for an object in the last object it traversed, in this case hippo. Since hippo does not contain anything, Zope backs up to hippo's immediate container, LargeAnimals. The feed script is not there, so Zope backs up to LargeAnimals' container, Zoo, where feed is finally found.

Now Zope has reached the end of the URL and has matched objects to every name in the URL. Zope recognizes that the last object found, feed, is callable, and calls it in the context of the second to last object found - the hippo object. This is how the feed script is called on the hippo object.

Likewise you can call the wash method on the hippo by visiting the URL Zoo/LargeAnimals/hippo/wash. In this case Zope acquires the wash method from the LargeAnimals folder.

Passing Arguments with an HTTP Query String

You can pass arguments to a URL, too. Just append them as standard query strings:

           http://my-zope-site:8080/Zoo/LargeAnimals/hippo/wash?soap=lye

Calling Scripts from Other Objects

You can call scripts from other objects, whether they are DTML objects, Page Templates, or Scripts (Python or Perl). The semantics of each language differ slightly, but the same rules of acquisition apply. You do not necessarily have to know what language is used in the script you are calling; you only need to pass it any parameters that it requires, if any.

Calling Scripts from DTML

As you saw in the chapter entitled Advanced DTML, you can call Zope scripts from DTML with the call tag. For example:

          <dtml-call updateInfo>

DTML will call the updateInfo script, whether it is implemented in Perl, Python, or any other language. You can also call other DTML objects and SQL Methods the same way.

If the updateInfo script requires parameters, either your script must have a name for the DTML namespace binding (see Binding Variables in the section "Using Python-based Scripts" below), so that the parameters will be looked up in the DTML namespace, or you must pass the parameters in an expression. For example::

You can also pass in any variables that are valid in the current DTML namespace. For example, if newColor and newPattern are defined using dtml-let, you could pass the variables as parameters like this:

          <dtml-call expr="updateInfo(color=newColor, pattern=newPattern)">

You can also pass variables that are defined automatically by DTML tags such as dtml-in. For example:

          <dtml-in all_animals prefix="seq">            <dtml-call expr="feed(animal=seq_item)">          </dtml-in>

This assumes that feed is a script and has a parameter called animal. The standard names used during DTML loops (sequence-item, sequence-key, et al.) are a bit cumbersome to spell out in a Python expr, because "sequence-item" would be interpreted as sequence minus item. To avoid this problem, we use the prefix attribute of the dtml-in tag, which uses the specified value ("seq") and an underscore ("_") instead of the customary "sequence-" string.

Calling scripts from Python and Perl

Calling scripts from other Python or Perl scripts works the same as calling scripts from DTML, except that you must always pass script parameters when you call a script from Python or Perl. For example, here is how you might call the updateInfo script from Python:

          new_color='brown'          context.updateInfo(color=new_color,                              pattern="spotted")

Note the use of the context variable to tell Zope to find updateInfo by acquisition.

From Perl you could do the same thing using standard Perl semantics for calling scripts:

          $new_color = 'brown';          $self->updateInfo(color => $new_color,                             pattern => "spotted");

Here we see that self is the way we refer to the current context in a Perl-based script.

Zope locates the scripts you call by using acquisition the same way it does when calling scripts from the web. Returning to our hippo feeding example of the last section, let's see how to vaccinate a hippo from Python and Perl. The figure below shows a slightly updated object hierarchy that contains two scripts, vaccinateHippo.py and vaccinateHippo.pl.

Figure 14-2 A collection of objects and scripts

Suppose vaccinateHippo.py is a Python script. Here is how you can call the vaccinate script on the hippo object from the vaccinateHippo.py script:

          context.Vet.LargeAnimals.hippo.vaccinate()

In other words, you simply access the object by using the same acquisition path as you would use if you called it from the web. The result is the same as if you visited the URL Zoo/Vet/LargeAnimals/hippo/vaccinate. Note that in this Python example, we do not bother to specify Zoo before Vet. We can leave Zoo out because all of the objects involved, including the script, are in the Zoo folder, so it is implicitly part of the acquisition chain.

Likewise, in the Perl version, vaccinateHippo.pl, you could say:

          $self->Vet->LargeAnimals->hippo->vaccinate();

Calling Scripts from Page Templates

Calling scripts from Page Templates is much like calling them by URL or from Python. Just use standard TALES path expressions as described in the chapter entitled Using Zope Page Templates. For example:

          <div tal:replace="here/hippo/feed" />

The inserted value will be HTML-quoted. You can disable quoting by using the structure keyword, as described in the chapter entitled Advanced Page Templates.

Page Templates do not really provide an equivalent to DTML's call tag. To call a script without inserting a value in the page, you can use define and ignore the variable assigned:

          <div tal:define="dummy here/hippo/feed" />

In a page template, here refers to the current context. It behaves much like the context variable in a Python-based Script. In other words, hippo and feed will both be looked up by acquisition.

If the script you call requires arguments, you must use a TALES python expression in your template, like so:

          <div tal:replace="python:here.hippo.feed(food='spam')" />

Just as in Path Expressions, the here variable refers to the acquisition context the Page Template is called in.

The python expression above is exactly like a line of code you might write in a Script (Python). The only difference is the name of the variable used to get the acquisition context. Don't be misled by the different terminology: context is context, whatever you call it. Unfortunately, the different names used in ZPT and Python Scripts evolved independently. (Note that as of this writing, the ZPT variable here is planned to become context in a future version of Zope, probably Zope 3.)

For further reading on using Scripts in Page Templates, refer to the chapter entitled Advanced Page Templates.

Calling Scripts: Summary and Comparison

Let's recap the ways to call a hypothetical updateInfo script on a foo object, with argument passing: from your web browser, from Python, from Perl, from DTML, and from Page Templates.

by URL:

          http://my-zope-server.com:8080/foo/updateInfo?amount=lots

from a Python script:

          context.foo.updateInfo(amount="lots")

from a Perl script:

          $self->foo->updateInfo(amount="lots");

from a Page Template:

          <span tal:content="here/foo/updateInfo"/>

from a Page Template, with arguments:

          <span tal:content="python:here.foo.updateInfo(amount='lots')"/>

from DTML:

          <dtml-with foo >           <dtml-var updateInfo>          </dtml-with>

from DTML, with arguments:

          <dtml-with foo>           <dtml-var expr="updateInfo(amount='lots')">          </dtml-with>

another DTML variant:

          <dtml-var expr="_['foo'].updateInfo()">

Regardless of the language used, this is a very common idiom to find an object, be it a script or any other kind of object: you ask the context for it, and if it exists in this context or can be acquired from it, it will be used.

Zope will throw a KeyError exception if the script you are calling cannot be acquired. If you are not certain that a given script exists in the current context, or if you want to compute the script name at run-time, you can use this Python idiom:

          updateInfo = getattr(context, "updateInfo", None)          if updateInfo is not None:              updateInfo(color="brown", pattern="spotted")          else:              # complain about missing script

The getattr function is a Python built-in. The first argument specifies an object, the second an attribute name. The getattr function will return the named attribute, or the third argument if the attribute cannot be found. So in the next statement we just have to test whether the updateInfo variable is None, and if not, we know we can call it.

Using Python-based Scripts

Earlier in this chapter you saw some examples of scripts. Now let us take a look at scripts in more detail.

The Python Language

Python is a high-level, object oriented scripting language. Most of Zope is written in Python. Many folks like Python because of its clarity, simplicity, and ability to scale to large projects.

There are many resources available for learning Python. The python.org web site has lots of Python documentation including a tutorial by Python's creator, Guido van Rossum.

Python comes with a rich set of modules and packages. You can find out more about the Python standard library at the python.org web site.

Another highly respected source for reference material is Python Essential Reference by David Beazley, published by New Riders.

Creating Python-based Scripts

To create a Python-based Script choose Script (Python) from the Product add list. Name the script hello, and click the Add and Edit button. You should now see the Edit view of your script.

This screen allows you to control the parameters and body of your script. You can enter your script's parameters in the parameter list field. Type the body of your script in the text area at the bottom of the screen.

Enter name="World" into the parameter list field, and type:

        return "Hello %s." % name

... in the body of the script. Our script is now equivalent to the following function definition in standard Python syntax:

        def hello(name="World"):            return "Hello %s." % name

The result should appear something like the below image:

Figure 14-3 Script editing view

You can now test the script by going to the Test tab as shown in the figure below.

Figure 14-4 Testing a Script

Leave the name field blank and click the Run Script button. Zope should return "Hello World." Now go back and try entering your name in the Value field and click the Run Script button. Zope should now say hello to you.

Since scripts are called on Zope objects, you can get access to Zope objects via the context variable, as described above in the section "Calling Scripts". For example, this script returns the number of objects contained by a given Zope object:

        ## Script (Python) "numberOfObjects"        ##        return len(context.objectIds())

The script calls context.objectIds(), a method in the Zope API, to get a list of the contained objects. objectIds is a method of Folders, so the context object should be a Folder-like object. The script then calls len() to find the number of items in that list. When you call this script on a given Zope object, the context variable is bound to the context object. So if you called this script by visiting the URL FolderA/FolderB/numberOfObjects, the context parameter would refer to the FolderB object.

When writing your logic in Python you'll typically want to query Zope objects, call other scripts and return reports. For example, suppose you want to implement a simple workflow system in which various Zope objects are tagged with properties that indicate their status. You might want to produce reports that summarize which objects are in which state. You can use Python to query objects and test their properties. For example, here is a script named objectsForStatus with one parameter, status:

        ## Script (Python) "objectsForStatus"        ##parameters=status        ##        """        Returns all sub-objects that have a given status        property.        """        results=[]        for object in context.objectValues():            if object.getProperty('status') == status:                results.append(object)        return results

This script loops through an object's sub-objects and returns all the sub-objects that have a status property with a given value. The lines at the top starting with a double hash ('##') are generated by Zope when editing a script via FTP. You can specify parameters and other things here (this is covered in more detail in the next section, Binding Variables).

You could then use this script from DTML to email reports. For example:

        <dtml-sendmail>        To: <dtml-var ResponsiblePerson>        Subject: Pending Objects        These objects are pending and need attention.        <dtml-in expr="objectsForStatus('Pending')">        <dtml-var title_or_id> (<dtml-var absolute_url>)        </dtml-in>        </dtml-sendmail>

This example shows how you can use DTML (or Page Templates) for presentation or report formatting, while Python handles the logic. This is a very important pattern that you will witness and use repeatedly in Zope.

Binding Variables

A set of special variables is created whenever a Python-based Script is called. These variables, defined on the script's Bindings view in the Zope Management Interface, are used by your script to access other Zope objects and scripts. They are not available in other Zope objects such as Perl scripts or DTML Documents, though there is a similar set of variables available in ZPT.

By default, the names of these binding variables are set to reasonable values and you should not need to change them. They are explained here so that you know how each special variable works, and how you can use these variables in your scripts.

Context

The Context binding defaults to the name context. This variable refers to the object that the script is called on.

Container

The Container binding defaults to the name container. This variable refers in which the script is defined.

Script

The Script binding defaults to the name script. This variable refers to the script object itself.

Namespace

The Namespace binding is left blank by default. If your script is called from a DTML Method, and you have chosen a name for this binding, then the named variable contains the DTML namespace explained in the chapter entitled Advanced DTML . Furthermore, if this binding is set, the script will search for its parameters in the DTML namespace when called from DTML without explicitly passing any arguments.

Subpath

The Subpath binding defaults to the name traverse_subpath. This is an advanced variable that you will not need for any of the examples in this book. If your script is traversed, meaning that other path elements follow it in a URL, then those path elements are placed in a list, from left to right, in this variable.

If you edit your scripts via FTP, you will notice that these bindings are listed in comments at the top of your script files. For example:

        ## Script (Python) "example"        ##bind container=container        ##bind context=context        ##bind namespace=        ##bind script=script        ##bind subpath=traverse_subpath        ##parameters=name, age        ##title=        ##        return "Hello %s you are %d years old." % (name, age)

You can change your script's bindings by changing these comments and then uploading your script. Note the implication that these comments are not merely comments: they carry semantic significance.

What are all these bindings good for, anyway? They can be used to control where Zope looks for the objects you need. We have been explaining the idea of context throughout this chapter; not surprisingly, in a Python-based Script, you can access its context through the context binding. Returning to our Zoo example, our feed script might contain a line like this:

        animal_id = context.getId()

In this example, we get the value "hippo," because hippo is the context and its id is "hippo". But we can call getId() on other variables, since nearly all Zope objects support this method. For example:

        folder_id = container.getId()        script_id = script.getId()

These values depend on the id of the script and the container it lives in. Calling the script in a different context will have no effect on these variables. Acquisition still applies when using container, but Zope will only try to acquire from the script's containers, not from the calling context.

Accessing the HTTP Request

What if we need to get user input, e.g. values from a form? We can find the REQUEST object, which represents a Zope web request, in the context. For example, if we visited our feed script via the URL Zoo/LargeAnimals/hippo/feed?food_type=spam, we could access the food_type variable as context.REQUEST.food_type. This same technique works with variables passed from forms.

Another way to get the REQUEST is to pass it as a parameter to the script. If REQUEST is one of the script's parameters, Zope will automatically pass the HTTP request and assign it to this parameter. We could then access the food_type variable as REQUEST.food_type.

String Processing in Python

One common use for scripts is to do string processing. Python has a number of standard modules for string processing. You cannot do regular expression processing within Python-based Scripts because of security restrictions. If you really need regular expressions, you can easily use them from External Methods, described in a subsequent section of this chapter. However, in a Script (Python), you do have access to the string module. You have access to the string module from DTML as well, but it is much easier to use from Python. Python 2.X also provides "string methods" which can perform most of the same duties as the string module.

Suppose you want to change all the occurrences of a given word in a DTML Document. Here is a script, replaceWord, that accepts two arguments, word and replacement. This will change all the occurrences of a given word in a DTML Document:

        ## Script (Python) "replaceWord"        ##parameters=word, replacement        ##        """        Replaces all the occurrences of a word with a        replacement word in the source text of a DTML        Document. Call this script on a DTML Document to use        it.         Note: you will need permission to edit a document in order        to call this script on the document.        This script assumes that the context is a DTML document,        which provides the document_src() and manage_edit() methods        described in Appendix B (API Reference).        """        import string        text=context.document_src()        text=string.replace(text, word, replacement)        context.manage_edit(text, context.title)

You can perform the same job by using the Python string method replace:

        ## Script (Python) "replaceWord"        ##parameters=word, replacement        ##        text=context.document_src()        text=text.replace(word, replacement)        context.manage_edit(text, context.title)

You can call this script from the web on a DTML Document to change the source of the document. For example, the URL Swamp/replaceWord?word=Alligator&replacement=Crocodile would call the replaceWord script on the document named Swamp and would replace all occurrences of the word Alligator with Crocodile.

You could also call this script from a DTML method, from a Page Template, or even from another Script, as described in this chapter under the heading "Calling Scripts from other Objects."

See the Python documentation for more information about manipulating strings from Python.

One thing that you might be tempted to do with scripts is to use Python to search for objects that contain a given word in their text or as a property. You can do this, but Zope has a much better facility for this kind of work, the Catalog. See the chapter entitled Searching and Categorizing Content for more information on searching with Catalogs.

Doing Math

Another common use of scripts is to perform mathematical calculations which would be unwieldy from DTML or ZPT. The math and random modules give you access from Python to many math functions. These modules are standard Python services as described on the Python.org web site.

math

Mathematical functions such as sin and cos.

random

Pseudo random number generation functions.

One interesting function of the random module is the choice function that returns a random selection from a sequence of objects. Here is an example of how to use this function in a script called randomImage:

        ## Script (Python) "randomImage"        ##        """        When called on a Folder that contains Image objects this        script returns a random image.        """        import random        return random.choice(context.objectValues('Image'))

Suppose you had a Folder named Images that contained a number of images. You could display a random image from the folder in DTML like so:

        <dtml-with Images>          <dtml-var randomImage>        </dtml-with>

This DTML calls the randomImage script on the Images folder. The result is an HTML IMG tag that references a random image in the Images Folder.

A ZPT equivalent to the above DTML script is:

        <span tal:replace="here/Images/randomImage"/>

Print Statement Support

Python-based Scripts have a special facility to help you print information. Normally printed data is sent to standard output and is displayed on the console. This is not practical for a server application like Zope since most of the time you do not have access to the server's console. Scripts allow you to use print anyway and to retrieve what you printed with the special variable printed. For example:

        ## Script (Python) "printExample"        ##        for word in ('Zope', 'on', 'a', 'rope'):            print word        return printed

This script will return:

        Zope        on        a        rope

The reason that there is a line break in between each word is that Python adds a new line after every string that is printed.

You might want to use the print statement to perform simple debugging in your scripts. For more complex output control you probably should manage things yourself by accumulating data, modifying it and returning it manually rather than relying on the print statement. And for control of presentation, you should return the script output to a Page Template or DTML page which then displays the return value appropriately.

Built-in Functions

Python-based Scripts give you a slightly different menu of built-ins than you find in normal Python. Most of the changes are designed to keep you from performing unsafe actions. For example, the open function is not available, which keeps you from being able to access the filesystem. To partially make up for some missing built-ins a few extra functions are available.

The following restricted built-ins work the same as standard Python built-ins: None, abs, apply, callable, chr, cmp, complex, delattr, divmod, filter, float, getattr, hash, hex, int, isinstance, issubclass, list, len, long, map, max, min, oct, ord, repr, round, setattr, str, tuple. For more information on what these built-ins do, see the online Python Documentation.

The range and pow functions are available and work the same way they do in standard Python; however, they are limited to keep them from generating very large numbers and sequences. This limitation helps protect against denial of service attacks as described previously.

In addition, these DTML utility functions are available: DateTime, and test. See Appendix A, DTML Reference for more information on these functions.

Finally to make up for the lack of a type function, there is a same_type function that compares the type of two or more objects, returning true if they are of the same type. So instead of saying:

        if type(foo) == type([]):            return "foo is a list"

... to check if foo is a list, you would instead use the same_type function to check this:

        if same_type(foo, []):            return "foo is a list"

Now let's take a look at External Methods which provide more power and fewer restrictions than Python-based Scripts.

Using External Methods

Sometimes the security constraints imposed by scripts, DTML and ZPT get in your way. For example, you might want to read files from disk, or access the network, or use some advanced libraries for things like regular expressions or image processing. In these cases you can use External Methods. We encountered External Methods briefly in the chapter entitled Using Basic Zope Objects . Now we will explore them in more detail.

To create and edit External Methods you need access to the filesystem. This makes editing these scripts more cumbersome since you can't edit them right in your web browser. However, requiring access to the server's filesystem provides an important security control. If a user has access to a server's filesystem they already have the ability to harm Zope. So by requiring that unrestricted scripts be edited on the filesystem, Zope ensures that only people who are already trusted have access.

External Method code is created and edited in files on the Zope server in the Extensions directory. This directory is located in the top-level Zope directory. Alternately you can create and edit your External Methods in an Extensions directory inside an installed Zope product directory, or in your INSTANCE_HOME directory if you have one. See the chapter entitled Installing and Starting Zope for more about INSTANCE_HOME.

Let's take an example. Create a file named Example.py in the Zope Extensions directory on your server. In the file, enter the following code:

      def hello(name="World"):          return "Hello %s." % name 

You've created a Python function in a Python module. But you have not yet created an External Method from it. To do so, we must add an External Method object in Zope.

To add an External Method, choose External Method from the product add list. You will be taken to a form where you must provide an id. Type "hello" into the Id field, type "hello" in the Function name field, and type "Example" in the Module name field. Then click the Add button. You should now see a new External Method object in your folder. Click on it. You should be taken to the Properties view of your new External Method as shown in the figure below.

Figure 14-5 External Method Properties view

Note that if you wish to create several related External Methods, you do not need to create multiple modules on the filesystem. You can define any number of functions in one module, and add an External Method to Zope for each function. For each of these External Methods, the module name would be the same, but function name would vary.

Now test your new script by going to the Test view. You should see a greeting. You can pass different names to the script by specifying them in the URL. For example, hello?name=Spanish+Inquisition.

This example is exactly the same as the "hello world" example that you saw for Python-based scripts. In fact, for simple string processing tasks like this, scripts offer a better solution since they are easier to work with.

The main reasons to use an External Method are to access the filesystem or network, or to use Python packages that are not available to restricted scripts.

For example, a Script (Python) cannot access environment variables on the host system. One could access them using an External Method, like so:

      def instance_home():         import os         return os.environ.get('INSTANCE_HOME')

Regular expressions are another useful tool that are restricted from Scripts. Let's look at an example. Assume we want to get the body of an HTML Page (everything between the body and /body tags):

      import re      pattern = r"<\s*body.*?>(.*?)</body>"      regexp = re.compile(pattern, re.IGNORECASE + re.DOTALL)      def extract_body(htmlstring):          """          If htmlstring is a complete HTML page, return the string          between (the first) <body> ... </body> tags          """          matched = regexp.search(htmlpage)          if matched is None: return "No match found"          body = matched.group(1)          return body 

Note that we import the re module and define the regular expression at the module level, instead of in the function itself; the extract_body() function will find it anyway. Thus, the regular expression is compiled once, when Zope first loads the External Method, rather than every time this External Method is called. This is a common optimization tactic.

Now put this code in a module called my_extensions.py. Add an External Method with an id of body_external_m; specify my_extensions for the Module Name to use and, extract_body for Function Name.

You could call this for example in a Script (Python) called store_html like this:

      ## Script (Python) "store_html"      ##      # code to get 'htmlpage' goes here...      htmlpage = "some string, perhaps from an uploaded file"      # now extract the body      body = context.body_external_m(htmlpage)      # now do something with 'body' ...

... assuming that body_external_m can be acquired by store_html. This is obviously not a complete example; you would want to get a real HTML page instead of a hardcoded one, and you would do something sensible with the value returned by your External Method.

Here is an example External Method that uses the Python Imaging Library (PIL) to create a thumbnail version of an existing Image object in a Folder. Enter the following code in a file named Thumbnail.py in the Extensions directory:

      def makeThumbnail(self, original_id, size=200):          """          Makes a thumbnail image given an image Id when called on a Zope          folder.          The thumbnail is a Zope image object that is a small JPG          representation of the original image. The thumbnail has an          'original_id' property set to the id of the full size image          object.          """          import PIL           from StringIO import StringIO          import os.path           # none of the above imports would be allowed in Script (Python)!          # Note that PIL.Image objects expect to get and save data          # from the filesystem; so do Zope Images. We can get around           # this and do everything in memory by using StringIO.          # Get the original image data in memory.          original_image=getattr(self, original_id)          original_file=StringIO(str(original_image.data))          # create the thumbnail data in a new PIL Image.           image=PIL.Image.open(original_file)          image=image.convert('RGB')          image.thumbnail((size,size))          # get the thumbnail data in memory.          thumbnail_file=StringIO()          image.save(thumbnail_file, "JPEG")           thumbnail_file.seek(0)          # create an id for the thumbnail          path, ext=os.path.splitext(original_id)          thumbnail_id=path + '.thumb.jpg'          # if there's an old thumbnail, delete it          if thumbnail_id in self.objectIds():              self.manage_delObjects([thumbnail_id])          # create the Zope image object for the new thumbnail          self.manage_addProduct['OFSP'].manage_addImage(thumbnail_id,                                                         thumbnail_file,                                                         'thumbnail image')          # now find the new zope object so we can modify           # its properties.          thumbnail_image=getattr(self, thumbnail_id)          thumbnail_image.manage_addProperty('original_id', original_id, 'string')

Notice that the first parameter to the above function is called self. This parameter is optional. If self is the first parameter to an External Method function definition, it will be assigned the value of the calling context (in this case, a folder). It can be used much like the context we have seen in Scripts (Python).

You must have PIL installed for this example to work. Installing PIL is beyond the scope of this book, but note that it is important to choose a version of PIL that is compatible with the version of Python that is used by your version of Zope.

To continue our example, create an External Method named makeThumbnail that uses the makeThumbnail function in the Thumbnail module.

Now you have a method that will create a thumbnail image. You can call it on a Folder with a URL like ImageFolder/makeThumbnail?original_id=Horse.gif This would create a thumbnail image named Horse.thumb.jpg.

You can use a script to loop through all the images in a folder and create thumbnail images for them. Create a Script (Python) named makeThumbnails:

      ## Script (Python) "makeThumbnails"      ##      for image_id in context.objectIds('Image'):          context.makeThumbnail(image_id)

This will loop through all the images in a folder and create a thumbnail for each one.

Now call this script on a folder with images in it. It will create a thumbnail image for each contained image. Try calling the makeThumbnails script on the folder again and you'll notice it created thumbnails of your thumbnails. This is no good. You need to change the makeThumbnails script to recognize existing thumbnail images and not make thumbnails of them. Since all thumbnail images have an original_id property you can check for that property as a way of distinguishing between thumbnails and normal images:

      ## Script (Python) "makeThumbnails"      ##      for image in context.objectValues('Image'):          if not image.hasProperty('original_id'):              context.makeThumbnail(image.getId())

Delete all the thumbnail images in your folder and try calling your updated makeThumbnails script on the folder. It seems to work correctly now.

Now with a little DTML you can glue your script and External Method together. Create a DTML Method called displayThumbnails:

      <dtml-var standard_html_header>      <dtml-if updateThumbnails>        <dtml-call makeThumbnails>      </dtml-if>      <h2>Thumbnails</h2>      <table><tr valign="top">      <dtml-in expr="objectValues('Image')">        <dtml-if original_id>          <td>            <a href="&dtml-original_id;"><dtml-var sequence-item></a><br>            <dtml-var original_id>          </td>         </dtml-if>      </dtml-in>      </tr></table>      <form>      <input type="submit" name="updateThumbnails" value="Update Thumbnails">      </form>      <dtml-var standard_html_footer>

When you call this DTML Method on a folder it will loop through all the images in the folder and display all the thumbnail images and link them to the originals as shown in the figure below.

Figure 14-6 Displaying thumbnail images

This DTML Method also includes a form that allows you to update the thumbnail images. If you add, delete or change the images in your folder you can use this form to update your thumbnails.

This example shows a good way to use scripts, External Methods and DTML together. Python takes care of the logic while the DTML handles presentation. Your External Methods handle external packages such as PIL while your scripts do simple processing of Zope objects. Note that you could just as easily use a Page Template instead of DTML.

Processing XML with External Methods

You can use External Methods to do nearly anything. One interesting thing that you can do is to communicate using XML. You can generate and process XML with External Methods.

Zope already understands some kinds of XML messages such as XML-RPC and WebDAV. As you create web applications that communicate with other systems you may want to have the ability to receive XML messages. You can receive XML a number of ways: you can read XML files from the file system or over the network, or you can define scripts that take XML arguments which can be called by remote systems.

Once you have received an XML message you must process the XML to find out what it means and how to act on it. Let's take a quick look at how you might parse XML manually using Python. Suppose you want to connect your web application to a Jabber chat server. You might want to allow users to message you and receive dynamic responses based on the status of your web application. For example suppose you want to allow users to check the status of animals using instant messaging. Your application should respond to XML instant messages like this:

        <message to="cage_monitor@zopezoo.org" from="user@host.com">          <body>monkey food status</body>        </message>

You could scan the body of the message for commands, call a script and return responses like this:

        <message to="user@host.com" from="cage_monitor@zopezoo.org">          <body>Monkeys were last fed at 3:15</body>        </message>

Here is a sketch of how you could implement this XML messaging facility in your web application using an External Method:

        # Uses Python 2.x standard xml processing packages.  See        # http://www.python.org/doc/current/lib/module-xml.sax.html for        # information about Python's SAX (Simple API for XML) support If        # you are using Python 1.5.2 you can get the PyXML package. See        # http://pyxml.sourceforge.net for more information about PyXML.        from xml.sax import parseString        from xml.sax.handler import ContentHandler        class MessageHandler(ContentHandler):            """            SAX message handler class            Extracts a message's to, from, and body            """            inbody=0            body=""            def startElement(self, name, attrs):                if name=="message":                    self.recipient=attrs['to']                    self.sender=attrs['from']                elif name=="body":                    self.inbody=1            def endElement(self, name):                if name=="body":                    self.inbody=0            def characters(self, content):                if self.inbody:                    self.body=self.body + content        def receiveMessage(self, message):            """            Called by a Jabber server            """            handler=MessageHandler()            parseString(message, handler)            # call a script that returns a response string            # given a message body string            response_body=self.getResponse(handler.body)            # create a response XML message            response_message="""              <message to="%s" from="%s">                <body>%s</body>              </message>""" % (handler.sender, handler.recipient, response_body)            # return it to the server            return response_message

The receiveMessage External Method uses Python's SAX (Simple API for XML) package to parse the XML message. The MessageHandler class receives callbacks as Python parses the message. The handler saves information its interested in. The External Method uses the handler class by creating an instance of it, and passing it to the parseString function. It then figures out a response message by calling getResponse with the message body. The getResponse script (which is not shown here) presumably scans the body for commands, queries the web applications state and returns some response. The receiveMessage method then creates an XML message using response and the sender information and returns it.

The remote server would use this External Method by calling the receiveMessage method using the standard HTTP POST command. Voila, you've implemented a custom XML chat server that runs over HTTP.

External Method Gotchas

While you are essentially unrestricted in what you can do in an External Method, there are still some things that are hard to do.

While your Python code can do as it pleases if you want to work with the Zope framework you need to respect its rules. While programming with the Zope framework is too advanced a topic to cover here, there are a few things that you should be aware of.

Problems can occur if you hand instances of your own classes to Zope and expect them to work like Zope objects. For example, you cannot define a class in your External Method and assign an instance of this class as an attribute of a Zope object. This causes problems with Zope's persistence machinery. You also cannot easily hand instances of your own classes over to DTML or scripts. The issue here is that your instances won't have Zope security information. You can define and use your own classes and instances to your heart's delight, just don't expect Zope to use them directly. Limit yourself to returning simple Python structures like strings, dictionaries and lists or Zope objects.

If you need to create new kinds of persistent objects, it's time to learn about writing Zope Products. Writing a Product is beyond the scope of this book. You can learn more by reading the Zope Developer's Guide

Using Perl-based Scripts

Perl-based Scripts allow you to script Zope in Perl. If you love Perl and don't want to learn Python to use Zope, these scripts are for you. Using Perl-based Scripts you can use all your favorite Perl modules and treat Zope like a collection of Perl objects.

The Perl Language

Perl is a high-level scripting language like Python. From a broad perspective, Perl and Python are very similar languages, they have similar primitive data constructs and employ similar programming constructs.

Perl is a popular language for Internet scripting. In the early days of CGI scripting, Perl and CGI were practically synonymous. Perl continues to be the dominant Internet scripting language.

Perl has a very rich collection of modules for tackling almost any computing task. CPAN (Comprehensive Perl Archive Network) is the authoritative guide to Perl resources.

Zope does not support Perl-based scripts in the default Zope installation. Perl-based scripts require you to have Perl installed, and a few other packages, and how to install these things is beyond the scope of this book. See the documentation that comes with Perl-based scripts from the above URL. There is also more information provided by Andy McKay available on Zope.org.

Creating Perl-based Scripts

Perl-based Scripts are quite similar to Python-based Scripts. Both have access to Zope objects and are called in similar ways. Here's the Perl hello world program:

        my $name=shift;        return "Hello $name.";

Let's take a look at a more complex example script by Monty Taylor. It uses the LWP::UserAgent package to retrieve the URL of the daily Dilbert comic from the network. Create a Perl-based Script named get_dilbert_url with this code:

        use LWP::UserAgent;        my $ua = LWP::UserAgent->new;        # retrieve the Dilbert page        my $request = HTTP::Request->new('GET','http://www.dilbert.com');        my $response = $ua->request($request);        # look for the image URL in the HTML        my $content = $response->content;        $content =~ m,(/comics/dilbert/archive/images/[^"]*),s;        # return the URL        return $content        

You can display the daily Dilbert comic by calling this script from DTML by calling the script inside an HTML IMG tag:

        <img src="&dtml-get_dilbert_url;">

However there is a problem with this code. Each time you display the cartoon, Zope has to make a network connection. This is inefficient and wasteful. You'd do much better to only figure out the Dilbert URL once a day.

Here's a script cached_dilbert_url that improves the situation by keeping track of when it last fetched the Dilbert URL with a dilbert_url_date property:

        my $context=shift;        my $date=$context->getProperty('dilbert_url_date');        if ($date==null or $now-$date > 1){            my $url=$context->get_dilbert_url();            $context->manage_changeProperties(              dilbert_url => $url              dilbert_url_time => $now            );        }        return $context->getProperty('dilbert_url');

This script uses two properties, dilbert_url and dilbert_url_date. If the URL gets too old, a new one is fetched. You can use this script from DTML just like the original script:

        <img src="&dtml-cached_dilbert_url;">

You can use Perl and DTML together to control your logic and your presentation.

Perl-based Script Security

Like DTML and Python-based Scripts, Perl-based Scripts constrain you in the Zope security system from doing anything that you are not allowed to do. Script security is similar in both languages, but there are some Perl specific constraints.

First, the security system does not allow you to eval an expression in Perl. For example, consider this script:

        my $context = shift;        my $input = shift;        eval $input

This code takes an argument and evaluates it in Perl. This means you could call this script from, say an HTML form, and evaluate the contents of one of the form elements. This is not allowed since the form element could contain malicious code.

Perl-based Scripts also cannot assign new variables to any object other than local variables that you declare with my.

Advanced Acquisition

https://engineering.purdue.edu/ECN/Support/KB/Docs/ZopeBook/Acquisition.whtml https://engineering.purdue.edu/ECN/Support/KB/Docs/ZopeBook/Acquisition.html

In the chapter entitled Acquisition , we introduced acquisition by containment, which we have been using throughout this chapter. In acquisition by containment, Zope looks for an object by going back up the containment hierarchy until it finds an object with the right id. In Chapter 7 we also mentioned context acquisition, and warned that it is a tricky subject capable of causing your brain to explode. If you are ready for exploding brains, read on.

Recall our Zoo example introduced earlier in this chapter.

Figure 14-7 Zope Zoo Example hierarchy

We have seen how Zope uses URL traversal and acquisition to find objects in higher containers. More complex arrangements are possible. Suppose you want to call the vaccinate script on the hippo object. What URL can you use? If you visit the URL Zoo/LargeAnimals/hippo/vaccinate Zope will not be able to find the vaccinate script since it isn't in any of the hippo object's containers.

The solution is to give the path to the script as part of the URL. Zope allows you to combine two or more URLs into one in order to provide more acquisition context! By using acquisition, Zope will find the script as it backtracks along the URL. The URL to vaccinate the hippo is Zoo/Vet/LargeAnimals/hippo/vaccinate. Likewise, if you want to call the vaccinate script on the kangarooMouse object you should use the URL Zoo/Vet/SmallAnimals/kangarooMouse/vaccinate.

Let's follow along as Zope traverses the URL Zoo/Vet/LargeAnimals/hippo/vaccinate. Zope starts in the root folder and looks for an object named Zoo. It moves to the Zoo folder and looks for an object named Vet. It moves to the Vet folder and looks for an object named LargeAnimals. The Vet folder does not contain an object with that name, but it can acquire the LargeAnimals folder from its container, Zoo folder. So it moves to the LargeAnimals folder and looks for an object named hippo. It then moves to the hippo object and looks for an object named vaccinate. Since the hippo object does not contain a vaccinate object and neither do any of its containers, Zope backtracks along the URL path trying to find a vaccinate object. First it backs up to the LargeAnimals folder where vaccinate still cannot be found. Then it backs up to the Vet folder. Here it finds a vaccinate script in the Vet folder. Since Zope has now come to the end of the URL, it calls the vaccinate script in the context of the hippo object.

Note that we could also have organized the URL a bit differently. Zoo/LargeAnimals/Vet/hippo/vaccinate would also work. The difference is the order in which the context elements are searched. In this example, we only need to get vaccinate from Vet, so all that matters is that Vet appears in the URL after Zoo and before hippo.

When Zope looks for a sub-object during URL traversal, it first looks for the sub-object in the current object. If it cannot find it in the current object it looks in the current object's containers. If it still cannot find the sub-object, it backs up along the URL path and searches again. It continues this process until it either finds the object or raises an error if it cannot be found. If several context folders are used in the URL, they will be searched in order from left to right.

Context acquisition can be a very useful mechanism, and it allows you to be quite expressive when you compose URLs. The path you tell Zope to take on its way to an object will determine how it uses acquisition to look up the object's scripts.

Note that not all scripts will behave differently depending on the traversed URL. For example, you might want your script to acquire names only from its parent containers and not from the URL context. To do so, simply use the container variable instead of the context variable in the script, as described above in the section "Using Python-based Scripts."

Context Acquisition Gotchas

Containment before context

It is important to realize that context acquisition supplements container acquisition. It does not override container acquisition.

One at a time

Another point that often confuses new users is that each element of a path "sticks" for the duration of the traversal, once it is found. Think of it this way: objects are looked up one at a time, and once an object is found, it will not be looked up again. For example, imagine this folder structure:

Figure 14-8 Acquisition example folder structure

Now suppose that the about_penguins page contains a link to Images/penguins.png. Shouldn't this work? Won't /Images/penguins.png succeed when /Content/Images/penguins.png fails? The answer is no. We always traverse from left to right, one item at a time. First we find Content, then Images within it; penguins.png appears in neither of those, and we have searched all parent containers of every element in the URL, so there is nothing more to search in this URL. Zope stops there and raises an error. Zope never looks in /Images because it has already found /Content/Images.

Readability

Context acquisition can make code more difficult to understand. A person reading your script can no longer simply look backwards up one containment hierarchy to see where an acquired object might be; many more places might be searched, all over the zope tree folder. And the order in which objects are searched, though it is consistent, can be confusing.

Fragility

Over-use of context acquisition can also lead to fragility. In object-oriented terms, context acquisition can lead to a site with low cohesion and tight coupling. This is generally regarded as a bad thing. More specifically, there are many simple actions by which an unwitting developer could break scripts that rely on context acquisition. These are more likely to occur than with container acquisition, because potentially every part of your site affects every other part, even in parallel folder branches.

For example, if you write a script that calls another script by a long and torturous path, you are assuming that the folder tree is not going to change. A maintenance decision to reorganize the folder hierarchy could require an audit of scripts in every part of the site to determine whether the reorganization will break anything.

Recall our Zoo example. There are several ways in which a zope maintainer could break the feed() script:

Inserting another object with the name of the method

This is a normal technique for customizing behavior in Zope, but context acquisition makes it more likely to happen by accident. Suppose that giraffe vaccination is controlled by a regularly scheduled script that calls Zoo/Vet/LargeAnimals/giraffe/feed. Suppose a content administrator doesn't know about this script and adds a DTML page called vaccinate in the giraffe folder, containing information about vaccinating giraffes. This new vaccinate object will be acquired before Zoo/Vet/vaccinate. Hopefully you will notice the problem before your giraffes get sick.

Calling an inappropriate path

if you visit Zoo/LargeAnimals/hippo/buildings/visitor_reception/feed, will the reception area be filled with hippo food? One would hope not. This might even be possible for someone who has no permissions on the reception object. Such URLs are actually not difficult to construct. For example, using relative URLs in standard_html_header can lead to some quite long combinations of paths.

Thanks to Toby Dickenson for pointing out these fragility issues on the zope-dev mailing list.

Calling DTML from Scripts

Often, you would want to call a DTML Method or DTML Document from a Script. For instance, a common pattern is to call a Script from an HTML form. The Script would process user input, and return an output page with feedback messages - telling the user her request executed correctly or signalling an error as appropriate.

Scripts are good at logic and general computational tasks, but ill suited for generating HTML. Therefore it makes sense to delegate the user feedback output to a DTML Method and call it from the Script.

Assume we have got an DTML Method a_dtml_method. We would call it from Script with:

      # grab the method and the REQUEST from the context      dtml_method = context.a_dtml_method      REQUEST = context.REQUEST      # call the dtml method, for parameters see below      s = dtml_method(client=context, REQUEST=REQUEST, foo='bar')      # s now holds the rendered html      return s      

Note that DTML Methods and Documents take optional client and REQUEST parameters. If a client is passed to a DTML Method, the method tries to resolve names by looking them up as attributes of the client object. By passing our context as a client, the method will look up names in that context. Also, we can pass it a REQUEST object and additional keyword arguments. The DTML Method will first try to look up variables in the keyword arguments, then the namespace, and finally in the REQUEST object. See the chapter "Advanced DTML", subchapter "DTML Namespaces," for details on namespaces, and Appendix B, API Reference for further information on DTML Methods / Documents.

Calling ZPT from Scripts

For the same reasons as outlined in the section "Calling DTML from Scripts" above, one might want to call Page Templates from Scripts. Assume we have this Page Template:

      Hello <span tal:replace="options/name | default">      World      </span>

Calling it from a script could be done with the following Script fragment:

      pt = context.hello_world_pt      s = pt(name="John Doe")      return s

The name parameter to the Page Template ends up in the options/name path expression. Of course, you can pass more than simple values to Page Templates this way. For instance, suppose we wanted to construct a list of objects and pass that to a Page Template for display. The list of objects could be constructed in an External Method. Place a file my_extensions.py in the Extensions directory, and for example add:

      class Giraffe:          __allow_access_to_unprotected_subobjects__ = 1          def __init__(self, name, neck_length=None):              self.name = name              self.n_length=neck_length          def neck_length(self):              n = self.n_length              if not n: return "unspecified"              if type(n) == type(0.0):                   return "%s meters" % n              return n      def giraffes(self):          # make a list of giraffes          glist = []          for name, neck in (('Guido', 1.2), ('Jim', 'long'), ('Barry', None)):              g = Giraffe(name, neck_length=neck)              glist.append(g)          # display the lot of them          pt = self.display_giraffes          return pt(giraffes=glist)

Add an External Method giraffes, module my_extensions, function giraffes. Also, add a Page Template display_giraffes containing the following snippet:

      <table border="1">        <tr>          <th>Name</th>          <th>Neck length</th>        </tr>        <tr tal:repeat="giraffe options/giraffes">          <td tal:content="giraffe/name">name</td>          <td tal:content="giraffe/neck_length">neck_length</td>        </tr>      </table>

If you go to Test tab of the giraffes External Method, you should see a table similar to the one in the figure below.

Figure 14-9 Giraffe table

In the my_extensions module (file my_extensions.py) we define a class Giraffes. The '__allow_access_to_unprotected_subobjects__ = 1' statement tells Zope that it is okay to grant access on giraffes to everyone - use only when you actually want this!

Then, in the giraffes function, we create some giraffe objects using hardcoded values, and hand the list of giraffes to a Page Template, which iterates through the list to display their data in a table.

Again, we use the options Page Template variable to access the list members, and access data and method attributes of the giraffe objects using path expressions like giraffe/name. The neck_length method gets called automatically in this process. You would want to use python expression syntax instead if you need to pass parameters to the giraffe objects methods, eg. use something like python:giraffe.neck_length(). See the chapters on "Zope Page Templates" and "Advanced Page Templates" for more details on path and python expressions.

Passing Parameters to Scripts

All scripts can be passed parameters. A parameter gives a script more information about what to do. When you call a script from the web, Zope will try to find the script's parameters in the web request and pass them to your script. For example, if you have a script with parameters dolphin and REQUEST Zope will look for dolphin in the web request, and will pass the request itself as the REQUEST parameter. In practical terms this means that it is easy to do form processing in your script. For example, here is a form:

       <form action="form_action">       Name of Hippo <input type="text" name="name"><br>       Age of Hippo <input type="text" name="age"><br>       <input type="submit">       </form>

You can easily process this form with a script named form_action that includes name and age in its parameter list:

       ## Script (Python) "form_action"       ##parameters=name, age       ##       "Process form"       age=int(age)       message= 'This hippo is called %s and is %d years old' % (name, age)       if age < 18:           message += '\n %s is not old enough to drive!' % name       return message

There is no need to process the form manually to extract values from it. Form elements are passed as strings, or lists of strings in the case of checkboxes and multiple-select input.

In addition to form variables, you can specify any request variables as script parameters. For example, to get access to the request and response objects just include REQUEST and RESPONSE in your list of parameters. Request variables are detailed more fully in Appendix B .

In the Python script given above, there is a subtle problem. You are probably expecting an integer rather than a string for age, but all form variables are passed as strings. Perl takes care of such things automagically, but Python does not. You could manually convert the string to an integer using the Python int built-in:

       age=int(age)

But this manual conversion may be inconvenient. Zope provides a way for you to specify form input types in the form, rather than in the processing script. Instead of converting the age variable to an integer in the processing script, you can indicate that it is an integer in the form itself:

       Age <input type="text" name="age:int">

The :int appended to the form input name tells Zope to automatically convert the form input to an integer. This process is called marshalling. If the user of your form types something that cannot be converted to an integer (such as "22 going on 23") then Zope will raise an exception as shown in the figure below.

Figure 14-10 Parameter conversion error

It's handy to have Zope catch conversion errors, but you may not like Zope's error messages. You should avoid using Zope's converters if you want to provide your own error messages.

Zope can perform many parameter conversions. Here is a list of Zope's basic parameter converters.

boolean

Converts a variable to true or false. Variables that are 0, None, an empty string, or an empty sequence are false, all others are true.

int

Converts a variable to an integer.

long

Converts a variable to a long integer.

float

Converts a variable to a floating point number.

string

Converts a variable to a string. Most variables are strings already so this converter is seldom used.

text

Converts a variable to a string with normalized line breaks. Different browsers on various platforms encode line endings differently, so this script makes sure the line endings are consistent, regardless of how they were encoded by the browser.

list

Converts a variable to a Python list.

tuple

Converts a variable to a Python tuple. A tuple is like a list, but cannot be modified.

tokens

Converts a string to a list by breaking it on white spaces.

lines

Converts a string to a list by breaking it on new lines.

date

Converts a string to a DateTime object. The formats accepted are fairly flexible, for example 10/16/2000, 12:01:13 pm.

required

Raises an exception if the variable is not present.

ignore_empty

Excludes the variable from the request if the variable is an empty string.

These converters all work in more or less the same way to coerce a form variable, which is a string, into another specific type. You may recognize these converters from the chapter entitled Using Basic Zope Objects , in which we discussed properties. These converters are used by Zope's property facility to convert properties to the right type.

The list and tuple converters can be used in combination with other converters. This allows you to apply additional converters to each element of the list or tuple. Consider this form:

       <form action="processTimes">        <p>I would prefer not to be disturbed at the following       times:</p>       <input type="checkbox" name="disturb_times:list:date"       value="12:00 AM"> Midnight<br>       <input type="checkbox" name="disturb_times:list:date"       value="01:00 AM"> 1:00 AM<br>       <input type="checkbox" name="disturb_times:list:date"       value="02:00 AM"> 2:00 AM<br>       <input type="checkbox" name="disturb_times:list:date"       value="03:00 AM"> 3:00 AM<br>       <input type="checkbox" name="disturb_times:list:date"       value="04:00 AM"> 4:00 AM<br>       <input type="submit">       </form>

By using the list and date converters together, Zope will convert each selected time to a date and then combine all selected dates into a list named disturb_times.

A more complex type of form conversion is to convert a series of inputs into records. Records are structures that have attributes. Using records, you can combine a number of form inputs into one variable with attributes. The available record converters are:

record

Converts a variable to a record attribute.

records

Converts a variable to a record attribute in a list of records.

default

Provides a default value for a record attribute if the variable is empty.

ignore_empty

Skips a record attribute if the variable is empty.

Here are some examples of how these converters are used:

       <form action="processPerson">       First Name <input type="text" name="person.fname:record"><br>       Last Name <input type="text" name="person.lname:record"><br>       Age <input type="text" name="person.age:record:int"><br>       <input type="submit">       </form>

This form will call the processPerson script with one parameter, person. The person variable will have the attributes fname, lname and age. Here's an example of how you might use the person variable in your processPerson script:

       ## Script (Python) "processPerson"       ##parameters=person       ##       " process a person record "       full_name="%s %s" % (person.fname, person.lname)       if person.age < 21:           return "Sorry, %s. You are not old enough to adopt an aardvark." % full_name       return "Thanks, %s. Your aardvark is on its way." % full_name

The records converter works like the record converter except that it produces a list of records, rather than just one. Here is an example form:

       <form action="processPeople">       <p>Please, enter information about one or more of your next of       kin.</p>       <p>First Name <input type="text" name="people.fname:records">       Last Name <input type="text" name="people.lname:records"></p>       <p>First Name <input type="text" name="people.fname:records">       Last Name <input type="text" name="people.lname:records"></p>       <p>First Name <input type="text" name="people.fname:records">       Last Name <input type="text" name="people.lname:records"></p>       <input type="submit">       </form>    

This form will call the processPeople script with a variable called people that is a list of records. Each record will have fname and lname attributes. Note the difference between the records converter and the list:record converter: the former would create a list of records, whereas the latter would produce a single record whose attributes fname and lname would each be a list of values.

The order of combined modifiers does not matter; for example, int:list is identical to list:int.

Another useful parameter conversion uses form variables to rewrite the action of the form. This allows you to submit a form to different scripts depending on how the form is filled out. This is most useful in the case of a form with multiple submit buttons. Zope's action converters are:

action

Appends the attribute value to the original form action of the form. This is mostly useful for the case in which you have multiple submit buttons on one form. Each button can be assigned to a script that gets called when that button is clicked to submit the form. A synonym for action is method.

default_action

Appends the attribute value to the original action of the form when no other action converter is used.

Here's an example form that uses action converters:

       <form action="employeeHandlers">       <p>Select one or more employees</p>       <input type="checkbox" name="employees:list" value="Larry"> Larry<br>       <input type="checkbox" name="employees:list" value="Simon"> Simon<br>       <input type="checkbox" name="employees:list" value="Rene"> Rene<br>       <input type="submit" name="fireEmployees:action"       value="Fire!"><br>       <input type="submit" name="promoteEmployees:action"       value="Promote!">       </form>

This form will call either the fireEmployees or the promoteEmployees script in the employeeHandlers folder, depending on which of the two submit buttons is used. Notice also how it builds a list of employees with the list converter. Form converters can be very useful when designing Zope applications.

Returning Values from Scripts

Scripts have their own variable scope. In this respect, scripts in Zope behave just like functions, procedures, or methods in most programming languages. If you call a script updateInfo, for example, and updateInfo assigns a value to a variable status, then status is local to your script -- it gets cleared once the script returns. To get at the value of a script variable, we must pass it back to the caller with a return statement.

Here is an example of how one might call a script from DTML and use a value returned from the script:

      <dtml-let status="updateInfo(color='brown', pattern='spotted')">        <dtml-if expr="status == 0">          Data updated successfully        <dtml-else>          An error occurred! The error status was: <dtml-var status>        </dtml-if>      </dtml-let>

Scripts can only return a single object. If you need to return more than one value, put them in a dictionary and pass that back.

Suppose you have a Python script compute_diets out of which you want to get values:

      ## Script (Python) "compute_diets"      d = {'fat': 10,           'protein': 20,           'carbohydrate': 40,          }      return d

The values would, of course, be calculated in a real application; in this simple example we will just hardcode some numbers.

You could call this script from DTML like this:

      <dtml-with compute_diets mapping>         This animal needs         <dtml-var fat>kg fat,         <dtml-var protein>kg protein, and         <dtml-var carbohydrate>kg carbohydrates.      </dtml-with>

Note the mapping attribute to the dtml-with tag - it tells DTML to expect a mapping (a dictionary in our case) instead of an object.

Script Security

All scripts that can be edited through the web are subject to Zope's standard security policies. The only scripts that are not subject to these security restrictions are scripts that must be edited through the filesystem. These unrestricted scripts include Python and Perl External Methods.

The chapter entitled Users and Security covers security in more detail. You should consult the Roles of Executable Objects and Proxy Roles sections for more information on how scripts are restricted by Zope security constraints.

Security Restrictions of Script (Python)

Scripts are restricted in order to limit their ability to do harm. What could be harmful? In general, scripts keep you from accessing private Zope objects, making harmful changes to Zope objects, hurting the Zope process itself, and accessing the server Zope is running on. These restrictions are implemented through a collection of limits on what your scripts can do.

Loop limits

Scripts cannot create infinite loops. If your script loops a very large number of times Zope will raise an error. This restriction covers all kinds of loops including for and while loops. The reason for this restriction is to limit your ability to hang Zope by creating an infinite loop.

Import limits

Scripts cannot import arbitrary packages and modules. You are limited to importing the Products.PythonScripts.standard utility module, the AccessControl module, those modules available via DTML (string, random, math, sequence), and modules which have been specifically made available to scripts by product authors. See Appendix B, API Reference for more information on these modules. If you want to be able to import any Python module, use an External Method, as described later in the chapter.

Access limits

You are restricted by standard Zope security policies when accessing objects. In other words the user executing the script is checked for authorization when accessing objects. As with all executable objects, you can modify the effective roles a user has when calling a script using Proxy Roles (see the chapter entitled Users and Security for more information). In addition, you cannot access objects whose names begin with an underscore, since Zope considers these objects to be private. Finally, you can define classes in scripts but it is not really practical to do so, because you are not allowed to access attributes of these classes! Even if you were allowed to do so, the restriction against using objects whose names begin with an underscore would prevent you from using your class's __init__ method. If you need to define classes, use External Methods (see below) or Zope Products (see the Zope Developer's Guide for more information about creating Products). You may, however, define functions in scripts, although it is rarely useful or necessary to do so. In practice, a Script in Zope is treated as if it were a single method of the object you wish to call it on.

Writing limits

In general you cannot directly change Zope object attributes using scripts. You should call the appropriate methods from the Zope API instead.

Despite these limits, a determined user could use large amounts of CPU time and memory using Python-based Scripts. So malicious scripts could constitute a kind of denial of service attack by using lots of resources. These are difficult problems to solve and DTML suffers from the same potential for abuse. As with DTML, you probably should not grant access to scripts to untrusted people.

The Zope API

One of the main reasons to script Zope is to get convenient access to the Zope API (Application Programmer Interface). The Zope API describes built-in actions that can be called on Zope objects. You can examine the Zope API in the help system, as shown in the figure below.

Figure 14-11 Zope API Documentation

Suppose you would like to have a script that takes a file you upload from a form and creates a Zope File object in a folder. To do this you need to know a number of Zope API actions. It's easy enough to read files in Python or Perl, but once you have the file you need to know what actions to call to create a new File object in a Folder.

There are many other things that you might like to script using the Zope API. Any management task that you can perform through the web can be scripted using the Zope API. This includes creating, modifying and deleting Zope objects. You can even perform maintenance tasks, like restarting Zope and packing the Zope database.

The Zope API is documented in Appendix B, API Reference as well as in the Zope online help. The API documentation shows you which classes inherit from which other classes. For example, Folder inherits from ObjectManager. This means that Folder objects have all the actions listed in the ObjectManager section of the API reference.

To get you started, and whet your appetite, we will go through some example Python scripts that demonstrate how you can use the Zope API.

Get all objects in a folder

The objectValues() method returns a list of objects contained in a folder. If the context happens not to be a folder, nothing is returned:

        objs = context.objectValues()

Get the id of an object

The id is the "handle" to access an object, and is set at object creation:

         id = context.getId()

Note that there is no setId() method - you have to either use the ZMI to rename them, set their id attribute via security-unrestricted code, or use the manage_renameObject or manage_renameObjects API methods exposed upon the container of the object you want to rename.

Get the Zope root folder

The root folder is the top level element in the Zope object database:

         root = context.getPhysicalRoot()

Get the physical path to an object

The getPhysicalPath() method returns a list contained the ids of the object's containment hierarchy:

         path_list = context.getPhysicalPath()         path_string = "/".join(path_list)

Get an object by path

restrictedTraverse() is the complement to getPhysicalPath(). The path can be absolute - starting at the Zope root - or relative to the context:

         path = "/Zoo/LargeAnimals/hippo"         hippo_obj = context.restrictedTraverse(path)

Change the content of an DTML Method or Document

You can actually change the content (and title) of a DTML Method or Document, exactly as if you edited it in the Zope management interface, by using its manage_edit() method:

         # context has to be a DTML method or document!         context.manage_edit('new content', 'new title')

Change properties of an object

You can use the manage_changeProperties method of any Zope object to change its properties:

         # context may be any kind of Zope object         context.manage_changeProperties({'title':'Another title'})

Get a property

getProperty() returns a property of an object. Many objects support properties (those that are derived from the PropertyManager class), the most notable exception being DTML Methods, which do not:

         pattern = context.getProperty('pattern')         return pattern

Change properties of an object

The object has to support properties and the property must exist:

         values = {'pattern' : 'spotted'}         context.manage_changeProperties(values)

Execute a DTML Method or DTML Document

This executes a DTML Method or Document and returns the result. Note that DTML Methods and Documents take optional client and REQUEST parameters. If a client is passed to a DTML Method, the method tries to resolve names by looking them up as attributes of the client object. By passing our context as a client, the method will look up names in that context. Also, we can pass it a REQUEST object and additional keyword arguments - the DTML Method will first try to resolve names in them. See the chapter entitled Advanced DTML, section "DTML Namespaces," for details:

         dtml_method = context.a_dtml_method         s = dtml_method(client=context, REQUEST={}, foo='bar')         return s      

Traverse to an object and add a new property

We get an object by its absolute path and add a property weight, and set it to some value. Again, the object must support properties for this to work. We introduce another neat trick in this example. Long method names can make lines in your scripts long and hard to read, so you can assign a shorter name for the method before using it:

         path = "/Zoo/LargeAnimals/hippo"         hippo_obj = context.restrictedTraverse(path)         add_method = hippo_obj.manage_addProperty         add_method('weight', 500, 'int')

Add a new object to the context

Scripts can add objects to folders, just like you do in the Zope management interface. The context has to be an object (i.e. a folder or another object derived from ObjectManager). The general pattern is:

         context.manage_addProduct['PackageName'].manage_addProductName(id)

manage_addProduct is a mapping in which we can look up a dispatcher - an object which gives us the necessary factory for creating a new object in the context. For most of the built-in Zope classes, the PackageName is OFSP, and the factory method is named after the product class itself. Once you have the factory method, you must pass it whatever arguments are needed for adding an object of this type. Some examples will make this clear.

Let's add a DTML Method to a folder:

         add_method = context.manage_addProduct['OFSP'].manage_addDTMLMethod         add_method('object_id', file="Initial contents of the DTML Method")

For any other product class that comes with Zope, we need only change the factory method and its arguments.

DTML Methods

manage_addDTMLMethod

DTML Documents

manage_addDTMLDocument

Images

manage_addImage

Files

manage_addFile

Folders

manage_addFolder

UserFolders

manage_addUserFolder

Version

manage_addVersion

To get a dispatcher for add-on Products which you download and install, replace OFSP with the directory which contains the product code. For example, if you have installed the famous Boring product, you could add one like so:

         add_method = context.manage_addProduct['Boring'].manage_addBoring         add_method(id='test_boring')

If the product author has been conscientious, the process for adding new instances of their product will be documented; but it will always look something like the above examples.

DTML versus Python versus Perl versus Page Templates

Zope gives you many ways to script. For small scripting tasks the choice of Python, Perl or DTML probably doesn't make a big difference. For larger, logic-oriented tasks you should use Python or Perl. You should choose the language you are most comfortable with. Of course, your boss may want to have some say in the matter too.

For presentation, Perl and Python should not be used; the choice then becomes whether to use DTML or ZPT.

Just for the sake of comparison, here is a simple presentational script suggested by Gisle Aas, the author of Perl-based Scripts, in four different languages.

In DTML:

      <dtml-in objectValues>        <dtml-var getId>: <dtml-var sequence-item>      </dtml-in>      done

In ZPT:

      <div tal:repeat="item here/objectValues"            tal:replace="python:'%s: %s\n' % (item.getId(), str(item))" />

In Python:

      for item in context.objectValues():          print "%s: %s" % (item.getId(), item)      print "done"      return printed

In Perl:

      my $context = shift;      my @result;      for ($context->objectValues()) {          push(@result, join(": ", $_->getId(), $_));      }      join("\n", @result, "done");

Despite the fact that Zope is implemented in Python, it sometimes (for better or worse) follows the Perl philosophy that "there's more than one way to do it".

Remote Scripting and Network Services

Web servers are used to serve content to software clients; usually people using web browser software. The software client can also be another computer that is using your web server to access some kind of service.

Because Zope exposes objects and scripts on the web, it can be used to provide a powerful, well organized, secure web API to other remote network application clients.

There are two common ways to remotely script Zope. The first way is using a simple remote procedure call protocol called XML-RPC. XML-RPC is used to execute a procedure on a remote machine and get a result on the local machine. XML-RPC is designed to be language neutral, and in this chapter you'll see examples in Python, Perl and Java.

The second common way to remotely script Zope is with any HTTP client that can be automated with a script. Many language libraries come with simple scriptable HTTP clients and there are many programs that let you you script HTTP from the command line.

Using XML-RPC

XML-RPC is a simple remote procedure call mechanism that works over HTTP and uses XML to encode information. XML-RPC clients have been implemented for many languages including Python, Perl, Java, JavaScript, and TCL.

In-depth information on XML-RPC can be found at the XML-RPC website.

All Zope scripts that can be called from URLs can be called via XML-RPC. Basically XML-RPC provides a system to marshal arguments to scripts that can be called from the web. As you saw earlier in the chapter Zope provides its own marshaling controls that you can use from HTTP. XML-RPC and Zope's own marshaling accomplish much the same thing. The advantage of XML-RPC marshaling is that it is a reasonably supported standard that also supports marshaling of return values as well as argument values.

Here's a fanciful example that shows you how to remotely script a mass firing of janitors using XML-RPC.

Here's the code in Python:

        import xmlrpclib        server = xmlrpclib.Server('http://www.zopezoo.org/')        for employee in server.JanitorialDepartment.personnel():            server.fireEmployee(employee)

In Perl:

        use Frontier::Client;        $server = Frontier::Client->new(url => "http://www.zopezoo.org/");        $employees = $server->call("JanitorialDepartment.personnel");        foreach $employee ( @$employees ) {          $server->call("fireEmployee",$server->string($employee));        }

In Java:

        try {            XmlRpcClient server = new XmlRpcClient("http://www.zopezoo.org/");            Vector employees = (Vector) server.execute("JanitorialDepartment.personnel");            int num = employees.size();            for (int i = 0; i < num; i++) {                Vector args = new Vector(employees.subList(i, i+1));                server.execute("fireEmployee", args);            }        } catch (XmlRpcException ex) {            ex.printStackTrace();        } catch (IOException ioex) {            ioex.printStackTrace();        }

Actually the above example will probably not run correctly, since you will most likely want to protect the fireEmployee script. This brings up the issue of security with XML-RPC. XML-RPC does not have any security provisions of its own; however, since it runs over HTTP it can leverage existing HTTP security controls. In fact Zope treats an XML-RPC request exactly like a normal HTTP request with respect to security controls. This means that you must provide authentication in your XML-RPC request for Zope to grant you access to protected scripts. The Python client at the time of this writing does not support control of HTTP Authorization headers. However it is a fairly trivial addition. For example, an article on XML.com Internet Scripting: Zope and XML-RPC includes a patch to Python's XML-RPC support showing how to add HTTP authorization headers to your XML-RPC client.

Remote Scripting with HTTP

Any HTTP client can be used for remotely scripting Zope.

On Unix systems you have a number of tools at your disposal for remotely scripting Zope. One simple example is to use wget to call Zope script URLs and use cron to schedule the script calls. For example, suppose you have a Zope script that feeds the lions and you'd like to call it every morning. You can use wget to call the script like so:

        $ wget --spider http://www.zopezope.org/Lions/feed

The spider option tells wget not to save the response as a file. Suppose that your script is protected and requires authorization. You can pass your user name and password with wget to access protected scripts:

        $ wget --spider --http-user=ZooKeeper --http-passwd=SecretPhrase http://www.zopezope.org/Lions/feed

Now let's use cron to call this command every morning at 8am. Edit your crontab file with the crontab command:

        $ crontab -e

Then add a line to call wget every day at 8 am:

        0 8 * * * wget -nv --spider --http_user=ZooKeeper --http_pass=SecretPhrase http://www.zopezoo.org/Lions/feed

The only difference between using cron and calling wget manually is that you should use the nv switch when using cron since you don't care about output of the wget command.

For our final example let's get really perverse. Since networking is built into so many different systems, it's easy to find an unlikely candidate to script Zope. If you had an Internet-enabled toaster you would probably be able to script Zope with it. Let's take Microsoft Word as our example Zope client. All that's necessary is to get Word to agree to tickle a URL.

The easiest way to script Zope with Word is to tell word to open a document and then type a Zope script URL as the file name as shown in Figure 14-12.

Figure 14-12 Calling a URL with Microsoft Word

Word will then load the URL and return the results of calling the Zope script. Despite the fact that Word doesn't let you POST arguments this way, you can pass GET arguments by entering them as part of the URL.

You can even control this behavior using Word's built-in Visual Basic scripting. For example, here's a fragment of Visual Basic that tells Word to open a new document using a Zope script URL:

        Documents.Open FileName:="http://www.zopezoo.org/LionCages/wash?use_soap=1&water_temp=hot" 

You could use Visual Basic to call Zope script URLs in many different ways.

Zope's URL to script call translation is the key to remote scripting. Since you can control Zope so easily with simple URLs you can easy script Zope with almost any network-aware system.

Conclusion

Zope provides scripting with Python, and optionally with Perl. With scripts you can control Zope objects and glue together your application's logic, data, and presentation. You can programmatically manage objects in your Zope folder hierarchy by using the Zope API. You can also perform serious programming tasks such as image processing and XML parsing.