WarningWarningWarning

This document is meant to describe the decorator syntax and the process that resulted in the decisions that were made. It does not attempt to cover the huge number of potential alternative syntaxes, nor is it an attempt to exhaustively list all the positives and negatives of each form.

Abstract

The current method for transforming functions and methods (for instance, declaring them as a class or static method) is awkward and can lead to code that is difficult to understand. Ideally, these transformations should be made at the same point in the code where the declaration itself is made. This PEP introduces new syntax for transformations of a function or method declaration.

Motivation

The current method of applying a transformation to a function or method places the actual transformation after the function body. For large functions this separates a key component of the function’s behavior from the definition of the rest of the function’s external interface. For example:

def foo(self):
    perform method operation
foo = classmethod(foo)

This becomes less readable with longer methods. It also seems less than pythonic to name the function three times for what is conceptually a single declaration. A solution to this problem is to move the transformation of the method closer to the method’s own declaration. The intent of the new syntax is to replace

def foo(cls):
    pass
foo = synchronized(lock)(foo)
foo = classmethod(foo)

with an alternative that places the decoration in the function’s declaration:

@classmethod
@synchronized(lock)
def foo(cls):
    pass

Modifying classes in this fashion is also possible, though the benefits are not as immediately apparent. Almost certainly, anything which could be done with class decorators could be done using metaclasses, but using metaclasses is sufficiently obscure that there is some attraction to having an easier way to make simple modifications to classes. For Python 2.4, only function/method decorators are being added.

PEP 3129 proposes to add class decorators as of Python 2.6.

Background

There is general agreement that syntactic support is desirable to the current state of affairs. Guido mentioned syntactic support for decorators in his DevDay keynote presentation at the 10th Python Conference, though he later said it was only one of several extensions he proposed there “semi-jokingly”. Michael Hudson raised the topic on python-dev shortly after the conference, attributing the initial bracketed syntax to an earlier proposal on comp.lang.python by Gareth McCaughan.

Class decorations seem like an obvious next step because class definition and function definition are syntactically similar, however Guido remains unconvinced, and class decorators will almost certainly not be in Python 2.4.

The discussion continued on and off on python-dev from February 2002 through July 2004. Hundreds and hundreds of posts were made, with people proposing many possible syntax variations. Guido took a list of proposals to EuroPython 2004, where a discussion took place. Subsequent to this, he decided that we’d have the Java-style @decorator syntax, and this appeared for the first time in 2.4a2. Barry Warsaw named this the ‘pie-decorator’ syntax, in honor of the Pie-thon Parrot shootout which occurred around the same time as the decorator syntax, and because the @ looks a little like a pie. Guido outlined his case on Python-dev, including this piece on some of the (many) rejected forms.

On the name ‘Decorator’

There’s been a number of complaints about the choice of the name ‘decorator’ for this feature. The major one is that the name is not consistent with its use in the GoF book. The name ‘decorator’ probably owes more to its use in the compiler area – a syntax tree is walked and annotated. It’s quite possible that a better name may turn up.

Design Goals

The new syntax should

• work for arbitrary wrappers, including user-defined callables and the existing builtins classmethod() and staticmethod(). This requirement also means that a decorator syntax must support passing arguments to the wrapper constructor
• work with multiple wrappers per definition
• make it obvious what is happening; at the very least it should be obvious that new users can safely ignore it when writing their own code
• be a syntax “that … [is] easy to remember once explained”
• not make future extensions more difficult
• be easy to type; programs that use it are expected to use it very frequently
• not make it more difficult to scan through code quickly. It should still be easy to search for all definitions, a particular definition, or the arguments that a function accepts
• not needlessly complicate secondary support tools such as language-sensitive editors and other “toy parser tools out there”
• allow future compilers to optimize for decorators. With the hope of a JIT compiler for Python coming into existence at some point this tends to require the syntax for decorators to come before the function definition
• move from the end of the function, where it’s currently hidden, to the front where it is more in your face

Andrew Kuchling has links to a bunch of the discussions about motivations and use cases in his blog. Particularly notable is Jim Huginin’s list of use cases.

Current Syntax

The current syntax for function decorators as implemented in Python 2.4a2 is:

@dec2
@dec1
def func(arg1, arg2, ...):
    pass

This is equivalent to:

def func(arg1, arg2, ...):
    pass
func = dec2(dec1(func))

without the intermediate assignment to the variable func. The decorators are near the function declaration. The @ sign makes it clear that something new is going on here.

The rationale for the order of application (bottom to top) is that it matches the usual order for function-application. In mathematics, composition of functions (g o f)(x) translates to g(f(x)). In Python, @g @f def foo() translates to foo=g(f(foo).

The decorator statement is limited in what it can accept – arbitrary expressions will not work. Guido preferred this because of a gut feeling.

The current syntax also allows decorator declarations to call a function that returns a decorator:

@decomaker(argA, argB, ...)
def func(arg1, arg2, ...):
    pass

This is equivalent to:

func = decomaker(argA, argB, ...)(func)

The rationale for having a function that returns a decorator is that the part after the @ sign can be considered to be an expression (though syntactically restricted to just a function), and whatever that expression returns is called. See declaration arguments.

Syntax Alternatives

There have been a large number of different syntaxes proposed – rather than attempting to work through these individual syntaxes, it’s worthwhile to break the syntax discussion down into a number of areas. Attempting to discuss each possible syntax individually would be an act of madness, and produce a completely unwieldy PEP.

@classmethod
def foo(arg1,arg2):
    pass

@accepts(int,int)
@returns(float)
def bar(low,high):
    pass

def @classmethod foo(arg1,arg2):
    pass

def @accepts(int,int),@returns(float) bar(low,high):
    pass

def foo @classmethod (arg1,arg2):
    pass

def bar @accepts(int,int),@returns(float) (low,high):
    pass

def foo(arg1,arg2) @classmethod:
    pass

def bar(low,high) @accepts(int,int),@returns(float):
    pass

def foo(arg1,arg2):
    @classmethod
    pass

def bar(low,high):
    @accepts(int,int)
    @returns(float)
    pass

decorate:
    classmethod
    def foo(arg1,arg2):
        pass

decorate:
    accepts(int,int)
    returns(float)
    def bar(low,high):
        pass

Current Implementation, History

Guido asked for a volunteer to implement his preferred syntax, and Mark Russell stepped up and posted a patch to SF. This new syntax was available in 2.4a2.

@dec2
@dec1
def func(arg1, arg2, ...):
    pass

This is equivalent to:

def func(arg1, arg2, ...):
    pass
func = dec2(dec1(func))

though without the intermediate creation of a variable named func.

The version implemented in 2.4a2 allowed multiple @decorator clauses on a single line. In 2.4a3, this was tightened up to only allowing one decorator per line.

A previous patch from Michael Hudson which implements the list-after-def syntax is also still kicking around.

After 2.4a2 was released, in response to community reaction, Guido stated that he’d re-examine a community proposal, if the community could come up with a community consensus, a decent proposal, and an implementation. After an amazing number of posts, collecting a vast number of alternatives in the Python wiki, a community consensus emerged (below). Guido subsequently rejected this alternate form, but added:

In Python 2.4a3 (to be released this Thursday), everything remains as currently in CVS. For 2.4b1, I will consider a change of @ to some other single character, even though I think that @ has the advantage of being the same character used by a similar feature in Java. It’s been argued that it’s not quite the same, since @ in Java is used for attributes that don’t change semantics. But Python’s dynamic nature makes that its syntactic elements never mean quite the same thing as similar constructs in other languages, and there is definitely significant overlap. Regarding the impact on 3rd party tools: IPython’s author doesn’t think there’s going to be much impact; Leo’s author has said that Leo will survive (although it will cause him and his users some transitional pain). I actually expect that picking a character that’s already used elsewhere in Python’s syntax might be harder for external tools to adapt to, since parsing will have to be more subtle in that case. But I’m frankly undecided, so there’s some wiggle room here. I don’t want to consider further syntactic alternatives at this point: the buck has to stop at some point, everyone has had their say, and the show must go on.

using:
    classmethod
    synchronized(lock)
def func(cls):
    pass

Examples

Much of the discussion on comp.lang.python and the python-dev mailing list focuses on the use of decorators as a cleaner way to use the staticmethod() and classmethod() builtins. This capability is much more powerful than that. This section presents some examples of use.

1. Define a function to be executed at exit. Note that the function isn’t actually “wrapped” in the usual sense.

   def onexit(f):
       import atexit
       atexit.register(f)
       return f
   
   @onexit
   def func():
       ...
   

   Note that this example is probably not suitable for real usage, but is for example purposes only.

2. Define a class with a singleton instance. Note that once the class disappears enterprising programmers would have to be more creative to create more instances. (From Shane Hathaway on python-dev.)

   def singleton(cls):
       instances = {}
       def getinstance():
           if cls not in instances:
               instances[cls] = cls()
           return instances[cls]
       return getinstance
   
   @singleton
   class MyClass:
       ...
   

3. Add attributes to a function. (Based on an example posted by Anders Munch on python-dev.)

   def attrs(**kwds):
       def decorate(f):
           for k in kwds:
               setattr(f, k, kwds[k])
           return f
       return decorate
   
   @attrs(versionadded="2.2",
          author="Guido van Rossum")
   def mymethod(f):
       ...
   

4. Enforce function argument and return types. Note that this copies the func_name attribute from the old to the new function. func_name was made writable in Python 2.4a3:

   def accepts(*types):
       def check_accepts(f):
           assert len(types) == f.func_code.co_argcount
           def new_f(*args, **kwds):
               for (a, t) in zip(args, types):
                   assert isinstance(a, t), \
                          "arg %r does not match %s" % (a,t)
               return f(*args, **kwds)
           new_f.func_name = f.func_name
           return new_f
       return check_accepts
   
   def returns(rtype):
       def check_returns(f):
           def new_f(*args, **kwds):
               result = f(*args, **kwds)
               assert isinstance(result, rtype), \
                      "return value %r does not match %s" % (result,rtype)
               return result
           new_f.func_name = f.func_name
           return new_f
       return check_returns
   
   @accepts(int, (int,float))
   @returns((int,float))
   def func(arg1, arg2):
       return arg1 * arg2
   

5. Declare that a class implements a particular (set of) interface(s). This is from a posting by Bob Ippolito on python-dev based on experience with PyProtocols.

   def provides(*interfaces):
        """
        An actual, working, implementation of provides for
        the current implementation of PyProtocols.  Not
        particularly important for the PEP text.
        """
        def provides(typ):
            declareImplementation(typ, instancesProvide=interfaces)
            return typ
        return provides
   
   class IBar(Interface):
        """Declare something about IBar here"""
   
   @provides(IBar)
   class Foo(object):
           """Implement something here..."""
   

Of course, all these examples are possible today, though without syntactic support.

(No longer) Open Issues

1. It’s not yet certain that class decorators will be incorporated into the language at a future point. Guido expressed skepticism about the concept, but various people have made some strong arguments (search for PEP 318 -- posting draft) on their behalf in python-dev. It’s exceedingly unlikely that class decorators will be in Python 2.4.

   PEP 3129 proposes to add class decorators as of Python 2.6.

2. The choice of the @ character will be re-examined before Python 2.4b1.

   In the end, the @ character was kept.

Copyright

This document has been placed in the public domain.