Acknowledgements

Talin wrote the Rationale below [1] as well as most of the section on ABCs vs. Interfaces. For that alone he deserves co-authorship. The rest of the PEP uses “I” referring to the first author.

Overloading isinstance() and issubclass()

During the development of this PEP and of its companion, PEP 3141, we repeatedly faced the choice between standardizing more, fine-grained ABCs or fewer, coarse-grained ones. For example, at one stage, PEP 3141 introduced the following stack of base classes used for complex numbers: MonoidUnderPlus, AdditiveGroup, Ring, Field, Complex (each derived from the previous). And the discussion mentioned several other algebraic categorizations that were left out: Algebraic, Transcendental, and IntegralDomain, and PrincipalIdealDomain. In earlier versions of the current PEP, we considered the use cases for separate classes like Set, ComposableSet, MutableSet, HashableSet, MutableComposableSet, HashableComposableSet.

The dilemma here is that we’d rather have fewer ABCs, but then what should a user do who needs a less refined ABC? Consider e.g. the plight of a mathematician who wants to define their own kind of Transcendental numbers, but also wants float and int to be considered Transcendental. PEP 3141 originally proposed to patch float.__bases__ for that purpose, but there are some good reasons to keep the built-in types immutable (for one, they are shared between all Python interpreters running in the same address space, as is used by mod_python [16]).

Another example would be someone who wants to define a generic function (PEP 3124) for any sequence that has an append() method. The Sequence ABC (see below) doesn’t promise the append() method, while MutableSequence requires not only append() but also various other mutating methods.

To solve these and similar dilemmas, the next section will propose a metaclass for use with ABCs that will allow us to add an ABC as a “virtual base class” (not the same concept as in C++) to any class, including to another ABC. This allows the standard library to define ABCs Sequence and MutableSequence and register these as virtual base classes for built-in types like basestring, tuple and list, so that for example the following conditions are all true:

isinstance([], Sequence)
issubclass(list, Sequence)
issubclass(list, MutableSequence)
isinstance((), Sequence)
not issubclass(tuple, MutableSequence)
isinstance("", Sequence)
issubclass(bytearray, MutableSequence)

The primary mechanism proposed here is to allow overloading the built-in functions isinstance() and issubclass(). The overloading works as follows: The call isinstance(x, C) first checks whether C.__instancecheck__ exists, and if so, calls C.__instancecheck__(x) instead of its normal implementation. Similarly, the call issubclass(D, C) first checks whether C.__subclasscheck__ exists, and if so, calls C.__subclasscheck__(D) instead of its normal implementation.

Note that the magic names are not __isinstance__ and __issubclass__; this is because the reversal of the arguments could cause confusion, especially for the issubclass() overloader.

A prototype implementation of this is given in [12].

Here is an example with (naively simple) implementations of __instancecheck__ and __subclasscheck__:

class ABCMeta(type):

    def __instancecheck__(cls, inst):
        """Implement isinstance(inst, cls)."""
        return any(cls.__subclasscheck__(c)
                   for c in {type(inst), inst.__class__})

    def __subclasscheck__(cls, sub):
        """Implement issubclass(sub, cls)."""
        candidates = cls.__dict__.get("__subclass__", set()) | {cls}
        return any(c in candidates for c in sub.mro())

class Sequence(metaclass=ABCMeta):
    __subclass__ = {list, tuple}

assert issubclass(list, Sequence)
assert issubclass(tuple, Sequence)

class AppendableSequence(Sequence):
    __subclass__ = {list}

assert issubclass(list, AppendableSequence)
assert isinstance([], AppendableSequence)

assert not issubclass(tuple, AppendableSequence)
assert not isinstance((), AppendableSequence)

The next section proposes a full-fledged implementation.

The abc Module: an ABC Support Framework

The new standard library module abc, written in pure Python, serves as an ABC support framework. It defines a metaclass ABCMeta and decorators @abstractmethod and @abstractproperty. A sample implementation is given by [13].

The ABCMeta class overrides __instancecheck__ and __subclasscheck__ and defines a register method. The register method takes one argument, which must be a class; after the call B.register(C), the call issubclass(C, B) will return True, by virtue of B.__subclasscheck__(C) returning True. Also, isinstance(x, B) is equivalent to issubclass(x.__class__, B) or issubclass(type(x), B). (It is possible type(x) and x.__class__ are not the same object, e.g. when x is a proxy object.)

These methods are intended to be called on classes whose metaclass is (derived from) ABCMeta; for example:

from abc import ABCMeta

class MyABC(metaclass=ABCMeta):
    pass

MyABC.register(tuple)

assert issubclass(tuple, MyABC)
assert isinstance((), MyABC)

The last two asserts are equivalent to the following two:

assert MyABC.__subclasscheck__(tuple)
assert MyABC.__instancecheck__(())

Of course, you can also directly subclass MyABC:

class MyClass(MyABC):
    pass

assert issubclass(MyClass, MyABC)
assert isinstance(MyClass(), MyABC)

Also, of course, a tuple is not a MyClass:

assert not issubclass(tuple, MyClass)
assert not isinstance((), MyClass)

You can register another class as a subclass of MyClass:

MyClass.register(list)

assert issubclass(list, MyClass)
assert issubclass(list, MyABC)

You can also register another ABC:

class AnotherClass(metaclass=ABCMeta):
    pass

AnotherClass.register(basestring)

MyClass.register(AnotherClass)

assert isinstance(str, MyABC)

That last assert requires tracing the following superclass-subclass relationships:

MyABC -> MyClass (using regular subclassing)
MyClass -> AnotherClass (using registration)
AnotherClass -> basestring (using registration)
basestring -> str (using regular subclassing)

The abc module also defines a new decorator, @abstractmethod, to be used to declare abstract methods. A class containing at least one method declared with this decorator that hasn’t been overridden yet cannot be instantiated. Such methods may be called from the overriding method in the subclass (using super or direct invocation). For example:

from abc import ABCMeta, abstractmethod

class A(metaclass=ABCMeta):
    @abstractmethod
    def foo(self): pass

A()  # raises TypeError

class B(A):
    pass

B()  # raises TypeError

class C(A):
    def foo(self): print(42)

C()  # works

Note: The @abstractmethod decorator should only be used inside a class body, and only for classes whose metaclass is (derived from) ABCMeta. Dynamically adding abstract methods to a class, or attempting to modify the abstraction status of a method or class once it is created, are not supported. The @abstractmethod only affects subclasses derived using regular inheritance; “virtual subclasses” registered with the register() method are not affected.

Implementation: The @abstractmethod decorator sets the function attribute __isabstractmethod__ to the value True. The ABCMeta.__new__ method computes the type attribute __abstractmethods__ as the set of all method names that have an __isabstractmethod__ attribute whose value is true. It does this by combining the __abstractmethods__ attributes of the base classes, adding the names of all methods in the new class dict that have a true __isabstractmethod__ attribute, and removing the names of all methods in the new class dict that don’t have a true __isabstractmethod__ attribute. If the resulting __abstractmethods__ set is non-empty, the class is considered abstract, and attempts to instantiate it will raise TypeError. (If this were implemented in CPython, an internal flag Py_TPFLAGS_ABSTRACT could be used to speed up this check [6].)

Discussion: Unlike Java’s abstract methods or C++’s pure abstract methods, abstract methods as defined here may have an implementation. This implementation can be called via the super mechanism from the class that overrides it. This could be useful as an end-point for a super-call in framework using cooperative multiple-inheritance [7], [8].

A second decorator, @abstractproperty, is defined in order to define abstract data attributes. Its implementation is a subclass of the built-in property class that adds an __isabstractmethod__ attribute:

class abstractproperty(property):
    __isabstractmethod__ = True

It can be used in two ways:

class C(metaclass=ABCMeta):

    # A read-only property:

    @abstractproperty
    def readonly(self):
        return self.__x

    # A read-write property (cannot use decorator syntax):

    def getx(self):
        return self.__x
    def setx(self, value):
        self.__x = value
    x = abstractproperty(getx, setx)

Similar to abstract methods, a subclass inheriting an abstract property (declared using either the decorator syntax or the longer form) cannot be instantiated unless it overrides that abstract property with a concrete property.

ABCs for Containers and Iterators

The collections module will define ABCs necessary and sufficient to work with sets, mappings, sequences, and some helper types such as iterators and dictionary views. All ABCs have the above-mentioned ABCMeta as their metaclass.

The ABCs provide implementations of their abstract methods that are technically valid but fairly useless; e.g. __hash__ returns 0, and __iter__ returns an empty iterator. In general, the abstract methods represent the behavior of an empty container of the indicated type.

Some ABCs also provide concrete (i.e. non-abstract) methods; for example, the Iterator class has an __iter__ method returning itself, fulfilling an important invariant of iterators (which in Python 2 has to be implemented anew by each iterator class). These ABCs can be considered “mix-in” classes.

No ABCs defined in the PEP override __init__, __new__, __str__ or __repr__. Defining a standard constructor signature would unnecessarily constrain custom container types, for example Patricia trees or gdbm files. Defining a specific string representation for a collection is similarly left up to individual implementations.

Note: There are no ABCs for ordering operations (__lt__, __le__, __ge__, __gt__). Defining these in a base class (abstract or not) runs into problems with the accepted type for the second operand. For example, if class Ordering defined __lt__, one would assume that for any Ordering instances x and y, x < y would be defined (even if it just defines a partial ordering). But this cannot be the case: If both list and str derived from Ordering, this would imply that [1, 2] < (1, 2) should be defined (and presumably return False), while in fact (in Python 3000!) such “mixed-mode comparisons” operations are explicitly forbidden and raise TypeError. See PEP 3100 and [14] for more information. (This is a special case of a more general issue with operations that take another argument of the same type).

class Sized(metaclass=ABCMeta):
    @abstractmethod
    def __hash__(self):
        return 0
    @classmethod
    def __instancecheck__(cls, x):
        return hasattr(x, "__len__")
    @classmethod
    def __subclasscheck__(cls, C):
        return hasattr(C, "__bases__") and hasattr(C, "__len__")

Strings

Python 3000 will likely have at least two built-in string types: byte strings (bytes), deriving from MutableSequence, and (Unicode) character strings (str), deriving from Sequence and Hashable.

Open issues: define the base interfaces for these so alternative implementations and subclasses know what they are in for. This may be the subject of a new PEP or PEPs (PEP 358 should be co-opted for the bytes type).

ABCs vs. Duck Typing

Does the introduction of ABCs mean the end of Duck Typing? I don’t think so. Python will not require that a class derives from BasicMapping or Sequence when it defines a __getitem__ method, nor will the x[y] syntax require that x is an instance of either ABC. You will still be able to assign any “file-like” object to sys.stdout, as long as it has a write method.

Of course, there will be some carrots to encourage users to derive from the appropriate base classes; these vary from default implementations for certain functionality to an improved ability to distinguish between mappings and sequences. But there are no sticks. If hasattr(x, "__len__") works for you, great! ABCs are intended to solve problems that don’t have a good solution at all in Python 2, such as distinguishing between mappings and sequences.

ABCs vs. Generic Functions

ABCs are compatible with Generic Functions (GFs). For example, my own Generic Functions implementation [4] uses the classes (types) of the arguments as the dispatch key, allowing derived classes to override base classes. Since (from Python’s perspective) ABCs are quite ordinary classes, using an ABC in the default implementation for a GF can be quite appropriate. For example, if I have an overloaded prettyprint function, it would make total sense to define pretty-printing of sets like this:

@prettyprint.register(Set)
def pp_set(s):
    return "{" + ... + "}"  # Details left as an exercise

and implementations for specific subclasses of Set could be added easily.

I believe ABCs also won’t present any problems for RuleDispatch, Phillip Eby’s GF implementation in PEAK [5].

Of course, GF proponents might claim that GFs (and concrete, or implementation, classes) are all you need. But even they will not deny the usefulness of inheritance; and one can easily consider the ABCs proposed in this PEP as optional implementation base classes; there is no requirement that all user-defined mappings derive from BasicMapping.

ABCs vs. Interfaces

ABCs are not intrinsically incompatible with Interfaces, but there is considerable overlap. For now, I’ll leave it to proponents of Interfaces to explain why Interfaces are better. I expect that much of the work that went into e.g. defining the various shades of “mapping-ness” and the nomenclature could easily be adapted for a proposal to use Interfaces instead of ABCs.

“Interfaces” in this context refers to a set of proposals for additional metadata elements attached to a class which are not part of the regular class hierarchy, but do allow for certain types of inheritance testing.

Such metadata would be designed, at least in some proposals, so as to be easily mutable by an application, allowing application writers to override the normal classification of an object.

The drawback to this idea of attaching mutable metadata to a class is that classes are shared state, and mutating them may lead to conflicts of intent. Additionally, the need to override the classification of an object can be done more cleanly using generic functions: In the simplest case, one can define a “category membership” generic function that simply returns False in the base implementation, and then provide overrides that return True for any classes of interest.