PEP 3132 – Extended Iterable Unpacking
- PEP
- 3132
- Title
- Extended Iterable Unpacking
- Author
- Georg Brandl <georg at python.org>
- Status
- Final
- Type
- Standards Track
- Created
- 30-Apr-2007
- Python-Version
- 3.0
- Post-History
Abstract
This PEP proposes a change to iterable unpacking syntax, allowing to specify a “catch-all” name which will be assigned a list of all items not assigned to a “regular” name.
An example says more than a thousand words:
>>> a, *b, c = range(5)
>>> a
0
>>> c
4
>>> b
[1, 2, 3]
Rationale
Many algorithms require splitting a sequence in a “first, rest” pair. With the new syntax,
first, rest = seq[0], seq[1:]
is replaced by the cleaner and probably more efficient:
first, *rest = seq
For more complex unpacking patterns, the new syntax looks even cleaner, and the clumsy index handling is not necessary anymore.
Also, if the right-hand value is not a list, but an iterable, it has to be converted to a list before being able to do slicing; to avoid creating this temporary list, one has to resort to
it = iter(seq)
first = it.next()
rest = list(it)
Specification
A tuple (or list) on the left side of a simple assignment (unpacking is not defined for augmented assignment) may contain at most one expression prepended with a single asterisk (which is henceforth called a “starred” expression, while the other expressions in the list are called “mandatory”). This designates a subexpression that will be assigned a list of all items from the iterable being unpacked that are not assigned to any of the mandatory expressions, or an empty list if there are no such items.
For example, if seq
is a sliceable sequence, all the following
assignments are equivalent if seq
has at least two elements:
a, b, c = seq[0], list(seq[1:-1]), seq[-1]
a, *b, c = seq
[a, *b, c] = seq
It is an error (as it is currently) if the iterable doesn’t contain enough items to assign to all the mandatory expressions.
It is also an error to use the starred expression as a lone assignment target, as in
*a = range(5)
This, however, is valid syntax:
*a, = range(5)
Note that this proposal also applies to tuples in implicit assignment
context, such as in a for
statement:
for a, *b in [(1, 2, 3), (4, 5, 6, 7)]:
print(b)
would print out
[2, 3]
[5, 6, 7]
Starred expressions are only allowed as assignment targets, using them anywhere else (except for star-args in function calls, of course) is an error.
Implementation
Grammar change
This feature requires a new grammar rule:
star_expr: ['*'] expr
In these two rules, expr
is changed to star_expr
:
comparison: star_expr (comp_op star_expr)*
exprlist: star_expr (',' star_expr)* [',']
Changes to the Compiler
A new ASDL expression type Starred
is added which represents a
starred expression. Note that the starred expression element
introduced here is universal and could later be used for other
purposes in non-assignment context, such as the yield *iterable
proposal.
The compiler is changed to recognize all cases where a starred expression is invalid and flag them with syntax errors.
A new bytecode instruction, UNPACK_EX
, is added, whose argument
has the number of mandatory targets before the starred target in the
lower 8 bits and the number of mandatory targets after the starred
target in the upper 8 bits. For unpacking sequences without starred
expressions, the old UNPACK_ITERABLE
opcode is kept.
Changes to the Bytecode Interpreter
The function unpack_iterable()
in ceval.c is changed to handle
the extended unpacking, via an argcntafter
parameter. In the
UNPACK_EX
case, the function will do the following:
- collect all items for mandatory targets before the starred one
- collect all remaining items from the iterable in a list
- pop items for mandatory targets after the starred one from the list
- push the single items and the resized list on the stack
Shortcuts for unpacking iterables of known types, such as lists or tuples, can be added.
The current implementation can be found at the SourceForge Patch tracker [SFPATCH]. It now includes a minimal test case.
Acceptance
After a short discussion on the python-3000 list [1], the PEP was accepted by Guido in its current form. Possible changes discussed were:
- Only allow a starred expression as the last item in the exprlist. This would simplify the unpacking code a bit and allow for the starred expression to be assigned an iterator. This behavior was rejected because it would be too surprising.
- Try to give the starred target the same type as the source
iterable, for example,
b
ina, *b = 'hello'
would be assigned the string'ello'
. This may seem nice, but is impossible to get right consistently with all iterables. - Make the starred target a tuple instead of a list. This would be
consistent with a function’s
*args
, but make further processing of the result harder.
References
- [SFPATCH]
- https://bugs.python.org/issue1711529
- [1]
- https://mail.python.org/pipermail/python-3000/2007-May/007198.html
Copyright
This document has been placed in the public domain.
Source: https://github.com/python-discord/peps/blob/main/pep-3132.txt
Last modified: 2021-09-17 18:18:24 GMT