Abstract

PEP 589 defines notation for declaring a TypedDict with all required keys and notation for defining a TypedDict with all potentially-missing keys, however it does not provide a mechanism to declare some keys as required and others as potentially-missing. This PEP introduces two new notations: Required[], which can be used on individual items of a TypedDict to mark them as required, and NotRequired[], which can be used on individual items to mark them as potentially-missing.

This PEP makes no Python grammar changes. Correct usage of required and potentially-missing keys of TypedDicts is intended to be enforced only by static type checkers and need not be enforced by Python itself at runtime.

Motivation

It is not uncommon to want to define a TypedDict with some keys that are required and others that are potentially-missing. Currently the only way to define such a TypedDict is to declare one TypedDict with one value for total and then inherit it from another TypedDict with a different value for total:

class _MovieBase(TypedDict):  # implicitly total=True
    title: str

class Movie(_MovieBase, total=False):
    year: int

Having to declare two different TypedDict types for this purpose is cumbersome.

This PEP introduces two new type qualifiers, typing.Required and typing.NotRequired, which allow defining a single TypedDict with a mix of both required and potentially-missing keys:

class Movie(TypedDict):
    title: str
    year: NotRequired[int]

This PEP also makes it possible to define TypedDicts in the alternative functional syntax with a mix of required and potentially-missing keys, which is not currently possible at all because the alternative syntax does not support inheritance:

Actor = TypedDict('Actor', {
    'name': str,
    # "in" is a keyword, so the functional syntax is necessary
    'in': NotRequired[List[str]],
})

Rationale

One might think it unusual to propose notation that prioritizes marking required keys rather than potentially-missing keys, as is customary in other languages like TypeScript:

interface Movie {
    title: string;
    year?: number;  // ? marks potentially-missing keys
}

The difficulty is that the best word for marking a potentially-missing key, Optional[], is already used in Python for a completely different purpose: marking values that could be either of a particular type or None. In particular the following does not work:

class Movie(TypedDict):
    ...
    year: Optional[int]  # means int|None, not potentially-missing!

Attempting to use any synonym of “optional” to mark potentially-missing keys (like Missing[]) would be too similar to Optional[] and be easy to confuse with it.

Thus it was decided to focus on positive-form phrasing for required keys instead, which is straightforward to spell as Required[].

Nevertheless it is common for folks wanting to extend a regular (total=True) TypedDict to only want to add a small number of potentially-missing keys, which necessitates a way to mark keys that are not required and potentially-missing, and so we also allow the NotRequired[] form for that case.

Specification

The typing.Required type qualifier is used to indicate that a variable declared in a TypedDict definition is a required key:

class Movie(TypedDict, total=False):
    title: Required[str]
    year: int

Additionally the typing.NotRequired type qualifier is used to indicate that a variable declared in a TypedDict definition is a potentially-missing key:

class Movie(TypedDict):  # implicitly total=True
    title: str
    year: NotRequired[int]

It is an error to use Required[] or NotRequired[] in any location that is not an item of a TypedDict. Type checkers must enforce this restriction.

It is valid to use Required[] and NotRequired[] even for items where it is redundant, to enable additional explicitness if desired:

class Movie(TypedDict):
    title: Required[str]  # redundant
    year: NotRequired[int]

It is an error to use both Required[] and NotRequired[] at the same time:

class Movie(TypedDict):
    title: str
    year: NotRequired[Required[int]]  # ERROR

Type checkers must enforce this restriction. The runtime implementations of Required[] and NotRequired[] may also enforce this restriction.

The alternative functional syntax for TypedDict also supports Required[] and NotRequired[]:

Movie = TypedDict('Movie', {'name': str, 'year': NotRequired[int]})

class _MovieBase(TypedDict):  # implicitly total=True
    title: str

class Movie(_MovieBase, total=False):
    year: int

class _MovieBase(TypedDict):
    title: str

class Movie(_MovieBase):
    year: NotRequired[int]

class Movie(TypedDict):
    title: str
    year: NotRequired[Annotated[int, ValueRange(-9999, 9999)]]  # ok

class Movie(TypedDict):
    title: str
    year: Annotated[NotRequired[int], ValueRange(-9999, 9999)]  # ok

class Movie(TypedDict):
    title: str
    year: NotRequired[int]

assert get_type_hints(Movie) == \
    {'title': str, 'year': int}
assert get_type_hints(Movie, include_extras=True) == \
    {'title': str, 'year': NotRequired[int]}

assert get_origin(Required[int]) is Required
assert get_args(Required[int]) == (int,)

assert get_origin(NotRequired[int]) is NotRequired
assert get_args(NotRequired[int]) == (int,)

assert Movie.__required_keys__ == frozenset({'title'})
assert Movie.__optional_keys__ == frozenset({'year'})

Backwards Compatibility

No backward incompatible changes are made by this PEP.

How to Teach This

To define a TypedDict where most keys are required and some are potentially-missing, define a single TypedDict as normal (without the total keyword) and mark those few keys that are potentially-missing with NotRequired[].

To define a TypedDict where most keys are potentially-missing and a few are required, define a total=False TypedDict and mark those few keys that are required with Required[].

If some items accept None in addition to a regular value, it is recommended that the TYPE|None notation be preferred over Optional[TYPE] for marking such item values, to avoid using Required[] or NotRequired[] alongside Optional[] within the same TypedDict definition:

Yes:

from __future__ import annotations  # for Python 3.7-3.9

class Dog(TypedDict):
    name: str
    owner: NotRequired[str|None]

Okay (required for Python 3.5.3-3.6):

class Dog(TypedDict):
    name: str
    owner: 'NotRequired[str|None]'

No:

class Dog(TypedDict):
    name: str
    # ick; avoid using both Optional and NotRequired
    owner: NotRequired[Optional[str]]

from typing import NotRequired, TypedDict

class Dog(TypedDict):
    name: str
    owner: NotRequired[str|None]

from __future__ import annotations  # for Python 3.7-3.9

from typing_extensions import NotRequired, TypedDict  # for Python <3.11 with (Not)Required

class Dog(TypedDict):
    name: str
    owner: NotRequired[str|None]

from typing import TypedDict  # oops: should import from typing_extensions instead
from typing_extensions import NotRequired

class Movie(TypedDict):
    title: str
    year: NotRequired[int]

assert Movie.__required_keys__ == frozenset({'title', 'year'})  # yikes
assert Movie.__optional_keys__ == frozenset()  # yikes

Reference Implementation

The mypy 0.930, pyright 1.1.117, and pyanalyze 0.4.0 type checkers support Required and NotRequired.

A reference implementation of the runtime component is provided in the typing_extensions module.

Rejected Ideas

class MyThing(TypedDict):
    opt1?: str  # may not exist, but if exists, value is string
    opt2: Optional[str]  # always exists, but may have None value

class MyThing(TypedDict):
    Optional[opt1]: str  # may not exist, but if exists, value is string
    opt2: Optional[str]  # always exists, but may have None value

class MyThing(TypedDict, total=False):
    req1: +int    # + means a required key, or Required[]
    opt1: str
    req2: +float

class MyThing(TypedDict):
    req1: int
    opt1: -str    # - means a potentially-missing key, or NotRequired[]
    req2: float

class MyThing(TypedDict):
    req1: int
    opt1: ~str    # ~ means a opposite-of-normal-totality key
    req2: float

>>> from typing import TypeVar
>>> (TypeVar('T', covariant=True), TypeVar('U', contravariant=True), TypeVar('V'))
(+T, -U, ~V)

class MyThing(TypedDict):
    req1: int
    opt1: str|Missing
    req2: float

class MyClass:
    attr: int|Missing

    def __init__(self, set_attr: bool) -> None:
        if set_attr:
            self.attr = 10

def foo(set_attr: bool) -> None:
    if set_attr:
        attr = 10
    reveal_type(attr)  # int|Missing

class Packet:
    data: Union[str, bytes]

def send_data(packet: Packet) -> None:
    if isinstance(packet.data, str):
        reveal_type(packet.data)  # str
        packet_bytes = packet.data.encode('utf-8')
    else:
        reveal_type(packet.data)  # bytes
        packet_bytes = packet.data
    socket.send(packet_bytes)

class Packet:
    data: Union[str, Missing]

def send_data(packet: Packet) -> None:
    if hasattr(packet, 'data'):
        reveal_type(packet.data)  # str
        packet_bytes = packet.data.encode('utf-8')
    else:
        reveal_type(packet.data)  # Missing? error?
        packet_bytes = b''
    socket.send(packet_bytes)

def send_data(packet_data: Optional[str]) -> None:
    packet_bytes: Union[str, Missing]
    if packet_data is not None:
        packet_bytes = packet.data.encode('utf-8')

    if 'packet_bytes' in locals():
        reveal_type(packet_bytes)  # bytes
        socket.send(packet_bytes)
    else:
        reveal_type(packet_bytes)  # Missing? error?

warning: Union[str, Missing]
import sys
if sys.version_info < (3, 6):
    warning = 'Your version of Python is unsupported!'

if 'warning' in globals():
    reveal_type(warning)  # str
    print(warning)
else:
    reveal_type(warning)  # Missing? error?

class MyThing(TypedDict, optional_as_missing=True):
    req1: int
    opt1: Optional[str]

class MyThing(TypedDict, optional_as_nullable=False):
    req1: int
    opt1: Optional[str]

References

[1]

https://mail.python.org/archives/list/typing-sig@python.org/message/4I3GPIWDUKV6GUCHDMORGUGRE4F4SXGR/

[2]

https://mail.python.org/archives/list/typing-sig@python.org/message/S2VJSVG6WCIWPBZ54BOJPG56KXVSLZK6/

[3]

https://bugs.python.org/issue46491

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.