Abstract

This PEP proposes the creation of a manylinux2014 platform tag to succeed the manylinux2010 tag introduced by PEP 513. It also proposes that PyPI and pip both be updated to support uploading, downloading, and installing manylinux2014 distributions on compatible platforms.

Rationale

CentOS 6 is now the oldest supported CentOS release, and will receive maintenance updates through November 30th, 2020, [1] at which point it will reach end-of-life, and no further updates such as security patches will be made available. All wheels built under the manylinux2010 images will remain at obsolete versions after that point.

Therefore, we propose the continuation of the existing manylinux standard, and that a new PEP 425-style platform tag called manylinux2014 be derived from CentOS 7 and that the manylinux toolchain, PyPI, and pip be updated to support it.

Similar to how PEP 571 and PEP 513 drew allowed shared libraries and their symbol versions from CentOS 5.11 and CentOS 6, respectively, a manylinux2014 platform tag will draw its libraries and symbol versions from CentOS 7, which will reach end-of-life on June 30th, 2024. [1]

The manylinuxYYYY pattern has a number of advantages that motivate continuing with the current status quo:

• Well-defined Docker images with clearly specified compatible libraries;
• No need to survey for compatibility issues across multiple releases;
• A single build image and auditwheel profile per architecture.

There are also some disadvantages:

• Requires drafting a new PEP for every new standard;
• Requires adding the new platform tag to installers (e.g., pip);
• Installers are unable to install a platform tag which predates a given release.

There are also challenges which would exist for any proposal, including the time and effort it takes to define, prepare and release the Docker images and corresponding auditwheel profiles. These challenges were experienced in the long rollout period for manylinux2010, which took approximately 1 year from PEP acceptance to compatible build environment published. [3]

However, if this PEP can be an indicator, the process is now well-defined and easily repeatable, which should increase the timeline for rollout of a newer, updated platform tag.

The manylinux2014 policy

The following criteria determine a linux wheel’s eligibility for the manylinux2014 tag:

1. The wheel may only contain binary executables and shared objects compiled for one of the following architectures supported by CentOS 7, or a CentOS 7 compatible base image (such as ubi7): [4]

   x86_64
   i686
   aarch64
   armv7l
   ppc64
   ppc64le
   s390x
   

   This list adds support for ARMv7 (armv7l), ARMv8 (aarch64) and PowerPC (ppc64, ppc64le) architectures supported by the CentOS Alternative Architecture Special Interest Group, as well as the IBM Z (s390x) architecture. [5]

2. The wheel’s binary executables or shared objects may not link against externally-provided libraries except those in the following list:

   libgcc_s.so.1
   libstdc++.so.6
   libm.so.6
   libdl.so.2
   librt.so.1
   libc.so.6
   libnsl.so.1
   libutil.so.1
   libpthread.so.0
   libresolv.so.2
   libX11.so.6
   libXext.so.6
   libXrender.so.1
   libICE.so.6
   libSM.so.6
   libGL.so.1
   libgobject-2.0.so.0
   libgthread-2.0.so.0
   libglib-2.0.so.0
   

   This list is identical to the externally-provided libraries originally allowed for manylinux2010, with one exception: libcrypt.so.1 was removed due to being deprecated in Fedora 30. libpythonX.Y remains ineligible for inclusion for the same reasons outlined in PEP 513.

   On Debian-based systems, these libraries are provided by the packages:

   Package	Libraries
   libc6	libdl.so.2, libresolv.so.2, librt.so.1, libc.so.6, libpthread.so.0, libm.so.6, libutil.so.1, libnsl.so.1
   libgcc1	libgcc_s.so.1
   libgl1	libGL.so.1
   libglib2.0-0	libgobject-2.0.so.0, libgthread-2.0.so.0, libglib-2.0.so.0
   libice6	libICE.so.6
   libsm6	libSM.so.6
   libstdc++6	libstdc++.so.6
   libx11-6	libX11.so.6
   libxext6	libXext.so.6
   libxrender1	libXrender.so.1

   On RPM-based systems, they are provided by these packages:

   Package	Libraries
   glib2	libglib-2.0.so.0, libgthread-2.0.so.0, libgobject-2.0.so.0
   glibc	libresolv.so.2, libutil.so.1, libnsl.so.1, librt.so.1, libpthread.so.0, libdl.so.2, libm.so.6, libc.so.6
   libICE	libICE.so.6
   libX11	libX11.so.6
   libXext:	libXext.so.6
   libXrender	libXrender.so.1
   libgcc:	libgcc_s.so.1
   libstdc++	libstdc++.so.6
   mesa	libGL.so.1

3. If the wheel contains binary executables or shared objects linked against any allowed libraries that also export versioned symbols, they may only depend on the following maximum versions:

   GLIBC_2.17
   CXXABI_1.3.7, CXXABI_TM_1 is also allowed
   GLIBCXX_3.4.19
   GCC_4.8.0
   

   As an example, manylinux2014 wheels may include binary artifacts that require glibc symbols at version GLIBC_2.12, because this an earlier version than the maximum of GLIBC_2.17.

4. If a wheel is built for any version of CPython 2 or CPython versions 3.0 up to and including 3.2, it must include a CPython ABI tag indicating its Unicode ABI. A manylinux2014 wheel built against Python 2, then, must include either the cpy27mu tag indicating it was built against an interpreter with the UCS-4 ABI or the cpy27m tag indicating an interpreter with the UCS-2 ABI. (PEP 3149 [7])
5. A wheel must not require the PyFPE_jbuf symbol. This is achieved by building it against a Python compiled without the --with-fpectl configure flag.

Compilation of Compliant Wheels

Like manylinux1, the auditwheel tool adds manylinux2014 platform tags to linux wheels built by pip wheel or bdist_wheel in a manylinux2014 Docker container.

Platform Detection for Installers

Platforms may define a manylinux2014_compatible boolean attribute on the _manylinux module described in PEP 513. A platform is considered incompatible with manylinux2014 if the attribute is False.

If the _manylinux module is not found, or it does not have the attribute manylinux2014_compatible, tools may fall back to checking for glibc. If the platform has glibc 2.17 or newer, it is assumed to be compatible unless the _manylinux module says otherwise.

Specifically, the algorithm we propose is:

def is_manylinux2014_compatible():
    # Only Linux, and only supported architectures
    from distutils.util import get_platform

    if get_platform() not in [
        "linux-x86_64",
        "linux-i686",
        "linux-aarch64",
        "linux-armv7l",
        "linux-ppc64",
        "linux-ppc64le",
        "linux-s390x",
    ]:
        return False

    # Check for presence of _manylinux module
    try:
        import _manylinux

        return bool(_manylinux.manylinux2014_compatible)
    except (ImportError, AttributeError):
        # Fall through to heuristic check below
        pass

    # Check glibc version. CentOS 7 uses glibc 2.17.
    # PEP 513 contains an implementation of this function.
    return have_compatible_glibc(2, 17)

Backwards compatibility with manylinux2010 wheels

As explained in PEP 513, the specified symbol versions for manylinux1 allowed libraries constitute an upper bound. The same is true for the symbol versions defined for manylinux2014 in this PEP. As a result, manylinux1 and manylinux2010 wheels are considered manylinux2014 wheels. A pip that recognizes the manylinux2014 platform tag will thus install manylinux2010 wheels for manylinux2014 platforms – even when explicitly set – when no manylinux2014 wheels are available.

PyPI Support

PyPI should permit wheels containing the manylinux2014 platform tag to be uploaded in the same way that it permits manylinux2010.

If technically feasible, PyPI should attempt to verify the compatibility of manylinux2014 wheels, but that capability is not a requirement for adoption of this PEP.

Package authors should not upload non-compliant manylinux2014 wheels to PyPI, and should be aware that PyPI may begin blocking non-compliant wheels from being uploaded.

References

[1] (1, 2)

CentOS Product Specifications (https://wiki.centos.org/About/Product)

[3]

Tracking issue for manylinux2010 rollout (https://github.com/pypa/manylinux/issues/179)

[4]

Red Hat Universal Base Image 7 (https://access.redhat.com/containers/?tab=overview#/registry.access.redhat.com/ubi7)

[5]

The CentOS Alternative Architecture Special Interest Group (https://wiki.centos.org/SpecialInterestGroup/AltArch)

[7]

SOABI support for Python 2.X and PyPy (https://github.com/pypa/pip/pull/3075)

[8]

auditwheel (https://github.com/pypa/auditwheel/)

Acceptance

PEP 599 was accepted by Paul Moore on July 31, 2019.

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.