The pip bootstrap

The Python installation includes an executable called “pip3” (see PEP 394 for naming rationale etc.) that attempts to import pip machinery. If it can then the pip command proceeds as normal. If it cannot it will bootstrap pip by downloading the pip implementation and setuptools wheel files. Hereafter the installation of the “pip implementation” will imply installation of setuptools and virtualenv. Once installed, the pip command proceeds as normal. Once the bootstrap process is complete the “pip3” command is no longer the bootstrap but rather the full pip command.

A bootstrap is used in the place of a the full pip code so that we don’t have to bundle pip and also pip is upgradeable outside of the regular Python upgrade timeframe and processes.

To avoid issues with sudo we will have the bootstrap default to installing the pip implementation to the per-user site-packages directory defined in PEP 370 and implemented in Python 2.6/3.0. Since we avoid installing to the system Python we also avoid conflicting with any other packaging system (on Linux systems, for example.) If the user is inside a PEP 405 virtual environment then the pip implementation will be installed into that virtual environment.

The bootstrap process will proceed as follows:

1. The user system has Python (3.4+) installed. In the “scripts” directory of the Python installation there is the bootstrap script called “pip3”.
2. The user will invoke a pip command, typically “pip3 install <package>”, for example “pip3 install Django”.
3. The bootstrap script will attempt to import the pip implementation. If this succeeds, the pip command is processed normally. Stop.
4. On failing to import the pip implementation the bootstrap notifies the user that it needs to “install pip”. It will ask the user whether it should install pip as a system-wide site-packages or as a user-only package. This choice will also be present as a command-line option to pip so non-interactive use is possible.
5. The bootstrap will and contact PyPI to obtain the latest download wheel file (see PEP 427.)
6. Upon downloading the file it is installed using “python setup.py install”.
7. The pip tool may now import the pip implementation and continues to process the requested user command normally.

Users may be running in an environment which cannot access the public Internet and are relying solely on a local package repository. They would use the “-i” (Base URL of Python Package Index) argument to the “pip3 install” command. This simply overrides the default index URL pointing to PyPI.

Some users may have no Internet access suitable for fetching the pip implementation file. These users can manually download and install the setuptools and pip tar files. Adding specific support for this use-case is unnecessary.

The download of the pip implementation install file will be performed securely. The transport from pypi.python.org will be done over HTTPS with the CA certificate check performed. This facility will be present in Python 3.4+ using Operating System certificates (see PEP XXXX).

Beyond those arguments controlling index location and download options, the “pip3” bootstrap command may support further standard pip options for verbosity, quietness and logging.

The “pip3” command will support two new command-line options that are used in the bootstrapping, and otherwise ignored. They control where the pip implementation is installed:

--bootstrap

Install to the user’s packages directory. The name of this option is chosen to promote it as the preferred installation option.

--bootstrap-to-system

Install to the system site-packages directory.

These command-line options will also need to be implemented, but otherwise ignored, in the pip implementation.

Consideration should be given to defaulting pip to install packages to the user’s packages directory if pip is installed in that location.

The “–no-install” option to the “pip3” command will not affect the bootstrapping process.

Modifications to publishing packages

An additional new Python package is proposed, “pypublish”, which will be a tool for publishing packages to PyPI. It would replace the current “python setup.py register” and “python setup.py upload” distutils commands. Again because of the measured Python release cycle and extensive existing Python installations these commands are difficult to bugfix and extend. Additionally it is desired that the “register” and “upload” commands be able to be performed over HTTPS with certificate validation. Since shipping CA certificate keychains with Python is not really feasible (updating the keychain is quite difficult to manage) it is desirable that those commands, and the accompanying keychain, be made installable and upgradeable outside of Python itself.

The existing distutils mechanisms for package registration and upload would remain, though with a deprecation warning.