PyTorch

Install Dependencies

Common

bash 
# Run this command from the PyTorch directory after cloning the source code
# using the "Get the PyTorch Source" section above
pip install --group dev
On Linux
bash 
pip install mkl-static mkl-include
# CUDA only: Add LAPACK support for the GPU if needed
# magma installation: run with active conda environment.
# specify CUDA version to install
.ci/docker/common/install_magma_conda.sh 12.4

# (optional) If using torch.compile with inductor/triton,
# install the matching version of triton. Run from the pytorch dir after cloning
# For Intel GPU support, please explicitly `export USE_XPU=1` before running.
make triton
On Windows
bash 
pip install mkl-static mkl-include
# Add these packages if torch.distributed is needed.
# Distributed package support on Windows is a prototype feature
# and is subject to changes.
conda install -c conda-forge libuv=1.51

Install PyTorch

On Linux

If you're compiling for AMD ROCm then first run this command:

bash 
# Only run this if you're compiling for ROCm
python tools/amd_build/build_amd.py

Install PyTorch

bash 
# the CMake prefix for conda environment
export CMAKE_PREFIX_PATH="${CONDA_PREFIX:-'$(dirname $(which conda))/../'}:${CMAKE_PREFIX_PATH}"
python -m pip install --no-build-isolation -v -e .

# the CMake prefix for non-conda environment, e.g. Python venv
# call following after activating the venv
export CMAKE_PREFIX_PATH="${VIRTUAL_ENV}:${CMAKE_PREFIX_PATH}"

On macOS

bash 
python -m pip install --no-build-isolation -v -e .

On Windows

If you want to build legacy python code, please refer to Building on legacy code and CUDA.

CPU-only builds

In this mode PyTorch computations will run on your CPU, not your GPU.

cmd 
python -m pip install --no-build-isolation -v -e .
note

OpenMP: The desired OpenMP implementation is Intel OpenMP (iomp). To link against iomp, manually download the library and set up the build environment by tweaking CMAKE_INCLUDE_PATH and LIB. The instructions here are an example for setting up both MKL and Intel OpenMP. Without these configurations, Microsoft Visual C OpenMP runtime (vcomp) will be used.

CUDA based build

In this mode PyTorch computations will leverage your GPU via CUDA for faster number crunching.

NVTX is needed to build PyTorch with CUDA. NVTX is a part of the CUDA distributive, where it is called "Nsight Compute". To install it onto an already-installed CUDA, run the CUDA installation once again and check the corresponding checkbox. Make sure that CUDA with Nsight Compute is installed after Visual Studio.

Currently, VS 2017 / 2019 and Ninja are supported as the CMake generator. If ninja.exe is detected in PATH, Ninja is used as the default generator; otherwise it uses VS 2017 / 2019. If Ninja is selected, the latest MSVC is used as the underlying toolchain.

Additional libraries such as Magma, oneDNN (MKLDNN / DNNL), and Sccache are often needed. Refer to the installation-helper to install them. You can also refer to the build_pytorch.bat script for other environment variable configurations.

cmd 
cmd

:: Set the environment variables after you have downloaded and unzipped
:: the mkl package, else CMake would throw `Could NOT find OpenMP`.
set CMAKE_INCLUDE_PATH={Your directory}\mkl\include
set LIB={Your directory}\mkl\lib;%LIB%

:: Read the previous section carefully before you proceed.
:: [Optional] Override the underlying toolset used by Ninja and VS with CUDA.
:: "Visual Studio 2019 Developer Command Prompt" runs automatically.
:: Requires CMake >= 3.12 when using the Visual Studio generator.
set CMAKE_GENERATOR_TOOLSET_VERSION=14.27
set DISTUTILS_USE_SDK=1
for /f "usebackq tokens=*" %i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -version [15^,17^) -products * -latest -property installationPath`) do call "%i\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%

:: [Optional] Override the CUDA host compiler
set CUDAHOSTCXX=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\cl.exe

python -m pip install --no-build-isolation -v -e .

Intel GPU builds

In this mode PyTorch with Intel GPU support will be built. Make sure the common prerequisites as well as the prerequisites for Intel GPU are properly installed and the environment variables are configured prior to starting the build. For build tool support, Visual Studio 2022 is required.

Then PyTorch can be built with the command:

cmd 
:: Set CMAKE_PREFIX_PATH to help find corresponding packages
:: %CONDA_PREFIX% only works after `conda activate custom_env`
if defined CMAKE_PREFIX_PATH (
    set "CMAKE_PREFIX_PATH=%CONDA_PREFIX%\Library;%CMAKE_PREFIX_PATH%"
) else (
    set "CMAKE_PREFIX_PATH=%CONDA_PREFIX%\Library"
)

python -m pip install --no-build-isolation -v -e .

Adjust Build Options (Optional)

You can adjust CMake variables through environment variables, which the build forwards to CMake (see cmake/EnvVarForwarding.cmake for the full list). For example, pointing the build at a specific Conda prefix, CuDNN, or BLAS:

On Linux
bash 
export CMAKE_PREFIX_PATH="${CONDA_PREFIX:-'$(dirname $(which conda))/../'}:${CMAKE_PREFIX_PATH}"
spin develop
On macOS
bash 
export CMAKE_PREFIX_PATH="${CONDA_PREFIX:-'$(dirname $(which conda))/../'}:${CMAKE_PREFIX_PATH}"
MACOSX_DEPLOYMENT_TARGET=11.0 spin develop

After a first spin develop, you can inspect or adjust the CMake cache interactively with ccmake build (or cmake-gui build) and rebuild with spin develop; your edits persist, because reconfiguration does not use --fresh and environment variables only seed cache entries that are not already set. The corollary: once a variable is in the cache, changing its environment variable no longer affects it — edit the cache directly (or delete build/CMakeCache.txt) to change it. The build type and compiler are the exception; the build always re-applies them.

Docker Image

Using pre-built images

You can also pull a pre-built docker image from Docker Hub and run with docker v23.0+.

bash 
docker run --gpus all --rm -ti --ipc=host pytorch/pytorch:latest
note

PyTorch uses shared memory to share data between processes, so if torch multiprocessing is used (e.g. for multithreaded data loaders) the default shared memory segment size the container runs with is not enough. You should increase shared memory size with either --ipc=host or --shm-size command line options to nvidia-docker run.

Building the image yourself

NOTE: Must be built with a Docker version >= 23.0.

The Dockerfile is supplied to build images with CUDA 12.6 support and cuDNN v9. You can pass PYTHON_VERSION=x.y make variable to specify which Python version Miniconda uses, or leave it unset to use the default, as the Dockerfile uses system Python.

bash 
make -f docker.Makefile
# images are tagged as docker.io/${your_docker_username}/pytorch

You can also pass the CMAKE_VARS="..." environment variable to specify additional CMake variables to be passed to CMake during the build. See cmake/EnvVarForwarding.cmake for the list of available variables.

bash 
make -f docker.Makefile

Building the Documentation

To build documentation in various formats, you will need Sphinx and the pytorch_sphinx_theme2.

Before you build the documentation locally, ensure torch is installed in your environment. For small fixes, you can install the nightly version as described in Getting Started.

For more complex fixes, such as adding a new module and docstrings for the new module, you might need to install torch from source. See Docstring Guidelines for docstring conventions.

bash 
cd docs/
pip install -r requirements.txt
make html
make serve

Run make to get a list of all available output formats. If you get a katex error run npm install katex. If it persists, try npm install -g katex.

note

If you see a numpy incompatibility error, run:

bash 
pip install 'numpy<2'