pixi manifest and install command

[uralik]

Description

This PR adds a non-breaking option of installing the entire env using pixi.

Why pixi?

• It uses uv for speed and has access to conda forge package for system dependencies needed here (that we used to get with conda).
• Entire env installation on rather slow filesystem takes 1-2 minutes with pixi.
• Manifest allows for very flexible tasking e.g. we can add a task for installing a local, editable titan that users can opt for if they need to tinker with the trainer.

Test plan

I have done testing on my end by building the env on fair infra cloud cluster:

pixi run install                                                                     
✨ Pixi task (install-pytorch in default):     uv pip install torch==2.9.0 --index-url https://download.pytorch.org/whl/cu128                  
Using Python 3.10.19 environment at: .pixi/envs/default                                                                                        
Resolved 25 packages in 405ms                                                                                                                  
Installed 25 packages in 13.33s                                                                                                                
 + filelock==3.20.0                                                                                                                            
 + fsspec==2025.12.0                                                                                                                           
 + jinja2==3.1.6                                                                                                                               
 + markupsafe==2.1.5                                                                                                                           
 + mpmath==1.3.0                                                                                                                               
 + networkx==3.4.2                                                                                                                             
 + nvidia-cublas-cu12==12.8.4.1                                                                                                                
 + nvidia-cuda-cupti-cu12==12.8.90                                                                                                             
 + nvidia-cuda-nvrtc-cu12==12.8.93                                                                                                             
 + nvidia-cuda-runtime-cu12==12.8.90                                                                                                           
 + nvidia-cudnn-cu12==9.10.2.21                                                                                                                
 + nvidia-cufft-cu12==11.3.3.83                                                                                                                
 + nvidia-cufile-cu12==1.13.1.3                                                                                                                
 + nvidia-curand-cu12==10.3.9.90                                                                                                               
 + nvidia-cusolver-cu12==11.7.3.90                                                                                                             
 + nvidia-cusparse-cu12==12.5.8.93                                                                                                             
 + nvidia-cusparselt-cu12==0.7.1                                                                                                               
 + nvidia-nccl-cu12==2.27.5                                                                                                                    
 + nvidia-nvjitlink-cu12==12.8.93                                                                                                              
 + nvidia-nvshmem-cu12==3.3.20                                                                                                                 
 + nvidia-nvtx-cu12==12.8.90                                                                                                                   
 + sympy==1.14.0                                                                                                                               
 + torch==2.9.0+cu128                                                                                                                          
 + triton==3.5.0                                                                                                                               
 + typing-extensions==4.15.0

✨ Pixi task (install-vllm-reqs in default):     uv pip install -r .github/packaging/vllm_reqs_12_8.txt                                        
Using Python 3.10.19 environment at: .pixi/envs/default                                                                                        
Resolved 139 packages in 796ms                                                                                                                 
Prepared 1 package in 1.63s                                                                                                                    
Uninstalled 3 packages in 400ms                                                                                                                
Installed 117 packages in 4.00s                                                                                                                
 + aiohappyeyeballs==2.6.1                                                                                                                     
 + aiohttp==3.13.1                                                                                                                             
 + aiosignal==1.4.0                                                                                                                            
 + annotated-types==0.7.0                                                                                                                      
 + anyio==4.11.0                                                                                                                               
 + astor==0.8.1                                                                                                                                
 + async-timeout==5.0.1                                                                                                                        
 + attrs==25.4.0                                                                                                                               
 + blake3==1.0.8                                                                                                                               
 + cachetools==6.2.1                                                                                                                           
 + cbor2==5.7.0                                                                                                                                
 + certifi==2025.10.5                                                                                                                          
 + cffi==2.0.0                                                                                                                                 
 + charset-normalizer==3.4.4                                                                                                                   
 + click==8.2.1                                                                                                                                
 + cloudpickle==3.1.1                                                                                                                          
 + cmake==4.1.0                                                                                                                                
 + compressed-tensors==0.10.2
 + cupy-cuda12x==13.6.0
 + depyf==0.19.0
 + dill==0.4.0
 + diskcache==5.6.3
 + distro==1.9.0
 + dnspython==2.8.0
 + einops==0.8.1
 + email-validator==2.3.0
 + exceptiongroup==1.3.0
 + fastapi==0.119.1
 + fastapi-cli==0.0.14
 + fastapi-cloud-cli==0.3.1
 + fastrlock==0.8.3
 - filelock==3.20.0
 + filelock==3.19.1
 + frozenlist==1.8.0
 - fsspec==2025.12.0
 + fsspec==2025.9.0
 + gguf==0.17.1
 + h11==0.16.0
 + hf-xet==1.1.10
 + httpcore==1.0.9
 + httptools==0.7.1
 + httpx==0.28.1
 + huggingface-hub==0.35.3
 + idna==3.11
 + interegular==0.3.3
 + jiter==0.11.1
 + jsonschema==4.25.1
 + jsonschema-specifications==2025.9.1
 + lark==1.2.2
 + llguidance==0.7.30
 + llvmlite==0.44.0
 + lm-format-enforcer==0.10.12
 + markdown-it-py==4.0.0
 + mdurl==0.1.2
 + mistral-common==1.8.5
 + msgpack==1.1.2
 + msgspec==0.19.0
 + multidict==6.7.0
 - networkx==3.4.2
 + networkx==3.3
 + ninja==1.13.0
 + numba==0.61.2
 + numpy==2.2.6
 + openai==1.90.0
 + opencv-python-headless==4.12.0.88
 + outlines-core==0.2.10
 + packaging==25.0
 + partial-json-parser==0.2.1.1.post6
 + pillow==12.0.0
 + prometheus-client==0.23.1
 + prometheus-fastapi-instrumentator==7.1.0
 + propcache==0.4.1
+ protobuf==6.33.0                                                                                                                            
 + psutil==7.1.1                                                                                                                               
 + py-cpuinfo==9.0.0                                                                                                                           
 + pybase64==1.4.2                                                                                                                             
 + pycountry==24.6.1                                                                                                                           
 + pycparser==2.23                                                                                                                             
 + pydantic==2.12.3                                                                                                                            
 + pydantic-core==2.41.4                                                                                                                       
 + pydantic-extra-types==2.10.6                                                                                                                
 + pygments==2.19.2                                                                                                                            
 + python-dotenv==1.1.1                                                                                                                        
 + python-json-logger==4.0.0                                                                                                                   
 + python-multipart==0.0.20                                                                                                                    
 + pyyaml==6.0.3                                                                                                                               
 + pyzmq==27.1.0                                                                                                                               
 + ray==2.50.1                                                                                                                                 
 + referencing==0.37.0                                                                                                                         
 + regex==2025.10.23                                                                                                                           
 + requests==2.32.5                                                                                                                            
 + rich==14.2.0                                                                                                                                
 + rich-toolkit==0.15.1                                                                                                                        
 + rignore==0.7.1                                                                                                                              
 + rpds-py==0.27.1                                                                                                                             
 + safetensors==0.6.2                                                                                                                          
 + scipy==1.15.3                                                                                                                               
 + sentencepiece==0.2.1                                                                                                                        
 + sentry-sdk==2.42.1                                                                                                                          
 + setuptools==80.10.1                                                                                                                         
 + setuptools-scm==9.2.2                                                                                                                       
 + shellingham==1.5.4                                                                                                                          
 + sniffio==1.3.1                                                                                                                              
 + soundfile==0.13.1                                                                                                                           
 + soxr==1.0.0                                                                                                                                 
 + starlette==0.48.0                                                                                                                           
 + tiktoken==0.12.0                                                                                                                            
 + tokenizers==0.22.1                                                                                                                          
 + tomli==2.3.0                                                                                                                                
 + tqdm==4.67.1                                                                                                                                
 + transformers==4.57.1                                                                                                                        
 + typer==0.20.0                                                                                                                               
 + typing-inspection==0.4.2                                                                                                                    
 + urllib3==2.5.0                                                                                                                              
 + uvicorn==0.38.0                                                                                                                             
 + uvloop==0.22.1                                                                                                                              
 + watchfiles==1.1.1                                                                                                                           
 + websockets==15.0.1                                                                                                                          
 + xgrammar==0.1.21                                                                                                                            
 + yarl==1.22.0

✨ Pixi task (install-vllm-deps in default):     uv pip install six && uv pip install "setuptools<80"                                          
Using Python 3.10.19 environment at: .pixi/envs/default                                                                                        
Resolved 1 package in 82ms                                                                                                                     
Installed 1 package in 26ms                                                                                                                    
 + six==1.17.0                                                                                                                                 
Using Python 3.10.19 environment at: .pixi/envs/default                                                                                        
Resolved 1 package in 81ms                                                                                                                     
Uninstalled 1 package in 327ms
Installed 1 package in 707ms
 - setuptools==80.10.1
 + setuptools==79.0.1

✨ Pixi task (install-vllm in default):     uv pip install vllm --no-cache-dir --index-url https://download.pytorch.org/whl/preview/forge      Using Python 3.10.19 environment at: .pixi/envs/default
Resolved 136 packages in 1.00s
Prepared 1 package in 30.81s
░░░░░░░░░░░░░░░░░░░░ [0/1] Installing wheels...                                                                                                warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance.
         If the cache and target directories are on different filesystems, hardlinking may not be supported.
         If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning.
Installed 1 package in 48.22s
 + vllm==0.10.1.dev0+g6d8d0a24c.d20260122

✨ Pixi task (install-torchstore in default):     uv pip install "git+https://github.com/meta-pytorch/torchstore.git@no-monarch-2025.12.17"    Using Python 3.10.19 environment at: .pixi/envs/default
Resolved 27 packages in 613ms
Installed 2 packages in 59ms
 + pygtrie==2.5.0
 + torchstore==0.0.0.dev0 (from git+https://github.com/meta-pytorch/torchstore.git@b28fcf4505797f3e15120e24c1895c0169acbd1f)

✨ Pixi task (install-forge in default):     uv pip install -e ".[dev]"                                                                        Using Python 3.10.19 environment at: .pixi/envs/default
    Updated https://github.com/meta-pytorch/torchstore.git (64e198590678df658feeeb8551736e0bbfa0e81d)
Resolved 195 packages in 4.15s
      Built forge @ file:///storage/home/kulikov/ram_coi/torchforge
      Built torchstore @ git+https://github.com/meta-pytorch/torchstore.git@64e198590678df658feeeb8551736e0bbfa0e81d
Prepared 5 packages in 2.85s
Uninstalled 1 package in 21ms
Installed 55 packages in 1.78s
 + absl-py==2.3.1
 + backports-asyncio-runner==1.2.0
 + cfgv==3.5.0
 + click-option-group==0.5.9
 + clusterscope==0.0.32
 + coverage==7.13.1
 + datasets==4.5.0
 + distlib==0.4.0
 + docker==7.1.0
 + docstring-parser==0.17.0
 + expecttest==0.3.0
 + forge==0.0.0 (from file:///storage/home/kulikov/ram_coi/torchforge)
 + gitdb==4.0.12
 + gitpython==3.1.46
 + grpcio==1.78.0rc2
 + hf-transfer==0.1.9
 + identify==2.6.16
 + importlib-metadata==8.7.1
 + iniconfig==2.3.0
 + markdown==3.10.1
 + multiprocess==0.70.18
 + mypy-extensions==1.1.0
 + nodeenv==1.10.0
 + omegaconf==2.4.0.dev4
 + opentelemetry-api==1.39.1
 + pandas==2.3.3
 + platformdirs==4.5.1
 + pluggy==1.6.0
 + pre-commit==4.5.1
 + pyarrow==23.0.0
 + pyre-extensions==0.0.32
 + pytest==9.0.2
 + pytest-asyncio==1.3.0
 + pytest-cov==7.0.0
 + pytest-timeout==2.4.0
 + python-dateutil==2.9.0.post0
 + pytz==2025.2
 + smmap==5.0.2
 + tabulate==0.9.0
 + tensorboard==2.20.0
 + tensorboard-data-server==0.7.2
 + torchdata==0.11.0
 + torchmonarch-nightly==2025.12.17
 - torchstore==0.0.0.dev0 (from git+https://github.com/meta-pytorch/torchstore.git@b28fcf4505797f3e15120e24c1895c0169acbd1f)
 + torchstore==0.0.0.dev0 (from git+https://github.com/meta-pytorch/torchstore.git@64e198590678df658feeeb8551736e0bbfa0e81d)
 + torchtitan==0.2.0
 + torchx-nightly==2026.1.22
 + typeguard==4.4.4
 + typing-inspect==0.9.0
 + tyro==1.0.5
 + tzdata==2025.3
 + virtualenv==20.35.4
 + wandb==0.24.0
 + werkzeug==3.1.5
 + xxhash==3.6.0
 + zipp==3.23.0

[allenwang28]

super cool, thanks @uralik !