Large-scale foundation models have changed how we build and operate ML systems. This post covers how ML infrastructure and MLOps practices evolved to support them—contrasting the classic era with modern paradigms, and examining the new patterns and workflows that emerged.
Today's LLMs don't fit on a single GPU. A 70B-parameter model needs ~140GB just for weights in FP16 -- nearly 2x what an A100 can hold. Training or serving these models means distributing work across multiple GPUs, and doing it wrong wastes most of your compute budget.
This guide covers practical strategies for scaling LLMs across multiple GPUs and nodes, drawing from Hugging Face's Ultra-Scale Playbook.
My 10-step process for turning a fresh macOS install into an AI engineering workstation.
# Install uv
brew install uv
# For new projects (modern workflow)
uv init # create project structure
uv add pandas numpy # add dependencies
uv run train.py # run your script
# For existing projects (legacy workflow)
uv venv # create virtual environment
uv pip install -r requirements.txt # install dependencies
uv run train.py # run your script
# Run tools without installing them
uvx ruff check . # run linter
uvx black . # run formatter