Day 26: Tensors: the PyTorch mental model, CPU vs GPU
From scalar Values to tensors
micrograd operated on single numbers. Real networks operate on tensors — n-dimensional arrays (Day 3's NumPy model, now GPU-capable and gradient-tracking). A scalar is a 0-D tensor, a vector 1-D, a matrix 2-D, a batch of RGB images 4-D (batch × channels × height × width). PyTorch tensors are NumPy arrays with two superpowers: they run on the GPU, and they record a computation graph for autograd.
import torch
x = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
x.shape # torch.Size([2, 2])
x.dtype # torch.float32
# a batch of 32 RGB 224x224 images
batch = torch.randn(32, 3, 224, 224)
# move computation to GPU if available
device = "cuda" if torch.cuda.is_available() else "cpu"
batch = batch.to(device)Why the GPU matters — and the ladder returns
A GPU has thousands of small cores built for the exact matrix multiplications neural nets are made of, doing in parallel what a CPU does sequentially. This is Stage 0 Day 1's storage/compute ladder again: keep data on the GPU, minimize CPU↔GPU transfers (H2D/D2H copies), because those copies are the slow rung. Stage 6A's Nsight profiling is largely about finding transfers that shouldn't be there.
The shape-mismatch tax
The overwhelming majority of PyTorch bugs are shape mismatches — a (32, 10) where a (10, 32) was expected. Get in the habit *now* of printing .shape at every step and reasoning about dimensions before running. Broadcasting (Day 4) applies to tensors exactly as it did to NumPy arrays.
Key terms
- Tensor
- An n-dimensional array, PyTorch's core data type — like a NumPy array but GPU-capable and autograd-tracked.
- Device
- Where a tensor lives and computes: 'cpu' or 'cuda' (GPU). Operations require all tensors on the same device.
- H2D / D2H copy
- Host-to-device / device-to-host memory transfer between CPU and GPU — a common performance bottleneck.
What are the two capabilities a PyTorch tensor has that a plain NumPy array does not?
