Shaders and GPU Programming

Shaders are programs that run in parallel on the GPU, designed to determine the color of each pixel in a scene. They are a prime example of how constraints drive creativity: each shader instance only knows its x and y position, yet people create high-fidelity graphics using math alone.

Source: Making Software

GPU vs CPU Architecture

CPUs minimize latency: a small number of powerful cores at fast clock speeds, optimized for processing a single instruction stream as quickly as possible. Modern multi-core CPUs: ~100 billion instructions/second.

GPUs maximize throughput: thousands of small, simple, relatively slow cores. Any single instruction is slower than on a CPU, but the sheer parallelism means tens of trillions of instructions per second.

GPU cores are arranged into compute units (or streaming multiprocessors / SMs) that dispatch work. These cores excel at specific operations like matrix multiplication and trigonometric functions.

Key constraints of GPU parallelism:

• Shader instances can't depend on each other's results
• No data passing between instances (but data can be passed down to all instances via uniforms)
• Cores are memory-less between tasks: no guarantee that consecutive work items are related

The Graphics Pipeline

Three main stages:

1. Vertex Shading

Runs once per vertex. Transforms vertex positions: rotation, scaling, translation. Uses matrix multiplication with uniforms (model matrix, view matrix, projection matrix) to convert from model space to world space to view space to screen space.

Varyings: Data passed from vertex shader to fragment shader gets interpolated across fragments. This is how vertex colors create smooth gradients and normals enable smooth lighting on curved surfaces.

2. Rasterization

Non-programmable step. Takes transformed triangles and determines which pixels they cover. For each covered pixel, generates a fragment containing all data the fragment shader needs (uniforms, textures, interpolated varyings, depth).

Depth testing: Checks the z-buffer to skip fragments hidden behind closer objects.
Anti-aliasing (MSAA): Samples multiple points within edge pixels to blend colors at triangle boundaries.

3. Fragment Shading

Runs once per fragment. Determines final pixel color using position, uniforms (resolution, time, light positions), textures, and interpolated varyings.

Example (gradients): Divide fragment position by resolution. Use the x component as red for a horizontal gradient. Use y for a vertical gradient. Add sin(time) for animated color cycling.

Phong Lighting Model (three components):

• Ambient: Uniform minimum light (constant × light color × object color)
• Diffuse: Brightness based on angle between surface normal and light direction (dot product)
• Specular: Reflective highlights based on angle between reflection direction and viewer direction, raised to a shininess power

Shader Languages and APIs

In practice, most people write shaders in high-level frameworks like Three.js which abstract away the API layer.

Compute Shaders

Run outside the graphics pipeline but can interact with it. Useful for GPU-accelerated computation that feeds back into rendering, e.g., physics simulations for particle systems where each particle's position is updated in parallel, then fed to the vertex shader.

CUDA extends this further: NVIDIA's compute API for running general-purpose computation on GPUs, widely used for AI/ML workloads. See llm-quantization for examples of custom CUDA kernels (cuTile) for LLM inference optimization.

Creative Coding

Beyond traditional game engines, shaders and GPU programming power creative coding: using code to produce visual art, simulations, and interactive experiences. Dhruv's viral Python farming simulation ("forget leetcode, I'm farming food with python," 274K views) is an example of creative coding applied to game-like simulations, where code produces visual output that is both technical and expressive.

Source: Dhruv (@haildhruv)

Related Topics

• llm-quantization - TurboQuant's custom CUDA kernels for GPU-optimized inference