Gaussian Elimination Algorithm
Gaussian elimination converts a system of linear equations into an equivalent upper triangular form through elementary row operations, then solves by back substitution.
Forward elimination: For each column (pivot column), find the row with the largest absolute value in that column (partial pivoting for numerical stability) and swap it to the pivot position. Then subtract multiples of the pivot row from all rows below to introduce zeros beneath the pivot.
Back substitution: Starting from the last equation (which now has only one unknown), solve for that variable. Substitute upward into each previous equation to find the remaining unknowns.
Example for a 3×3 system: given the augmented matrix [A|b], after two elimination steps the matrix has the upper triangular form:
[ a₁₁ a₁₂ a₁₃ | b₁ ] [ 0 a₂₂' a₂₃'| b₂'] [ 0 0 a₃₃''| b₃'']
Then x₃ = b₃''/a₃₃'', x₂ = (b₂' − a₂₃''·x₃)/a₂₂', x₁ = (b₁ − a₁₂·x₂ − a₁₃·x₃)/a₁₁.
Types of Solutions in Linear Systems
A system of n linear equations in n unknowns has exactly one of three possible outcomes:
- Unique solution: The coefficient matrix has full rank (n pivots). Every variable takes a single definite value. Geometrically, two lines meet at a point (2D) or three planes meet at a single point (3D).
- No solution (inconsistent): Gaussian elimination produces a row [0, 0, …, 0 | c] with c ≠ 0 — a contradiction. In 2D the lines are parallel; in 3D two planes may be parallel or the three planes may form a triangular prism with no common point.
- Infinitely many solutions (dependent): A row becomes all zeros (including the constant), meaning one equation is redundant. The system has at least one free variable and a family of solutions parametrised by that variable. In 2D the two lines coincide; in 3D two or more planes share an entire line or plane.
Partial Pivoting and Numerical Stability
When computing by hand you can choose any non-zero pivot, but computers use partial pivoting: before eliminating column j, swap rows to put the element with the largest absolute value in position (j, j). This prevents dividing by a very small number, which would amplify rounding errors and produce wildly inaccurate results.
Without pivoting, a system like the one below is numerically disastrous for small ε:
εx + y = 1 x + y = 2
Partial pivoting swaps the rows first, making the computation stable regardless of ε. All solutions produced by this solver use partial pivoting.