Python NumPy: Solving LinAlgError: Singular matrix
The numpy.linalg.LinAlgError: Singular matrix is a common error in linear algebra operations using NumPy, particularly when attempting to compute a matrix inverse (np.linalg.inv()) or solve a system of linear equations (np.linalg.solve()). This error indicates that the matrix involved in the operation is "singular," meaning its determinant is zero, and therefore, it does not have a true inverse in standard matrix algebra.
This guide will clearly explain what a singular matrix is, demonstrate common scenarios that lead to this LinAlgError (such as linearly dependent rows/columns or specific matrix constructions), and provide robust solutions, including checking matrix invertibility, understanding problematic operations like multiplying a vector by its transpose, and using the Moore-Penrose pseudo-inverse (np.linalg.pinv()) for non-invertible matrices or np.linalg.lstsq() for solving systems with singular matrices.
Understanding Singular Matrices and Invertibility
What is a Singular Matrix? (Determinant is Zero)
A square matrix is called singular (or degenerate) if its determinant is equal to zero. Some key properties and implications of singular matrices include:
- They do not have a unique multiplicative inverse.
- Their rows (and columns) are linearly dependent. This means at least one row can be expressed as a linear combination of other rows (e.g., one row is a multiple of another, or one row is the sum of two others).
- The system of linear equations
Ax = beither has no solution or infinitely many solutions ifAis singular.
Why Singular Matrices Cannot Be Inverted
The formula for the inverse of a 2x2 matrix A = [[a, b], [c, d]] is (1/det(A)) * [[d, -b], [-c, a]]. If det(A) = 0, this formula involves division by zero, making the inverse undefined. This principle extends to larger matrices.
Reproducing the Error with numpy.linalg.inv()
Attempting to invert a singular matrix using np.linalg.inv() will raise the LinAlgError.
import numpy as np
# A singular matrix (second row is a multiple of the first, or columns are linearly dependent)
singular_matrix = np.array([
[2, 4],
[2, 4] # Row 2 is 1 * Row 1
])
print("Singular Matrix:")
print(singular_matrix)
print(f"Determinant: {np.linalg.det(singular_matrix)}")
try:
# ⛔️ numpy.linalg.LinAlgError: Singular matrix
inverse_matrix_error = np.linalg.inv(singular_matrix)
print(inverse_matrix_error)
except np.linalg.LinAlgError as e:
print(f"Error: {e}")
Output:
Singular Matrix:
[[2 4]
[2 4]]
Determinant: 0.0
Error: Singular matrix
Solution 1: Ensure the Matrix is Invertible
The most direct solution is to ensure the matrix you are trying to invert is indeed invertible (non-singular).
Checking the Determinant with numpy.linalg.det()
A non-singular matrix has a non-zero determinant.
import numpy as np
# Example of an invertible matrix
invertible_matrix = np.array([
[1, 2],
[3, 5] # Rows/columns are linearly independent
])
determinant = np.linalg.det(invertible_matrix)
print(f"Invertible Matrix:\n{invertible_matrix}")
print(f"Determinant: {determinant}")
Output:
Invertible Matrix:
[[1 2]
[3 5]]
Determinant: -1.0000000000000004
Due to floating-point precision, det might be very close to 0, like -2.0000000000000004, which is effectively non-zero.
Example of an Invertible Matrix
If the matrix is invertible, np.linalg.inv() will succeed.
import numpy as np
# invertible_matrix from above
invertible_matrix = np.array([
[1, 2],
[3, 5] # Rows/columns are linearly independent
])
# ✅ Calculate the inverse
inverse_matrix_correct = np.linalg.inv(invertible_matrix)
print("Inverse of the invertible matrix:")
print(inverse_matrix_correct)
print()
# Verify: A @ A_inv should be close to the identity matrix
identity_check = np.dot(invertible_matrix, inverse_matrix_correct)
print("Product of matrix and its inverse (should be close to identity):")
print(np.round(identity_check, decimals=5)) # Round to handle floating point inaccuracies
Output:
Inverse of the invertible matrix:
[[-5. 2.]
[ 3. -1.]]
Product of matrix and its inverse (should be close to identity):
[[1. 0.]
[0. 1.]]
Common Pitfall: Creating a Singular Matrix by Multiplying a 1D Vector by its Transpose
A common way to inadvertently create a singular matrix is by performing an outer product of a 1D vector with itself (i.e., column_vector @ row_vector_transpose or v[:, np.newaxis] @ v[np.newaxis, :]). The resulting matrix will have rank 1, meaning its rows/columns are linearly dependent.
import numpy as np
vector_1d = np.array([1, 2, 3, 4])
# Create a column vector and a row vector for outer product
column_vec = vector_1d[:, np.newaxis] # Shape (4, 1)
row_vec = vector_1d[np.newaxis, :] # Shape (1, 4)
# Outer product results in a matrix where rows/columns are multiples of each other
outer_product_matrix = column_vec @ row_vec # Or np.outer(vector_1d, vector_1d)
print("Vector (1D):", vector_1d)
print("Outer product matrix (rank 1, singular):")
print(outer_product_matrix)
print(f"Determinant of outer product matrix: {np.linalg.det(outer_product_matrix)}")
try:
# ⛔️ numpy.linalg.LinAlgError: Singular matrix
np.linalg.inv(outer_product_matrix)
except np.linalg.LinAlgError as e:
print(f"Error inverting outer product: {e}")
Output:
Vector (1D): [1 2 3 4]
Outer product matrix (rank 1, singular):
[[ 1 2 3 4]
[ 2 4 6 8]
[ 3 6 9 12]
[ 4 8 12 16]]
Determinant of outer product matrix: 0.0
Error inverting outer product: Singular matrix
Each row in outer_product_matrix is a scalar multiple of vector_1d. Such matrices are always singular (if dimension > 1).
Solution 2: Using the Moore-Penrose Pseudo-Inverse (numpy.linalg.pinv())
If your matrix is singular or non-square but you still need a form of "inverse" for solving least-squares problems or other specific applications, NumPy provides the Moore-Penrose pseudo-inverse via np.linalg.pinv(). This function computes a generalized inverse even for singular matrices.
import numpy as np
# singular_matrix
singular_matrix = np.array([
[2, 4],
[2, 4] # Row 2 is 1 * Row 1
])
# ✅ Compute the pseudo-inverse
pseudo_inverse = np.linalg.pinv(singular_matrix)
print("Singular Matrix:")
print(singular_matrix)
print("Pseudo-inverse using np.linalg.pinv():")
print(pseudo_inverse)
Output:
Singular Matrix:
[[2 4]
[2 4]]
Pseudo-inverse using np.linalg.pinv():
[[0.05 0.05]
[0.1 0.1 ]]
The pseudo-inverse has useful properties, but it's not the same as the true inverse (which doesn't exist for singular matrices). Understand its mathematical implications before using it as a direct replacement for np.linalg.inv().
Scenario 2: Error with numpy.linalg.solve()
The numpy.linalg.solve(a, b) function solves the linear matrix equation a @ x = b for x. It requires the coefficient matrix a to be square and non-singular (invertible).
Reproducing the Error
import numpy as np
# Singular coefficient matrix 'a'
a_singular = np.array([
[2, 4],
[2, 4]
])
b_vector = np.array([10, 20]) # Ordinates
try:
# ⛔️ numpy.linalg.LinAlgError: Singular matrix
x_solution_error = np.linalg.solve(a_singular, b_vector)
print(x_solution_error)
except np.linalg.LinAlgError as e:
print(f"Error with np.linalg.solve: {e}")
Output:
Error with np.linalg.solve: Singular matrix
Solution: Use an Invertible Coefficient Matrix
If a is invertible, solve() will work.
import numpy as np
# invertible_matrix
invertible_matrix = np.array([
[1, 2],
[3, 5] # Rows/columns are linearly independent
])
b_vector_solve = np.array([5, 11]) # Example b
# ✅ Solve with an invertible matrix 'a'
x_solution_correct = np.linalg.solve(invertible_matrix, b_vector_solve)
print("Solution x for Ax=b with invertible A:")
print(x_solution_correct)
# Re-check with the example from the original article:
a_orig_solve = np.array([[1,2],[3,4]])
b_orig_solve = np.array([1,2])
x_orig_solve = np.linalg.solve(a_orig_solve, b_orig_solve)
print(f"Solution for solve example: {x_orig_solve}")
Output:
Solution x for Ax=b with invertible A:
[-3. 4.]
Solution for solve example: [0. 0.5]
Alternative: Using numpy.linalg.lstsq() for Least-Squares Solution
If your matrix a is singular or not square, np.linalg.solve() will fail. In such cases, numpy.linalg.lstsq(a, b) can be used to find the least-squares solution to a @ x = b. This finds an x that minimizes the Euclidean 2-norm || b - a @ x ||^2.
import numpy as np
# a_singular and b_vector
a_singular = np.array([
[2, 4],
[2, 4]
])
b_vector = np.array([10, 20]) # Ordinates
# ✅ Use np.linalg.lstsq for singular or non-square 'a'
# rcond=None silences a future warning about rcond default value change.
solution_tuple = np.linalg.lstsq(a_singular, b_vector, rcond=None)
x_least_squares = solution_tuple[0] # The solution vector is the first element of the returned tuple
print("Least-squares solution x using np.linalg.lstsq:")
print(x_least_squares)
print()
b_vector_article = np.array([1,2])
solution_tuple_article = np.linalg.lstsq(a_singular, b_vector_article, rcond=None)
x_least_squares_article = solution_tuple_article[0]
print(f"Least-squares for article's b: {x_least_squares_article}")
Output:
Least-squares solution x using np.linalg.lstsq:
[1.5 3. ]
Least-squares for article's b: [0.15 0.3 ]
np.linalg.lstsq returns a tuple containing the solution, residuals, rank, and singular values.
Conclusion
The numpy.linalg.LinAlgError: Singular matrix error is a fundamental indication in NumPy that a matrix lacks a true inverse, typically because its determinant is zero due to linearly dependent rows or columns.
- When using
np.linalg.inv(): Ensure your matrix is non-singular (has a non-zero determinant). Check for common pitfalls like outer products of 1D vectors. - If you must work with a singular matrix and need an "inverse-like" quantity,
np.linalg.pinv()computes the Moore-Penrose pseudo-inverse. - When using
np.linalg.solve(a, b): The coefficient matrixamust be square and non-singular. - If
ainsolveis singular or non-square, usenp.linalg.lstsq(a, b)to find a least-squares solution.
Understanding matrix properties and choosing the appropriate NumPy linear algebra function is key to avoiding this error and obtaining meaningful results.