Confusion Matrix: Read Results and Metrics

Choosing an evaluation metric starts with understanding which predictions were correct and which classes were confused. A confusion matrix organizes those results by actual and predicted class.

I’ll explain how class order shapes the matrix and how error costs guide a choice between precision and recall.

TL;DR

A confusion matrix counts the true class against the predicted class, so each cell shows where a classifier was right or wrong.

  • Read rows as actual labels and columns as predicted labels in scikit-learn.
  • Precision measures predicted positives that were correct. Recall measures actual positives that were found.
  • Use explicit label order when turning matrix positions into TP, FP, FN, and TN.

What Is a Confusion Matrix?

In scikit-learn, rows hold actual classes and columns hold predictions. Cell C[i,j] counts samples known to be class i and assigned to class j.

I print both axis labels because a square table alone does not show its direction.

Choose the positive class before calculating binary metrics. In this example, Yes is positive and No is negative, so the four cells mean:

Actual classPredicted NoPredicted Yes
NoTrue negative (TN): 3False positive (FP): 2
YesFalse negative (FN): 1True positive (TP): 4

Positive and negative name the chosen classes. A false positive is an actual No assigned to Yes, while a false negative is an actual Yes assigned to No.

An email filter can choose which error to reduce based on whether hiding legitimate mail or letting spam through costs more.

How to Read a Confusion Matrix Step by Step

Start with the class order and axis labels, then read the diagonal and off-diagonal counts before choosing a summary metric. The example uses ten labeled cases so the counts and rates can be checked by hand.

Step 1: Make the class order explicit

Scikit-learn sorts observed labels when the labels argument is omitted. I set No then Yes because the example treats Yes as positive. The function receives actual labels before predicted labels.

Step 2: Build the count matrix with scikit-learn

Install scikit-learn in the Python environment with this command, save the program as confusion_matrix_demo.py, and run the command shown in the output below. The code creates raw counts and the rates for each actual class.

.venv/bin/python -m pip install scikit-learn
from sklearn.metrics import confusion_matrix

actual = ["No", "No", "No", "No", "No", "Yes", "Yes", "Yes", "Yes", "Yes"]
predicted = ["No", "No", "Yes", "No", "Yes", "No", "Yes", "Yes", "Yes", "Yes"]
labels = ["No", "Yes"]

counts = confusion_matrix(actual, predicted, labels=labels)
row_rates = confusion_matrix(actual, predicted, labels=labels, normalize="true")
tn, fp, fn, tp = counts.ravel()

print("Rows: actual No, Yes; columns: predicted No, Yes")
print(counts)
print("Row-normalized rates:")
print(row_rates)
print(f"Accuracy: {(tp + tn) / counts.sum():.2f}")
print(f"Yes precision: {tp / (tp + fp):.2f}")
print(f"Yes recall: {tp / (tp + fn):.2f}")
ninad@codeforgeek:~$ .venv/bin/python confusion_matrix_demo.py
Rows: actual No, Yes; columns: predicted No, Yes
[[3 2]
 [1 4]]
Row-normalized rates:
[[0.6 0.4]
 [0.2 0.8]]
Accuracy: 0.70
Yes precision: 0.67
Yes recall: 0.80
Terminal output with the No and Yes confusion matrix, row-normalized rates, accuracy, precision and recall
The terminal prints the example’s raw counts beside its row-normalized rates.

I ran the script and got three true negatives, two false positives, one false negative, and four true positives. The output reports 0.70 accuracy, 0.67 precision for Yes, and 0.80 recall for Yes, matching the counts in the matrix.

Step 3: Choose the metric that matches the error cost

The matrix supplies counts for metrics that answer different questions.

MetricFormulaQuestion
Accuracy(TP + TN) / (TP + TN + FP + FN)What fraction of all predictions was correct?
PrecisionTP / (TP + FP)Of the predicted positives, how many were correct?
Recall (sensitivity)TP / (TP + FN)Of the actual positives, how many were found?
SpecificityTN / (TN + FP)Of the actual negatives, how many were correctly rejected?
F1 score2 × precision × recall / (precision + recall)What is the harmonic mean of precision and recall?

In this example, precision is 4 / (4 + 2), or 0.67, because six cases were predicted Yes and four were correct. Recall is 4 / (4 + 1), or 0.80, because the five actual Yes cases include four found by the classifier. The F1 score is 0.73 and combines the two positive-class rates, leaving true negatives out.

Choose precision when false alarms carry a cost, and choose recall when missed positives matter more. F1 summarizes both rates but does not encode task-specific cost.

On a test set with 95 No cases and five Yes cases, predicting No every time gives 95% accuracy and 0% Yes recall.

Step 4: Compare counts with normalized rates

I compared the count output with normalize=”true” in the same script. The No row has rates of 0.60 correct and 0.40 mistaken, while the Yes row has 0.80 correct and 0.20 missed.

Normalize optionWhat it divides byWhat sums to one
trueEach actual-class rowEach row
predEach predicted-class columnEach column
allThe total sample countThe full matrix

Use counts when sample totals matter, and use rates to compare classes with different support. Keep both because a rate of 80% can represent four of five cases or four hundred of five hundred.

Edge Cases and Next Checks

A correct reading depends on valid labels, a visible class order, and the distribution of the evaluation set. I ran .venv/bin/python confusion_matrix_empty_labels.py, which passes labels=[] to confusion_matrix. It returned ValueError: ‘labels’ should contain at least one label.

Scikit-learn ValueError when confusion_matrix receives an empty labels list
The traceback points to the empty labels argument.

Include every class of interest even when no current prediction uses it, so the matrix retains a zero count for that class.

CaseWhat it meansNext check
Only one class appearsThe default matrix may be one by one.Pass the full label list to retain the expected shape.
No predicted positivesPrecision has a zero denominator.Review scikit-learn’s zero_division policy and report the support.
Uneven class countsAccuracy can be high while a small class is missed.Inspect per-class recall and the row totals.
Numeric target valuesThe task is regression rather than classification.Use an error metric such as MAE or RMSE instead.

For multiple classes, an actual Dog predicted as Cat appears in row Dog and column Cat, making a miss for Dog and a false alarm for Cat.

Per-class countWhere to find it for the target class
True positiveThe class’s diagonal cell
False positiveOther entries in its predicted-class column
False negativeOther entries in its actual-class row
True negativeCells outside that row and column

Multi-label tasks allow one sample to carry several classes, so use scikit-learn’s multilabel_confusion_matrix rather than one N-by-N table. It can return one binary matrix per class or per sample.

Scikit-learn’s classification_report produces a per-class summary. Its support field gives the number of actual examples represented by each class.

AverageHow classes contribute
MacroEach class receives equal weight
WeightedEach class is weighted by its support
MicroCounts are pooled before the metric is calculated

When a rare class carries a different cost, report its per-class score beside the chosen average.

Every confusion matrix reflects one chosen threshold. Changing a probability cutoff can raise recall while producing more false positives, even when probability scores stay fixed. Tune that cutoff on validation data, not the final test set, to avoid overfitting.

A confusion matrix evaluates classification labels rather than continuous regression targets. Keep the test set untouched until metric and threshold choices are complete.

Conclusion

Keep the class definition with the reported matrix, and retain the test split for later comparisons.

The scikit-learn confusion_matrix API documents label ordering and normalization. Its threshold guide explains how changing a decision cutoff changes predicted labels. For the next evaluation task, compare validation results with CodeForGeek’s cross-validation guide, then check metric definitions with its AI and machine-learning questions.

FAQ

These answers cover the label, metric, and task boundaries that come up when reading a confusion matrix.

Can a confusion matrix be used for regression?

A confusion matrix compares actual and predicted class labels, so it applies to classification. For continuous regression targets, use a metric such as mean absolute error or root mean squared error.

What does a confusion matrix tell you?

It counts correct and incorrect predictions by actual and predicted class. The diagonal shows correct counts, while off-diagonal cells show which classes were confused.

How do I choose between precision and recall?

Use precision when false positives are costly, and recall when false negatives are costly. Check the error counts and your task’s costs before relying on either score.

Ninad Pathak
Ninad Pathak

Ninad Pathak is a founding member and Editor in Chief at CodeForGeek. He writes about practical AI tooling and developer workflows, and outside work you will likely find him with a book, hot brewed coffee, and ambient music.

Articles: 80