Linear Regression Model for Data Points

A set of measurements can show how one value changes as another changes, and a fitted line can help estimate the outcome. A linear regression model fits coefficients for a line that estimates a numeric target from input features.

I’ll calculate the slope and intercept, then use residuals and held-out error to assess the fit.

TL;DR

Linear regression fits coefficients for a line that estimates a numeric target from one or more input features.

• For one feature, write the model as y = b + wx.
• The slope w is the fitted change in y for one unit of x.
• Residuals show observed minus predicted values, so inspect them rather than trusting a score alone.

What Is a Linear Regression Model?

A linear regression model estimates a numeric outcome from input data by fitting a linear equation.

For a single feature, the equation is ŷ = b + wx, where x is the input, ŷ is the prediction, b is the intercept, and w is the slope. This form makes the role of each fitted coefficient clear.

A training example pairs one input with its measured outcome. The fitting procedure chooses coefficients to reduce prediction errors under a chosen loss function.

These pairs show association. A causal claim requires a separate study design.

Multiple inputs add a weighted term for each feature: ŷ = b + w₁x₁ + w₂x₂ + … + wₖxₖ. Each weight describes the model’s contribution for that feature while other inputs are held fixed.

The intercept can be mathematically required even when x = 0 has no practical meaning in the data.

The one-feature equation makes each coefficient visible. With multiple inputs, each feature contributes its weighted term to the prediction.

How to Fit a Linear Regression Model to Data Points

For a small dataset, ordinary least squares chooses the slope and intercept that minimize the sum of squared residuals. The four pairs below let you see the calculation and check each resulting prediction without installing a package.

Step 1: Set up paired observations

Each row must pair one input with the target measured for that same case. The example uses four simple pairs. They are deliberately small so the arithmetic and output can be checked by hand.

Keep the units attached to the measurements. If x is measured in hours and y in dollars, the fitted slope has units of dollars per hour. A row with a missing or mismatched target does not form a valid pair and needs a deliberate data-cleaning decision before fitting.

Step 2: Calculate the slope and intercept

Ordinary least squares computes the slope as the sum of (x − mean x)(y − mean y), divided by the sum of (x − mean x)². The intercept is mean y − slope × mean x. The denominator is zero if every input value is identical, so a slope cannot be estimated from those observations.

# linear_fit_demo.py
points = [(1, 2), (2, 3), (3, 5), (4, 4)]
x_mean = sum(x for x, _ in points) / len(points)
y_mean = sum(y for _, y in points) / len(points)
slope = sum((x-x_mean)*(y-y_mean) for x,y in points) / sum((x-x_mean)**2 for x,_ in points)
intercept = y_mean - slope*x_mean
print(f"slope: {slope:.2f}")
print(f"intercept: {intercept:.2f}")
for x, observed in points:
    predicted = intercept + slope*x
    print(f"x={x}, observed={observed}, predicted={predicted:.2f}, residual={observed-predicted:.2f}")

slope: 0.80
intercept: 1.50
x=1, observed=2, predicted=2.30, residual=-0.30
x=2, observed=3, predicted=3.10, residual=-0.10
x=3, observed=5, predicted=3.90, residual=1.10
x=4, observed=4, predicted=4.70, residual=-0.70

The fitted equation is ŷ = 1.50 + 0.80x. The slope is positive, so predictions rise by 0.80 target units for each one-unit increase in x. The intercept is the predicted value at x = 0, though the data here begin at x = 1.

Step 3: Read the predictions and residuals

A prediction is the line’s estimated target at an input value. A residual is observed minus predicted, which preserves the direction of the error: a positive residual means the point lies above the line, and a negative residual means it lies below.

In the output, the third point has the largest absolute residual, 1.10. That flags a comparatively poor fit at x = 3, and the other residuals help show whether the miss reflects a wider issue. Residuals should be checked across the input range for curvature, changing spread, and unusual points.

The fitting formula squares residuals, so errors with larger magnitudes have greater influence on the chosen line. That property is useful when large misses matter, but it also makes the fit sensitive to outliers. Check unusual observations against their source before deciding whether to correct, retain, or analyze them separately.

Step 4: Validate before using the model

Reserve observations before fitting and score those predictions after training. Training error describes the data that shaped the line. For time-ordered data, preserve chronology so future observations do not enter training.

Mean absolute error averages error magnitude in target units. Mean squared error gives larger misses extra weight.

R-squared compares residual variation with variation around the target mean. Read it alongside the evaluation data and task.

I prefer a held-out error measure alongside a residual plot because the score summarizes size while the plot can reveal a systematic shape the summary hides. For a tiny dataset like this demonstration, a separate test set would be too small to give a stable estimate, so it teaches the calculation rather than demonstrating model quality.

How Is a Linear Regression Line Estimated?

Ordinary least squares has a closed-form solution for a single feature. Other estimators use iterative updates or change the loss being minimized, which matters when data size or outlier behavior changes the problem.

For the four observations in the code example, the direct formula is transparent and reproducible. Gradient descent would add learning-rate and stopping choices without improving that small demonstration. In larger problems, standardize features where scale affects optimization, monitor convergence, and compare the result against a trusted baseline.

Least absolute deviations minimizes absolute residuals rather than their squares. It reduces the influence of a single large error on the objective, though it changes the estimator and the interpretation of the fitted center. Choose it when that objective matches the decision, then evaluate it on held-out observations using a metric aligned with the cost of errors.

What Can Linear Regression Be Used For?

Linear regression estimates numeric outcomes when a weighted combination of features is useful for explanation or prediction. The model is a starting point, and the use case determines which inputs, validation split, and error measure make sense.

• Estimate a house price from area and other recorded attributes, then validate on later sales or a held-out sample.
• Describe how measured output changes with time, while checking for seasonal or curved residual structure.
• Estimate demand from price and campaign variables, while avoiding features that would only be known after the forecast date.
• Quantify an association between measurements, with study design guiding any causal interpretation.

In each case, define the target and the moment when a prediction is made before choosing features. That prevents leakage, where a model receives information unavailable at the intended prediction time. Compare against a simple baseline such as the training-target mean so a complex model must earn its added cost.

How Do You Read a Regression Trend Line?

A trend line summarizes the fitted direction across the observed feature range. Its slope describes predicted change per unit of input, while the residuals show how observations sit around those predictions.

Uneven spacing of x values does not by itself make ordinary least squares invalid. Every observation contributes one squared residual, although observations at extreme x values can have greater influence on the estimated slope. A scatter plot and influence diagnostics help distinguish sparse coverage from an influential observation.

People often ask whether clustered x values pull a line toward that region. The answer depends on the estimation objective and the design, not a simple count of how close points look on a chart. The discussion of unevenly spaced x values illustrates why “accuracy” needs a defined metric and target population.

What Should You Check After Fitting?

After fitting, inspect residuals against fitted values and the input, review influential observations, and evaluate predictive error on an appropriate holdout. Use the table to match a symptom with a next check.

Grouped observations may need group-based splits, while time-ordered data need chronological splits. Compare metrics on the same evaluation cases, and check unusual records against their source before deciding whether to correct or retain them.

When Does a Linear Regression Model Fit Poorly?

A straight-line fit can miss systematic structure or fail assumptions needed for inference. Diagnose the data before changing models. The table pairs common symptoms with a useful check.

Linear regression estimates a conditional mean relationship under its modeling setup. Statistical tests and confidence intervals rely on additional assumptions, including an appropriate specification and error behavior. Prediction can still be useful when assumptions for classical inference are not met, but the uncertainty estimate and conclusions need suitable treatment.

Multiple regression adds features, but does not automatically improve a model. Redundant inputs can make individual coefficients difficult to interpret, and adding variables can improve training fit even when test performance worsens. Compare models on the same held-out observations and keep features available at prediction time.

Extrapolation means predicting outside the range of inputs used to fit the model. A line continues indefinitely on paper, but the underlying relationship may change beyond the observed range. Report the input range and treat out-of-range predictions as a separate risk rather than presenting them as routine estimates.

Conclusion

Residuals show how far observed values sit from the fitted line. Check them across the observed range to see whether the line describes the data well.

For prediction, evaluate errors on observations held out from fitting.

The CodeForGeek JavaScript regression walkthrough gives a related implementation. For evaluation splits, see the cross-validation guide.

The scikit-learn LinearRegression documentation defines its estimator. Google’s linear regression lesson explains model training.

FAQ

These short answers address common questions about the model’s output and scope.