Linear regression is one of the most common and easiest machine learning models to understand. It belongs to the regression family because its goal is to predict a continuous numeric value, such as cost, length of stay, or resource utilization.
Instead of simply describing what happened in the past, linear regression tries to answer a predictive question:
“Based on what we know, what value should we expect in the future?”
What Is the Main Idea?
Linear regression assumes there is a relationship between an input variable (x) and an output variable (y). It attempts to find the straight line that best represents that relationship.
The simplest form of the model is:
y = β₀ + β₁x
Where:
- y = the value we want to predict (dependent or target variable)
- x = the input or explanatory variable (independent variable)
- β₀ = the intercept (the starting value when x = 0)
- β₁ = the slope (how much y changes when x increases by one unit)
A positive slope means that as x increases, y tends to increase. A negative slope means that as x increases, y tends to decrease.
Simple vs. Multiple Linear Regression
Simple Linear Regression
Uses one input variable to predict one output.
Example:
- Input (x): Patient age
- Output (y): Annual healthcare cost
Question:
Does healthcare cost generally increase as patient age increases?
Multiple Linear Regression
Uses more than one input variable.
Example:
Predict annual healthcare cost using:
- Age
- Number of chronic conditions
- Number of emergency department visits
- Gender
- Risk score
The equation becomes:
y = β₀ + β₁x₁ + β₂x₂ + … + βₙxₙ
In real healthcare analytics, multiple regression is often more useful because healthcare outcomes are usually influenced by many factors rather than just one.
Healthcare Analytics Examples
Linear regression can be used to estimate or forecast many healthcare-related values:
- Predicting a member’s annual healthcare cost.
- Estimating next year’s PMPM (Per Member Per Month) spending.
- Forecasting emergency department utilization.
- Studying the relationship between the number of chronic conditions and total claim costs.
- Exploring whether higher HCC burden is associated with increased resource utilization.
For example, if historical data show that members with more chronic conditions generally have higher annual costs, linear regression can help estimate future costs for similar populations.
What Is the “Best-Fit Line”?
Imagine plotting patient age on the horizontal axis and annual healthcare cost on the vertical axis. Each patient becomes a point on the graph.
Linear regression tries to draw one straight line that passes as close as possible to all of those points.
This line is called the best-fit line because it represents the overall trend in the data.
What Are Residuals?
A residual is simply the difference between the actual value and the value predicted by the model.
Residual = Actual Value − Predicted Value
Example:
- Actual annual healthcare cost: $6,200
- Predicted annual healthcare cost: $5,800
Residual = $400
A good regression model tries to make these residuals as small as possible.
What Is Ordinary Least Squares (OLS)?
Most linear regression models use a method called Ordinary Least Squares (OLS). OLS finds the best-fit line by minimizing the sum of the squared residuals (prediction errors). In simple terms, it looks for the line that keeps the overall prediction errors as small as possible.
Why are the errors squared?
- Negative and positive errors do not cancel each other out.
- Larger errors are penalized more heavily than smaller ones.
- The result is a line that provides the best overall fit to the data.
What Is R² (R-Squared)?
R² measures how well the model explains the variation in the data.
The value ranges from 0 to 1:
- R² = 0 → The model explains almost none of the variation.
- R² = 0.50 → The model explains about half of the variation.
- R² = 1.00 → A perfect fit (rare in real-world data).
A higher R² generally means the regression model fits the data better, although it does not guarantee the model is appropriate or useful.
Key Assumptions of Linear Regression
Linear regression works best when several assumptions are reasonably satisfied:
- Linearity – The relationship between x and y is approximately linear.
- Independence – Observations are independent of each other.
- Homoscedasticity – The spread of the errors remains relatively constant.
- Normality of Errors – The residuals are approximately normally distributed.
In practice, analysts often check these assumptions before relying on the results.
Classification vs. Regression
I find it helpful to remember one simple question:
Am I trying to predict a category or a number?
| Business Question | Method |
|---|---|
| Will a patient be readmitted? | Classification |
| Is this claim potentially fraudulent? | Classification |
| What will the annual healthcare cost be? | Regression |
| What is the projected PMPM spending? | Regression |
If the answer is a group or label, use classification.
If the answer is a continuous numeric value, use regression.
My Personal Way to Remember It
I try not to think about linear regression as a complicated mathematical formula. Instead, I think of it as answering a simple healthcare business question:
“Can we use patterns from historical data to estimate a future numeric value?”
For me, the mathematics explains how the model works, but the business question explains why we use it.
🌱 Study Note: Before learning the formula, understand the purpose. Linear regression is simply a tool for finding the best line that helps estimate future values from past data.