Overview
Diamonds: Predicting Price is a regression analysis project that models diamond prices from physical and quality attributes, written end to end in R. The analysis covers a dataset of ~54,000 diamonds with features including carat weight, cut grade, color grade, clarity grade, and physical dimensions (length, width, depth).
This was my earliest end-to-end data science project. I'm including it here not as a showcase of technical sophistication, but as evidence of where I started — and because the analytical thinking it required is genuinely interesting.
The full R notebook with all exploratory analysis, visualizations, and statistical modeling is available below.
The Analysis
The central challenge is that diamond price is not linear in any single feature. Carat weight has the strongest correlation with price, but two diamonds with the same carat weight can differ in price by an order of magnitude based on cut, color, and clarity. These quality grades interact with each other and with size in complex ways.
The analysis proceeded in several phases:
Exploratory data analysis: Distribution analysis revealed that carat weight clusters at round values (0.5, 0.75, 1.0, 1.5) due to market conventions, and that price distributions are right-skewed. The EDA also surfaced a data quality problem worth handling before modeling: a small number of diamonds have a recorded length, width, or depth of exactly zero, which is physically impossible. Those were converted to NAs rather than silently left in as valid measurements.
Feature ordering: The raw quality grades (cut: Fair → Ideal, color: J → D, clarity: I1 → IF) are ordinal categories, and color ships in an order that runs best-to-worst while the others run worst-to-best. Reordering it to match makes the model coefficients read consistently across all three grades instead of one of them pointing the opposite way.
Modeling: The analysis works through four families on an 80/20 train-test split — simple and multivariate linear regression, regression decision trees (with complexity-parameter tuning), random forests, and gradient boosting machines — fitting each against both the 4 C's alone and the full feature set, and scoring every one by MAE, RMSE, and adjusted R² on the held-out data. Linear regression is the interpretable baseline; the ensembles capture the non-linear interactions more accurately at the cost of interpretability. A final comparison table ranks all of them side by side.
What I Learned
The most important thing this project taught me wasn't a modeling technique — it was that fitting the model is the small part. Most of the work was upstream of it: deciding what a zero-length diamond means, getting the ordinal grades into a consistent order, and choosing which features a given model should even see. By the time the data was right, swapping between a linear model and a random forest was a few lines. That ratio has held on every project since.
It also gave me an early appreciation for confounding. Plot price against cut on its own and the trend is missing — Ideal diamonds don't command an obvious premium over Fair ones, which is the opposite of what the grade is supposed to mean. The explanation isn't gemological, it's compositional: buyers chasing large stones tend to trade away cut quality, so the poorly cut diamonds in the data are also the biggest ones, and carat swamps the effect being measured. Any single feature plotted against price in this dataset is really showing you carat until you control for it.