Case study · 2025–present
Tinted
Computer vision that gets skin tone right across the full range, then recommends makeup that actually matches.
The problem
Most beauty tech gets skin tone wrong for anyone who isn't light-skinned, because camera white balance and lighting swamp the signal. Tinted corrects for that and classifies tone across the full Monk scale.
Decisions, and what they cost
Every architecture is a set of trade-offs. These are the ones I made, the alternatives I rejected, and why.
Classical CV preprocessing in LAB color space
vs. raw RGB into a modelLighting is the dominant error source, and correcting it deterministically beats hoping a model learns invariance. LAB separates lightness from color so classification works on the right axes.
Monk Skin Tone scale
vs. the older Fitzpatrick scaleMonk was built for inclusive tech, with real coverage of deeper skin tones. That's the exact failure mode this project exists to avoid.
CLIP shade matching with embeddings precomputed at import
vs. a hand-built rules engine, or embedding per requestCLIP matches visual similarity without labelling thousands of products, and precomputing the whole catalogue reduces per-request work to one matrix multiply. There's a Lab-distance fallback if the model fails to load, so the feature degrades instead of dying.
A four-check quality gate that refuses the image
vs. always returning an answerBlur, over/under-exposure, yaw and pitch are each checked with a specific, actionable error message. For a product whose whole purpose is getting tone right, a clear refusal beats a confidently wrong reading. The pitch estimator is still marked provisional in the code pending calibration across the eval set.
Per-patch outlier rejection with a hard floor on survivors
vs. averaging every sampled patch17 landmark patches are sampled on flat malar and forehead skin, then discarded if they read as an edge, a specular highlight, or deep shadow, with a 10 percent trimmed mean over what remains. If fewer than five patches survive, the request fails rather than guessing.
Sliding-window rate limits plus a fail-closed daily spend cap
vs. trusting upstream quotasPer-minute and per-day limits with correct Retry-After headers and bounded memory, on top of a cap that meters real token usage against model pricing and stops at a dollar ceiling. A public demo calling a paid API needs a hard spending floor, not good intentions.
Evidence it works
77-test pytest suite across the preprocessing and classification pipeline.
Deployed end to end: FastAPI on Render, Next.js 16 and React 19 on Vercel, behind a typed API client.
The seven-stage pipeline streams over SSE from a worker thread, so the UI narrates each stage instead of showing a spinner.
Classification returns the distance to all ten Monk reference tones, not just the winner, so the UI can show a range with match scores and let the user override.
Interface colours are held to measured WCAG AA contrast, and the palette is deliberately near-neutral because a tinted background shifts perceived skin tone and would corrupt the exact judgement the product exists to make.
What I'd do differently
I'd add a labeled eval set of diverse faces with per-tone accuracy reporting, the same eval discipline I applied to the research agent.