A dual-method study exploring the tension between mathematical truth and human perception in artwork color analysis.
Try the live extractor →The answer depends on who you ask. A pixel-counting algorithm sees one truth. A human eye sees another. An AI trained on millions of images sees something else entirely.
This study applies two fundamentally different methods to the same artworks—and finds that the most interesting discoveries live in the gap between them.
Python + Pillow. Image resized to 400×400, then quantized to 5 colors via MEDIANCUT. Colors sorted by pixel count—most area first. Same input always yields the same output.
Images sent to a vision-capable AI model. It considers visual perception, artistic intent, and compositional emphasis—not just pixel counts.
“The deterministic palette reveals colors that are mathematically present but that we may not consciously perceive. The AI produces palettes that conform to our visual experience. Neither is correct in isolation.”
Together, they provide a richer understanding of how color operates. The deterministic results offer unexpected starting points; the AI results capture the emotional and material qualities we want to evoke.
Each artist's palette represents a distinct emotional and material character. Hover over the color strips to reveal hex codes.
"White Center" yields the golds and magentas we perceive. But the dark compositions reveal deep teals (#20332F, #274942) beneath apparent blackness—the quiet tonal architecture beneath Rothko's luminous surfaces. Note how "Orange and Yellow" is dominated by whites and creams: the canvas and framing consume more pixel area than the vivid orange we remember.
Frankenthaler's soak-stain method produces palettes of remarkable range across works. The algorithm perfectly captures her technique: large washes soaking into raw canvas mean a single hue and its variants dominate each piece. The "Red Study" extracts an almost monochromatic spectrum from #A35750 through #DE7774.
A prime example of the algorithm revealing mathematical truth vs. visual perception. "Fantasia" is full of vivid yellows, blues, and greens, yet the palette extracts warm greys and muted neutrals—the painting's heavily textured, interwoven brushwork means muted, mixed tones genuinely occupy more pixel area than any single pure hue. The vivid colors are present but fragmented across the surface, each too small to dominate a MEDIANCUT partition. The "Blossoms" painting, dominated by pink, extracts as dusty mauves and browns.
Branch's large-scale atmospheric paintings are visually dominated by bold reds, blues, and yellows—but the algorithm extracts the vast blended regions where those colors meet: olive greens, burnt siennas, warm greys. The vivid primaries that define the visual experience occupy relatively small pixel areas. A clear case where the algorithm's output diverges dramatically from human perception.
Technology is a tool, not a replacement for critical thinking. The deterministic algorithm represents craft and precision; the AI represents intuition and association. Both are necessary.
The most interesting insights emerge from the space between them. By maintaining a ground truth alongside AI interpretation, we preserve the ability to be surprised—and to find colors we never knew were there.