Color Palette
Extraction

A dual-method study exploring the tension between mathematical truth and human perception in artwork color analysis.

Tao Sun  ·  Ferdinand × Tao Project  ·  2026

Try the live extractor →
The Question

What colors are really in a painting?

The answer depends on who you ask. A pixel-counting algorithm sees one truth. A human eye sees another. An AI trained on millions of images sees something else entirely.

This study applies two fundamentally different methods to the same artworks—and finds that the most interesting discoveries live in the gap between them.

Process

Two paths, one image

Artwork Input
Deterministic
Resize 400×400
→
MEDIANCUT → 5 Colors
→
Count Pixels & Sort
AI-Based
Visual Analysis
→
Perceptual Palette
→
Artistic Description

MEDIANCUT Extraction — Step by Step

Source
Step 1 / 5
Generating synthetic painting from artist palette…
Comparison

Deterministic vs. AI

Method 1

Deterministic Algorithm

Python + Pillow. Image resized to 400×400, then quantized to 5 colors via MEDIANCUT. Colors sorted by pixel count—most area first. Same input always yields the same output.

Fully reproducible
Sorted by dominant frequency
Backgrounds & neutrals can dominate
Output: swatches + hex codes
Method 2

AI-Based Analysis

Images sent to a vision-capable AI model. It considers visual perception, artistic intent, and compositional emphasis—not just pixel counts.

Probabilistic (varies per run)
Matches human perception
Captures mood and materiality
Output: palettes + descriptions
Key Finding

“The deterministic palette reveals colors that are mathematically present but that we may not consciously perceive. The AI produces palettes that conform to our visual experience. Neither is correct in isolation.”

Together, they provide a richer understanding of how color operates. The deterministic results offer unexpected starting points; the AI results capture the emotional and material qualities we want to evoke.

Four Color Worlds

Each artist's palette represents a distinct emotional and material character. Hover over the color strips to reveal hex codes.

Mark Rothko

5 works analyzed — luminous color fields
White Center (Yellow, Pink and Lavender on Rose)
#E6B443
#D373A6
#D061A2
#EFDA88
#BB6298
Orange and Yellow
#FFFFFF
#EE6F32
#F1A843
#FCFFFD
#FCF2D3
Dark Composition
#310C0D
#181718
#20332F
#274942
#375C54

"White Center" yields the golds and magentas we perceive. But the dark compositions reveal deep teals (#20332F, #274942) beneath apparent blackness—the quiet tonal architecture beneath Rothko's luminous surfaces. Note how "Orange and Yellow" is dominated by whites and creams: the canvas and framing consume more pixel area than the vivid orange we remember.

Helen Frankenthaler

4 works analyzed — soak-stain technique
Teal & Pink Wash
#3F6E6C
#9A7363
#D99F82
#789C89
#E7BE7D
Red Study
#A35750
#B2644F
#56413B
#DE7774
#C7685E
Green Botanical
#5E853E
#71964C
#83A762
#3A5B34
#A0BD82

Frankenthaler's soak-stain method produces palettes of remarkable range across works. The algorithm perfectly captures her technique: large washes soaking into raw canvas mean a single hue and its variants dominate each piece. The "Red Study" extracts an almost monochromatic spectrum from #A35750 through #DE7774.

Augusto Giacometti

2 works analyzed — textured surfaces
Fantasia (all-over floral)
#979082
#BCB292
#505361
#C3CACC
#DED7C8
Pink Blossoms
#95747F
#693E2F
#825C52
#9F8D8D
#AEA5A3

A prime example of the algorithm revealing mathematical truth vs. visual perception. "Fantasia" is full of vivid yellows, blues, and greens, yet the palette extracts warm greys and muted neutrals—the painting's heavily textured, interwoven brushwork means muted, mixed tones genuinely occupy more pixel area than any single pure hue. The vivid colors are present but fragmented across the surface, each too small to dominate a MEDIANCUT partition. The "Blossoms" painting, dominated by pink, extracts as dusty mauves and browns.

Winston Branch

5 works analyzed — atmospheric abstraction
Red & Blue Atmospheric
#383936
#AC331A
#90734E
#DBD8D2
#CDC1B1
Earth & Crimson
#727953
#8A3439
#302F29
#BDBCB9
#B2A389
Deep Blue Nocturne
#273851
#1B1A38
#D5CFC2
#51787E
#E1DED6

Branch's large-scale atmospheric paintings are visually dominated by bold reds, blues, and yellows—but the algorithm extracts the vast blended regions where those colors meet: olive greens, burnt siennas, warm greys. The vivid primaries that define the visual experience occupy relatively small pixel areas. A clear case where the algorithm's output diverges dramatically from human perception.

Philosophy

Why two methods?

Technology is a tool, not a replacement for critical thinking. The deterministic algorithm represents craft and precision; the AI represents intuition and association. Both are necessary.

The most interesting insights emerge from the space between them. By maintaining a ground truth alongside AI interpretation, we preserve the ability to be surprised—and to find colors we never knew were there.