ImageBench

ImageBench V1 —

192 evaluations across 6 categories

Benchmark V1 verdicts are produced by VLM judges and can contain mistakes. Treat PASS/FAIL labels as machine-assisted assessments, and inspect the images yourself. Learn more about the methodology.

Generation Details

Source-backed model context, size, cost, and request settings for this ImageBench V1 run.

replicate/google/imagen-4

API

Imagen 4 is Google DeepMind's flagship text-to-image model, offered on Replicate as the standard-quality tier. It emphasizes fine detail rendering, strong prompt adherence, style versatility (photorealistic and abstract), and notably improved text/typography rendering, with output up to 2K resolution.

Maker
Google DeepMind
Family
Imagen 4
Model Size
not disclosed
unknown
Cost
$0.04 / image
high
Run Target
replicate/google/imagen-4
Effective Request
aspect_ratio: 1:1 · seed: 7
47.8
Overall
46%
Capability
49.2
Est. Preference
89
Pass
103
Fail
10.3s
Avg Latency
5.0s
Min Latency
44.0s
Max Latency
Text Rendering20%Spatial Reasoning63%Human realism26%Truthfulness33%Professional Studio67%Graphical design50%Preference49%Latency31%

All 192 generations

Text Rendering20%

Typography Style33%

Writing accuracy17%

Spatial Reasoning63%

Attributes Binding100%

Compositionality78%

Counting33%

Negation67%

Relative Position58%

Scale & Proportions44%

Human realism26%

Faces & Expressions25%

Full Body0%

Hands58%

Multi-Subject17%

Truthfulness33%

Photorealism0%

Physics & Reflections25%

World Knowledge50%

Professional Studio67%

Camera & Lighting67%

Color Precision67%

Photorealism67%

Graphical design50%

Data Visualisation0%

Layout & Design22%

Style Diversity83%