ImageBench V1 —

192 evaluations across 6 categories

Benchmark V1 verdicts are produced by VLM judges and can contain mistakes. Treat PASS/FAIL labels as machine-assisted assessments, and inspect the images yourself. Learn more about the methodology.

Generation Details

Source-backed model context, size, cost, and request settings for this ImageBench V1 run.

bfl/flux-1.1-pro-ultra

API

FLUX1.1 [pro] ultra is the high-resolution mode of FLUX1.1 [pro], announced November 2024, generating up to 4 megapixels — four times the standard resolution — without sacrificing prompt adherence.

Maker: Black Forest Labs
Family: FLUX.1
Model Size: not disclosed
unknown
Cost: $0.06 / image
high
Run Target: bfl/flux-pro-1.1-ultra
Effective Request: aspect_ratio: 1:1 · output_format: png · seed: 7
Sources: Introducing FLUX1.1 [pro] Ultra and Raw Modes BFL pricing

48.0

Overall

45%

Capability

50.7

Est. Preference

87

Pass

105

Fail

13.9s

Avg Latency

8.0s

Min Latency

45.0s

Max Latency

All 192 generations

Text Rendering67%

Typography Style100%

Writing accuracy58%

Spatial Reasoning40%

Attributes Binding67%

Compositionality56%

Counting22%

Negation11%

Relative Position42%

Scale & Proportions44%

Human realism31%

Faces & Expressions33%

Full Body8%

Hands67%

Multi-Subject0%

Truthfulness30%

Photorealism0%

Physics & Reflections42%

World Knowledge25%

Professional Studio70%

Camera & Lighting67%

Color Precision83%

Photorealism33%

Graphical design58%

Data Visualisation0%

Layout & Design56%

Style Diversity75%