ImageBench

ImageBench V1 —

192 evaluations across 6 categories

Benchmark V1 verdicts are produced by VLM judges and can contain mistakes. Treat PASS/FAIL labels as machine-assisted assessments, and inspect the images yourself. Learn more about the methodology.

Generation Details

Source-backed model context, size, cost, and request settings for this ImageBench V1 run.

local/flux-3-klein-fp8

Local

FLUX 3 Image Klein is Black Forest Labs' 5.6B open-weights image model (released 2026-10-15) covering text-to-image, editing with up to ten reference images and bounding-box layout control in one model, with a Qwen3-VL-4B-Instruct text encoder and output up to 4K. This entry runs the distilled flow-map checkpoint (16 steps, no CFG) in FP8 precision. Released under the FLUX Kommunity License (non-commercial, with a commercial-output exception for small companies).

Maker
Black Forest Labs
Family
FLUX 3
Model Size
5.6B
high
Cost
local run; no API price
not_applicable
Run Target
gx10/flux3-klein-fp8
Effective Request
response_format: b64_json · size: 1024x1024 · seed: 7 · num_inference_steps: 16 · guidance: none (distilled flow map; CFG is rejected)
58.8
Overall
60%
Capability
57.2
Est. Preference
116
Pass
76
Fail
25.2s
Avg Latency
25.0s
Min Latency
26.7s
Max Latency
Text Rendering87%Spatial Reasoning54%Human realism55%Truthfulness33%Professional Studio96%Graphical design58%Preference57%Latency5%

All 192 generations

Text Rendering87%

Typography Style100%

Writing accuracy83%

Spatial Reasoning54%

Attributes Binding78%

Compositionality67%

Counting33%

Negation44%

Relative Position58%

Scale & Proportions44%

Human realism55%

Faces & Expressions75%

Full Body17%

Hands83%

Multi-Subject33%

Truthfulness33%

Photorealism33%

Physics & Reflections50%

World Knowledge17%

Professional Studio96%

Camera & Lighting100%

Color Precision100%

Photorealism67%

Graphical design58%

Data Visualisation0%

Layout & Design44%

Style Diversity83%