ImageBench

ImageBench V1 —

192 evaluations across 6 categories

Benchmark V1 verdicts are produced by VLM judges and can contain mistakes. Treat PASS/FAIL labels as machine-assisted assessments, and inspect the images yourself. Learn more about the methodology.

Generation Details

Source-backed model context, size, cost, and request settings for this ImageBench V1 run.

local/flux-3-klein-base-fp8

Local

FLUX 3 Image Klein is Black Forest Labs' 5.6B open-weights image model (released 2026-10-15) covering text-to-image, editing with up to ten reference images and bounding-box layout control in one model, with a Qwen3-VL-4B-Instruct text encoder and output up to 4K. This entry runs the undistilled base checkpoint (50 steps at CFG 3.5) in FP8 precision. Released under the FLUX Kommunity License (non-commercial, with a commercial-output exception for small companies).

Maker
Black Forest Labs
Family
FLUX 3
Model Size
5.6B
high
Cost
local run; no API price
not_applicable
Run Target
gx10/flux3-klein-base-fp8
Effective Request
response_format: b64_json · size: 1024x1024 · seed: 7 · num_inference_steps: 50 · guidance_scale: 3.5
54.9
Overall
63%
Capability
47.3
Est. Preference
120
Pass
72
Fail
164.3s
Avg Latency
163.6s
Min Latency
165.5s
Max Latency
Text Rendering80%Spatial Reasoning56%Human realism55%Truthfulness41%Professional Studio96%Graphical design67%Preference47%Latency0%

All 192 generations

Text Rendering80%

Typography Style100%

Writing accuracy75%

Spatial Reasoning56%

Attributes Binding56%

Compositionality67%

Counting56%

Negation44%

Relative Position75%

Scale & Proportions33%

Human realism55%

Faces & Expressions75%

Full Body8%

Hands92%

Multi-Subject33%

Truthfulness41%

Photorealism33%

Physics & Reflections50%

World Knowledge33%

Professional Studio96%

Camera & Lighting100%

Color Precision92%

Photorealism100%

Graphical design67%

Data Visualisation0%

Layout & Design67%

Style Diversity83%