ImageBench

ImageBench V1 —

192 evaluations across 6 categories

Benchmark V1 verdicts are produced by VLM judges and can contain mistakes. Treat PASS/FAIL labels as machine-assisted assessments, and inspect the images yourself. Learn more about the methodology.

Generation Details

Source-backed model context, size, cost, and request settings for this ImageBench V1 run.

local/flux-3-klein-base-nvfp4

Local

FLUX 3 Image Klein is Black Forest Labs' 5.6B open-weights image model (released 2026-10-15) covering text-to-image, editing with up to ten reference images and bounding-box layout control in one model, with a Qwen3-VL-4B-Instruct text encoder and output up to 4K. This entry runs the undistilled base checkpoint (50 steps at CFG 3.5) in NVFP4 precision. Released under the FLUX Kommunity License (non-commercial, with a commercial-output exception for small companies).

Maker
Black Forest Labs
Family
FLUX 3
Model Size
5.6B
high
Cost
local run; no API price
not_applicable
Run Target
gx10/flux3-klein-base-nvfp4
Effective Request
response_format: b64_json · size: 1024x1024 · seed: 7 · num_inference_steps: 50 · guidance_scale: 3.5
55.3
Overall
64%
Capability
46.4
Est. Preference
123
Pass
69
Fail
80.4s
Avg Latency
80.1s
Min Latency
81.4s
Max Latency
Text Rendering80%Spatial Reasoning67%Human realism50%Truthfulness41%Professional Studio89%Graphical design71%Preference46%Latency0%

All 192 generations

Text Rendering80%

Typography Style100%

Writing accuracy75%

Spatial Reasoning67%

Attributes Binding89%

Compositionality78%

Counting44%

Negation56%

Relative Position67%

Scale & Proportions67%

Human realism50%

Faces & Expressions75%

Full Body0%

Hands92%

Multi-Subject17%

Truthfulness41%

Photorealism33%

Physics & Reflections58%

World Knowledge25%

Professional Studio89%

Camera & Lighting100%

Color Precision92%

Photorealism33%

Graphical design71%

Data Visualisation0%

Layout & Design78%

Style Diversity83%