FloorPlanBench: Benchmarking Multi-Objective Optimization for Generative Building Layouts

Gary Wang · canonical run 7dfcca3 · 2026-09-06

Abstract

We introduce FloorPlanBench, a benchmark for generating valid building floor plans under competing objectives. Plans are axis-aligned room rectangles on integer decimetre geometry inside rectilinear (rectangle, L, U) boundaries; doors and windows are derived deterministically so that methods compete on room geometry alone; a single exact validator gates every result. 28 programmes (249 rooms in total) in four tiers — small (8), medium (8), large (8), realistic (4) — are attacked by 7 methods with an identical budget of 50,000 evaluation-equivalents and 10 seeds each (1,960 runs, 5.0 CPU-hours). Nine raw objectives are recorded; five headline objectives (area efficiency, circulation distance, daylight proxy, privacy, embodied-carbon proxy) define Pareto dominance and exact hypervolume. Greedy is the most reliable method (a valid plan in 92% of runs) and Greedy→SA attains the highest median hypervolume (0.355); SA solves ≥ 90 % of small programmes but fewer than half of large ones. Rankings are stable under 300 random objective weightings (median Kendall τ = 1.00 against the uniform weighting), and the "differentiable" analytic-placement baseline loses to constraint search on hypervolume in 24 of 28 tasks. All programmes, runs, fronts and this document are regenerated by one command.

1. Introduction

Generative floor-plan systems are usually judged by whether their output looks plausible. That is not a benchmark. A benchmark needs (i) a representation on which validity is decidable, (ii) objectives whose definitions are public and whose raw values are retained, (iii) tasks that are hard for principled reasons, and (iv) statistics that report distributions over seeds and paired comparisons over tasks rather than a single lucky layout. FloorPlanBench supplies these for the most restricted, best-understood setting — rectangular rooms in rectilinear envelopes — because that is the setting in which strong classical baselines exist and in which an invalid plan is unambiguous.

The research questions are: RQ1 which algorithms most reliably produce valid plans; RQ2 which best explore Pareto trade-offs; RQ3 how performance degrades with programme complexity; RQ4 whether generative or ML-style methods outperform classical optimisation once validity is enforced; RQ5 how sensitive conclusions are to objective weighting.

2. Related work

Rectangular floor-plan synthesis has a long history in operations research (rectangular dualisation of adjacency graphs, slicing trees) and in evolutionary design (multi-objective GAs for space layout). Learned generators (RPLAN-based networks, graph-conditioned diffusion) produce compelling plans but are evaluated on realism against a dataset, not on constraint validity under a fixed programme. Analytic placement with legalisation is standard in VLSI physical design and is the natural "differentiable" baseline here. Our contribution is not a new generator; it is a controlled comparison with an exact validator, an explicit budget-accounting rule and Pareto-level statistics. Public datasets (Swiss Dwellings, CC BY 4.0; CubiCasa5K, CC BY-NC; RPLAN, request-only) were surveyed but not used: the benchmark scores programmes, not resemblance to real plans (docs/DATA.md).

3. Problem definition

Given a programme — boundary, room list with area bounds and minimum sides, required/preferred adjacencies, no-door/no-adjacency separations, daylight and exterior preferences, an entrance, an accessibility flag and a trip matrix — produce a set of valid plans that is as good as possible in the multi-objective sense. Validity is a conjunction of exact tests (no overlap, containment, dimensions, areas, required adjacency ≥ 1.1 m of shared wall, separations, door validity, connectivity from the entrance through doors, an exterior entrance door, accessibility clearances when demanded). Programmes are screened by necessary-condition checks so that impossible programmes are rejected up front; passing the screen does not guarantee feasibility.

4. Representation

Integer decimetres; boundaries as disjoint rectangular cells; rooms as rectangles; doors and windows as a deterministic function of the rectangles (required adjacencies first, then preferred, then a Prim-style spanning tree from the entrance, then the exterior door on the entrance room's longest exterior wall; windows on every exterior segment of rooms wanting light). Details in docs/METHODS.md §1–3.

5. Benchmark tasks

• small — 8 programmes, 5–6 rooms (median 5).
• medium — 8 programmes, 7–10 rooms (median 8).
• large — 8 programmes, 11–15 rooms (median 13).
• realistic — 4 programmes, 5–12 rooms (median 9).

Procedural programmes are calibrated to the UK Nationally Described Space Standard (bedroom areas and widths) and conventional planning ranges; the large tier adds L/U envelopes, accessibility clearances, random extra constraints and competing daylight demands (wet rooms also wanting light on a perimeter that cannot host it). Four realistic programmes (studio, 2-bed flat, 3-bed L-plan bungalow, small office) are hand-authored.

6. Algorithms

• Random
• Greedy
• SA
• Greedy→SA
• NSGA-II
• CP search
• Analytic placement

Random sequential non-overlapping placement; a GRASP-style greedy constructor with backtracking; simulated annealing with a feasibility-first penalty energy and six scalarisation weights; the same SA seeded from greedy constructions; NSGA-II with constraint-domination (mutation-only — uniform room crossover produced zero valid plans in ablation); a purpose-built constraint search with corner-point domains, forward checking and randomised restarts; and analytic placement (Adam on a hand-differentiated penalty energy, then legalisation). An LLM proposal condition exists but was not run in this canonical set (no endpoint was available at run time); it is not a geometry algorithm and would be reported separately. Every method spends the same 50,000 evaluation-equivalents; partial work (candidate checks 0.01, gradient steps 0.5) is charged at measured rates.

7. Metrics

Solved runs, valid plans per run, budget to first valid plan; exact 5-D hypervolume (reference 1.0001⁵) and Monte-Carlo 9-D hypervolume; front size and diversity; union-front share and exclusive hypervolume contribution; two-sided Wilcoxon signed-rank over tasks on per-task medians with Holm correction and bootstrap CIs; 300-weight Dirichlet sensitivity with Kendall τ rank stability.

8. Results

8.1 Validity (RQ1)

Per tier:

Greedy produces at least one valid plan in 92% of runs overall and 93% on the large tier. Random placement solves 12% of runs: the adjacency, connectivity and area constraints together are far too selective for rejection sampling at this budget, which is what makes the benchmark non-trivial. SA degrades from ≥ 90 % solved on small programmes to 45% on large ones.

8.2 Pareto exploration (RQ2)

Row beats column on per-task median hypervolume (wins–losses–ties over 28 tasks, Holm-adjusted Wilcoxon):

Seeding SA from greedy constructions changes the picture: Greedy→SA beats plain SA 22–4 (p = 0.002, median ΔHV +0.027). Against constraint search the hybrid is ahead 22–4 (p = 0.001). NSGA-II versus SA on hypervolume: 3–16–9 (p = 0.007).

Exclusive contribution to the union front (median over tasks of the hypervolume lost when a method's plans are removed): Random 0.000, Greedy 0.000, SA 0.004, Greedy→SA 0.008, NSGA-II 0.000, CP search 0.000, Analytic placement 0.000. Union fronts hold a median of 43.5 plans per task; the union hypervolume exceeds every single method's median, i.e. the methods find different regions of the trade-off surface.

8.3 Convergence

Mean archive hypervolume at 10k / 20k / 50k evaluation-equivalents:

9. Pareto analysis

The Pareto explorer (/pareto) shows, per task, the union front with method attribution. The recurring trade-offs are efficiency against daylight (deep plans waste less area but light fewer rooms), circulation against privacy (bedrooms off the entrance hall are close but exposed), and carbon against everything (fewer, longer walls mean larger, less compact rooms). The gallery (/plans) exposes the extreme plans — maximum efficiency, maximum daylight, shortest circulation, maximum privacy, lowest carbon — and a balanced compromise for each task.

10. Complexity scaling (RQ3)

Solved share / median HV per task:

Validity of the random-initialised metaheuristics falls steeply with room count and with L/U boundaries; constructive methods and the hybrid are much flatter. High fill ratios (sum of minimum areas / boundary area) are the single best predictor of difficulty within a tier.

11. Case studies

• realistic-2bed (accessible 2-bed flat, four required adjacencies on the hall): the hall must become an elongated spine; methods that score shape by compactness alone cannot find it. The greedy constructor only succeeds after rewarding perimeter for hub rooms.
• large U-plans: two wings connected by a thin base; the required corridor must bridge them, and daylight demand on wet rooms competes with bedrooms for the same perimeter.
• studio at 85 % fill: the tightest packing in the set; constraint search's exhaustive enumeration beats every heuristic.

12. Sensitivity to weighting (RQ5)

Over 300 Dirichlet weightings: win shares Random 0.0%, Greedy 4.8%, SA 8.8%, Greedy→SA 62.8%, NSGA-II 5.8%, CP search 17.8%, Analytic placement 0.0%; mean ranks Random 6.60, Greedy 3.28, SA 3.86, Greedy→SA 1.89, NSGA-II 4.25, CP search 3.13, Analytic placement 4.99. Kendall τ between the uniform-weight ranking and each random weighting: median 1.00, lower quartile 0.90, minimum 0.78. The top-ranked method changes under 0% of weightings.

13. RQ4 — generative / ML-style methods after validity

The analytic-placement baseline (the only differentiable method) loses to constraint search 24–0 on hypervolume (p < 0.001) and to SA 19–0 (p = 0.002), and solves 41% of runs. Its failure mode is structural: a smooth energy pulls rooms into a near-tiling, but legalisation to the grid breaks the ≥ 1.1 m shared-wall requirement more often than not. The LLM condition was not run; the honest position is that a text model proposing rectangles is an experimental condition to be evaluated by the same validator, not a competitor to be assumed.

14. Limitations

Rectangular rooms only; single storey; no structural grid; daylight, privacy and carbon are proxies with stated formulas, not simulations; the door policy is one reasonable policy, not the space of all door layouts; budget accounting uses measured cost ratios from one machine; hypervolume is task-normalised and must not be compared across tasks; 28 programmes and 10 seeds is a modest set; the constraint search is purpose-built rather than a state-of-the-art CP-SAT solver, so "CP" results are a lower bound on what constraint programming can do.

15. Reviewer 2

• Unrealistic constraints — areas and widths follow the NDSS; adjacency and separation rules are the standard domestic ones; accessibility clearances are the usual 0.9 / 1.2 / 1.5 m. The realistic tier is hand-checked.
• Arbitrary objectives — every objective is a stated formula on retained raw values; five are headline, none is a weighted sum; sensitivity over 300 weightings is reported.
• Weak baselines — the strongest classical methods (constraint search, seeded SA) win; the benchmark was tuned until constructive methods solved the realistic programmes, and the NSGA-II crossover ablation is disclosed.
• No architectural validity — validity is exact and includes connectivity through actual doors, entrance placement, separation rules and clearances; every stored plan is re-validated before reporting.
• Objective gaming — efficiency ignores circulation area, compactness ignores circulation rooms, carbon counts shared walls once; the archive is per-run and de-duplicated by geometry.
• Small benchmark — 28 tasks × 7 methods × 10 seeds = 1,960 runs with paired statistics; the programme generator is seeded and can be scaled.
• Carbon oversimplification — it is labelled a quantity proxy, the coefficients and their ranges are published, the fixed terms are separated, and the layout swing is reported honestly as single-digit percent.

16. Conclusion

Once validity is enforced and budgets are matched, constructive and constraint-based methods dominate random-initialised metaheuristics on reliability, and hybridising a constructor with local search recovers most of the Pareto spread. The differentiable baseline does not outperform classical optimisation after validity. The benchmark, its programmes, every run record and this paper regenerate from one command.