RoomTessera started from one simple sentence: import a floor plan, get a 3D view automatically, then place furniture freely. It sounds like a recognition problem. Only after building it did the real weight show up: what happens once recognition gets it wrong.

The project went through 16 rounds of explicit feedback. The early rounds added features: rooms, furniture, materials, lighting, saving. Almost every later round dealt with the same thing — walls. A thick load-bearing wall came out too wide, wall joints protruded, some corners turned into diagonals. Every round said recognition was still wrong, and every fix was followed by the next floor plan.

16rounds of explicit feedback
15procedural furniture categories
10cmwall-line snap grid
20undo steps

What automatic recognition promises

What automatic recognition can do is specific: estimate the extent of the plan, find the floor outline, split it into rooms, compute areas. On a clean plan those results are already enough to start furnishing. The user does not have to draw an empty box first, and furniture has boundaries to sit inside.

The catch is that floor plans were not drawn for algorithms. They have line weights, dimensions, furniture covering the walls, shadows, sometimes even perspective. A wall's centre line and its visible edge are not the same line. A thick load-bearing wall looks wider on the plan, so the algorithm easily takes the whole thickness as wall and produces a room that is one wall too fat.

Why walls come out crooked

The typical failures come in three kinds:

  • Thick walls get wider. Walls on the plan have thickness; recognition takes the outer edge as the centre line and eats a strip out of the room.
  • Joints protrude. When two walls meet, the segment endpoints are not merged correctly, so a small block appears in 3D.
  • Corners turn into diagonals. Slight perspective or a skewed scan, and the algorithm fits a line that should be horizontal as a diagonal; the room tilts with it.

None of these is fully solved by a bigger model. The input itself lacks depth, and the semantics of a wall have no single answer. A more honest framing: automatic recognition produces an editable draft, not a finished plan.

Demote automation to a draft

The turning point was reclassifying the recognition output from a result into a draft. The interface stopped pretending it must be right and let people edit directly: horizontal and vertical wall lines, click or drag to erase, clear all automatic walls, 10cm grid snapping, 20-step undo.

It does not look impressive, but it moved the tool from demo to usable. The user no longer argues with the algorithm; they straighten the segment that is wrong. Automatic recognition lays out 80% of the spatial relations; hand-drawn walls supply the last 20% of precision.

The value of automatic recognition is not doing the work for you. It is making you do less of it.

Keeping the original plan as an underlay matters too. People need to compare against the source image to decide where a wall is, not patch it from memory in 3D. With the underlay, the grid, the wall lines and the room labels in one plan view, correction becomes an action you can reason about.

RoomTessera 3D interface: the floor plan on the left, a 3D living room with furniture on the right.
RoomTessera 3D / floor plan, 3D view and furniture panel on one screen.Project screenshot

The correction interface is the product

Beyond wall lines, the project added several things that made the draft usable:

  • Plan and 3D views. Fix the structure in plan, read the space in 3D; both views share one data model.
  • Collision feedback. While dragging furniture, walls, other furniture and the space boundary are checked in real time; overlaps turn red.
  • Smart placement. New furniture first finds a spot where it fits, then the user adjusts, instead of being dragged in from the origin.
  • Local save and export. Plans live in the browser, can be imported and exported as JSON, and the current view can be exported as a PNG.
  • Materials and lighting. 6 floors, 6 wall colours, 3 lighting moods, 4 whole-home styles, so a draft turns into something presentable quickly.

Not one of these makes recognition more accurate. They all answer the same question: after recognition gets it wrong, can the user keep working without leaving the page?

Local-first

RoomTessera is front end only: recognition, rendering and storage all run in the browser, with no backend and no account. Floor plans and designs are never uploaded to a server. That is not a technical boast; it is the minimum trust condition for this class of tool — a home floor plan is private, and nobody should have to register just to try something.

Local-first imposes constraints as well: with no server-side model, recognition has to be heuristic; with no cloud sync, plans have to be exportable; with no background jobs, all computation has to finish in a time the front end can tolerate. Those constraints made the product clearer: it is a local workbench, not an AI design platform.

What to measure

Judged only by wall pixel accuracy, this project looks like a failure. More useful metrics:

  • Correction time. From import to a usable plan, how many wall segments and how many minutes of hand editing?
  • Recoverability of failure. Once a wall is crooked, can the user straighten it in one or two steps instead of starting over?
  • First-run usability. With nothing changed, can furniture be placed in the generated 3D right away?
  • Spatial readability. With 8 rooms and 15 furniture categories, is the frame still legible?

These are product metrics, not model metrics. They also explain why training a stronger recognition model is not the only answer: what the user actually needs is a sense of control.

Conclusion

The hard part of image-to-3D is not turning an image into geometry; it is making that geometry editable, correctable and savable. Automatic recognition saves you a few strokes, but it cannot make decisions for you. An “AI-generated 3D” with no correction interface becomes a dead end on real floor plans very quickly.

RoomTessera is still v0.1: wall recognition is heuristic, complex plans fail, and the manual tools need polishing. But at least it admits that, and it puts correction into the product instead of hiding it behind a recognition accuracy figure.

References / Live demo: roomtessera-3d; source: NorwinYu/roomtessera-3d. Project page: RoomTessera 3D. This is a development postmortem, not an evaluation of recognition accuracy.