Geometric encoding
The input drawing and canonical templates are sampled into line segments and encoded by a shared backbone.
Overview
Abstract
We introduce TempFit, a novel approach to Panoptic Symbol Spotting on CAD drawings. Existing methods treat symbol spotting as a per-primitive classification or mask-prediction task, with no explicit constraint enforcing that the primitives assigned to one instance form a spatially connected group. Consequently, predictions may merge primitives from distant symbols or fragment a coherent symbol across multiple instances.
Inspired by the professional retrieve-and-transform CAD workflow, we reformulate the task as Template Fitting: each foreground instance is detected by retrieving and aligning a canonical template from a library, enforcing structural completeness by construction. Our autoregressive transformer unifies template retrieval and pose prediction, treats the template library as an explicit geometric memory, parses background regions in parallel, and synthesizes foreground instances while attending to template-specific geometry. Experiments on FloorPlanCAD and IndustryCAD demonstrate state-of-the-art performance, with particularly large gains on complex industrial drawings.
Approach
Template fitting in a unified model
The drawing and template library are encoded into geometric tokens, then processed jointly for parallel background segmentation and sequential foreground template fitting.
Hybrid prediction
Background tokens predict category masks in parallel, while instance tokens retrieve templates and predict their transformations.
Back projection
Predicted background masks and fitted foreground templates are projected onto the input primitives to produce the panoptic result.
Results
Panoptic symbol spotting across CAD domains
Representative results on architectural floorplans and complex industrial drawings. Colors denote semantic categories, while the highlighted insets reveal fine-grained instance details.
Dataset 01
FloorPlanCAD
Dataset 02
IndustryCAD
Application
From spotted symbols to 3D scenes
The predicted template-and-transformation pairs can be lifted back to 3D and directly assembled into an industrial scene, connecting symbol spotting with downstream digital-twin reconstruction.