Comparison between the ground-truth annotation, the initial SAM mask, and our refined mask on the Figurines scene. In this cluttered tabletop scene with many small objects, the initial SAM mask often produces fragmented or overly fine-grained predictions, whereas our method yields cleaner object-level masks with more consistent identities.
Comparison between the ground-truth annotation, the initial SAM mask, and our refined mask on the Ramen scene. While the initial SAM mask over-segments fine-grained texture patterns such as those on the table, our method suppresses these spurious fragments and recognizes them as part of a single coherent object.
Comparison between the ground-truth annotation, the initial SAM mask, and our refined mask on the Blue Sofa scene. While the initial SAM mask produces fragmented predictions such as sunglasses and piggy-like doll, our method consolidates them into a cleaner and more coherent object-level segmentation.
Comparison between the ground-truth annotation, the initial SAM mask, and our refined mask on the Teatime scene. When looking at the sheep and bear dolls, over-segmented masks are merged into object-level units.
Comparison between the ground-truth annotation, the initial SAM mask, and our refined mask on the Table scene. While the initial SAM mask tends to produce unstable boundaries and fragmented parts for neighboring objects, our method refines them into clearer object-level segments with improved consistency.
Reliable instance-level scene understanding is a fundamental prerequisite for object-level interactions and high-fidelity 3D representations. While current methods often leverage 2D foundation segmentation models to obtain these priors, their 2D-centric design typically yields fragmented masks and inconsistent predictions across different views. To address these issues, we propose a novel framework that produces consistent 2D instance masks to guide the optimization of 3D Gaussian Splatting (3DGS) feature fields. Our framework consists of three main stages. (1) Multi-Cue Extraction that generates synergistic semantic, geometric, and structural priors from input images. (2) Multi-Cue-Guided Mask Merging process that consolidates fragmented masks using a composite merge score derived from semantic, depth, and edge cues. (3) Cross-View Mask Matching that establishes globally consistent identity assignments across all viewpoints. By transforming viewpoint-specific segments into coherent 3D primitives, our approach enables stable 3D instance segmentation and effective downstream editing tasks. Experiments demonstrate that our method significantly improves cross-view consistency and segmentation stability over existing baselines while maintaining high-fidelity photometric reconstruction.
Given multiview images, we extract initial SAM masks and multi-cue priors. These masks are refined via MCM using a composite merge score. Following cross-view mask matching, refined 2D features are backprojected and lifted into 3D Gaussians. Consequently, the model outputs a view-consistent object-level feature field for high-fidelity 3D scene understanding.
This figure demonstrates how faithfully our refinement results represent the scene and maintain view consistency across different viewpoints. We compare our model's performance with three baselines.
* For clearer visualization, this figure enhances edge visibility and overlays object IDs on the masks.
Compared to our baseline model (GaussianGrouping), which exhibits over-segmentation, unclear mask area, and view-inconsistent predictions, our approach successfully produces coherent, clear, and view-consistent object masks across changing viewpoints.
To be updated.
@article{YourPaperKey2024,
title={Your Paper Title Here},
author={First Author and Second Author and Third Author},
journal={Conference/Journal Name},
year={2024},
url={https://your-domain.com/your-project-page}
}