Acknowledgement
This research was financially supported by the Ministry of Trade, Industry & Energy (MOTIE, Korea) under the "Scale-up Technology Commercialization Program" (P0026386), supervised by the Korea Institute for Advancement of Technology (KIAT), and by the Technology Development Program (00507790) funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea).
References
- T. Brown et al., "Language models are few-shot learners," Advances in neural information processing systems, vol. 33, pp. 1877-1901, Dec. 2020, DOI: 10.18653/v1/2021.mrl-1.1.
- Z. Yang, L. Li, K. Lin, J. Wang, C.-C. Lin, Z. Liu, and L. Wang, "The dawn of lmms: Preliminary explorations with gpt-4v(ision)," arXiv preprint arXiv:2309.17421, 2023, DOI: 10.48550/arXiv.2309.17421.
- Q. Feng, V. Ablavsky, and S. Sclaroff, "Cityflow- nl: Tracking and retrieval of vehicles at city scale by natural language descriptions," arXiv preprint arXiv:2101.04741, Jan., 2021, DOI: 10.48550/arXiv.2101.04741.
- L. Kong et al., "Robo3d: Towards robust and reliable 3d perception against corruptions," in Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, pp. 19994-20006, 2023, DOI: 10.1109/ICCV51070.2023.01830.
- D. Z. Chen, A. X. Chang, and M. Nießner, "Scanrefer: 3d object localization in rgb-d scans using natural language," in European conference on computer vision, Glasgow, UK, pp. 202-221, 2020, DOI: 10.1007/978-3-030-58565-5_13.
- P. Achlioptas, A. Abdelreheem, F. Xia, M. Elhoseiny, and L. Guibas, "Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes," in European conference on computer vision, Glasgow, UK, pp. 422-440, 2020, DOI: 10.1007/978-3-030-58452-8_25.
- A. Dai et al., "Scannet: Richly-annotated 3d reconstructions of indoor scenes," in Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, USA, pp. 2432-2443, 2017, DOI: 10.1109/CVPR.2017.261.
- Z. Zhu et al., "3d -Vista: Pre-trained transformer for 3d vision and text alignment," in Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, pp. 2911-2921, 2023, DOI: 10.1109/ICCV51070.2023.00272.
- J. Yang et al., "Llm-grounder: Open-vocabulary 3d visual grounding with large language model as an agent," in 2024 IEEE International Conference on Robotics and Automation, Yokohama, Japan, pp. 7694-7701, 2024, DOI: 10.1109/ICRA57147.2024.10610443.
- Z. Yuan et al., "Visual programming for zero-shot open-vocabulary 3d visual grounding," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 20623-20633, 2024, DOI: 10.1109/CVPR52733.2024.01949.
- R. Xu et al., "Vlm-grounder: A vlm agent for zero-shot 3d visual grounding," in Conference on Robot Learning, vol. 270, pp. 3961-3985, 2025, DOI: 10.48550/arXiv.2410.13860.
- R. Li, S. Li, L. Kong, X. Yang, and J. Liang, "Zero-shot 3d visual grounding from vision-language models," arXiv preprint arXiv: 2505.22429, May 2025, DOI: 10.48550/arXiv.2505.22429.
- J. H. Jung et al., "Is 'right' right? enhancing object orientation understanding in multimodal large language models through egocentric instruction tuning," in 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, pp. 14257-14267, 2025, DOI: 10.1109/CVPR52734.2025.01330.
- A. Jain, N. Gkanatsios, I. Mediratta, and K. Fragkiadaki, "Bottom up top down detection transformers for language grounding in images and point clouds," in European Conference on Computer Vision, Tel Aviv, Israel, pp. 417-433, 2022, DOI: 10.1007/978-3-031-20059-5_24.
- O. Unal, C. Sakaridis, S. Saha, and L. Van Gool, "Four ways to improve verbovisual fusion for dense 3d visual grounding," in European Conference on Computer Vision, Milan, Italy, pp. 196-213, 2024, DOI: 10.1007/978-3-031-73116-7_12.
- S. Peng et al., "Openscene: 3d scene understanding with open vocabularies," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Vancouver, Canada, pp. 815-824, 2023, DOI: 10.1109/CVPR52729.2023.00085.
- J. Kerr, C. M. Kim, K. Goldberg, A. Kanazawa, and M. Tancik, "Lerf: Language embedded radiance fields," in Proceedings of the IEEE/CVF international conference on computer vision, Paris, France, pp. 19729-19739, 2023, DOI: 10.1109/ICCV51070.2023.01807.
- S. Tong et al., "Eyes wide shut? exploring the visual shortcomings of multimodal llms," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, USA, pp. 9568-9578, 2024, DOI: 10.1109/CVPR52733.2024.00914.
- T.-Y. Lin et al., "Microsoft coco: Common objects in context," in European conference on computer vision, Zurich, Switzerland, pp. 740-755, 2014, DOI: 10.1007/978-3-319-10602-1_48.
- C. Schuhmann et al., "Laion-5b: An open large-scale dataset for training next generation image-text models," Advances in neural information processing systems, vol. 35, pp. 25278-25294, 2022, DOI: 10.5555/3600270.3602103.
- Y. Huang et al., "Egoexolearn: A dataset for bridging asynchronous ego-and exo-centric view of procedural activities in real world," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 22072-22086, 2024, DOI: 10.1109/CVPR52733.2024.02084.
- Y. Mu et al., "Embodiedgpt: Vision-language pre-training via embodied chain of thought," Advances in Neural Information Processing Systems, vol. 36, pp. 25081-25094, 2023, DOI: 10.5555/3666122.3667212.
- J. L. Schonberger and J.-M. Frahm, "Structure-from-motion revisited," in Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, USA, pp. 4104-4113, 2016, DOI: 10.1109/CVPR.2016.445.
- T. Ren et al., "Grounded sam: Assembling open-world models for diverse visual tasks," arXiv preprint arXiv:2401.14159, Jan. 2024, DOI: 10.48550/arXiv.2401.14159.
- S. Liu et al., "Grounding dino: Marrying dino with grounded pre-training for open-set object detection," in European conference on computer vision, Milan, Italy, pp. 38-55, 2024, DOI: 10.1007/978-3-031-72970-6_3.
- N. Ravi et al., "Sam 2: Segment anything in images and videos," in International Conference on Learning Representations, arXiv preprint arXiv:2408.00714, 2024, DOI: 10.48550/arXiv.2408.00714.
- A. Marafioti et al., "Smolvlm: Redefining small and efficient multimodal models," in Second Conference on Language Modeling, 2025, DOI: 10.48550/arXiv.2504.05299.