Recently, the research team from the Institute of Intelligent Machines at the Hefei Institutes of Physical Science of the Chinese Academy of Sciences has introduced IPENS, an innovative framework integrating Neural Radiance Fields (NeRF) with Segment Anything Model 2 (SAM2).
"Our system allows computers to 'see' and understand plants,”said HUANG He, who led the team, "It makes plant phenotyping faster and more accurate, reconstructing detailed 3D models of different parts of crops like rice and wheat."
Their findings have been published in Plant Phenomics.
In modern intelligent breeding, plant phenotypic analysis stands as one of the core technologies. However, traditional methods rely heavily on manual labeling and measurements, which are slow, costly, and prone to human error.
In this study, IPENS combines two advanced AI technologies. Neural Radiance Fields (NeRF), which reconstructs 3D scenes from ordinary photos, and the Segment Anything Model 2 (SAM2), which automatically identifies and segments objects in images. Together, they enables the system to rapidly generate accurate 3D representations of plant structures without manual labeling.
In experiments with rice and wheat, IPENS automatically extracted and reconstructed detailed 3D models with high accuracy, completing each process in just three minutes—dramatically improving efficiency. Moreover, the system showed excellent cross-species adaptability, proving effective in analyzing diverse crop organs.
This breakthrough offers an efficient, low-cost, and non-invasive solution for acquiring plant phenotypic data, significantly enhancing both the accuracy and efficiency of botanical analysis, according to the team.
This research was supported by the National Key Research and Development Program of China and the Strategic Priority Research Program of the Chinese Academy of Sciences.
Overall technical workflow of the IPENS phenotyping extraction framework (Image by SONG Wentao)
Visualization of 3D reconstructions by IPENS across multiple crop species. (Image by SONG Wentao )
