Accurate color matching is essential for achieving aesthetically realistic outcomes in dental crown and bridge restorations. Traditional visual methods, however, are often affected by lighting variations and observer subjectivity, which can result in inconsistent color matching, including metameric mismatches under changing illumination. Hyperspectral imaging can provide the spectral reflectance of teeth, but existing spatial-scanning or filter-based systems are typically bulky and time-consuming, limiting their suitability for real-time clinical use. Snapshot hyperspectral imaging captures multispectral information in a single exposure, yet many existing reconstruction algorithms rely primarily on data-driven approaches without explicitly accounting for the underlying physics of image formation, limiting their generalizability and interpretability.

Recently, a team led by Prof. Junfei Shen at Sichuan University published a research titled “Physics‑informed multimodal learning for snapshot dental spectral reflectance prediction” in Quantitative Biology. By developing a physics-informed multimodal deep-learning framework that incorporates optical imaging priors into a modular reconstruction pipeline, the team integrated attention mechanisms to predict hyperspectral reflectance from linear RGB images. Based on a dedicated dataset of over 4,000 paired RGB-hyperspectral images of dental samples collected under diverse illumination conditions, the framework demonstrated accurate and robust spectral reconstruction.

Figure 1 illustrates the overall experimental scheme of this paper. The workflow begins with a beam-splitter-based optical setup that simultaneously captures RGB and hyperspectral images of a tooth or calibration target under the same illumination, facilitating precise registration and calibration. The ColorChecker is captured by an RGB camera for device characterization. The image formation process is then modeled in terms of the illumination spectrum, object reflectance, and camera sensitivity, and three sequentially connected neural network modules are used to invert this physical model: a camera spectral sensitivity estimation network (MSNet) that takes a ColorChecker’s linear RGB image as input; an illumination spectrum estimation subnetwork (ISNet) that predicts the illumination spectrum from the white board’s linear RGB image, guided by the estimated camera sensitivity; and a hyperspectral reflectance reconstruction subnetwork (HRNet) that recovers the tooth’s spectral reflectance from its linear RGB image using the estimated sensitivity and illumination as physical priors. Finally, the reconstructed hyperspectral reflectance is compared against the ground truth from a spectral camera using MSE, SSIM, and visual inspection. The authors note that the framework provides a fast, reliable, and physically consistent approach to dental color matching and may also be extended to broader biomedical imaging applications, including tissue characterization and wound assessment.
DOI：10.1002/qub2.70030