The ultimate goal of spectral imaging is to achieve high spectral accuracy, so that the spectral information can be used to calculate colorimetrically accurate images for any combination of illuminant and observer. A new spectral reconstruction method, called the matrix R method, was developed to reconstruct spectral reflectance factor accurately while simultaneously achieving high colorimetric performance for a defined illuminant and observer. The method combines the benefits of both colorimetric and spectral transformations. Tristimulus values were predicted by a colorimetric transformation from multi-channel camera signals, while spectral reflectance factor was estimated by a spectral transformation from the same signals. The method reconstructed reflectance factor by combining the fundamental stimulus from the predicted tristimulus values with the metameric black from the estimated spectral reflectance, based on the Wyszecki hypothesis. The experimental results verified the new method as a promising technique for building a spectral image database.