Context. Recent developments in time domain astronomy, like the Zwicky Transient Facility, have made possible a daily scan of the entire visible sky, leading to the discovery of hundreds of new transients every night. Among these detections, 10 to 15 are supernovae (SNe), which have to be classified prior to cosmological use. The Spectral Energy Distribution machine (SEDm), a low resolution (R ∼ 100) Integral Field Spectrograph, has been designed, built, and operated to spectroscopically observe and classify targets detected by the ZTF main camera. Aims. As the current pysedm pipeline can only handle isolated point sources, it is limited by contamination when the transient is too close to its host galaxy core; this can lead to an incorrect typing and ultimately bias the cosmological analyses, and affect the SN sample homogeneity in terms of local environment properties. We present a new scene modeler to extract the transient spectrum from its structured background, aiming at improving the typing efficiency of the SEDm. Methods. HyperGal is a fully chromatic scene modeler, which uses archival pre-transient photometric images of the SN environment to generate a hyperspectral model of the host galaxy; it is based on the cigale SED fitter used as a physically-motivated spectral interpolator. The galaxy model, complemented by a point source for the transient and a diffuse background component, is projected onto the SEDm spectro-spatial observation space and adjusted to observations; the SN spectrum is ultimately extracted from this multi-component model. The full procedure, from scene modeling to transient spectrum extraction and typing, is validated on 5000 simulated cubes built from actual SEDm observations of isolated host galaxies, covering a large variety of observing conditions and scene parameters. Results. We introduce the contrast c as the transient-to-total flux ratio at SN location, integrated over the ZTF r band. From estimated contrast distribution of real SEDm observations, we show that HyperGal correctly classifies ∼ 95% of SNe Ia, and up to 99% for contrast c 0.2, representing more than 90% of the observations. Compared to the standard point-source extraction method (without the hyperspectral galaxy modeling step), HyperGal correctly classifies 20% more SNe Ia between 0.1 < c < 0.6 (50% of the observation conditions), with less than 5% of SN Ia misidentifications. The false positive rate is less than 2% for c > 0.1 (> 99% of the observations), which represents half as much as the standard extraction method. Assuming a similar contrast distribution for core-collapse SNe, HyperGal classifies 14% additional SNe II and 11% additional SNe Ibc. Conclusions. HyperGal proves to be extremely effective to extract and classify SNe in the presence of strong contamination by the host galaxy, providing a significant improvement with respect to the single point source extraction.