The Planck and Herschel missions are currently measuring the far-infrared to millimeter emission of dust, which combined with existing IR data, will for the first time provide the full spectral energy distribution (SED) of the galactic interstellar medium dust emission, from the mid-IR to the mm range, with an unprecedented sensitivity and down to spatial scales ∼30 . Such a global SED will allow a systematic study of the dust evolution processes (e.g. grain growth or fragmentation) that directly affect the SED because they redistribute the dust mass among the observed grain sizes. The dust SED is also affected by variations of the radiation field intensity. Here we present a versatile numerical tool, DustEM, that predicts the emission and extinction of dust grains given their size distribution and their optical and thermal properties. In order to model dust evolution, DustEM has been designed to deal with a variety of grain types, structures and size distributions and to be able to easily include new dust physics. We use DustEM to model the dust SED and extinction in the diffuse interstellar medium at high-galactic latitude (DHGL), a natural reference SED that will allow us to study dust evolution. We present a coherent set of observations for the DHGL SED, which has been obtained by correlating the IR and HI-21 cm data. The dust components in our DHGL model are (i) polycyclic aromatic hydrocarbons; (ii) amorphous carbon and (iii) amorphous silicates. We use amorphous carbon dust, rather than graphite, because it better explains the observed high abundances of gas-phase carbon in shocked regions of the interstellar medium. Using the DustEM model, we illustrate how, in the optically thin limit, the IRAS/Planck HFI (and likewise Spitzer/Herschel for smaller spatial scales) photometric band ratios of the dust SED can disentangle the influence of the exciting radiation field intensity and constrain the abundance of small grains (a < ∼ 10 nm) relative to the larger grains. We also discuss the contributions of the different grain populations to the IRAS, Planck (and similarly to Herschel) channels. Such information is required to enable a study of the evolution of dust as well as to systematically extract the dust thermal emission from CMB data and to analyze the emission in the Planck polarized channels. The DustEM code described in this paper is publically available.