This paper focuses on issues related to energy efficient Digital Signal Processing (DSP) -taking a Systemsto-Silicon approach. This implies that achieving maximum energy efficiency in a DSP system is a distributed task from the systems level (compilers, algorithm design etc.) to a silicon (transistor, process technologies etc.) level. It addresses how we view this problem and also uses specific application areas such as hearing devices and medical imaging to illustrate the needs and concepts involved in this context.