Currency recognition has always been a troublesome task for blind and visually impaired people (BVIP). The problem is more severe in developing countries such as India, where there is still a lack of robust currency recognition systems. BVIP primarily relies on size variations and patterns such as intaglio printings for recognizing the underlying currency denominations. Most of the current Indian legal tenders resemble in size, thus making the identification process more strenuous. Also, the engraved patterns are not as distinctive as BVIP standards, and they fade over time. For an automated paper currency recognition system, issues such as folded or partial views, uneven illumination, and background clutter make it non-trivial and challenging. This paper ventures to present an end-to-end and robust framework for assisting BVIP in recognizing the Indian paper currency denomination. This paper presents a lightweight network, IPCRNet, useful in a resource-constrained environment such as low/medium level smartphones. The proposed network is based on Dense connection, Multi-Dilation, and Depth-wise separable convolution layers. Additionally, we congregated one of the most diversified Indian paper currency image dataset with more than 50,000 images belonging to almost all denominations in circulation. A customized and publically available android application, "Roshni-Currency recognizer", has also been introduced. The experimental results on multiple datasets demonstrate the superiority of the proposed model. IPCRNet improves the classification accuracy by more than 2% on the proposed dataset compared to the state-of-the-art networks.