Progress in materials science can benefit significantly from the use of modern computational and data-driven methods. Thus, in the present-day research environment, traditional trial-and-error type approaches to materials design are increasingly being replaced by computation-guided experimental design. The advent of materials informatics further adds a unique dimension with the application of state-of-the-art machine learning techniques on the generated data to yield accurate learning models. In this chapter, we describe a rational design approach centred around high-throughput computations, machine learning and targeted experimentation aimed at discovering new and advanced polymer dielectrics for energy storage capacitor applications. Density functional theory computations were performed on a few hundred polymers from a selected chemical space to estimate their dielectric constants and band gaps, two properties that provide useful initial screening criteria for capacitor dielectrics. Synthesis and characterization was done for a few screened candidates to validate the computations and provide initial promising candidates. Further, machine learning techniques were applied on the computational data to yield crucial correlations between polymer attributes and properties as well as regression-based property prediction models, which enabled swift expansion of knowledge to unexplored regions of the chemical space. Synthesis of many of the promising polymers thus identified, formation of thin films, impressive dielectric breakdown and loss characteristics, along with computationally validated and desirable dielectric constants and band gaps makes this a story of successful co-design of novel polymer dielectrics.