On-chip silicon photonic filters acting as spectral shapers for the input spectrum are important in various applications including optical communication, computing, and spectroscopy sensing. We used the photonics inverse design framework, aided by the Particle Swarm Optimization (PSO) algorithm, to obtain miniature and efficient filtering structures in the Silicon on Insulator (SOI) platform. These optical filters are less than 10 μm, or just a few wavelengths, in size along any dimension. The investigated structures demonstrate good light filtering characteristic, with various filter types and wavelength ranges. The performance of designed filters is comparable to widely-used mm-long Bragg grating filters, while being a tiny fraction of their size. We also study a class of filters to directly resolve the issue of removing back reflected light without using a circulator or adding a third port (as is generally done) using the optimization process. Our structures are expected to have a much wider range of applications than grating-type filters, primarily due to their miniature footprint, independence from external devices like circulators for back-reflection control, and greater tolerance to thermal effects. Our work also highlights the possibility of designing and fabricating photonic components as a next step, by inversely designing components that best match a given set of requirements instead of depending on general conventional devices.