In this paper, the boundary generating curves and the numerical range of Kac-Sylvester matrices up to the order $9$ are characterized. Based on the obtained results and on several computational experiments performed with the Mathematica and MatLab programs, we conjecture that the found types of algebraic curves, namely ellipses and ovals, will appear for an arbitrary order.