One of the fastest growing areas of interest in quantum computing is its use within machine learning methods, in particular through the application of quantum kernels. Despite this large interest, there exist very few proposals for relevant physical platforms to evaluate quantum kernels. In this article, we propose and simulate a protocol capable of evaluating quantum kernels using Hong-Ou-Mandel (HOM) interference, an experimental technique that is widely accessible to optics researchers. Our proposal utilises the orthogonal temporal modes of a single photon, allowing one to encode multi-dimensional feature vectors. As a result, interfering two photons and using the detected coincidence counts, we can perform a direct measurement and binary classification. This physical platform confers an exponential quantum advantage also described theoretically in other works. We present a complete description of this method and perform a numerical experiment to demonstrate a sample application for binary classification of classical data.