The Brownian motion of a particle with higher-derivative dynamics (HDD) coupling with a bath consisting of harmonic oscillators is investigated. The Langevin equation and corresponding Fokker-Planck equation for the Brownian motion of the HDD particle are derived. As a case study, we particularly consider a stochastic Pais-Uhlenbeck oscillator. It is found that the Boltzmann distribution is pathological while this distribution is the steady solution to the Fokker-Planck equation.