The higher-spin (HS) algebras relevant to Vasiliev's equations in various dimensions can be interpreted as the symmetries of the minimal representation of the isometry algebra. After discussing this connection briefly, we generalize this concept to any classical Lie algebra and consider the corresponding HS algebras. For sp 2N and so N , the minimal representations are unique so we get unique HS algebras. For sl N , the minimal representation has one-parameter family, so does the corresponding HS algebra. The so N HS algebra is what underlies the Vasiliev theory while the sl 2 one coincides with the 3D HS algebra hs [λ]. Finally, we derive the explicit expression of the structure constant of these algebras -more precisely, their bilinear and trilinear forms. Several consistency checks are carried out for our results.