The observation of a high-mobility two-dimensional electron gas between two insulating complex oxides, especially LaAlO3/SrTiO3, has enhanced the potential of oxides for electronics. The occurrence of this conductivity is believed to be driven by polarization discontinuity, leading to an electronic reconstruction. In this scenario, the crystal orientation has an important role and no conductivity would be expected, for example, for the interface between LaAlO3 and (110)-oriented SrTiO3, which should not have a polarization discontinuity. Here we report the observation of unexpected conductivity at the LaAlO3/SrTiO3 interface prepared on (110)-oriented SrTiO3, with a LaAlO3-layer thickness-dependent metal-insulator transition. Density functional theory calculation reveals that electronic reconstruction, and thus conductivity, is still possible at this (110) interface by considering the energetically favourable (110) interface structure, that is, buckled TiO2/LaO, in which the polarization discontinuity is still present. The conductivity was further found to be strongly anisotropic along the different crystallographic directions with potential for anisotropic superconductivity and magnetism, leading to possible new physics and applications.