We study a four-dimensional system modified from a three-dimensional chaotic circuit by adding a memristor, which is a new fundamental electronic element with promising applications. Although the system has a line of infinitely many equilibria, our studies show that when the strength of the memristor increases, it can exhibit rich interesting dynamics, such as hyperchaos, long period-1 orbits, transient hyperchaos, as well as non-attractive behaviors frequently interrupting hyperchaos. To verify the existence of hyperchaos and reveal its mechanism, a horseshoe with two-directional expansion is studied rigorously in detail by the virtue of the topological horseshoe theory and the computer-assisted approach of a Poincaré map. At last, the system is implemented with an electronic circuit for experimental verification.