We measure the effective optical depth of He II Lyα absorption τ eff,HeII at 2.3 < z < 3.5 in 17 UV-transmitting quasars observed with UV spectrographs on the Hubble Space Telescope (HST). The median τ eff,HeII values increase gradually from 1.95 at z = 2.7 to 5.17 at z = 3.4, but with a strong sightline-to-sightline variance. Many ≃ 35 comoving Mpc regions of the z > 3 intergalactic medium (IGM) remain transmissive (τ eff,HeII < 4), and the gradual trend with redshift appears consistent with density evolution of a fully reionized IGM. These modest optical depths imply average He II fractions of x HeII < 0.01 and He II ionizing photon mean free paths of ≃ 50 comoving Mpc at z ≃ 3.4, thus requiring that a substantial volume of the helium in the Universe was already doubly ionized at early times; this stands in conflict with current models of He II reionization driven by luminous quasars. Along 10 sightlines we measure the coeval H I Lyα effective optical depths, allowing us to study the density dependence of τ eff,HeII at z ∼ 3. We establish that the dependence of τ eff,HeII on increasing τ eff,HI is significantly shallower than expected from simple models of an IGM reionized in He II. This requires higher He II photoionization rates in overdense regions or underdense regions being not in photoionization equilibrium. Moreover, there are very large fluctuations in τ eff,HeII at all τ eff,HI which greatly exceed the expectations from these simple models. These data present a distinct challenge to scenarios of He II reionization -an IGM where He II appears to be predominantly ionized at z ≃ 3.4, and with a radiation field strength that may be correlated with the density field, but exhibits large fluctuations at all densities.