The source responsible for reionizing the universe at z > 6 remains uncertain. While an energetically adequate population of star-forming galaxies may be in place, it is unknown whether a large enough fraction of their ionizing radiation can escape into the intergalactic medium. Attempts to measure this escape fraction in intensely star-forming galaxies at lower redshifts have largely yielded upper limits. In this paper, we present new Hubble Space Telescope Cosmic Origins Spectrograph and archival Far-Ultraviolet Spectroscopic Explorer (FUSE) far-UV spectroscopy of a sample of 11 Lyman Break Analogs (LBAs), a rare population of local galaxies that strongly resemble the high-z Lyman Break galaxies. We combine these data with Sloan Digital Sky Survey optical spectra and Spitzer photometry. We also analyze archival FUSE observations of 15 typical UV-bright local starbursts. We find evidence of small covering factors for optically thick neutral gas in three cases. This is based on two independent pieces of evidence: a significant residual intensity in the cores of the strongest interstellar absorption-lines tracing neutral gas and a small ratio of extinction-corrected Hα to UV plus far-IR luminosities. These objects represent three of the four LBAs that contain a young, very compact (∼10 2 pc), and highly massive (∼10 9 M ) dominant central object (DCO). These three objects also differ from the other galaxies in showing a significant amount of blueshifted Lyα emission, which may be related to the low covering factor of neutral gas. All four LBAs with DCOs in our sample show extremely high velocity outflows of interstellar gas, with line centroids blueshifted by about 700 km s −1 and maximum outflow velocities reaching at least 1500 km s −1 . We show that these properties are consistent with an outflow driven by a powerful starburst that is exceptionally compact. We speculate that such extreme feedback may be required to enable the escape of ionizing radiation from star-forming galaxies.