The electromagnetic instabilities driven by the pickup ions in the outer heliosheath play a crucial role in interpreting the energetic neutral atom (ENA) ribbon observed by the Interstellar Boundary EXplorer (IBEX). Previous studies on the stability of the outer heliosheath pickup ions focused on ring-like velocity distributions with pickup angles equal or close to 90○. In this study, we investigate the instabilities driven by a ring-beam velocity distribution of the pickup ions in the outer heliosheath using linear instability analysis. The modes propagating along (both parallel and anti-parallel to) the background magnetic field are examined, and the emphasis is on how the instabilities change with the pickup angle, α, varying from zero to 90○. The parallel thermal spread of the pickup ions is chosen to make the plasma parameters lie in the Alfvén cyclotron ‘stability gap’ to exclude the well-studied Alfvén cyclotron instability. Our linear instability analysis reveals that the unstable modes all have left helicity but may propagate parallel or anti-parallel to the background magnetic field. At small α, the parallel-propagating modes are unstable in two separate wavenumber (and frequency) ranges. The two unstable ranges merge into one when α increases to 82○. In addition, the anti-parallel-propagating modes become unstable when α is above  5○, but the parallel-propagating modes have larger growth rates until α becomes larger than αc = 82.8○. The growth rates of the parallel-propagating modes quickly drop below zero when α goes beyond αc, while the anti-parallel-propagating modes remain unstable until α approaches 90○.

The stability of the pickup ions in the outer heliosheath is a vital factor in the generation of the energetic neutral atom (ENA) ribbon observed by the Interstellar Boundary EXplorer according to the secondary ENA mechanism. Most previous studies of the pickup ion stability assumed simple, idealized velocity distributions of the pickup ions and focused on the parallel-propagating modes only. This paper takes a more realistic multicomponent pickup ion velocity distribution given by the global modeling of neutral atoms in the heliosphere and investigates the role of the oblique mirror waves, in addition to the parallel modes. Both linear kinetic instability analysis and hybrid simulations are performed. In contrast to a recent investigation using the same distribution that demonstrated the growth of parallel waves but reported an insignificant contribution of oblique modes, our study reveals substantial growth of the oblique mirror instability. The oblique mirror modes and the parallel/quasi-parallel ion cyclotron waves grow simultaneously with different growth rates. The pickup ion scattering by two types of excited waves together is more pronounced than by either type of wave alone. More importantly, our two-dimensional hybrid simulation results demonstrate that the development of the mirror instability not only produces its own pickup ion scattering, but also leads to the growth of extra ion cyclotron waves (in a quasi-linear manner), which further enhances the pickup ion scattering. The results suggest an important role of the mirror modes that should not be ignored in the stability study of the outer heliosheath pickup ions.

One-dimensional hybrid simulations are performed to investigate the plasma instabilities driven by the pickup ions in the outer heliosheath, which are believed to have a ring-beam velocity distribution in general. The modes propagating parallel and antiparallel to the background magnetic field are studied for pickup ion ring-beam distributions of different pickup angles, following the linear kinetic instability analysis presented in our previous work. The simulation results first confirm the unstable modes predicted by the linear analysis and further reveal how the pickup ions are scattered by the excited waves. While the maximum growth rate of the low-frequency, left-helicity waves occurs at the pickup angle of 82○ (consistent with the linear analysis results), the saturation level of the fluctuating magnetic field is the highest around 45○ pickup angle. For the pickup ion parameters examined, the pitch-angle spread of the pickup ions at the end of the simulations decreases with increasing pickup angle. Our study goes beyond previous studies of the pickup-ion-driven instabilities in the outer heliosheath which mainly focused on ring-like distributions of around 90○ pickup angle. With the caveats that the pickup ion parameters are in the Alfvén cyclotron stability gap and that the one-dimensional simulations only include parallel and anti-parallel-propagating modes, our results show that the scattering of pickup ions at small pickup angles is limited to one hemisphere so they can drift away from the ribbon direction as required by the spatial retention scenario of the energetic neutral atom ribbon observed by the Interstellar Boundary EXplorer (IBEX).

Scattering of pickup ion ring-beam distributions in the outer heliosheath is a fundamental element in the spatial retention scenario of the energetic neutral atom (ENA) ribbon observed by the Interstellar Boundary EXplorer (IBEX). According to our earlier linear instability analysis, pickup ion ring-beam distributions trigger magnetic field-aligned, right-hand polarized unstable waves in two separate frequency ranges which are near and far above the proton cyclotron frequency, respectively. We have performed hybrid simulations to study the unstable waves near the proton cyclotron frequency. However, the high-frequency waves well above the proton cyclotron frequency are beyond the reach of hybrid simulations. In this paper, particle-in-cell simulations are carried out to study the parallel- and antiparallel-propagating high-frequency waves excited by the outer heliosheath pickup ions at different pickup angles as well as the scattering of the pickup ions by the waves excited. In the early stages of the simulations, the results confirm the excitation of the parallel-propagating, right-hand polarized high-frequency waves as predicted by the earlier linear analysis. Later in the simulations, enhanced antiparallel-propagating modes also emerge. Furthermore, the evolution of the pickup ion ring-beam distributions of the selected pickup angles reveals that the high-frequency waves do not significantly contribute to the pickup ion scattering. These results are favourable regarding the plausibility of the spatial retention scenario of the IBEX ENA ribbon.

The energetic neutral atom ribbon observed by the Interstellar Boundary Explorer spacecraft is believed to originate from the pickup ions in the outer heliosheath. The outer heliosheath pickup ions generally have a ring-beam velocity distribution at a certain pickup angle, α, the angle at which these ions are picked up by the interstellar magnetic field. The pickup ion ring-beam distributions can drive unstable waves of different propagation angles with respect to the background interstellar magnetic field, θ. Previous studies of the outer heliosheath pickup ion dynamics were mainly focused on ring-like pickup ion distributions with α ≈ 90° and/or the parallel- and antiparallel-propagating unstable waves (θ = 0° and 180°). The present study carries out linear kinetic instability analysis to investigate both the parallel and oblique unstable modes (0° ≤ θ ≤ 180°) driven by ring-beam pickup ion distributions of different pickup angles between 0° and 90°. Our linear instability analysis reveals that ring-beam pickup ion distributions can excite oblique mirror waves as well as parallel/quasi-parallel and oblique right- and left-helicity waves. The maximum growth rate among all the instabilities belongs to the parallel-propagating left-helicity waves at most pickup angles. Furthermore, the evolution of the unstable mirror waves by varying pickup angle indicates that as the pickup angle increases, the maximum growth rate of the mirror modes increases, while its propagation angle decreases.