New Zealand responded to the COVID-19 pandemic with a combination of border restrictions and an Alert Level system that included strict stay-at-home orders. These interventions were successful in containing the outbreak and ultimately eliminating community transmission of COVID-19. The timing of interventions is crucial to their success. Delaying interventions for too long may both reduce their effectiveness and mean that they need to be maintained for a longer period of time. Here, we use a stochastic branching process model of COVID-19 transmission and control to simulate the epidemic trajectory in New Zealand and the effect of its interventions during its COVID-19 outbreak in March-April 2020. We use the model to calculate key outcomes, including the peak load on the contact tracing system, the total number of reported COVID-19 cases and deaths, and the probability of elimination within a specified time frame. We investigate the sensitivity of these outcomes to variations in the timing of the interventions. We find that a delay to the introduction of Alert Level 4 controls results in considerably worse outcomes. Changes in the timing of border measures have a smaller effect. We conclude that the rapid response in introducing stay-at-home orders was crucial in reducing the number of cases and deaths and increasing the probability of elimination.