When should episodic memories be stored and retrieved to support event understanding? Traditional list-learning memory experiments make it obvious when to store and retrieve memories, but it is less obvious when to do this in naturalistic settings. To address this question, we trained a memory-augmented neural network to predict upcoming events, in an environment where situations (sets of parameters governing transitions between events) sometimes reoccurred. The model was allowed to learn a policy for when to consult episodic memory, and we explored how this learned policy varied as a function of the task environment. We found that the learned retrieval policy is shaped by internal uncertainty about upcoming events, the level of penalty associated with incorrect predictions, the confusability of stored memories, the presence of a “familiarity signal” indicating the availability of relevant memories, and the presence of statistical regularities (prototypical events). With regard to encoding policy, we found that selectively storing episodic memories at the end of an event (but not mid-event) leads to better subsequent event prediction performance and less incorrect recall. Additionally, we found that the model can integrate information over long timescales even without the hippocampus; it can link information over many event segments via episodic memory; and it shows classic schema-consistent memory effects when the upcoming time point has a prototypical event. Overall, these modeling results provide a normative explanation of several existing behavioral and neuroimaging findings regarding the use of episodic memory in naturalistic settings, and lead to a wide range of testable predictions.