We present a model-free reinforcement learning (RL) predictive model with a supervised learning non-linear reconstructor for adaptive optics (AO) control with a pyramid wavefront sensor (P-WFS). First, we analyse the additional problems of training an RL control method with a P-WFS compared to the Shack-Hartmann WFS. From those observations, we propose our solution: a combination of model-free RL for prediction with a non-linear reconstructor based on neural networks with a U-net architecture. We test the proposed method in simulation of closed-loop AO for an 8m telescope equipped with a 32x32 P-WFS and observe that both the predictive and non-linear reconstruction add additional benefits over an optimised integrator.