Parking occupancy prediction (POP) plays a vital role in many parking-related smart services for better parking management. However, an issue hinders its mass deployment: many parking facilities cannot collect enough data to feed data-hungry machine learning models. To tackle the challenges in small-sample POP, we propose an approach named Adaptation and Learning to Learn (ALL) by adopting the capability of advanced deep learning and federated learning. ALL integrates two novel ideas: (1) Adaptation: by leveraging the Asynchronous Advantage Actor-Critic (A3C) reinforcement learning technique, an auto-selector module is implemented, which can group and select data-scarce parks automatically as supporting sources to enable the knowledge adaptation in model training; and (2) Learning to learn: by applying federated meta-learning on selected supporting sources, a meta-learner module is designed, which can train a high-performance local prediction model in a collaborative and privacy-preserving manner. Results of an evaluation with 42 parking lots in two Chinese cities (Shenzhen and Guangzhou) show that, compared to state-of-the-art baselines: (1) the auto-selector can reduce the model variance by about 17.8%; (2) the meta-learner can train a converged model 102x faster; and (3) finally, ALL can boost the forecasting performance by about 29.8%. Through the integration of advanced machine learning methods, i.e., reinforcement learning, meta-learning, and federated learning, the proposed approach ALL represents a significant step forward in solving small-sample issues in parking occupancy prediction.