A UAV-aided wireless power transfer and data collection network is studied, where it is assumed that when the harvested energy at the sensor node (SN) cannot surpass its circuit activation threshold or the received data rate at UAV falls below a minimal required rate threshold, the information outage occurs. The closed-form expressions of energy outage probability and rate outage probability are derived at first, and then the overall outage probability and coverage performance of the system are analyzed. Based on which, an optimization problem is formulated to minimize the overall outage probability by optimizing UAV's elevation angle and the time splitting (TS) factor. Since the problem is non-convex and has no known solution, an alternating optimization (AO)-based algorithm with Golden-section (GS) based linear search method is designed to find the global optimal solution. In order to explore the maximum coverage area of the UAV for a given tolerable outage probability, another optimization problem is also formulated to maximize the coverage range by optimizing UAV's elevation angle. By using Karush-Kuhn-Tucker (KKT) conditions, the closed-form solution of the optimal elevation angle for maximizing the coverage area is derived. Monte Carlo simulations verify the accuracy of the derived closed-form expression of the overall outage probability and the semi-closed-form expressions of the optimum UAV's elevation angle and TS factor. It shows that there exist a unique optimum elevation angle and the TS factor to achieve the minimum overall outage probability, and significant performance gain can be obtained by using our proposed optimization scheme. The developed theoretical results can be useful to the design of UAV-aided wireless communication systems with wireless power transfer.