Unmanned aerial vehicles (UAVs) have been envisioned as a promising technique to provide relaying and mobile edge computing (MEC) services for ground user equipment (UE). In this paper, we propose a UAV-assisted MEC architecture in dynamic environment, where a UAV flies with a fixed trajectory and may act as a MEC server to process the tasks offloaded from the UE or act as a relay to help the UE to offload their tasks to the ground base station (BS). The objective of this work is to maximize the long-term number of completed tasks of the UE. An optimization problem is formulated to optimize the task offloading decisions of the UE. Considering the random demands of the UE, a deep reinforcement learning- (DRL-) based algorithm is proposed to solve the formulated nonconvex optimization problem. Simulation results verify the effectiveness and correctness of the proposed algorithm.