Unmanned aerial vehicles (UAV) can be applied in many Internet of Things (IoT) systems, e.g., smart farms, as a data collection platform. However, the UAV-IoT wireless channels may be occasionally blocked by trees or high-rise buildings. Intelligent reflecting surface (IRS) can be applied to improve the wireless channel quality by smartly reflecting the signal via a large number of low-cost passive reflective elements. This paper aims to minimize the energy consumption of the system by jointly optimizing the deployment and trajectory of the UAV. The problem can be formulated as a mixed-integerand-nonlinear-programming (MINLP), which is difficult to be addressed by the traditional solution, which may be easily fall into the local optimal. To address this issue, we propose a Joint Optimization framework of depLoyment and Trajectory (JOLT), where an adaptive whale optimization algorithm (AWOA) is applied to optimize the deployment of the UAV, and an elastic ring self-organizing map (ERSOM) is introduced to optimize the trajectory of the UAV. Specifically, in AWOA, a variable-length population strategy is applied to find the optimal number of stop points, and a nonlinear parameter a and a partial mutation rule are introduced to balance the exploration and exploitation. In ERSOM, a competitive neural network is also introduced to learn the trajectory of the UAV by competitive learning, and a ring structure is presented to avoid the trajectory intersection. Extensive experiments are carried out to show the effectiveness of the proposed JOLT framework.