Unmanned Aerial Vehicles (UAVs) are abundantly becoming a part of society, which is a trend that is expected to grow even further. The quadrotor is one of the drone technologies that is applicable in many sectors and in both military and civilian activities, with some applications requiring autonomous flight. However, stability, path planning, and control remain significant challenges in autonomous quadrotor flights. Traditional control algorithms, such as proportional-integral-derivative (PID), have deficiencies, especially in tuning. Recently, machine learning has received great attention in flying UAVs to desired positions autonomously. In this work, we configure the quadrotor to fly autonomously by using agents (the machine learning schemes being used to fly the quadrotor autonomously) to learn about the virtual physical environment. The quadrotor will fly from an initial to a desired position. When the agent brings the quadrotor closer to the desired position, it is rewarded; otherwise, it is punished. Two reinforcement learning models, Q-learning and SARSA, and a deep learning deep Q-network network are used as agents. The simulation is conducted by integrating the robot operating system (ROS) and Gazebo, which allowed for the implementation of the learning algorithms and the physical environment, respectively. The result has shown that the Deep Q-network network with Adadelta optimizer is the best setting to fly the quadrotor from the initial to desired position.

Heat waves often occur during sunny days in tropical regions during summer season, with temperatures sometimes reaching over 42 degrees Celsius. Because overheat waves can cause heatstroke for human beings who work outside, umbrellas are usually used to cut off sunshine to protect them. In the paper, an umbrella prototype is designed, which can drive a fan to provide cooling effect for persons who work outside in tropical regions. The designed umbrella uses solar cell attached on the above surface of the umbrella to convert solar energy into electricity, which is used to drive a fan or charge a battery. Moreover, the battery can also drive the fan when sunshine intensity is weak, or provide a charging port with 5 voltages for portable devices, such as mobile phones. The key component to realize these functions is an electronic control module, which includes charging circuit and discharging circuit. The former increases output voltage from the solar cell to the desired voltage to charge the battery; the latter can control electricity from the battery to drive the DC motor fan, LED indicator, and charging port.