The traditional automatic driving behavior decision algorithm needs to manually set complex rules, resulting in long vehicle decision-making time, poor decision-making effect, and no adaptability to the new environment. As one of the main methods in the field of machine learning and intelligent control in recent years, reinforcement learning can learn reasonable and effective policies only by interacting with the environment. Firstly, this paper introduces the current research status of automatic driving technology and the current mainstream automatic driving control methods. Then, it analyzes the characteristics of convolutional neural network, reinforcement learning method ( Q -learning), and deep Q network (DQN) and deep deterministic policy gradient (DDPG). Compared with the DQN algorithm based on value function, the DDPG algorithm based on action policy can well solve the continuity problem of action space. Finally, the DDPG algorithm is used to solve the control problem of automatic driving. By designing a reasonable reward function, deep convolution network, and exploration policy, the intelligent vehicle can avoid obstacles and, finally, achieve the purpose of avoiding obstacles and running the whole process in a 2D environment.

Aiming at the demand of industrial instrument reading, this study proposes a method of industrial instrument classification and reading recognition based on YOLOv3. Given that industrial meters can be divided into pointer meters and digital meters according to the dial type, this method conducts a reading study for each of the two types of meters. Firstly, the YOLOv3 model is trained to recognize and detect the meter types and classify the meters according to the values of the obtained classes. The pointer meter uses a Hough circle to detect the dial, extracts the scale and the pointer, calculates the angle between the 0 scale line and the pointer, and obtains the reading of the pointer meter. The digital meter extracts the digits by finding the contours of the dial and the digit area and then uses a support vector machine (SVM) to identify the extracted digits and output the readings of the digital meter. Through the test, the mean average precision (mAP) of the recognition model in this study is 93.73%. The absolute error of pointer meter reading is less than 0.1 in general, and the maximum relative error is 0.35%. The accuracy of the digital meter reading is 99.7%. The proposed method can accurately read the value of the instrument and meet the needs of industrial production.

The traditional Deep Deterministic Policy Gradient (DDPG) algorithm has been widely used in continuous action spaces, but it still suffers from the problems of easily falling into local optima and large error fluctuations. Aiming at these deficiencies, this paper proposes a dual-actor-dual-critic DDPG algorithm (DN-DDPG). First, on the basis of the original actor-critic network architecture of the algorithm, a critic network is added to assist the training, and the smallest Q value of the two critic networks is taken as the estimated value of the action in each update. Reduce the probability of local optimal phenomenon; then, introduce the idea of dual-actor network to alleviate the underestimation of value generated by dual-evaluator network, and select the action with the greatest value in the two-actor networks to update to stabilize the training of the algorithm process. Finally, the improved method is validated on four continuous action tasks provided by MuJoCo, and the results show that the improved method can reduce the fluctuation range of error and improve the cumulative return compared with the classical algorithm.

No abstract