Natural Language Description Generation Method of Intelligent Image Internet of Things Based on Attention Mechanism

Abstract

With the rapid development of Internet of Things technology, the image data on the Internet are growing at an amazing speed. How to describe the semantic content of massive image data is facing great challenges. Attentional mechanisms originate from the study of human vision. In cognitive science, due to bottlenecks in information processing, humans selectively attend to a portion of all information while ignoring the rest of the visible information. This study mainly discusses the natural language description generation method of Internet of Things intelligent image based on attention mechanism. In this study, a CMOS sensor based on Internet of Things technology is used for image data acquisition and display. FPGA samples cis16bit parallel port data, writes FIFO, stores image data, and then transmits it to host computer for display through network interface. In order to minimize the value of cross-entropy loss function, maximum-likelihood estimation is used to maximize the joint probability of word sequences in the language model when sentence descriptions are generated using the encoder-decoder framework. At each moment, in addition to image features, additional text features are input. Image feature vector and text feature vector are weighted and summed by attention mechanism at each time. In decoding, the attention mechanism gives each image region feature weight, and the long-term and short-term memory network decodes in turn, but the long-term and short-term memory network has limited decoding ability. We use bidirectional long-term and short-term memory network instead of long-term memory network, and dynamically focus on context information through forward LSTM and reverse LSTM. The specificity of the proposed network is 5% higher than that of the 3D convolution residual link network. The results show that the performance of image description model is improved by inputting image context and text context into long-term memory network decoder.

Large language models assisted multi-effect variants mining on cerebral cavernous malformation familial whole genome sequencing

“…Firstly, the initial text-coding module will be upgraded into a time-dependent version so that the progression of diseases can be accommodated [1] . Secondly, an image encoding module will be added to cope with visual data [37] . Finally, a transformer is conceived so that various modalities, such as texts and images, can be handled simultaneously [2] .…”

A bolt defect detection method for transmission lines based on improved YOLOv5

“…How to solve the problem of ensuring that there are no network differences after transplanting PC end algorithms to drone onboard devices is also a need to consider. [ (Qian et al, 2023;Matta et al, 2012;Woo et al, 2018;Li et al, 2019;Liu X. et al, 2020;Ge and Chen, 2020;Li C. et al, 2022;Zhang Y. et al, 2022;Ouyang and Yu, 2022;Huang et al, 2023)]…”

Bidirectional Long-Short-Term Memory with Attention Mechanism for Emotion Analysis in Textual Content

“…Early studies on emotion analysis and detection primarily employed traditional machine learning algorithms, such as support vector machines (SVM), naive Bayes, and decision trees [8]. These methods rely on handcrafted features to represent the input text, such as bag-of-words, n-grams, partof-speech tags, and sentiment lexicons [8][9]. Although these techniques have shown promising results in various emotion analysis tasks, they often fail to capture the complex and longrange dependencies present in natural language, resulting in suboptimal performance.…”