FPGA Implementation of a Convolutional Neural Network and Its Application for Pollen Detection upon Entrance to the Beehive

Abstract

The condition of a bee colony can be predicted by monitoring bees upon hive entrance. The presence of pollen grains gives beekeepers significant information about the well-being of the bee colony in a non-invasive way. This paper presents a field-programmable-gate-array (FPGA)-based pollen detector from images obtained at the hive entrance. The image dataset was acquired at native entrance ramps from six different hives. To evaluate and demonstrate the performance of the system, various densities of convolutional neural networks (CNNs) were trained and tested to find those suitable for pollen grain detection at the chosen image resolution. We propose a new CNN accelerator architecture that places a pre-trained CNN on an SoC FPGA. The CNN accelerator was implemented on a cost-optimized Z-7020 FPGA with 16-bit fixed-point operations. The kernel binarization and merging with the batch normalization layer were applied to reduce the number of DSPs in the multi-channel convolutional core. The estimated average performance was 32 GOPS for a single convolutional core. We found that the CNN with four convolutional and two dense layers gave a 92% classification accuracy, and it matched those declared for state-of-the-art methods. It took 8.8 ms to classify a 512 × 128 px frame and 2.4 ms for a 256 × 64 px frame. The frame rate of the proposed method outperformed the speed of known pollen detectors. The developed pollen detector is cost effective and can be used as a real-time image classification module for hive status monitoring.

Labeled dataset for bee detection and direction estimation on entrance to beehive

“…No extra artificial background was added behind the landing boards. The aim was to acquire records of bee at the entrance of beehives using native landing boards, and then train a convolutional neural network model for pollen detection [1] , bee detection, tracking [2] and behavior identification. Camera was stationary mounted to horizontally adjustable stick and a tripod for stability, as shown in Fig.…”

Key-Point-Descriptor-Based Image Quality Evaluation in Photogrammetry Workflows

“…Photogrammetry offers a sophisticated method to extract precise geometric information from photographs. This technology enables the creation of detailed three-dimensional (3D) models of objects [1][2][3][4] or scenes [5][6][7][8], transforming the way industries and sciences approach visualization and analysis [9][10][11][12][13][14][15]. Utilization of photogrammetry spans a diverse range of fields, underlining its significance in contemporary applications that demand accuracy and detail [16][17][18][19][20][21][22].…”

“…The first five methods(1)(2)(3)(4)(5) are the baseline image quality evaluation methods. The rest of the methods(6)(7)(8)(9)(10)(11)(12)(13)(14) are proposed key-point-descriptor-based image quality evaluation methods. Suffixes "64" and "128" near the descriptor name denote the dimensionality of the descriptor.…”

Reduced-Parameter YOLO-like Object Detector Oriented to Resource-Constrained Platform

“…For the accelerator, it is finally to be deployed in the actual application environment, which is an important research topic for hardware accelerators. Prior to this, there have been many studies on the practical application of neural network accelerators [ 12 , 13 , 14 , 15 ] and the design of neural network accelerators [ 16 , 17 , 18 ]. However, most of the existing algorithm models are transformed and arranged on the FPGA, and the model design is not combined with the hardware architecture.Therefore, in this work, we design a YOLO algorithm and deploy the algorithm model to run on FPGAs.…”