Current Trends and Future Directions of Large Scale Image and Video Annotation: Observations From Four Years of BIIGLE 2.0

Abstract

Marine imaging has evolved from small, narrowly focussed applications to large-scale applications covering areas of several hundred square kilometers or time series covering observation periods of several months. The analysis and interpretation of the accumulating large volume of digital images or videos will continue to challenge the marine science community to keep this process efficient and effective. It is safe to say that any strategy will rely on some software platform supporting manual image and video annotation, either for a direct manual annotation-based analysis or for collecting training data to deploy a machine learning–based approach for (semi-)automatic annotation. This paper describes how computer-assisted manual full-frame image and video annotation is currently performed in marine science and how it can evolve to keep up with the increasing demand for image and video annotation and the growing volume of imaging data. As an example, observations are presented how the image and video annotation tool BIIGLE 2.0 has been used by an international community of more than one thousand users in the last 4 years. In addition, new features and tools are presented to show how BIIGLE 2.0 has evolved over the same time period: video annotation, support for large images in the gigapixel range, machine learning assisted image annotation, improved mobility and affordability, application instance federation and enhanced label tree collaboration. The observations indicate that, despite novel concepts and tools introduced by BIIGLE 2.0, full-frame image and video annotation is still mostly done in the same way as two decades ago, where single users annotated subsets of image collections or single video frames with limited computational support. We encourage researchers to review their protocols for education and annotation, making use of newer technologies and tools to improve the efficiency and effectivity of image and video annotation in marine science.

FathomNet: A global image database for enabling artificial intelligence in the ocean

“…Imaging, a non-extractive method for ocean observation, enables the identification of many animals to the species level, elucidates community structure and spatial relationships in a variety of habitats, and reveals fine-scale behavior of animal groups 10 . However, processing visual data, particularly data with complex scenes and organisms that require expert classifications, is a resource-intensive process that cannot be scaled without significant investment, capacity building, and advances in automation 11 , 12 .…”

Eine Online‐Plattform für die Verarbeitung digitaler visueller Daten zur Zustandsbeschreibung von Beschichtungssystemen an maritimen Stahlbauten

“…Die webbasierte Annotations‐ und Bewertungsplattform BIIGLE 2.0 (BioImage Indexing, Graphical Labelling and Exploration) ist aus Sicht der Software‐Anforderungen und ‐Entwicklung im Detail in [5–7] beschrieben worden. Teile der Architektur der Plattform sind in Bild 2 dargestellt.…”

“…and Exploration) ist aus Sicht der Software-Anforderungen und -Entwicklung im Detail in[5][6][7] beschrieben worden. Teile der Architektur der Plattform sind in Bild 2 dargestellt.…”

A Digital Light Microscopic Method for Diatom Surveys Using Embedded Acid-Cleaned Samples

“…The resulting gigapixel-sized virtual slide image usually depicts hundreds to thousands of individual diatom valves. Taxonomic identification and counting are performed by manually locating and labelling diatom valves in the virtual slide using the web-browser-based image annotation system BIIGLE 2.0 [22,23], a tool that allows the collaborative and asynchronous annotation of samples.…”