Ai in Production: Video Analysis and Machine Learning for Expanded Live Events Coverage

Wright, Craig; Allnutt, Jack; Campbell, Rosie; Evans, Michael J.; Forman, Ronan; Gibson, James; Jolly, S.; Kerlin, Lianne; Lechelt, Susan; Phillipson, Graeme; Shotton, Matthew

doi:10.5594/jmi.2020.2967204

Cited by 15 publications

(6 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The BBC has created a proof-of-concept system for automated coverage of live events. In this work, the AI-based system performs shot framing (wide, mid and close-up shots), sequencing, and shot selection automatically (Wright et al 2020). However, the initial results show that the algorithm needs some improvement if it is to replace human operators.…”

Section: Ai For the Creative Industriesmentioning

confidence: 99%

Artificial intelligence in the creative industries: a review

2021

View full text Add to dashboard Cite

This paper reviews the current state of the art in artificial intelligence (AI) technologies and applications in the context of the creative industries. A brief background of AI, and specifically machine learning (ML) algorithms, is provided including convolutional neural networks (CNNs), generative adversarial networks (GANs), recurrent neural networks (RNNs) and deep Reinforcement Learning (DRL). We categorize creative applications into five groups, related to how AI technologies are used: (i) content creation, (ii) information analysis, (iii) content enhancement and post production workflows, (iv) information extraction and enhancement, and (v) data compression. We critically examine the successes and limitations of this rapidly advancing technology in each of these areas. We further differentiate between the use of AI as a creative tool and its potential as a creator in its own right. We foresee that, in the near future, ML-based AI will be adopted widely as a tool or collaborative assistant for creativity. In contrast, we observe that the successes of ML in domains with fewer constraints, where AI is the ‘creator’, remain modest. The potential of AI (or its developers) to win awards for its original creations in competition with human creatives is also limited, based on contemporary technologies. We therefore conclude that, in the context of creative industries, maximum benefit from AI will be derived where its focus is human-centric—where it is designed to augment, rather than replace, human creativity.

show abstract

Section: Ai For the Creative Industriesmentioning

confidence: 99%

Artificial intelligence in the creative industries: a review

2021

View full text Add to dashboard Cite

show abstract

“…The BBC has created a proof-of-concept system for automated coverage of live events. In this work, the AI-based system performs shot framing (wide, mid and close-up shots), sequencing, and shot selection automatically [230]. However, the initial results show that the algorithm needs some improvement if it is to replace human operators.…”

Section: Ai For the Creative Industriesmentioning

confidence: 99%

Artificial Intelligence in the Creative Industries: A Review

Anantrasirichai,

Bull

2020

Preprint

View full text Add to dashboard Cite

This paper reviews the current state of the art in Artificial Intelligence (AI) technologies and applications in the context of the creative industries. A brief background of AI, and specifically Machine Learning (ML) algorithms, is provided including Convolutional Neural Network (CNNs), Generative Adversarial Networks (GANs), Recurrent Neural Networks (RNNs) and Deep Reinforcement Learning (DRL). We categorise creative applications into five groups related to how AI technologies are used: i) content creation, ii) information analysis, iii) content enhancement and post production workflows, iv) information extraction and enhancement, and v) data compression. We critically examine the successes and limitations of this rapidly advancing technology in each of these areas. We further differentiate between the use of AI as a creative tool and its potential as a creator in its own right. We foresee that, in the near future, machine learning-based AI will be adopted widely as a tool or collaborative assistant for creativity. In contrast, we observe that the successes of machine learning in domains with fewer constraints, where AI is the 'creator', remain modest. The potential of AI (or its developers) to win awards for its original creations in competition with human creatives is also limited, based on contemporary technologies. We therefore conclude that, in the context of creative industries, maximum benefit from AI will be derived where its focus is human centric -where it is designed to augment, rather than replace, human creativity.

show abstract

“…[12]. Wright et al [70] describe and evaluate Ed, a system for automated camera and framing selection for live events. These methods are complementary to ours.…”

Section: Post-filming Video Directing and Editingmentioning

confidence: 99%

LookOut! Interactive Camera Gimbal Controller for Filming Long Takes

et al. 2022

View full text Add to dashboard Cite

The job of a camera operator is challenging, and potentially dangerous, when filming long moving camera shots. Broadly, the operator must keep the actors in frame while safely navigating around obstacles and while fulfilling an artistic vision. We propose a unified hardware and software system that distributes some of the camera operator’s burden, freeing the operator up to focus on safety and aesthetics during a take. Our real-time system provides solo operators with end-to-end control so that they can balance on-set responsiveness to action against planned storyboards and framing while looking where they are going. By default, we film without a field monitor. Our LookOut system is built around a lightweight commodity camera gimbal mechanism, with heavy modifications to the controller, which would normally just provide active stabilization. Our control algorithm reacts to speech commands, video, and a premade script. Specifically, our automatic monitoring of the live video feed saves the operator from distractions. In preproduction, an artist uses our graphical user interface (GUI) to design a sequence of high-level camera “behaviors.” Those can be specific, based on a storyboard, or looser objectives, such as “frame both actors.” Then, during filming, a machine-readable script, exported from the GUI, ties together with the sensor readings to drive the gimbal. To validate our algorithm, we compared tracking strategies, interfaces, and hardware protocols and collected impressions from (a) filmmakers who used all aspects of our system and (b) filmmakers who watched footage filmed using LookOut.

show abstract

Ai in Production: Video Analysis and Machine Learning for Expanded Live Events Coverage

Cited by 15 publications

References 5 publications

Artificial intelligence in the creative industries: a review

Artificial intelligence in the creative industries: a review

Artificial Intelligence in the Creative Industries: A Review

LookOut! Interactive Camera Gimbal Controller for Filming Long Takes

Contact Info

Product

Resources

About