Multitask deep co-design for extended depth of field and depth from defocus

Abstract

Deep co-design methods have been proposed to optimize simultaneously optical and neural network parameters for many separate tasks such as high dynamic range, extended depth of field (EDOF), depth from defocus (DfD), object detection or pose estimation. In contrast, we study the multi-task co-design of an imaging system for two antagonist tasks: EDOF and DfD. We model and optimize a chromatic Cooke triplet using differentiable ray tracing, and we compare the performances for DfD and EDOF tasks, in a single, parallel and collaborative optimization scheme. We show how one task can benefit from the result of the other task. We also explore the benefit of the local positional information to process images with spatially varying point spread functions related to optical field aberrations.