Purpose: Solution to the problem of optimizing the determinants of matrices with a modulus of entries < 1. Developing a theory of such matrices based on preliminary research results. Methods: Extreme solutions (in terms of the determinant) are found by minimizing the absolute values of orthogonal matrix elements, and their subsequent classification. Results: Matrices of orders equal to prime Fermat numbers have been found. They are special, as their absolute determinant maximums can be reached on a simple structure. We provide a precise evaluation of the determinant maximum for these matrices and formulate a conjecture about it. We discuss the close relation between the solutions of extremal problems with the limitation on the matrix column orthogonality and without it. It has been shown that relative maximums of orthogonality-limited matrix determinants correspond to absolute maximums of orthogonality-unlimited matrix determinants. We also discuss the ways to build extremal matrix families for the orders equal to Mersenne numbers. Practical relevance: Maximum determinant matrices are used extensively in the problems of error-free coding, compression and masking of video information. Programs for maximum determinant matrix search and a library of constructed matrices are used in the mathematical network “mathscinet.ru” along with executable online algorithms.