The applications of non-square binary matrices span many domains including mathematics, error-correction coding, machine learning, data storage, navigation signals, and cryptography. In particular, they are employed in the McEliece and Niederreiter public-key cryptosystems. For the parity check matrix of these cryptosystems, a systematic non-square binary matrix [Formula: see text] with dimensions [Formula: see text], [Formula: see text], [Formula: see text], there exist [Formula: see text] distinct inverse matrices. This article presents an algorithm to generate these matrices as well as a method to construct a random inverse matrix. Then it is extended to non-square matrices in arbitrary fields. This overcomes the limitations of the Moore-Penrose and Gauss-Jordan methods. The application to public-key cryptography is also discussed.