Mobile camera modules are manufactured by aligning and assembling multiple differently shaped part lenses. Therefore, selecting the part lenses to assemble from candidates (called cavities) and determining the directional angle of each part lens for assembly have been important issues to maximize production yield. Currently, this process is manually conducted by experts at the manufacturing site, and the manual assembly condition optimization carries the risk of reduced production yield and increased failure cost as it largely depends on one’s expertise. Herein, we propose an AI framework that determines the optimal assembly condition including the combination of part lens cavities and the directional angles of part lenses. To achieve this, we combine the genetic algorithm with convolutional bidirectional long-term short-term memory (C-BLSTM). To the best of our knowledge, this is the first study on lens module production finding the optimal combination of part lens cavities and directional angles at the same time using machine learning methods. Based on experimental results using real-world datasets collected by lens module manufacturers, the proposed framework outperformed existing algorithms with an F1 score of 0.89. Moreover, the proposed method (S2S-AE) for predicting the directional angles exhibited the best performance compared to existing algorithms with an accuracy of 78.19%.