With the increasing complexity of 3-D semiconductor structures, the use of optical critical dimension (OCD) metrology has become a popular solution due to its accuracy and fast inference time. Machine learning has been widely adopted in this field to further improve the efficiency and precision of OCD metrology. Especially for high aspect ratio structures such as DRAM and VNAND, where the required computing power for physical modeling increases exponentially, the importance of machine learning with reference data is crucial. However, one significant challenge of the machine learningbased metrology under rapidly changing process condition is the limitation of available labeled data, which causes overfitting and decreases recipe reliability in the manufacturing process as the cost of wafer consumption increases. To utilize machine learning algorithms in mass production, the development of robust algorithms that can be optimized with few-shot data is required. In this paper, we propose a few-shot machine learning algorithm that includes i) wafer-level statistical information-based data augmentation and ii) anomaly detection to automatically remove data with measurement errors. The proposed algorithm shows superior accuracy, repeatability, and in-wafer uniformity compared to the benchmark algorithm in tests with manufacturing phase data. Additionally, this robustness can be sustained with the minimum amount of data in metrology, as only 9 reference training data are used on three design of experiment (DoE) wafers. The proposed optimized solution is expected to contribute to the reduction of measurement costs and production yields of highly complicated 3D semiconductor structures.