This paper proposes a new data-driven method to generate three-dimensional fragility surfaces for post-earthquake damage assessment of reinforced concrete (RC) shear walls (SWs) using image-based damage features. A research database comprised of 212 images corresponding to 66 damaged reinforced concrete shear walls tested under quasi-static cyclic loads is utilized. The walls are categorized into three damage states defined based on different load points along the backbone curve. Convolutional kernel-based filters are then employed to measure the crack patterns and crushed areas from images of the damaged walls. A set of 360,000 sub-cracks from the 212 images is analyzed using Gaussian mixture modeling to distinguish between shear and flexural cracking. These two types of cracking, along with crushing, are the prominent characteristics used to determine the damage states of the RCSWs. The extracted features of shear and flexural cracking are also compressed into a unique cracking index using the Principal Component Analysis theory for dimensionality reduction. Using the compressed feature of cracking and crushing, a new methodology for generating three-dimensional fragility surfaces, entitled the Box-counting method, is introduced, and the damage surfaces are developed based on visual damage features. The damage surfaces are finally formulated using statistical models or machine learning regression learners. The proposed fragility surface can be used for postearthquake damage state identification, risk assessment, and loss estimation of damaged RCSWs.