The fourth industrial revolution (“Industry 4.0”) has caused an escalating need for smart technologies in manufacturing industries. Companies are examining various cutting-edge technologies to realize smart manufacturing and construct smart factories and are devoting efforts to improve their maturity level. However, productivity improvement is rarely achieved because of the large variety of new technologies and their wide range of applications; thus, elaborately setting improvement goals and plans are seldom accomplished. Fortunately, many researchers have presented guidelines for diagnosing the smartness maturity level and systematic directions to improve it, for the eventual improvement of productivity. However, most research has focused on mass production industries wherein the overall smartness maturity level is already high (e.g., high-level automation). These studies thus have limited applicability to the shipbuilding industry, which is basically a built-to-order industry. In this study, through a technical demand survey of the shipbuilding industry and an investigation of existing smart manufacturing and smart factories, the keywords of connectivity, automation, and intelligence were derived and based on these keywords, we developed a new diagnostic framework for smart shipyard maturity level assessment. The framework was applied to eight shipyards in South Korea to diagnose their smartness maturity level, and a data envelopment analysis (DEA) was performed to confirm the usefulness of the diagnosis results. By comparing the DEA models, the results with the smart level as an input represents the actual efficiency of shipyards better than the results of conventional models.