In this study, the effect of the notch locations on the Charpy-V toughness values of the all-welded joint obtained using robotic flux-cored arc welding was investigated with respect to microstructures at the notch locations. Charpy impact tests were performed through the thickness with notch location at the centerline as well as off-set regions of the weld metal in addition to the microhardness measurements conducted. The detailed weld metal characterization was conducted using a stereo microscope, optical microscope, and scanning electron microscope at the same location where the Charpy tests and microhardness tests were performed. The sub-zero impact toughness test results indicated that the columnar weld metal regions exhibited low toughness values while the centerline microstructure consisting of mainly reheated regions displayed much higher toughness values even at the test temperature of −60 °C, satisfying the toughness requirement of the requested 47 J value. It is concluded that a small variation of the through-thickness notch position may result in different toughness values for the same weld metal. On this basis, the notching procedure of the Charpy-V samples for the multi-pass weld metal should be conducted with care and obtained results should be explained with respective notch position and microstructure.