Before Automated Driving Systems (ADS) with full driving automation (SAE Level 5) are placed into practical use, the issue of calibrating drivers’ initial trust in Level 5 ADS to an appropriate degree to avoid inappropriate disuse or improper use should be resolved. This study aimed to identify the factors that affected drivers’ initial trust in Level 5 ADS. We conducted two online surveys. Of these, one explored the effects of automobile brands and drivers’ trust in automobile brands on drivers’ initial trust in Level 5 ADS using a Structural Equation Model (SEM). The other identified drivers’ cognitive structures regarding automobile brands using the Free Word Association Test (FWAT) and summarized the characteristics that resulted in higher initial trust among drivers in Level 5 ADS. The results showed that drivers’ trust in automobile brands positively impacted their initial trust in Level 5 ADS, which showed invariance across gender and age. In addition, the degree of drivers’ initial trust in Level 5 ADS was significantly different across different automobile brands. Furthermore, for automobile brands with higher trust in automobile brands and Level 5 ADS, drivers’ cognitive structures were richer and varied, which included particular characteristics. These findings suggest the necessity of considering the influence of automobile brands on calibrating drivers’ initial trust in driving automation.