Secretory immunoglobulin A (IgA) plays a crucial role in the mucosal immunity for preventing the invasion of the exogenous antigens, however, little has been understood about the neutralizing activity of serum IgA. Here, to examine the role of IgA antibodies against COVID-19 illnesses, we determined the neutralizing activity of serum/plasma IgG and IgA purified from previously SARS-CoV-2-infected and COVID-19 mRNA-vaccine-receiving individuals. We found that serum/plasma IgA possesses substantial but rather modest neutralizing activity against SARS-CoV-2 compared to IgG with no significant correlation with the disease severity. Neutralizing IgA and IgG antibodies achieved the greatest activity at approximately 25 and 35 days after symptom onset, respectively. However, neutralizing IgA activity quickly diminished and went down below the detection limit approximately 70 days after onset, while substantial IgG activity was observed till 200 days after onset. The total neutralizing activity in sera/plasmas of those with COVID-19 largely correlated with that in purified-IgG and purified-IgA and levels of anti-SARS-CoV-2-S1-binding IgG and anti-SARS-CoV-2-S1-binding IgA. In individuals who were previously infected with SARS-CoV-2 but had no detectable neutralizing IgA activity, a single dose of BNT162b2 or mRNA-1273 elicited potent serum/plasma neutralizing IgA activity but the second dose did not further strengthen the neutralization antibody response. The present data show that the systemic immune stimulation with natural infection and COVID-19 mRNA-vaccines elicit both SARS-CoV-2-specific neutralizing IgG and IgA response in serum, but the IgA response is modest and diminishes faster compared to IgG response.