The emerging COVID-19 caused by SARS-CoV-2 infection poses severe challenges to global public health. Serum antibody testing is becoming one of the critical methods for the diagnosis of COVID-19 patients. We investigated IgM and IgG responses against SARS-CoV-2 nucleocapsid (N) and spike (S) protein after symptom onset in the intensive care unit (ICU) and non-ICU patients. 130 blood samples from 38 COVID-19 patients were collected. The levels of IgM and IgG specific to N and S protein were detected by ELISA. A series of blood samples were collected along the disease course from the same patient, including 11 ICU patients and 27 non-ICU patients for longitudinal analysis. N and S specific IgM and IgG (N-IgM, N-IgG, S-IgM, S-IgG) in non-ICU patients increased after symptom onset. N-IgM and S-IgM in some non-ICU patients reached a peak in the second week, while N-IgG and S-IgG continued to increase in the third week. The combined detection of N and S specific IgM and IgG could identify up to 75% of SARS-CoV-2 infected patients in the first week. S-IgG was significantly higher in non-ICU patients than in ICU patients in the third week. In contrast, N-IgG was significantly higher in ICU patients than in non-ICU patients. The increase of S-IgG positively correlated with the decrease of C-reactive protein (CRP) in non-ICU patients. N and S specific IgM and IgG increased gradually after symptom onset and can be used for detection of SARS-CoV-2 infection. Analysis of the dynamics of S-IgG may help to predict prognosis.