Glycans on the surface of bacteria have diverse and essential biological functions and have widely been employed for treating various bacterial infectious diseases. Furthermore, these glycans comprise various functional groups, such as O-, N-, and carboxyl-modified, which significantly increase the diversity of glycan structures. These functional groups are not only crucial for glycans’ structural identity but are also essential for their biological functions. Therefore, a clear understanding of the biological functions of these modified groups in corresponding bacterial glycans is crucial for their medical applications. Thus far, the activities of functional groups in some biomedical active carbohydrates have been elucidated. It has been shown that some functional groups are key constituents of biologically active bacterial glycans, while others are actually not essential and may even mask the functions of the glycans. This paper reviews the structures of naturally occurring side-chain functional groups in glycans located on the bacterial surface and their roles in immunological responses.