We theoretically study the atomic structure and energetics of silicon and silicon-nitrogen impurities in graphene. Using density-functional theory, we get insight into the atomic structures of the impurities, evaluate their formation energies and assess their abundance in realistic samples. We find that nitrogen, as well as oxygen and hydrogen, are trapped at silicon impurities, considerably altering the electronic properties of the system. Furthermore, we show that nitrogen doping can induce local magnetic moments resulting in spin-dependent transport properties,even though neither silicon nor nitrogen impurities are magnetic by themselves. To simulate large systems with many randomly distributed impurities, we derive tight-binding models that describe the effects of the impurities on graphene π electron structure. Then by using the linear-scaling real-space Kubo-Greenwood method, we evaluate the transport properties of large-scale systems with random distribution of impurities, and find the fingerprint-like scattering cross sections for each impurity type. The transport properties vary widely, and our results indicate that some of the impurities can even induce strong localization in realistic graphene samples.