Consider the Hilbert scheme of points on a higher dimensional affine space. Its component is elementary if it parameterizes irreducible subschemes. We characterize reduced elementary components in terms of tangent spaces and provide a computationally efficient way of finding such components. As an example, we find an infinite family of elementary and generically smooth components on the affine four-space. We analyse singularities and formulate a conjecture which would imply the non-reducedness of the Hilbert scheme. Our main tool is a generalization of the Bia lynicki-Birula decomposition for this singular scheme.