In the present study, we clarify the micro-to mesoscopic deformation behavior of semicrystalline polymers by the finite element homogenization method. The crystalline plasticity theory using a penalty method for the inextensibility constraint in the chain direction and the nonaffine molecular chain network theory were used to the representation of the deformation behavior of crystalline and amorphous phases, respectively, in the composite microstructure of a semicrystalline polymer. Various directional tensions are applied to the two dimensional plane-strain unit cell model of a composite microstructure. The results reveal a highly anisotropic deformation behavior caused by the rotation of the chain direction and lamella interface, which depends on tensile direction and manifests as substantial hardening/softening in the early stage of deformation. The mesoscopic structure of a semicrystalline polymer was modeled using a voronoi polygon comprised of composite microstructures with different lamella interface directions. The initial isotropy of the response of the mesoscopic scale was verified. Due to their interaction with the surrounding grains, the individual grains of the mesoscopic scale show a conservative response as compared with that of the unit cell, and a very nonuniform response depending on the location of the respective grain is observed; these are typical of the mesoscopic response of semicrystalline polymers.