The development, universality and miniaturization of electronic devices leads to the search for new piezoelectric materials, among which recently, polymers play an increasingly important role. In this work, composites based on two types of polyethylene—high density polyethylene (HDPE), and medium density polyethylene (MDPE)—and aluminosilicate fillers were obtained by extrusion process. This method allowed obtaining flexible electrets in the form of a thin film after polarization under a constant electric field of 100 V/μm. The morphology of the composites was characterized by scanning electron microscopy, whereas the crystallinity was determined by X-ray diffraction. The mechanical properties and thermal stability of the composites were examined by means of tensile tests and thermogravimetry, respectively. The piezoelectric characteristics were appointed by measuring the electric charge and the voltage in the polarized samples. Piezoelectric coefficients, and the stability of electrets over time were also determined. Moreover, the effect of film orientation on piezoelectric properties was investigated. Composites with appropriate morphology (i.e., well dispersed filler particles in the polymer matrix and formed holes) were obtained which ensured permanent electrical polarization. It was found that the best piezoelectric, mechanical properties and thermal stability exhibits HDPE composite with 5% of aluminosilicate filler.