We report on the formation of bendable and edge-on poly[3-(4-carboxybutyl)thiophene-2,5-diyl] (P3CT) polymers thin layer used as a hole modification layer (HML) in the inverted perovskite solar cell. The aggregations of 2D layer-like P3CT polymers in dimethylformamide (DMF) solution can be formed via aromatic stacking interactions and/or hydrogen-bonding interactions with the different concentration from 0.01 wt% to 0.02 wt%, which highly influences the photovoltaic performance of the inverted perovskite solar cells. The atomic-force microscopic images and water droplet contact angle images show that the P3CT polymers modify the surface properties of the transparent conductive substrate and thereby dominating the formation of perovskite crystalline thin films, which play important roles in the highly efficient and stable perovskite solar cells. It is noted that the VOC (JSC) of the encapsulated solar cells values are maintained to be higher than 1.115 V (22 mA/cm2) after 104 days when an optimized stacked and hydrogen-bonded P3CT polymer is used as the HML. On the other hand, the solar cell showed a high long-term stability by maintaining 85% of the initial power conversion efficiency in the ambient air for 103 days.