A hierarchical all‐solid‐state electrolyte based on nitrile materials (SEN) is prepared via in situ synthesis method. This hierarchical structure is fabricated by in situ polymerizing the cyanoethyl polyvinyl alcohol (PVA‐CN) in succinonitrile (SN)‐based solid electrolyte that is filled in the network of polyacrylonitrile (PAN)‐based electrospun fiber membrane. The crosslinked PVA‐CN polymer framework is uniformly dispersed in the SN‐based solid electrolyte, which can strongly enhance its mechanical strength and keeps it in a quasi‐solid state even over the melting point. The electrospun fiber membrane efficiently reduces the thickness of SEN film besides a further improvement in strength. Because of the unique hierarchical structure and structure similarity among the raw materials, the prepared SEN film exhibits high room‐temperature ionic conductance (0.30 S), high lithium ion transference number (0.57), favorable mechanical strength (15.31 MPa), excellent safety, and good flexibility. Furthermore, the in situ synthesis ensures an excellent adhesion between SEN and electrodes, which leads to an outstanding electrochemical performance for the assembled LiFePO4/SEN/Li cells. Both the superior performance of SEN and the simple fabricating process of SEN‐based all‐solid‐state cells make it potentially as one of the most promising electrolyte materials for next generation lithium‐ion batteries.