This study linked the chemical potential change to high specific filtration resistance (SFR) of gel layer, and then proposed a novel membrane fouling mechanism regarding gel layer filtration, namely, Flory-Huggins based filtration mechanism. A mathematical model for this mechanism was theoretically deduced. Agar was used as a model polymer for gel formation. Simulation of the mathematical model for agar gel showed that volume fraction of polymer and Flory-Huggins interaction parameter were the two key factors governing the gel SFR, whereas, pH and ionic strength were not related with the gel SFR. Filtration tests of gel layer showed that the total SFR value, effects of pH and ionic strength on the gel SFR well agreed with the perditions of model’s simulation, indicating the real occurrence of this mechanism and the feasibility of the proposed model. This mechanism can satisfactorily explain the extremely high SFR of gel layer, and improve fundamental insights into membrane fouling regarding gel layer filtration.