In recent years, significant attention has been paid towards the study and application of mixed matrix nanofibrous membranes for water treatment. The focus of this study is to develop and characterize functional polysulfone (PSf)-based composite nanofiltration (NF) membranes comprising two different oxides, such as graphene oxide (GO) and zinc oxide (ZnO) for arsenic removal from water. PSf/GO- and PSf/ZnO-mixed matrix NF membranes were fabricated using the electrospinning technique, and subsequently examined for their physicochemical properties and evaluated for their performance for arsenite–As(III) and arsenate–As(V) rejection. The effect of GO and ZnO on the morphology, hierarchical structure, and hydrophilicity of fabricated membranes was studied using a scanning electron microscope (SEM), small and ultra-small angle neutron scattering (USANS and SANS), contact angle, zeta potential, and BET (Brunauer, Emmett and Teller) surface area analysis. Fourier transform infrared spectroscopy (FTIR), X-ray diffraction (XRD), and X-ray photoelectron spectroscopy (XPS) were used to study the elemental compositions and polymer-oxide interaction in the membranes. The incorporation of GO and ZnO in PSf matrix reduced the fiber diameter but increased the porosity, hydrophilicity, and surface negative charge of the membranes. Among five membrane systems, PSf with 1% ZnO has the highest water permeability of 13, 13 and 11 L h−1 m−2 bar−1 for pure water, As(III), and As(V)-contaminated water, respectively. The composite NF membranes of PSf and ZnO exhibited enhanced (more than twice) arsenite removal (at 5 bar pressure) of 71% as compared to pristine PSf membranes, at 43%, whereas both membranes showed only a 27% removal for arsenate.