A method for the optical data encryption and decryption based on the sweeping computational ghost imaging is proposed. This method is governed on reconstructing the ghost imaging by one-by-one sweeping of rows or columns of the random generated matrices. Proposed encryption in this paper is defined by these sweeping row and column matrices and basic mathematical operators between them. Introduced encryption can be employed as symmetric and asymmetric encryptions. In the symmetric system, cross-operator and the row and column matrices were assumed as private keys, defining the four basic operators as decryption keys. Whereas, in the asymmetric system, permutation of three possible cases was considered as private keys, leaving cross-operator and row and column selection sequence as public keys. The number of pixel changing Rate (NPCR) as a parameter that evaluates the strength of image encryption versus differential attacks was calculated for each case which show high security data transfer to the user. In the asymmetric systems, even with eavesdropping 50% of data, no useful information was obtained from the image or data. In the symmetric systems, with eavesdropping 100% of data, no useful information was obtained by multiple attacking the encrypted data. Offered high security along with achievable high speeds and compact data packages classify sweeping computational ghost imaging among the best applicable methods for optical data encryption.