In recent years secret permutations have been widely used for protecting different types of multimedia data, including speech files, digital images and videos. Based on a general model of permutation-only multimedia ciphers, this paper performs a quantitative cryptanalysis on the performance of these kind of ciphers against plaintext attacks. When the plaintext is of size M × N and with L different levels of values, the following quantitative cryptanalytic findings have been concluded under the assumption of a uniform distribution of each element in the plaintext: 1) all permutation-only multimedia ciphers are practically insecure against known/chosen-plaintext attacks in the sense that only O (log L (M N )) known/chosen plaintexts are sufficient to recover not less than (in an average sense) half elements of the plaintext; 2) the computational complexity of the known/chosen-plaintext attack is only O(n · (M N ) 2 ), where n is the number of known/chosen plaintexts used. When the plaintext has a non-uniform distribution, the number of required plaintexts and the computational complexity is also discussed. to demonstrate the real performance of the known-plaintext attack for a typical permutation-only image cipher.