A primitive prime divisor of an element a n of a sequence (a m ) m≥0 is a prime p that divides a n , but does not divide a m for all m < n. The Zsigmondy set Z of the sequence is the set of n such that a n has no primitive prime divisors. Let f : X → X be a self-morphism of a variety, let D be an effective divisor on X, and let P ∈ X, all defined overQ. We consider the Zsigmondy set Z(X, f, P, D) of the sequence defined by the arithmetic intersection of the f -orbit of P with D. Under various assumptions on X, f , D, and P , we use Vojta's conjecture with truncated counting function to prove that the set of points f n (P ) with n ∈ Z(X, f, P, D) is not Zariski dense in X. The classical definition does not include the requirement that a m = 0, but including this condition allows for cleaner statements of theorems. Note that if some a m = 0, then a m is divisible by every prime, so if we didn't exclude such a m , then a n with n > m would never have a primitive prime divisor.