Several experiments reported hints for the violation of lepton flavor or lepton flavor universality in processes involving muons. Most prominently, there is the hint for a non-zero rate of the flavor violating Higgs decay h → τ µ at the LHC, as well as the hint for lepton flavor universality violation in rare B meson decays at LHCb. In addition, also the long standing discrepancy in the anomalous magnetic moment of the muon motivates new physics connected to muons. A symmetry which violates lepton flavor universality, is Lµ − Lτ : the difference of muon-number and tau-number. We show that adding vector-like fermions to a Lµ − Lτ theory generates naturally an effect in the anomalous magnetic moment of the muon and h → τ µ, while effects in other τ → µ transitions are systematically suppressed by symmetry arguments. We find that if Lµ − Lτ is gauged it is possible to also accommodate the discrepant b → sµµ data while predicting a τ → 3µ and a modified h → µµ rate within reach of upcoming experiments.