We discuss some general aspects of commutators of local operators in Lorentzian CFTs, which can be obtained from a suitable analytic continuation of the Euclidean operator product expansion (OPE). Commutators only make sense as distributions, and care has to be taken to extract the right distribution from the OPE. We provide explicit computations in two and four-dimensional CFTs, focusing mainly on commutators of components of the stress-tensor. We rederive several familiar results, such as the canonical commutation relations of free field theory, the local form of the Poincaré algebra, and the Virasoro algebra of two-dimensional CFT. We then consider commutators of light-ray operators built from the stress-tensor. Using simplifying features of the light sheet limit in four-dimensional CFT we provide a direct computation of the BMS algebra formed by a specific set of light-ray operators in theories with no light scalar conformal primaries. In four-dimensional CFT we define a new infinite set of light-ray operators constructed from the stress-tensor, which all have well-defined matrix elements. These are a direct generalization of the two-dimensional Virasoro light-ray operators that are obtained from a conformal embedding of Minkowski space in the Lorentzian cylinder. They obey Hermiticity conditions similar to their two-dimensional analogues, and also share the property that a semi-infinite subset annihilates the vacuum.