The purpose of this paper is to study cohomology and deformations of O-operators on Lie triple systems. We define a cohomology of an O-operator T as the Lie-Yamaguti cohomology of a certain Lie triple system induced by T with coefficients in a suitable representation. Then we consider infinitesimal and formal deformations of O-operators from cohomological viewpoint. Moreover we provide relationships between O-operators on Lie algebras and associated Lie triple systems.

The aim of this paper is to introduce the notion of a mock-Lie bialgebra which is equivalent to a Manin triple of mock-Lie algebras. The study of a special case called coboundary mock-Lie bialgebra leads to the introduction the mock-Lie Yang-Baxter equation on a mock-Lie algebra which is an analogue of the classical Yang-Baxter equation on a Lie algebra. Note that a skew-symmetric solution of mock-Lie Yang-Baxter equation gives a mock-Lie bialgebra. Finally, the notation of O-operators are studied to construct skew-symmetric solution of mock-Lie Yang-Baxter equation.