In this paper, we studied some conservative problems, and constructed a low memory approach to its solution. By means of component-wise approximation approach, a diagonal update to the Jacobian matrix was derived. Numerical examples are given to illustrate the proposed scheme.