This paper investigates the problem of dynamic investment and consumption in a market, where a risky asset evolves with jumps and capital gains are taxed. In addition, the investor's behavior of tax evasion is taken into account, and tax evasion is subject to penalty when it is uncovered by audits. Using dynamic programming approach, we derive an analytical solution for an investor with the CRRA utility. We find the following: (1) jumps in the risky asset do not affect the optimal tax evasion strategy; (2) jump risk lessens the optimal fraction of wealth in the risky asset; (3) tax evasion can be reduced by increasing the fine and/or the frequency of tax audits; (4) the effects of the jumps, audits and penalty on the optimal consumption are determined by the degree of risk aversion of the investor.