SummaryA general topology optimization method, which is capable of simultaneous design of density and orientation of anisotropic material, is proposed by introducing orientation design variables in addition to the density design variable. In this work, the Cartesian components of the orientation vector are utilized as the orientation design variables. The proposed method supports continuous orientation design, which is out of the scope of discrete material optimization approaches, as well as design using discrete angle sets. The advantage of this approach is that vector element representation is less likely to fail into local optima because it depends less on designs of former steps, especially compared with using the angle as a design variable (Continuous Fiber Angle Optimization) by providing a flexible path from one angle to another with relaxation of orientation design space. An additional advantage is that it is compatible with various projection or filtering methods such as sensitivity filters and density filters because it is free from unphysical bound or discontinuity such as the one at θ = 2π and θ = 0 seen with direct angle representation. One complication of Cartesian component representation is the point‐wise quadratic bound of the design variables; that is, each pair of element values has to reside in a given circular bound. To overcome this issue, we propose an isoparametric projection method, which transforms box bounds into circular bounds by a coordinate transformation with isoparametric shape functions without having the singular point that is seen at the origin with polar coordinate representation. A new topology optimization method is built by taking advantage of the aforementioned features and modern topology optimization techniques. Several numerical examples are provided to demonstrate its capability. Copyright © 2014 John Wiley & Sons, Ltd.