In this paper we are concerned with the performance of structural stability of torsion in square cross section of a beam with holes. The critical load is defined as the smallest load at which the equilibrium of the structure fails to be stable as the load is slowly increased from zero. The beams subjected to torsion are frequently encountered in general structures and these forces influence to the stability of structure. The boundary element method is found to be very efficient and accurate for the analysis of torsion problems including complex boundary conditions with respect to its simplicity and generality. In this paper, it is required to derive the boundary element formulation for torsion problem and integrate directly on the discrete boundary. To investigate the validity of the developed computer program, three distinctly solid cross-sections which are elliptical, rectangular and triangular one are analyzed, and comparisons are made with analytical approaches where these can also be used.