Strong party discipline is a core feature of Westminster parliamentary systems. Parties typically compel Members of Parliament (MPs) to support the party position regardless of MPs' individual preferences. Rebellion, however, does occur. Using an original dataset of MP votes and speeches in the British House of Commons from 1992 to 2015, coupled with new estimations of MPs' ideological positions within their party, we find evidence that MPs use rebellion to strategically differentiate themselves from their party. The strategy that MPs employ is contingent upon an interaction of ideological extremity with party control of government. Extremists are loyal when their party is in the opposition, but these same extremists become more likely to rebel when their party controls government. Additionally, they emphasize their rebellion through speeches. Existing models of rebellion and party discipline do not account for government agenda control and do not explain these patterns.