Andrographis paniculata is an important herbal medicine widely used in several Asian countries for the treatment of various diseases due to its broad range of pharmacological activities. The present study reports that A. paniculata extracts potently inhibit the growth of liver (HepG2 and SK-Hep1) and bile duct (HuCCA-1 and RMCCA-1) cancer cells. A. paniculata extracts with different contents of major diterpenoids, including andrographolide, 14-deoxy-11,12-didehydroandrographolide, neoandrographolide, and 14-deoxyandrographolide, exhibited a different potency of growth inhibition. The ethanolic extract of A. paniculata at the first true leaf stage, which contained a high amount of 14-deoxyandrographolide but a low amount of andrographolide, showed a cytotoxic effect to cancer cells about 4 times higher than the water extract of A. paniculata at the mature leaf stage, which contained a high amount of andrographolide but a low amount of 14-deoxyandrographolide. Andrographolide, not 14-deoxy-11,12-didehydroandrographolide, neoandrographolide, or 14-deoxyandrographolide, possessed potent cytotoxic activity against the growth of liver and bile duct cancer cells. The cytotoxic effect of the water extract of A. paniculata at the mature leaf stage could be explained by the present amount of andrographolide, while the cytotoxic effect of the ethanolic extract of A. paniculata at the first true leaf stage could not. HuCCA-1 cells showed more sensitivity to A. paniculata extracts and andrographolide than RMCCA-1 cells. Furthermore, the ethanolic extract of A. paniculata at the first true leaf stage increased cell cycle arrest at the G0/G1 and G2/M phases, and induced apoptosis in both HuCCA-1 and RMCCA-1 cells. The expressions of cyclin-D1, Bcl-2, and the inactive proenzyme form of caspase-3 were reduced by the ethanolic extract of A. paniculata in the first true leaf stage treatment, while a proapoptotic protein Bax was increased. The cleavage of poly (ADP-ribose) polymerase was also found in the ethanolic extract of A. paniculata in the first true leaf stage treatment. This study suggests that A. paniculata could be a promising herbal plant for the alternative treatment of intrahepatic cholangiocarcinoma.