Simple and convergent synthesis of a tetra- and a trisaccharide portions of an antitumor compound Julibroside J(28), isolated from Albizia julibrissin, that showed significant in vitro antitumor activity against HeLa, Bel-7402 and PC-3M-1E8 cancer cell lines is reported. The tetrasaccharide has been synthesized as its p-methoxyphenyl glycoside starting from commercially available D-glucose, L-rhamnose and L-arabinose. The trisaccharide part has been synthesized from commercially available N-acetyl D-glucosamine, D-fucose and D-xylose using simple protecting group manipulations. Sulfuric acid immobilized on silica has been used successfully as a Brönsted acid catalyst for the crucial glycosylation steps.