Metagenomics has opened up a vast pool of genes for putative, yet uncharacterized, enzymes. It widens our knowledge on the enzyme diversity world and discloses new families for which a clear classification is still needed, as is exemplified by glycoside hydrolase family-3 (GH3) proteins. Herein, we describe a GH3 enzyme (GlyA1) from resident microbial communities in strained ruminal fluid. The enzyme is a β-glucosidase/β-xylosidase that also shows β-galactosidase, β-fucosidase, α-arabinofuranosidase, and α-arabinopyranosidase activities. Short cello- and xylo-oligosaccharides, sophorose and gentibiose, are among the preferred substrates, with the large polysaccharide lichenan also being hydrolyzed by GlyA1. The determination of the crystal structure of the enzyme in combination with deletion and site-directed mutagenesis allowed identification of its unusual domain composition and the active site architecture. Complexes of GlyA1 with glucose, galactose, and xylose allowed picturing the catalytic pocket and illustrated the molecular basis of the substrate specificity. A hydrophobic platform defined by residues Trp-711 and Trp-106, located in a highly mobile loop, appears able to allocate differently β-linked bioses. GlyA1 includes an additional C-terminal domain previously unobserved in GH3 members, but crystallization of the full-length enzyme was unsuccessful. Therefore, small angle x-ray experiments have been performed to investigate the molecular flexibility and overall putative shape. This study provided evidence that GlyA1 defines a new subfamily of GH3 proteins with a novel permuted domain topology. Phylogenetic analysis indicates that this topology is associated with microbes inhabiting the digestive tracts of ruminants and other animals, feeding on chemically diverse plant polymeric materials.