Recent breakthroughs in graph representation learning (GRL) methods have vastly advanced many real-world applications where graph structures naturally arise, such as drug discoveries, molecular property predictions, and social recommendation systems. The rapid developments of GRL with applications in specific scientific domains are largely enabled by graph learning libraries such as PyTorch Geometric, which provide infrastructures that allow domain scientists to share domain-specific benchmarking datasets, and GRL researchers to adapt and design specialized GRL methods for these particular tasks. Meanwhile, over the past two decades, network biology has demonstrated superior value in harvesting biological insights from network data, such as identifying genes' associated functions, diseases, and traits. Nevertheless, the GRL community faces a significant barrier in working with biological networks because of the tedious and specialized (pre-)processing steps required to set up machine learning (ML)-ready benchmarking datasets. Here, we present nleval, a Python package containing reusable modules that enable researchers to effortlessly set up PyG-compatible ML-ready datasets using data downloaded from public databases. We expect nleval to help network biologists set up custom benchmarking datasets for answering specific biological questions of their interests and help GRL researchers adapt these datasets for designing new specialized GRL architectures.