Working memory (WM) is a memory system with a limited capacity that can process and store information temporarily in the performing of cognitive tasks. Despite WM is known to be influenced by age, the difficulty of tasks and trained or not from behavior studies, little is known about their relationships from the aspect of the brain functional network. Our goal was to explore the factor of aging-related changes of WM with brain functional networks. Methods: In this study, 25 healthy elderly and 23 healthy young volunteers were recruited for electroencephalogram (EEG) recording during the visual WM task with four difficulty levels (1-4 backs). In each back, we repeat the experiment with four sessions, and we add training sections between session one and session two as well as between session two and session three. However, we remove any training section between session three and session four in order to evaluate the impact of forgetting on WM in different age groups. After the experiment, we utilized graph theoretical analysis to characterize the brain functional network in three frequency bands (alpha, beta, and theta). Results: From the well-designed experiment, we found that physiological aging influences brain network connectivity and makes the functional brain network less differentiated. Moreover, there is an inverse relationship between alpha activity and WM load for the elderly group, which is absent in the young group. At the same time, theta band activity will be correlated with behavioral performance for the elderly group with WM training between sessions, which is also absent in the young group. To further study the influence of difficulty of tasks and training on the WM, we distinguish the tasks with quantified topological characteristics, and the classification results manifest that the training is more effective for the young group. Finally, through the establishment of a brain map before and after training, we find that the right parietal lobe plays an important role in the training of WM for the elderly group whereas the beta band plays an important role in WM for both the elderly group and the young group. Conclusion: Taken together, our findings