Electronic Health Records (EHRs) play a crucial role in shaping predictive healthcare models, yet they encounter challenges such as significant data gaps and class imbalances. Traditional Graph Neural Networks (GNNs) approaches have limitations in fully leveraging neighbourhood data or demanding intensive computational regularisation. To address this challenge, we introduce CliqueFluxNet, a novel framework that innovatively constructs a patient similarity graph to maximise cliques, thereby highlighting strong inter-patient connections. At the heart of CliqueFluxNet lies its stochastic edge fluxing strategy — a dynamic process involving random edge addition and removal during training. This strategy aims to enhance the model's generalizability and mitigate overfitting. Our empirical analysis, conducted on MIMIC-III and eICU datasets, demonstrates significant progress in representation learning, particularly in scenarios with limited data availability. Qualitative assessments further underscore CliqueFluxNet’s effectiveness in extracting meaningful EHR representations, solidifying its potential for advancing GNN applications in healthcare analytics.