Nowadays, endophytic fungi represent a rich source of biological active compounds. In the current study, twelve endophytic fungal species were isolated from Avicennia marina leaves. From the isolates, Aspergillus niger, Penicillium rubens and Alternaria alternata recorded the highest isolation frequency (80%), relative density (12.5%) and antimicrobial activity. The antimicrobial and anticancer activities of P. rubens were more effective than those of A. niger and A. alternata; therefore, its identification was confirmed via the ITS rRNA gene. Filtrate extracts of P. rubens, A. alternata and A. niger were analyzed using GC-MS and showed different detected constituents, such as acetic acid ethyl ester, N-(4,6-Dimethyl-2-pyrimidinyl)-4-(4-nitrobenzylideneamino) benzenesulfonamide, 1,2-benzenedicarboxylic acid, hexadecanoic acid and octadecanoic acid. Filtrate extract of P. rubens exhibited the presence of more compounds than A. alternata and A. niger. Bacillus subtilis, Staphylococcus aureus, Escherichia coli, Pseudomonas aeruginosa, Candida albicans and Aspergillus fumigatus were more inhibited by P. rubens extract than A. alternata or A. niger, with inhibition zones of 27.2 mm, 22.21 mm, 26.26 mm, 27.33 mm, 28.25 mm and 8.5 mm, respectively. We observed negligible cytotoxicity of P. rubens extract against normal cells of human lung fibroblasts (WI-38 cell line), unlike A. alternata and A. niger extracts. Proliferation of prostate cancer (PC-3) was inhibited using P. rubens extract, exhibiting mortality levels of 75.91% and 76.2% at 200 µg/mL and 400 µg/mL of the extract. Molecular docking studies against the crystal structures of C. albicans (6TZ6) and the cryo-EM structure of B. subtilis (7CKQ) showed significant interactions with benzenedicarboxylic acid and N-(4,6-dimethyl-2-pyrimidinyl)-4-(4-nitrobenzylideneamino) benzenesulfonamide as a constituent of P. rubens extract. N-(4,6-dimethyl-2-pyrimidinyl)-4-(4-nitrobenzylideneamino) benzenesulfonamide had the highest scores of −6.04905 kcal/mol and −6.590 kcal/mol towards (6tz6) and (7CKQ), respectively.