Conformational behaviour of hydrophobically modified (HM) polymer gels in solutions of nonionic surfactants is studied theoretically. A HM gel contains hydrophobic side chains (stickers) grafted to its subchains. Hydrophobic stickers are capable to aggregate into joint micelles with surfactant molecules. Micelles containing more than one sticker serve as additional physical cross-links of the network, and their formation causes gel shrinking. In the proposed theoretical model, the interior of the gel/surfactant complex is treated as an array of densely packed spherical polymer brushes consisting of gel subchains tethered to the surface of the spherical sticker/surfactant micelles. Effect of stickers length and grafting density, surfactant concentration and hydrophobicity on gel swelling as well as on hydrophobic association inside it is analyzed. It is shown that increasing surfactant concentration can result in a gel collapse, which is caused by surfactant-induced hydrophobic aggregation of stickers, and a successive gel reswelling. The latter should be attributed to a growing fraction of surfactants in joint aggregates and, hence, increasing number of micelles containing only one sticker and not participating in gel physical cross-linking. In polyelectrolyte (PE) gels hydrophobic aggregation is opposed by osmotic pressure of mobile counterions, so that at some critical ionization degree hydrophobic association is completely suppressed. Hydrophobic modification of polymers is shown to open new ways for controlling gel responsiveness. In particular, it is discussed that incorporation of photosensitive groups into gel subchains and/or surfactant tail could give a possibility to vary the gel volume by light. Since hydrophobic aggregation regularities in gels and solutions are common, we hope our findings will be useful for design of polymer based self-healing materials as well.