In this paper, the effects of altering the organizational setting of distributed adaptive search processes in the course of search are investigated. We put particular emphasis on the complexity of interactions between partial search problems assigned to search agents. Employing an agent-based simulation based on the framework of NK landscapes we analyze different temporal change modes of the organizational set-up. The organizational properties under change include, for example, the coordination mechanisms among search agents. Results suggest that inducing organizational dynamics has the potential to increase the effectiveness of distributed adaptive search processes with respect to various performance measures like the final performance achieved at the end of the search, the chance to find the optimal solution of the search problem, or the average performance per period achieved during the search process. However, results also indicate that the mode of temporal change in conjunction with the complexity of the search problem considerably affects the order of magnitude of these beneficial effects. In particular, results suggest that organizational dynamics induces a shift towards more exploration, i.e., discovery of new areas in the fitness landscape, and less exploitation, i.e., stepwise improvement.