We investigate an agent-based model of opinion dynamics with two types of social response: conformity and independence. Conformity is introduced to the model analogously as in the Sznajd model or q-voter model, which means that only unanimous group exerts peer pressure on individuals. The novelty, in relation to previous versions of the q-voter model, is memory possessed by each agent and external noise T , which plays the role of social temperature. Each agent has its own memories of past experiences related to the social costs and benefits of being independent or conformist. If an agent was awarded in past more for being independent, it will have a greater tendency to be independent than conformist and vice versa. We will show that depending on the social temperature T the system spontaneously organizes into one of two regimes. Below a certain critical social temperature T c , all agents in the society acquire personal traits (so called person state). Some of them become permanent conformists and others start to behave forever independently. This means that initially homogeneous population becomes heterogeneous, and agents respond differently to social influence. For T > T c , all agents with equal probabilities behave independently or conform to peer pressure (so called situation state). This regime change between person and situation state, which reminds the idea of an annealed vs. quenched disorder, affects also public opinion. Particularly interesting results are obtained for individualistic societies, in which public opinion is non-monotonic function of T , which means that there is an optimal social temperature for which an agreement in the society is the highest.