Take-down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum. AbstractWe introduce a simulation algorithm which allows the off-lattice simulation of various phenomena observed in heteroepitaxial growth (see e.g. [Politi et al., Phys. Rep. 324 (2000) 271-404]) like a critical layer thickness for the appearance of misfit dislocations, or self-assembled island formation in 1 + 1 dimensions. The only parameters of the model are deposition flux, simulation temperature and an interaction potential between the particles of the system. 

Equilibrium states of large layered neural networks with differentiable activation function and a single, linear output unit are investigated using the replica formalism. The quenched free energy of a student network with a very large number of hidden units learning a rule of perfectly matching complexity is calculated analytically. The system undergoes a first order phase transition from unspecialized to specialized student configurations at a critical size of the training set. Computer simulations of learning by stochastic gradient descent from a fixed training set demonstrate that the equilibrium results describe quantitatively the plateau states which occur in practical training procedures at sufficiently small but finite learning rates.

Equilibrium statistical physics is applied to layered neural networks with differentiable activation functions. A first analysis of off-line learning in soft-committee machines with a finite number (K) of hidden units learning a perfectly matching rule is performed. Our results are exact in the limit of high training temperatures (β → 0). For K = 2 we find a second order phase transition from unspecialized to specialized student configurations at a critical size P of the training set, whereas for K ≥ 3 the transition is first order. Monte Carlo simulations indicate that our results are also valid for moderately low temperatures qualitatively. The limit K → ∞ can be performed analytically, the transition occurs after presenting on the order of N K/β examples. However, an unspecialized metastable state persists up to P ∝ N K 2 /β.

No abstract

Equilibrium statistical physics is applied to layered neural networks with differentiable activation functions. A first analysis of off-line learning in soft-committee machines with a finite number (K) of hidden units learning a perfectly matching rule is performed. Our results are exact in the limit of high training temperatures (β → 0). For K = 2 we find a second order phase transition from unspecialized to specialized student configurations at a critical size P of the training set, whereas for K ≥ 3 the transition is first order. Monte Carlo simulations indicate that our results are also valid for moderately low temperatures qualitatively. The limit K → ∞ can be performed analytically, the transition occurs after presenting on the order of N K/β examples. However, an unspecialized metastable state persists up to P ∝ N K 2 /β.