Back propagation(BP) is used to solve real world problems which use the concept of multilayer perceptron(MLP). BP have the disadvantage of trapped in local minima, slow convergence rate and more error prone. To optimize BP Algorithm, Particle swarm optimization(PSO) and Genetic algorithm(GA) is used. Limitation of PSO_Hill and PSO_A* is overcomes when these algorithms are combined and on the basis of strength of these two algorithm we proposed a new PSO_Hill_A* algorithm which is used to optimize and enhance learning process in terms of convergence rate and accuracy. GA is a kind of method to simulate and to search the optimal solution, GA can have four operations including Encoding, selecting, crossover and Mutation. To optimize and improve BP, we proposed two architecture: 1) Use of PSO_Hill_A* before and after hidden layer. 2) Use of GA before and after hidden layer.