In this paper an algorithm to solve the problem of automatic face recognition is presented. The novelty of the algorithm is the ability to combine the computer vision tasks with Particle Swarm Optimization (PSO) to improve the execution time and to obtain better recognition results. The efficiency of face recognition system is improved by using a fitness function to measure the similarity of an input face compared with a database of faces. The use of the fitness function helps to obtain more accurate results in a faster way. The results obtained are excellent even when the system was tested in uncontrolled environments. A comparison of the results obtained with the algorithm without PSO versus the algorithm using PSO is also presented. The algorithm is also implemented on the colored images of the human faces.