Face recognition is still an active pattern analysis topic. Faces have already been treated as objects or textures, but human face recognition system takes a different approach in face recognition. People refer to faces by their most discriminant features. People usually describe faces in sentences like ``She's snub-nosed'' or ``he's got long nose'' or ``he's got round eyes'' and so like. These most discriminant features have been extracted by comparing a face with average face formed in one's mind. We have mathematically formulated the approach and placed importance upon the most discriminant features. We have explained feature processing and classification parts in details. We also explained the train and test phases of the proposed algorithm. We have compared the proposed classification part with 1-NN classifier to show the strength of the algorithm and reported the results. We have also compared the whole proposed algorithm with a well-known face recognition method, Eigenfaces and achieved promising results in different cases.