The face expression recognition is used in countless application areas for example security, computer vision and medical science etc. The facial expressions are used to communicate in a non-verbal way (i.e., using eye contact, facial expressions etc.). Emotions play an important role in facial expression recognition which helps to identify what an individual is feeling. Various AI research in the field of facial recognition system is being carried since a decade. Many of the machine learning algorithms are also being used to identify the facial expression which helps them to train and test using the facial expression to get a correct output of the given expression. This paper presents a new facial expression recognition system, local directional feature structure (LDFS). LDFS uses different features of the face (i.e., eyebrows, nose, mouth, eyes). The face is detected and aligned using the edge detection. The task of the edge detection is to detect the face, face alignment and position variations of the face. The edge detection extracts the specific features for the identification of the emotions. Two types of datasets have been used for the qualitative and quantitative experiments on the face expression mainly the CK+ and Jaffe dataset. This approach for our model shows an improvement when compared to the existing system.