@@ -64,6 +61,7 @@ The main research questions came up for the project were based on each individua
*How to evaluate lecture performance by tracking their behavior during a lecture and analyzing the quality of the lecture content which is delivered by the lecturer?”*
## Individual Objectives
The main objectives of the research project could be further split into individual objectives which elaborates purpose of the project in much detail.
...
...
@@ -98,6 +96,26 @@ The main objectives of the research project could be further split into individu
## Summary of Individual Components
### Attendance Register
There are three processes, namely, attendance marking, getting feedback, and displaying information. To identify students, video is recorded from a front-mounted camera and it is used to capture a frame. OpenCV was used to record as well as capture a frame. The image enhancement algorithm clears out the captured image to run the face detection algorithm. After detecting the faces in the frame, the system runs the face detection algorithm to identify students. OpenCV and Scikit-learn were used for the processing of images and the classification of face images. Detected students' attendance details were stored in the database as well as late coming activities and skipping activities. The mobile app helps achieve the task of gathering face recognition data. Face data collection is a mandatory step in the registration process once a student downloads and sets up the application. The user interface, similar to setting the face to unlock in smartphones, makes the system exceptional in terms of usability. After a lecture, using the mobile application, reasons for student behaviors track down. The lecturer's dashboard in the mobile application presents a detailed student attendance, students' feedback, and reasons.
### Monitoring Student Behavior
Once the students are recognized, the overall student behavior will be evaluated based on Activity, Emotion recognition, and gaze estimation components. A pre-trained deep learning model was used for identifying the student emotions under categories such as *Happy*, *Sad*, *Anger*, *Surprise*, and *Neutral*. The Activity Recognition was also based on a deep learning model. This model was trained under 3 classes: *phone checking*, *listening*, and *note-taking*. Prior to running this model, the students needed to be separately identified. Hence, an object detection model named MobileNet-SSD was used. To perform the student gaze estimation, a combination of facial landmarks, deep learning and the application of *perspective-n-point* algorithm was used. The head gaze was calculated based on directions such as *looking up and right, looking up and left, looking down and right, looking down* and left and *looking front*. Some threshold values were set to vertical and horizontal directions by calculating the angle relative to the respective planes.
Once the above stages are completed, the data visualization process takes place. For this the lecture video will be divided into 5 equal time segments and all the student behavioral aspects will be measured for the given time segments. These calculations will be saved into the database for future reference. In order to find the correlation between the student behavior and lecturer performance, the python data manipulation and analysis library named *pandas* is used.
### Lecture Summarizing
Once the lecture is Completed the original audio file will be saved in the Database. As the initial step of "lecture summarizing" the audio file will be processed and the unnecessary parts such as background noises, long pauses will be removed from the audio file using the spectral grating algorithm. Then the filtered audio will be converted into a text format using the "speech-to-text approach". This text file will be used to summarize the lecture and identify important things such as exam dates, assignment details, etc., mentioned by the lecturer. As the 1st step of this process using three keywords: Exam, assignment, and important, the notices, exam/assignments dates mentioned will be identified and saved in a different document.
Then the rest of the document will be summarized using extraction-based summarization. After the summarization, the summarized lecture will be converted into a pdf and provided to students. The important notices and important points of the lecture will be displayed separately. The recorded lecture will also be provided as they can use the recording if they want to know more details after reading the summary.
### Monitoring Lecturer Performance
The concept of this component is not to judge the lecturer by their actions but to distribute accurate data among the higher management in order to help them make better decisions in lecturer evaluation. Once the lecture is initiated, the back camera will start monitoring the lecturer and start tracking the body movements by detecting the postures. This procedure will categorize the lecturers' behavior under 3 postures which are sitting, standing, and walking. By analyzing those postures by calculating from the number of frames that the video has captured, showing the percentage of each posture with analyzing each frame when the lecture has been ongoing. The main purpose of the above process is to identify the behavior of the lecturer in the classroom. In the emergence of the lecture, the lecturers’ audio data will be recorded and converted to text for further analysis. It will be analyzed under two main methods named quantitative and qualitative analysis
