# Taking the classroom monitoring to the next level with SLPES

## Works well for Administration, Lecturers and Students

## Research Project Scope

## Literature Survey

### Class Dojo

Software that encourages positive classroom behavior by awarding points for participation

##### Best For:

- Teachers
- School leaders
- Families

##### Features

Useful for classrooms, Real-time chat facility.

### TopHat

The Top Hat app motivates students to engage with course content, participate in class and ultimately master your material.

Run discussions in your classroom, assign interactive readings and quiz for understanding, all with one easy-to-use interface.

##### Features

- Get real-time feedback on student progress.
- Leverage students' devices in class to increase participation.
- Upload your slides easily and add interactive questions.
- Take attendance effortlessly and prevent absent students from bending the rules.
- Launch real-time polls to gauge student understanding.

### AI-enabled classrooms in China

A High School in Hangzhou, China has installed cameras which are designed to mark attendance automatically and track activities that students are engaging in, including reading, writing or listening.

In addition, students’ real-time emotions such as fear, happiness, sadness, surprise, disgust, anger and neutral are also tracked.

### EduSense: Practical Classroom Sensing at Scale

EduSense is a comprehensive, open source, sensing system that produces a plethora of theoretically-motivated visual and audio features correlated with effective instruction, which can feed professional development tools in much the same way as a Fitbit sensor reports step count to an end user app

The system consists of 3 main components as follows.

- Body Segmentation, Keypoints and Interframe tracking
- Facial Landmarks
- Speech Detection

## Research Gap

| AI Systems in China | TopHat | ClassDojo | EduSense | SLPES
--- | --- | --- | --- | --- | ---
Attendance Register |  |  |  |  |
Monitoring Student Behavior |  |  |  |  |
Lecture Summarizing |  |  |  |  |
Monitoring Lecturer Performance |  |  |  |  |

## Research Problem

- “What are the main problems faced by university students and lecturers in the classroom environment and how to address those issues with the aid of technologies in the domain of Artificial Intelligence, Machine Learning & Computer Vision?”

## Research Objectives

## Main Objectives

Tracking student attendance using facial detection and facial recognition, notify and getting opinion from the students who are absent and students who are leaving the lecture in the during a specified time period.

Monitoring student behavior within the classroom during lecture periods to identify potential problem areas in retaining student attention and adapting the lecturing styles to suit the student needs.

Summarize the converted text and present as a document and identify the important points of the lecture to display them highlighted.

Monitoring lecturers’ behavior by analyzing the lecture audio data recorded in a lecture hall during lecture hours.

## Specific Objectives

- Tracking student attendance using face detection and facial recognition.
- Notify and get opinion from the students who are absent and students who are leaving the lecture during a specified time period.
- Provide an overview for the lecturer regarding student behavior during the lecture periods through a web interface.
- Identifying the student activities, emotion expressions and concentration levels of students within 20-minute lecture duration using activity recognition, emotion recognition and gaze estimation by employing computer vision technologies.
- Convert the filtered audio through lecture audio into the text format using the speech-to-text approach in order to identify the important parts of the lecture and to summarize the lecture.
- Summarize the converted text and provide the summary in a pdf format for students.
- Monitoring lecturers’ evaluation by analyzing the audio data recorded in a lecture hall during the lecture hours.
- Monitoring the lecturers’ interaction by tracking their posture movements such as sitting, standing and walking.

## Methodology

As mentioned in the System Diagram, this system will mainly consist of a mobile and web application. When the students and the lecturer enter the classroom, their behavior will be monitored by two separate video-cameras and a microphone. The collected data will transmit to the server through local machine and stores the gathered data in the MongoDB cloud database service.

While the video camera tracks both facial expression and body movements, the microphone will record the lecture for further analysis. The student attendance will register by capturing facial identity using deep learning and computer vision mechanisms. Students will be monitored by capturing their facial and body movements by Computer Vision and Deep Learning algorithms.

The lecturer audio and video details will be gathered and stored for analyzing as such lecture summarizing and monitoring lecturer performance using machine learning, deep learning, and computer vision technologies. While the students have access to the mobile application, the lecturers and the higher management have the authority to manage the web-application. Frontend components will communicate with the backend using Django REST API.

## Domains

Deep Learning
Computer Vision
Machine Learning
Natural Language Processing

## Tools

## Research Project Milestones

Proposal

### Description

The Research group was given the opportunity to elaborate the research problem, research objectives and tentative methodology of the proposed study. The presentation was carried out by each individual describing their components

### Date

Feb 27th, 2020.

### Marks

10

Progress presentation 1

### Description

This presentation was aimed to show the progress of the ongoing porcess, such that 50% of the complete product had to be presented. Each individual had to clearly mention what they have done and the actual percentage of the work completed

### Date

July 13, 2020

### Marks

15%
Progress presentation 2

### Description

The main purpose of this presentation was to show the progress of the project. It was expected that 90% of the project to be completed.

### Date

September 24, 2020

### Marks

18%
Demo

### Description

The research project group had to design a Research Poster to provide a basic understand about the Research. The content included were Introduction, Literature Survey, Research Problem, Research Objectives, Methodology, Results and Discussion and Conclusion

### Date

September 24, 2020

### Marks

12%
Final Assessment

### Description

A completed product demonstration was expected

### Date

To be Informed

### Marks

10%
Viva

### Description

Each student will be individual evaluated.

### Date

To be Informed

### Marks

10%

## Documents

## Project Charter

## Project Proposal

## Research Paper

## Final Thesis

View more

## Presentation

## Proposal Presentation

## Progress Presentation - I

## Progress Presentation - II

# Final Presentation

## About Us

#### Supervisors

#### Samantha Rajapaksha

Senior Lecturer

#### Ms. Dinuka Wijendra

Lecturer

#### Group Members

#### Ishan Seneviratne

Group Leader

#### Sohan Perera

Member

#### Sachith Fernando

Member

#### Linisha Siriwardana

Member

## Contact

Make contact via the details below to get in touch with us.

### Our Address

Sri Lanka Institute of Information Technology, New Kandy Rd, Malabe 10115

### Our Email

cdap.slpes@gmail.com

Loading
Your message has been sent. Thank you!