A Virtual Scribe System Using Convolutional Neural Network to Aid the Physically Challenged

Abstract

Around the world 30% of the people are suffering from various disabilities, many of them are physically and visually impaired. Such physically and visually challenged students often require a scribe to assist them during exams and also finding a scribe is a tedious task. Speech recognition technology is one of the emerging and fastest growing engineering technologies. It has become an integral part of our lives nowadays in many forms like Voice based assistants, Navigation, Searches, and education. Speech recognition system can be used as a replacement for scribes. Though the most commonly available speech recognition systems like Ok Google, Siri has very good accuracy for general speech, they are not suitable in domain-specific applications like recognizing chemical equations. So the aim is to develop an automatic speech recognition system for chemical equations in order to make the examination process easier for the physically challenged. This system uses a conventional neural network to recognize speech. It takes a speech signal recorded using a microphone containing chemical equation and converts it into text in real time. Additionally, this system can also be used for similar tasks like voice-based dialing, voice-based data entry and documentation. In the future, this system can be extended to recognize mathematical equations.