A team of Electrical Engineering seniors at Texas A&M University at Qatar (TAMUQ), a Qatar Foundation (QF) partner university, are undertaking the development of an innovative Arabic language-learning tool.
‘Allemny’, which means 'teach me' in Arabic, seeks to utilize speech-activated mobile phone technology in a game to support those learning Arabic as a foreign language by giving immediate user feedback.
Hossam El Husseini explained to The Foundation why the students are working on this project. “We’re all native Arabic speakers,” he said, turning to his fellow students Elsherif Mahmoud and Mohamed Ahmed, adding: “and we want more people to have a knowledge of the language. There are already mobile games out there but very few are speech-controlled.
“Everyone has smartphones, so a game to help people learn to speak Arabic would be really useful. There are a few apps which use speech recognition technologies, or mobile assisted pronunciation training as it’s known, for English, but none for Arabic,” he said.
The group were inspired to get involved in the research, which started in March 2016 and is supported by QF Research & Development member Qatar National Research Fund (QNRF), in classes led by Dr Beena Ahmed, Assistant Professor, TAMUQ, whose research interests include speech processing, and who is mentoring the year-long Undergraduate Research Experience Project (UREP).
The project has no impact on overall study grades. “Taking part in this initiative gives me the opportunity to work on an actual project and see tangible benefits,” said Mahmoud. “During classes, lessons are theoretical, so we can often end up focusing on what we need to do to achieve a higher grade – in this work we are creating something. This is important to us as engineers.”
Ahmed added: “It’s an opportunity to give back to the community. It also helps you decide which area you would like to focus on when you graduate, or not as the case may be. Of course, this depends on employment opportunities, but it’s a great practical experience.”
When the project needed support to develop a game that could help users learn how to pronounce simple Arabic words, the students signed up.
“With speech recognition technologies already available, we want to develop a simple game which can give live feedback to players through the gameplay to help them learn,” Mahmoud added, although one of the initial challenges was to come up with a name for the game.
“We put together a list of Arabic words and phrases and together with Dr Beena came up with the name Allemny.”
Dr Beena explained: “My role in the project is as a mentor, to help the team keep to the timeframe they have set by following the correct avenues of study, and to offer guidance. It’s up to the students to develop the game and do the research.”
Utilizing gaming technology
The TAMUQ team is quick to point out that none of them are computer programmers, although all of the students did spend the summer of 2013 interning at QF member and Hamad bin Khalifa University research institute Qatar Computer Research Institute, where they worked on various projects including the institute’s natural language processing initiatives.
“We needed a game for the purpose of our research and since we had one year for the full project, we did not have time to develop a new game,” El Husseini explained. “Therefore we needed something we could reuse - a game which everybody knew already and was familiar with, was engaging, was open source, could run on mobile technology, and wouldn’t need too much development on our part.”
With a tough list of requirements to meet, the team eventually decided on a version of an iconic 1980s arcade video game that sees a character move around
Instead of using four directional buttons, the speech-activated Allemny uses different Arabic words as input commands. The player has to say the correct command word out loud to get their character to move in the direction they want. The resulting game is entertaining, fun, and fast-paced, encouraging players to keep playing.
Before the game starts, players choose words from a list of basic conversational words. Players can also listen to an example of the pronunciation of each word before beginning to play.
Words can be chosen from a list of around 40 plus nouns and simple verbs, such as colors, landmarks, and directions, decided upon by the team for their use in everyday speech. Once a player becomes familiar with a set of words, they can progress by choosing new command words in the game.
“There are really two stages to the project,” Mahmoud explained. “There’s the development of the game, use of automatic speech recognition, and getting any bugs ironed out through internal testing among the group. For example, the game should not correct native Arabic speakers.
“For the second stage, we’ll need both Arabic and non-Arabic speakers to continue to test the game, particularly as the final game is for this latter audience, and that’s when we will appeal for testers from across QF partner universities.”
As part of the application for human subject testing, under the UREP requirements, the team has already finalized the wording for posters, leaflets, social media, etc, that will be used to source a selected test group.
“We’ll target the Arabic learning programs in QF for testers,” El Husseini added, “probably from September 2016 onward. We don’t need that many subjects for the research: 20 native Arabic speakers and 20 non-native Arabic speakers.”
Mahmoud explained: “We’re currently testing the specificity and sensitivity of the system to ensure it is accurate, and then we’ll be ready for external testers.
“The beauty of using apps is that they give immediate feedback to us as programmers. We can monitor the success rates of the players but more importantly the performance of the game by also tracking general interest and the number of returning players. This makes it an effective tool to us as researchers.”
Following testing, the game should be made available for general download, which will facilitate wider user feedback and development. It could also be developed to include different and more complex phrases.
By using speech recognition technology already found on smartphones and tablets, the team is increasing access to conversational Arabic language skills and Allemny will help non-native Arabic learners speak with success.