With help from Matt Boslem at Immotion Games, the test game is now successfully working and ready for testing.
The video game that is used for the purpose of the testing is a simple map created in Unity terrain editor and then boxes have been implemented on top of this base map. Participants complete the game by collecting the boxes in a specific order. To encourage discussion, the boxes are not labelled, but flash red when corrected incorrectly and green when collected correctly. The game includes visual prompts to assist both hearing and D/deaf players as well as sound cues.
The game connects via the host’s IP address through the wi-fi through an existing Unity plug-in called Mirror. Mirror is an old Unity framework that uses the computer’s IP address and decentralises it – this allowed for a multiplayer gaming experience.
Over the past few months I have been working on my speech to text tool. On my own, I had lots of coding and play-ability issues with my game and accessibility tool. With the help of Warren Hilton at The University of Salford, a solution has been found that will allow me to obtain testing data that will allow my dissertation to progress.
The main problem that occurred with my speech to text tool was not with giving my open source unity game the speech to text ability, but to allow cross-network communication so that the speech appeared on the other player’s screen. Cross network communication is a complex task to carry out which was outside of my abilities and outside of the abilities of people who have helped me throughout the project.
Microsoft Azure allowed me to create a simple unity application that recognised speech and outputted it as text; “Quickstart: Recognise speech from a microphone input“. Firstly, this was looked into – the possibility of using this in conjunction with my first person shooter open source unity game. This is when the issue occurred with multi-device conversation through the collaborative gaming network.
Due to the multi-device conversation issue being outside of my coding abilities, further research was carried out into alternative methods of testing my speech to text tool. Using Microsoft Azure, a translator tool has been found which allows speech to text communication running as a separate console; “Quickstart: Multi-device conversation“. The console can be run in “presentation mode” which allows constant speech to text, uninterrupted – this allows game-play and speech to text to run simultaneously. This method appeared to be the best solution to my problems to allow my research to keep moving and to allow for testing to commence in the very near future.
After battling through other university assignments, this week, my project is back on the move.
Some problems surfaced with my laptop not being powerful enough to run the game smoothly, however, the simple test level ran perfectly. For the time being, I plan on working on the simple test level as I would rather the game run smoothly and efficiently than have a really professional looking game.
In preparation for the poster presentation in January, I plan on implementing my speech to text tool onto a simple unity game. This will allow me to present the testing to the people who are attending the poster presentation.
I will be tidying up my written dissertation and making sure the literature review, advisory group section and theory are well written and presentable before moving onto the creation of my in game accessibility tool.
Work is progressing on my dissertation project. Over the past few weeks, work has been carried out on my research paper, including, theory sections and literature reviews. My first draft of the literature review is finished.
On advise from my dissertation supervisor, I have contacted the Games Design and Production course leader at the University of Salford for their input into the project and hopefully some collaboration with fellow students.
Along with this, Naomi Sharples, who once worked for the D/deaf nursing course at Salford University, has been contacted via LinkedIn also.