Proposal slides - February 24, 2004
In many schools, sighted children have access to story books, regular games, and educational and entertaining computer games. Blind children, on the other hand, often have very little to do because many of these activities use visual stimuli. Because of this, blind children also do not have the same opportunities for exercise and play and can as a result lack muscle tone. It would be nice if there were games that were more accessible.
The primary goal behind this project was to use inexpensive equipment to create a fun and enjoyable game for children with visual impairments. I also wanted to create a game that involved some physical activity so that children would be exercising while playing. An educational aspect to the game would be nice as well.
The game that I chose to create is a very common children's game: Whack-a-Mole. For those who are unfamiliar with the game, the basic idea is that mechanical moles pop up out of holes and the player hits them on the head with a mallet to put them back into their holes. An example of what this looks like can be seen here.
In order to get Whack-a-Mole to work for blind children, I had to replace the visual part of the game with audio. Essentially, the idea was to provide spatial aural cues instead of spatial visual cues. I initially wanted to use a webcam as an input device for the game. I intended to have the moles represented as colored sponges, and the player would hit the sponges with a soft bat. The webcam would capture image data, and image processing code would determine which sponge (mole) had been hit.
I later switched to the idea of using multiple mice as input. The main reasons for this were that I was curious about the effectiveness of this kind of input scheme, and I wanted to evaluate it as an interface. Using multiple mice also seemed like a fairly natural and inexpensive solution, since all I really needed was to be able to distinguish between which mouse was being clicked (hit).
Accordingly, I implemented the system using four USB mice connected to a PC via a four-port USB hub for input, and two speakers for sound output. One speaker is placed behind the leftmost mole, and the other is placed behind the rightmost mole. Audio panning is used to interpolate the positions of the middle moles. The player uses a soft toy bat to hit the moles. Here is a picture of my setup (minus the speakers).
My code was written in C#, and uses the Single Display Groupware Toolkit (SDGT) (from the University of Calgary's GroupLab led by Saul Greenberg) for multiple mouse input. SDGT works as a C# wrapper for Windows raw input. For sound, I used DirectSound.
I ran across several issues when trying to implement this system, and I will discuss each of them in turn. There were issues with:
The mice
One of the problems with using mice is that mice buttons are small and can be difficult to hit with a bat. The solution is fairly simple: just put something on top of the buttons so that there is a larger contact area and the player has a larger target. I did this by cutting up some foam and attaching it to the buttons using rubber cement.
Multiple input devices
One of the problems with using mice as input for the game is that the mouse is already a major input device for PCs. Having many of them connected to a PC can lead to problems in controlling the system. For example, if you have a designated system mouse and a button click arrives from one of the other mice, you need to make sure the button click is not registered as a system mouse click at the system mouse's location. Some of these issues are addressed by SDGT.
USB
USB devices are plug-n-play, and there is no guarantee on the order in which mice are attached to the system. The implication of this is that there must be an explicit calibration step to determine the relative ordering of the mice (moles), which can make accessibility difficult. There needs to be a mapping between mouse IDs and mole IDs.
Audio panning
A problem with using audio panning to change the mole that is originating sounds is that it is sometimes difficult to distinguish between four distinct panning positions with two speakers. Three positions are easy: hard left, hard right, and center. My system is currently set up to use four moles and four panning positions. These panning positions are also explicitly calibrated, and may sound different to different users.
It also can be difficult to distinguish between the locations of sounds playing simultaneously. For example, if two moles, the leftmost and rightmost ones, come up at the same time and play the same sound, then it seems like the sound is coming from the center, where there is no mole. I experimented with using different mole sounds, but I did not include that in this version of the game. Full spatial audio and mole placement might also be a useful enhancement.
My evaluation of the multiple-mouse interface is that currently it is not ideal, primarily because of the requirement for explicit calibration. The setup is fairly simple, and if there were some guarantee on the ordering of the mouse IDs then the system would be easier to work with. Nevertheless, it still works. Because of the requirement for calibration, I decided to focus on making the gameplay itself accessible; the calibration must be done by someone other than the player (presumably a teacher).
The game flow is fairly simple. When the program is started, the user is presented with the following menu:
Pressing the Game Setup button proceeds to prompt the user for mouse and speaker calibration, and Quit, of course, quits the program. After pressing the Game Setup button, the user is presented with a calibration dialog box (at least two mice must be connected to the PC):
The user is asked to click a button first on the system mouse, and then on each of the mole mice in sequence from left to right. Pressing the ESC key cancels out of calibration. If the calibration is successful, the user is presented with a speaker calibration dialog box:
A sound is played, and the user is asked to position the slider until it seems like the sound is coming from the specified mole (from left to right). If the user cancels out of speaker calibration, default panning values are used: -10000 (hard left), a "gradient" between -1000 and 1000 for the middle mice (depending on how many mice are attached), and 10000 (hard right).
Once both mouse calibration and speaker calibration have been successfully completed, the Play! button on the main form is activated, and the user can click on it to play the game.
The gameplay form itself is very simple. It just tells the user to press ESC to quit the game. The rest of the interface from here is in audio.
The game starts out by welcoming the player, and waiting (for up to some specified number of seconds) for the player to whack a mole with the bat to start the game. If the player does not do this, the game enters a training mode which teaches them how to play the game. The training mode gives some instructions and brings each mole up in sequence and lets the player hit them. Once training mode is over, the player again has the ability to start a game, or go through training mode again by waiting.
The game currently consists of four levels, with ten moles popping up in each. With each level, the moles hide more quickly, making the player have to respond more quickly. If the player whacks enough moles in a level, they will advance to the next level. Otherwise, the game will be over, and they have the option to try again by whacking a mole.
Click here for information on downloading and installing Whack-a-Mole.
Thanks to: