I want all the games to be as accessible as we can make them. That includes at least access by using 2 switches. We’ll want to allow the colors to be adjusted like Tar Heel Reader so that students who are VI have a better chance of reading it. Further, I’d like to make it accessible to students who are blind but that will be very challenging (perhaps the new WAI-ARIA stuff is applicable here).

The page colors, logo image, and title text should be easily changeable from the administrative back end so that we can customize the look for different schools and age groups.

I’d like to have an optional area of the screen where we can plug in a static or animated “reward zone”. This area might be a rectangle across the top, bottom, or side of the page. Or it might be in the empty spaces. The point of a reward zone is to have something happen when you get the right answer. We’d have a list of plug-ins that could be selected from the administrative back end. If several are enabled then the player gets to choose or they are randomly selected. I think this could enhance the game playing experience.

All of the animations could either be constantly active or, if that is too distracting, active only for a few seconds after a correct answer.

Here are some things I can think of to include in the reward zone. I’m sure students could come up with better ideas.

• A high-school football theme might include a lighted scoreboard that counts up.
• A save the turtles theme might have animated turtle sprites that can’t climb over a curb. You’re helping them get out of the road by solving puzzles. (You can stop these animated gifs with the Escape key)
  
• A zombie theme might have zombies wandering around the screen. One explodes each time you solve a puzzle.
  
• A flower might sprout up.
• Fireworks might go off.
• Maybe a trashy environment gets cleaned up.
• These could optionally include sound effects.

I think the reward zone plug-in could be responsible for the players perception of the score. Of course behind the scenes the system is tracking attempts, hints, right, wrong, time, etc. But in a football theme the score might be 6 points for correct without a hint, 3 points if only 1 hint is required, and only 1 point if more hints are required. In the zombie game the score might be displayed in little zombie silhouettes.

Implementation ideas

The database has records for students, groups of students (classes, whatever), themes (color, logo, etc.), rewards (which plugin to use), etc. After login the back-end knows what theme to use for the displays and what plug-ins are available to this student.

On the client (in javascript) we’ll handle the mouse (drag and drop, click, etc.) and keyboard (switch or full typing) input. We’ll submit answers to the back-end to get feedback using Ajax. The results will come back with right/wrong and some instructive feedback. The JS in the page will display these appropriately. As a side effect of displaying the right/wrong information the main JS code announces the result using a pub-sub model. If a plug-in subscribes to this information it can trigger the reward appropriately. We could also publish a “busy” event when the mouse is moving or a key is pressed. This way the animation could stop (or start) when the user is idle.

In the template on the server, we only have to fill in the script tag to load the plug-in js and provide a sized div to specify where it fits in the page. The plug-in js handles displaying itself and responding to events.

It would be fun to use the new canvas tag in html5 for the animation.

Lots of other situations could be handled similarly.

We could build or buy a platform that can tilt (say) 20 degrees in two directions under computer control. I don’t think it has to be very high performance at all. We roll the kid’s chair onto the platform and strap it down.

John suggests a platform with a center pivot, springs on 2 sides and stepper motors with cables on the other 2 sides. Missy says we should ask Disney who have all this figured out.

A group of our students go out to a theme park (or the state fair) and record wide-angle HD video, acceleration and good audio. Then we synchronize playback on a big screen (or screens). Perhaps a sub woofer adds some low frequency shake (or maybe the platform could do this). I don’t think we need any “wash out” algorithm for the motion platform. Simply, display the best approximation to the direction of the gravity vector that the tilt platform can manage. We might want to rotate the image to undo some of the tilt. That would be an interesting study on its own.

The log flume ride would be perfect for this. We could spray the rider with some water at the end!

Then we can invite kids in for a virtual carnival or theme park field trip. That would be a ton of fun and an exciting project. Later groups of students could do fully computer generated graphics and sound as more advanced projects.

John suggests we could record historic train rides and such for more educational content.

We link to Tar Heel Reader by having kids read about the experience beforehand and write about it afterward!

I’m going to try to resist immediately jumping on this until I find some $ to pay for it.

My friend Michelle’s son Alex is autistic. We visited them on Saturday night. While Michelle and the rest of us were doing whatever, Alex was playing with an R2D2 toy.

This toy performs actions based on speech recognition.

Alex: “Hey R2″
R2: affirmative beep (sounds like “Boo-Boop”)
Alex: “Do yamgine?”
R2: negative beep (sounds like “Bee-Bawp”)
Alex: “Do you ‘magine?”
R2: negative beep
Michelle: “Do you remember?”
R2: affirmative beep
Michelle: “Darth Vader?”
R2: scared noise and shaking head and moving in circles (sounds like “wa-a-a-hoo!”)

REPEAT w/o michelle for like an hour

the kid was HIGHLY motivated to say the right stuff, and kept at it.

what if there was a web framework (through flash or silverlight?) and a teacher could program in like the vocab, and a storyline (maybe somehting like tarheel reader could help create some default storylines (TANGENT: we should try to have some of your projects seed other as possible) and then the kid tries to say it, this would be for kids with speech issues.

maybe the kid’s a detective, and has to go around quesitoning people, maybe at first it acknowledges bits and pieces (i.e.”Hey ” “where were you?” “?”) then later it only acknowledges at the end

maybe how tolerant it is of sound deviating depends on the kid’s level and their recent success rate…

I love it! This could make a very interesting student project and would take our work in a different and exciting direction.

Who would benefit? students with visual impairment – and students that need more cuing than they get from visual supports on the printed page

Use vibration feedback in TTS, screen readers, digital auditory text

• To cue for bold text, boxed info, important information
• Software could “read” the text/page and insert vibration to cue the student

But more importantly, to allow the student to tag the auditory text – when they “re-read” the text, the tags trigger vibration. Use different types of vibration (intensity, pattern, etc) for different types of tags.

• Tag as a highlighter, tag a phrase or a paragraph
• Tag specific info they need to capture (main characters, literary action)
• Tag for references
• Tag as notetaking – retrieve only tagged information for later?

Karen suggested long ago and I failed to blog about it, that the WiiMote could be useful for “keeping beat with reading. Kid reads along with program, keeps beat. Shakes the controller on the key words. That way you know the kid is reading.”

Seems to me there is great synergy between these ideas. This would make a great student research project.

I have this feeling that there is a fun game idea in here for Maze Day (and beyond). I’m not sure what it is. Perhaps you press a square on the DDR Pad to hear one phrase “mahna mahna, dee dee da dee dee”. Then you have to press another one to go to the next phrase? Or perhaps it is a joint effort between two kids? One doing “mahna mahna” and the other doing the dee’s? Maybe the background is playing in a loop and you have to queue the “mahna mahna” at the right time like Guitar Hero?

Give it a listen and then give me some ideas in the comments.

How about putting the samples on the 16 pads you get with 2 DDR pads? Then we could have a game that was something like Guitar Hero where you put the samples in the right places. DJ Hero? Or maybe DJ Revolution? I think our blind visitors would have a ton of fun on Maze Day trying to “sing” along with Harder, Better, Faster, Stronger by stepping on the DDR pads. Think Daft Hands or Daft Bodies on the DDR pad.

Or how about making an AAC device more fun by changing the pitch of the voice like this? Or somehow linking a “word wall” to music so when you put words together in appropriate ways they go with the music?

Suppose you could provide a list of words or word combinations to a web site and it would automagically assign them musical pitches like this. What cool games could we make where students play with words while making music? How could we integrate reading into it? Perhaps the “score” (the words to be spoken) is presented something like the Guitar Hero notes so you have to at least recognize the words to know which to play.

How do we make such a game switch accessible?

I’m thinking out loud here. Help me out folks, some of you have to be more musical and creative than I am. Post a comment.

Come see if you can find “it” at Maze Day 2009 on the University of North Carolina Campus at Chapel Hill. What are we looking for? We’ll be looking for bus stops, intersections, building entrances and much more. Using Trekker Breeze, students, teachers, parents and peers will participate in a “Talking GPS” scavenger hunt on campus. During the activity we will use the newest in GPS technology for the blind and visually impaired to find out “What’s around.” Join us to learn about this exciting technology and be ready to have some fun in the process.