I want all the games to be as accessible as we can make them. That includes at least access by using 2 switches. We’ll want to allow the colors to be adjusted like Tar Heel Reader so that students who are VI have a better chance of reading it. Further, I’d like to make it accessible to students who are blind but that will be very challenging (perhaps the new WAI-ARIA stuff is applicable here).

The page colors, logo image, and title text should be easily changeable from the administrative back end so that we can customize the look for different schools and age groups.

I’d like to have an optional area of the screen where we can plug in a static or animated “reward zone”. This area might be a rectangle across the top, bottom, or side of the page. Or it might be in the empty spaces. The point of a reward zone is to have something happen when you get the right answer. We’d have a list of plug-ins that could be selected from the administrative back end. If several are enabled then the player gets to choose or they are randomly selected. I think this could enhance the game playing experience.

All of the animations could either be constantly active or, if that is too distracting, active only for a few seconds after a correct answer.

Here are some things I can think of to include in the reward zone. I’m sure students could come up with better ideas.

• A high-school football theme might include a lighted scoreboard that counts up.
• A save the turtles theme might have animated turtle sprites that can’t climb over a curb. You’re helping them get out of the road by solving puzzles. (You can stop these animated gifs with the Escape key)
  
• A zombie theme might have zombies wandering around the screen. One explodes each time you solve a puzzle.
  
• A flower might sprout up.
• Fireworks might go off.
• Maybe a trashy environment gets cleaned up.
• These could optionally include sound effects.

I think the reward zone plug-in could be responsible for the players perception of the score. Of course behind the scenes the system is tracking attempts, hints, right, wrong, time, etc. But in a football theme the score might be 6 points for correct without a hint, 3 points if only 1 hint is required, and only 1 point if more hints are required. In the zombie game the score might be displayed in little zombie silhouettes.

Implementation ideas

The database has records for students, groups of students (classes, whatever), themes (color, logo, etc.), rewards (which plugin to use), etc. After login the back-end knows what theme to use for the displays and what plug-ins are available to this student.

On the client (in javascript) we’ll handle the mouse (drag and drop, click, etc.) and keyboard (switch or full typing) input. We’ll submit answers to the back-end to get feedback using Ajax. The results will come back with right/wrong and some instructive feedback. The JS in the page will display these appropriately. As a side effect of displaying the right/wrong information the main JS code announces the result using a pub-sub model. If a plug-in subscribes to this information it can trigger the reward appropriately. We could also publish a “busy” event when the mouse is moving or a key is pressed. This way the animation could stop (or start) when the user is idle.

In the template on the server, we only have to fill in the script tag to load the plug-in js and provide a sized div to specify where it fits in the page. The plug-in js handles displaying itself and responding to events.

It would be fun to use the new canvas tag in html5 for the animation.

Who would benefit? students with visual impairment – and students that need more cuing than they get from visual supports on the printed page

Use vibration feedback in TTS, screen readers, digital auditory text

• To cue for bold text, boxed info, important information
• Software could “read” the text/page and insert vibration to cue the student

But more importantly, to allow the student to tag the auditory text – when they “re-read” the text, the tags trigger vibration. Use different types of vibration (intensity, pattern, etc) for different types of tags.

• Tag as a highlighter, tag a phrase or a paragraph
• Tag specific info they need to capture (main characters, literary action)
• Tag for references
• Tag as notetaking – retrieve only tagged information for later?

Karen suggested long ago and I failed to blog about it, that the WiiMote could be useful for “keeping beat with reading. Kid reads along with program, keeps beat. Shakes the controller on the key words. That way you know the kid is reading.”

Seems to me there is great synergy between these ideas. This would make a great student research project.

How about putting the samples on the 16 pads you get with 2 DDR pads? Then we could have a game that was something like Guitar Hero where you put the samples in the right places. DJ Hero? Or maybe DJ Revolution? I think our blind visitors would have a ton of fun on Maze Day trying to “sing” along with Harder, Better, Faster, Stronger by stepping on the DDR pads. Think Daft Hands or Daft Bodies on the DDR pad.

Or how about making an AAC device more fun by changing the pitch of the voice like this? Or somehow linking a “word wall” to music so when you put words together in appropriate ways they go with the music?

Suppose you could provide a list of words or word combinations to a web site and it would automagically assign them musical pitches like this. What cool games could we make where students play with words while making music? How could we integrate reading into it? Perhaps the “score” (the words to be spoken) is presented something like the Guitar Hero notes so you have to at least recognize the words to know which to play.

How do we make such a game switch accessible?

I’m thinking out loud here. Help me out folks, some of you have to be more musical and creative than I am. Post a comment.

Check out CalcuType online and let us know what you think.

You can type using the virtual keyboard, the standard keyboard, or a mouse. The “mover” and “chooser” keys are adjustable and William included a 3rd “completer” key for quick access to completions. However you’re typing the program will suggest completions much like many other available tools. The key difference is when you type a mathematical expression it will suggest the answer as a completion. So if you type (3+5)*2 it will suggest =16. It handles complicated mathematical expressions including square root and trig functions.

One other neat feature William implemented is the ability to write on a worksheet represented as an HTML form.

It still needs work but we’re at the stage where we need feedback from users who need such a tool.

These pages are from a typical book at the site.

My friends Karen Erickson and Gretchen Hanser of the Center for Literacy and Disability Studies tell me that children with limited ability to speak or handle traditional books have few opportunities to learn to read and write. While typically developing children have access to dozens or even hundreds of books, severely disabled kids often have access to very few. Imagine being fifteen years old, unable to speak or to reliably use your hands to manipulate a book and getting your first opportunity to read. Books about baby farm animals aren’t going to cut it. You might be interested NASCAR, fashion, or spacecraft but beginner books on those subjects are rare and even more rarely accessible. Could we do something to make creating accessible books easier?

At the Tar Heel Reader site you can select a book and then advance through the pages with just about any of the characters commonly generated by switch interfaces (let me know if you need something not currently supported). You can have each page read out loud by a computer-generated voice if you wish. Also, each book is automatically available for download as a PowerPoint, Impress, or Flash show.

If you don’t find a book to suit your reader’s interest, making a new book is very easy. The book creator wizard guides you through the process of finding pictures on Flickr, and providing a simple sentence to go with each. Then, with the click of a button, you can create a new book that will immediately be available for reading or download.

How it works

I started hacking PHP using the Flickr API and quickly had a primitive book creator wizard working. When I started thinking about all the supporting infrastructure I would need to provide logins, searches, and multiple authors, it hit me that WordPress already had everything I’d need. A blog post by Steve Winton encouraged me to examine the possibilities further. I’m very impressed with the ease of integrating the wizard and switch-accessible reading into WordPress by only hacking on the theme.

I started with the default theme and hacked the css a bit to create more space on the page. Then I created an empty page for the wizard and gave it a custom template containing my PHP code. Working within the page framework in this way gave me access to all the great WordPress functions while leaving me free to program the page however I wished. In the screen shot to the left you can see the wizard in use. The two rows of pictures at the top are the result of a Flickr search for the tag squirrels. The two boxes near the bottom are the pages constructed so far with an area in each for the short sentence for that page. I used the cool Hoverbox Image Gallery css to make the Flickr search images zoom when you move your cursor over them.

The wizard would be much cooler if it used AJAX but the current version is simply a single html form that does a POST to the page URL. At the very top of the code I call the WordPress auth_redirect function to make sure the author is logged in. Then I check to see if the request method is GET or POST. A GET request signals starting to write; all other calls will use method POST. I used the phpFlickr class to manage access to Flickr. When the author has completed the book, pressing the Create my book button uses the WordPress wp_insert_post function to create post representing the book and then redirects the browser to the newly created post.

Each book is simply a multi-page post using the WordPress <!–nextpage–> tag. They have a regular structure that allows me to parse them with a regular expression when invoking the wizard for editing a book. I customized the book display by modifying the single.php file to make a singlebook.php file that is used for posts in the Book category. The singlebook template eliminates the sidebar and other navigation to unclutter the layout. Switch enabling the reading is done with a bit of JavaScript that advances to the next page on a KeyPress event. Speech enabling the books is implemented with a Python CGI script that uses pyibmtts to convert the caption to a Flash movie that is embedded in the page.

Teachers sometimes use slide show tools such as PowerPoint to make accessible books for their disabled students. To support this and to allow reading books offline each book may be downloaded in Microsoft PowerPoint, OpenOffice Impress, and Adobe Flash format. I generate the Impress format XML using the PHP DOM functions and then convert to PowerPoint and/or Flash using soffice running as a server with the PyODConverter as described in another post.

Future directions

The book on Jeff Gordon illustrates a possible future direction for these books. I used the built-in WordPress editor to modify the last page to include a YouTube video instead of a static picture. I think we could extend the wizard to include YouTube searches if there is enough interest.

Karen suggested including sentence frames to encourage authors to create easy to read texts. I didn’t understand her well enough to try to implement something. Hey, I’m just a computer guy!

Gretchen would like a way to make whatever book the class is currently reading available in this format. I think we could do something with a web cam and a bit of uploading assistance.

Developing this prototype was both fun and educational for me. If we get enough interest from teachers perhaps I’ll get a group of students to program an enhanced version this fall.

Ideas: Content for Route 66, maybe Nascar?
Games that are switch accessible, maybe racing?
Maybe some FYS students could go? How does that work?
Coordinate with Software Engineering and maybe a parallel ET class for CS students.

Scratch for interactive content generation?

Karen has a group at Forest View elementary, kids are using computers to make content of various kinds. For example recording themselves reading books for 1st graders. They could be a good group to get interested in content for device users. Visuals + Audio and simple text could be exciting to author and use.

Look at A to Z phonics website, content isn’t that good but reading level is appropriate, see what books should look like.

Check out Dirty Bertie

Take pictures from good book, get object name and descriptors from teacher and generate really simple sentences like “Butterflies can ______” or “Pirates are ______” to make lots of content quickly. Share on a web site. Make it easy to produce and share content. Patterns like “The noun is verbing” and “The adjective noun is verbing”. Generate text for beginning level reader.

Choose a topic, add descriptors, and generate sentences. Makes a “PowerPoint” or whatever for the teacher to use. Pictures from Flickr or somewhere. Teacher provides topic, gets a bunch of pictures, provides descriptors, system fills words into sentence frames. Make adding pictures easy. Site provides text to speech converted to mp3 and embedded. How about sounds for blind kids?

Something like SamiSays for recording “homework” answers from AAC devices that don’t interface to computers. Teacher/parent plugs device into line-in and computer records audio to send to teacher. Kid listens to questions and answers using their device, app sends mp3 to teacher with the results. Enables kid to do homework independently. Email access too. Use a VERY small number of words as controls for the computer by recognizing them in certain contexts. Provide independent computer access. email via mp3.

Software specs
Developer LiveCD
Hardware specs