My friend Michelle’s son Alex is autistic. We visited them on Saturday night. While Michelle and the rest of us were doing whatever, Alex was playing with an R2D2 toy.
This toy performs actions based on speech recognition.
Alex: “Hey R2″
R2: affirmative beep (sounds like “Boo-Boop”)
Alex: “Do yamgine?”
R2: negative beep (sounds like “Bee-Bawp”)
Alex: “Do you ‘magine?”
R2: negative beep
Michelle: “Do you remember?”
R2: affirmative beep
Michelle: “Darth Vader?”
R2: scared noise and shaking head and moving in circles (sounds like “wa-a-a-hoo!”)
REPEAT w/o michelle for like an hour
the kid was HIGHLY motivated to say the right stuff, and kept at it.
what if there was a web framework (through flash or silverlight?) and a teacher could program in like the vocab, and a storyline (maybe somehting like tarheel reader could help create some default storylines (TANGENT: we should try to have some of your projects seed other as possible) and then the kid tries to say it, this would be for kids with speech issues.
maybe the kid’s a detective, and has to go around quesitoning people, maybe at first it acknowledges bits and pieces (i.e.”Hey ” “where were you?” “?”) then later it only acknowledges at the end
maybe how tolerant it is of sound deviating depends on the kid’s level and their recent success rate…
I love it! This could make a very interesting student project and would take our work in a different and exciting direction.
Who would benefit? students with visual impairment – and students that need more cuing than they get from visual supports on the printed page
Use vibration feedback in TTS, screen readers, digital auditory text
To cue for bold text, boxed info, important information
Software could “read” the text/page and insert vibration to cue the student
But more importantly, to allow the student to tag the auditory text – when they “re-read” the text, the tags trigger vibration. Use different types of vibration (intensity, pattern, etc) for different types of tags.
Tag as a highlighter, tag a phrase or a paragraph
Tag specific info they need to capture (main characters, literary action)
Tag for references
Tag as notetaking – retrieve only tagged information for later?
Karen suggested long ago and I failed to blog about it, that the WiiMote could be useful for “keeping beat with reading. Kid reads along with program, keeps beat. Shakes the controller on the key words. That way you know the kid is reading.”
Seems to me there is great synergy between these ideas. This would make a great student research project.
I have this feeling that there is a fun game idea in here for Maze Day (and beyond). I’m not sure what it is. Perhaps you press a square on the DDR Pad to hear one phrase “mahna mahna, dee dee da dee dee”. Then you have to press another one to go to the next phrase? Or perhaps it is a joint effort between two kids? One doing “mahna mahna” and the other doing the dee’s? Maybe the background is playing in a loop and you have to queue the “mahna mahna” at the right time like Guitar Hero?
Give it a listen and then give me some ideas in the comments.
Paul posted a really nice video about using Tar Heel Reader over at YouTube. The puppet and the stop motion self assembly of the switch interface are great! Check out Reading with Franz.
I saw iDaft today and think it rocks! You play the samples by typing the corresponding keys on your keyboard. Not surprising technically, its just Flash. But it makes me think about combining fun, music, and literacy. What could we do with music and samples like this to make fun and even educational games for kids with disabilities?
How about putting the samples on the 16 pads you get with 2 DDR pads? Then we could have a game that was something like Guitar Hero where you put the samples in the right places. DJ Hero? Or maybe DJ Revolution? I think our blind visitors would have a ton of fun on Maze Day trying to “sing” along with Harder, Better, Faster, Stronger by stepping on the DDR pads. Think Daft Hands or Daft Bodies on the DDR pad.
Or how about making an AAC device more fun by changing the pitch of the voice like this? Or somehow linking a “word wall” to music so when you put words together in appropriate ways they go with the music?
Suppose you could provide a list of words or word combinations to a web site and it would automagically assign them musical pitches like this. What cool games could we make where students play with words while making music? How could we integrate reading into it? Perhaps the “score” (the words to be spoken) is presented something like the Guitar Hero notes so you have to at least recognize the words to know which to play.
How do we make such a game switch accessible?
I’m thinking out loud here. Help me out folks, some of you have to be more musical and creative than I am. Post a comment.
Come see if you can find “it” at Maze Day 2009 on the University of North Carolina Campus at Chapel Hill. What are we looking for? We’ll be looking for bus stops, intersections, building entrances and much more. Using Trekker Breeze, students, teachers, parents and peers will participate in a “Talking GPS” scavenger hunt on campus. During the activity we will use the newest in GPS technology for the blind and visually impaired to find out “What’s around.” Join us to learn about this exciting technology and be ready to have some fun in the process.
I haven’t used CVtypes recently so it hasn’t gotten any of my (very limited) attention. I see that Minh-Tri Pham has forked a version that he is maintaining over at Google Code. I recommend folks use and support his version. Next time I need it, I’ll probably adopt his.
Maze Day is for visually impaired and blind students in grades K-12, their parents and teachers. Your students will enjoy fun and educational computer applications developed especially for them. UNC students will learn how well their accessible applications work with real users. And everyone will have a good time!
This is a follow up to my post about Accessible Math Ideas from over a year ago. I finally got a smart high school student, William Condon from the NC School of Science and Math, to implement word prediction with built-in math.
Check out CalcuType online and let us know what you think.
You can type using the virtual keyboard, the standard keyboard, or a mouse. The “mover” and “chooser” keys are adjustable and William included a 3rd “completer” key for quick access to completions. However you’re typing the program will suggest completions much like many other available tools. The key difference is when you type a mathematical expression it will suggest the answer as a completion. So if you type (3+5)*2 it will suggest =16. It handles complicated mathematical expressions including square root and trig functions.
One other neat feature William implemented is the ability to write on a worksheet represented as an HTML form.
It still needs work but we’re at the stage where we need feedback from users who need such a tool.
In one bowl stir together dry ingredients. Combine the wet ingredients in another bowl. Add them all at once to the dry ingredients stirring until blended but slightly lumpy. Add additional buttermilk to thing the batter. Pour about 1/4 cup batter onto 325 degree griddle sprayed with PAM. Cook until goldenbrown, turning to cook other side when pancakes have a bubbly surface and slightly dry edges.