Geeks making the world a bit better.

Entries Tagged Mac

Outfox: speech, sound, and more for Firefox

Quick net

Outfox is a Firefox extension that allows in-page JavaScript to access local platform services and devices such as text-to-speech synthesis, sound playback and game controllers. You can read all about it at its Google Code page and discuss it in its Google Group.

Motivation

In our continuing pursuit of the best platform for delivering accessible games and tools to people with disabilities we are now focusing on Firefox to see what we can accomplish from inside the browser. Up until recently, we’ve used Python. Its a great language and we’ve delivered thousands of copies of our apps to people all over the world but I kept thinking we could do better. For example, the ability to author games in Hark the Sound has enabled teachers to do things we never imagined but sharing games is so complicated that is almost never happens. In contrast, the exciting response we’ve seen to Tar Heel Reader with 419 books in 3 languages after just 10 weeks online has convinced me that browser-based apps can have a big place in our work.

Firefox is a nearly perfect platform; it runs everywhere we want to be and it is easy to program but it has a few deficiencies that limit what we can do. Many of our users are blind or visually impaired. For these kids, speech and sound are a must. Speech and sound are important for kids who can see as well. For example, many beginning readers benefit from having text read to them. Unfortunately, sound in the browser gets little respect.

We used SoundManager2 with a web-based text-to-speech server in Tar Heel Reader. It works well enough but the delays associated with fetching speech as mp3s over the web and the requirement to be online to get speech look bad in comparison locally hosted program.

Outfox

Outfox provides a simple programming interface to JavaScript running in the browser so it can access services on the computer that aren’t normally available. In this first release, we enable access to speech and multi-channel sound with callbacks into JavaScript at the beginning and ends of words and sounds. Soon we expect to extend it to include joysticks, DDR pads, and such. With access to these capabilities I believe we can implement browser-based versions of our most popular games such as Hark the Sound, Sonic Zoom, and Descent into Madness. If things work as I hope, teachers will have access to hundreds of curriculum-related games and a simple way to write and share their own. I hope we’ll get cross-over between the amazing community behind Tar Heel Reader and the community of teachers of the visually impaired and orientation and mobility teachers who use our games for blind children. When we start seeing books about games and games about books, we’ll know that synergy is taking hold.

I’m excited about offering Firefox with speech and sound to my students to see what amazing things they’ll think of doing. How about self-voicing web pages and maps? How about books with interactive, switch-accessible games embedded?

This initial version is likely buggy on at least one of the three operating systems we’re supporting. Please give it a try. Or better yet, download the source, and help us make it better.

Also, help us get it out of the sandbox at Mozilla Addons by trying and commenting on it.

Making CVtypes work on the Mac

I’m writing this down because I don’t see complete working (for me at least) directions anywhere else.

  1. Install xcode from the Mac OS X install CD
  2. MacPorts (everyone calls it DarwinPorts)
  3. Get and unpack the opencv source tarball
  4. cd opencv; mkdir build; cd build
  5. ../configure CPPFLAGS=”-I/opt/local/include” LDFLAGS=”-L/opt/local/lib” –without-python
  6. make
  7. sudo make install

I also needed the macam webcam driver to use my Logitech webcam. I believe the notebooks have a camera built in so you should be good to go on one of those.

I’m getting tons of error messages like these as my app is running and I’m only getting about 5 frames per second. What’s up with that?

2008-02-14 09:00:23.983 Python[85901:6d37] *** Break on _NSLockError() to debug.
2008-02-14 09:00:24.179 Python[85901:880f] *** -[NSLock unlock]: lock ( '(null)') unlocked from thread which did not lock it

Also, I note that video doesn’t start until I move the mouse. Are these macam bugs?

Bringing OS X compatibility to our apps

I regularly get requests from teachers at schools that are Mac based for versions of our software that will work for them. I want every kid to have access to our stuff so I bought a Mac mini for development. I’d rather be running OS X in a virtual machine but Apple won’t let me do that. So I’ve got the mini sitting here with its video out connected to one of the inputs on my right projector and I hacked a python wrapper for synergy server to switch the projector source when I move the mouse off my Ubuntu desktop onto the mini. It works great. Combine that with MacFusion for file sharing and routing the mini’s audio output into the line-in on my Ubuntu machine and it is just about as good as a VM would be.

Pygame seems to work fine on the mini so most of what I need is there. Good deal.

But it sure is frustrating to try to get anything else going. First there is hardly any information online. What I find is often out of date and useless because so much seems to change from release to release. Then when something doesn’t work I don’t know who to ask. In contrast on Windows or Ubuntu there are huge and active communities of hackers like me who 1) figure stuff out and 2) write it down where Google can find it. The problem is usually culling through millions of results. On queries related to OS X the problem is I get 5 hits, 3 of which are asking the same question with no answer. The documentation at Apple is horrible and either not indexed by Google or not indexed well. I seriously don’t know why software developers put up with it. I guess they like being pioneers. I just want to get the job done…