CamKeys is a simple camera-based alternative keyboard

| tags: motor impaired, enabling technology

People with severely impaired movement often use one or two switches to control their computers and/or communication devices. Even though the switch is arguably the simplest electrical component, when it becomes an assistive technology it is likely to cost the end user hundred's of dollars. On the other hand, a fantastically complex electronic device, the USB camera, is available at discount stores for a few dollars. CamKeys is a simple python program that makes a cheap camera emulate any number of switches. It simply generates key events when user-specified regions of the camera image change.

CamKeys was developed by UNC undergraduate student Arthur Greenside using python, wxPython, and CVtypes a ctypes-based wrapper for OpenCV. This project was supported by a grant from the Mozilla foundation.

CamKeys is simple to use. Start it up and point the camera at a background with some visual texture. We've used tablecloths, printed images, and wood grain successfully. Just about anything other than a featureless field will work fine. Now press the left mouse button and stretch out a small rectangle in the image. When you release the mouse button, a dialog will popup asking for a key to type that box. Repeat for each different virtual button you'd like to create. Now click on some other window to give it keyboard focus. When your hand (or anything) enters one of the watch boxes you'll see the box outline change from green to blue and the key you entered will be virtually pressed. When you leave the watch box the key will be released. You can save and restore configurations using the menus. You can also enable a click sound on key activation.

Since CamKeys generates what look like key presses it can be used with any software that responds to key events. Later, we'll add a direct path to communicate with Jambu enabling switch users to type on Steve Lee's alternative keyboard. For now, we're using it control Daniel Parker's Tar Heel Typer to provide an alternative pencil to kids learning to write.

For now CamKeys is available only as source code you can download from cvs on SourceForge. It is known to work on Windows and Ubuntu with cameras from Logitech. I expect it to work on others OS's and with other cameras. After I get past Maze Day on December 6th, I'll make a proper installer.

As of 18 November 2007, CamKeys has several known limitations:

  • It doesn't properly handle keys that require 2 events such as pressing shift then A to get a capital A
  • On Windows the OpenCV camera input seems limited to about 15 frames per second. On Linux, I have to use setpwc -f 30 to get my Logictech Pro 4000 to go at 30Hz otherwise it is limited to 10 FPS. My Logitech Pro 5000 seems to ignore our request for a small image.
  • Press and release are currently tied to entering and leaving the watch box. For some users it might be better to allow a single press and release when the box is entered.
  • We don't yet have provisions for controlling the mouse.
  • CamKeys is a weak name. I hate naming things.