RIApalooza 2: RIAs Beyond the Mouse and Keyboard Recap

1 Comment

On May 8th I as at RIApalooza Two, an unconference event for Rich Internet Application enthusiasts. Adam Flater and I gave a talk titled ‘RIA Beyond the Mouse and Keyboard’. Adam and I talked about how computing user interfaces are evolving and moving away from the current status quo of mouse and keyboard.

Adam talked about and demoed examples including his use of the mac accelerometer as an input device, his Lego Mindstorm robot controlled via Flash, and his mobile device, the Tesla Roadster.

I showed a couple videos of innovative uses of technologies such as augmented reality, touch computing, and the Wii balance board. Here’s a link to some of those videos. I demoed controlling a virtual 3D car with the Wii balance board, controlling a 3D earth using head tracking, combining Augmented Reality with Twitter, and my PostIt board with handwriting recognition. These are all part of a holistic vision of what constitutes a more natural user interface, moving away from the more artificial interaction that are the current state of input devices.

Here are some photos taken from Anthony Hand’s camera. My favorite picture has to be where I’m caught picking my nose – great.

Popularity: 3% [?]

Concept Virtual Corkboard With Handwriting Recognition for Large Interactive Walls


I’ve been playing with the idea of an inexpensive large collaborative wall. Part of the requirements for my large interactive touch screen is that it has to be multitouch and provide a more natural way of interfacing with it other than mouse and keyboard. Johnny Lee has shown that you can build one pretty cheaply. You can set one up using just a couple Wii remotes, a projection screen, and a homemade IR pen. You can then create a virtual whiteboard application pretty easily.

The problem with virtual whiteboards is that all the drawings and text are just graphics. They don’t mean anything to the system. The text isn’t really text. It would be better if you could write the letters and have them be interpreted as real text.

Using an open source Flash based mouse gesture library, I put together a concept virtual corkboard that lets you write letters with your mouse (or IR pen) and put that text on virtual PostIt notes that you can place anywhere on the board.

Check out the video:

Of course, this isn’t the only way to tackle this problem. One could just do OCR on the drawings and figure out the text that way like the mobile app EverNote, but this is just one solution.

Open demo in lightbox

Open demo in new browser

Popularity: 15% [?]