Concept Virtual Corkboard With Handwriting Recognition for Large Interactive Walls
Apr 13
Labs, Technology flash, flex, gesture, nui, ria, touch, wii 7 Comments
I’ve been playing with the idea of an inexpensive large collaborative wall. Part of the requirements for my large interactive touch screen is that it has to be multitouch and provide a more natural way of interfacing with it other than mouse and keyboard. Johnny Lee has shown that you can build one pretty cheaply. You can set one up using just a couple Wii remotes, a projection screen, and a homemade IR pen. You can then create a virtual whiteboard application pretty easily.
The problem with virtual whiteboards is that all the drawings and text are just graphics. They don’t mean anything to the system. The text isn’t really text. It would be better if you could write the letters and have them be interpreted as real text.
Using an open source Flash based mouse gesture library, I put together a concept virtual corkboard that lets you write letters with your mouse (or IR pen) and put that text on virtual PostIt notes that you can place anywhere on the board.
Check out the video:
Of course, this isn’t the only way to tackle this problem. One could just do OCR on the drawings and figure out the text that way like the mobile app EverNote, but this is just one solution.
Open demo in lightboxPopularity: 40% [?]
Related posts:
Pek Pongpaet is the VP of Technology and Product for SpotOn Inc. Pongpaet’s expertise ranges from product design and development, and martial arts. Prior to SpotOn, Pongpaet worked at Accenture Technology Labs in the research department coming up with next generation user interfaces. At Roundarch, a technology and strategy consulting firm, Pongpaet’s work included envisioning and designing the dashboard of the future for the Tesla Model S electric car.
Apr 13, 2009 @ 04:58:04
You are definitely onto something here!
The handwriting recognition reminded me of Graffiti from my old Palm days (daze?). I was able to use it without even watching the video.
This would be great to sit down sometime with a bunch of people and brainstorm using it! For one, I can see this as a great tool for modeling Web site hierarchy, blog categories, etc.!
mp/m
Apr 13, 2009 @ 20:03:16
Hey Mike, the hand writing library was definitely based on Graffiti and I had no trouble picking it up as well. Glad you enjoyed it.
Apr 14, 2009 @ 09:19:49
To my mind, there are 2 issues with this:
One, will people be willing to learn to write in a new way — an “in place” style — in order to put text on a board. With the Palm Pilot, people seemed willing to do so, perhaps surprisingly. But then again, that was a small personal device, a fetish object to be mastered. You could practice in private — mistakes weren’t visible to others. With a wall things are quite different, as it’s a public affordance. Business users really hate to look silly at work.
Two, are we to the point yet where the tech supports a writing experience that isn’t crappy. I’ve built the Wii-based IR pen protoype that Johnny suggested — nowhere near enough resolution. I’ve built other systems as well; I’ve used Smart boards, and even those things you can suction-cup onto a whiteboard. All these are basically science fair projects compared to a real piece of chalk or a dry-erase pen. People make *very* fine-grained motor movements when writing naturally and un-selfconsciously. They also lift up and put down the instrument onto the surface really quickly and precisely, so timing is crucial. You can’t ask them to squeeze a button every time you want them to make a mark. Some very small movements (like at the top of a t — or was it an f?) turn into meaningful, visible changes in the writing output. Our tech needs to really capture these movements.
An in-place writing system like you’re suggesting could really contribute something valuable if it could get people to write BIGGER — make bigger movements, easier for the system can digest — while preventing the hassle where users have to physically move along the board because their BIG writing is getting so expansive. So I like the compression aspect.
Brandon
Apr 14, 2009 @ 15:47:57
Cool. I could see Twitter and a search for people be involved in this….
Apr 29, 2009 @ 02:14:41
Pek, can I be your new #1 fan?
May 10, 2009 @ 22:14:34
Hi Dan, thanks for your vote of confidence. Love your work as well.