With all the buzz going on with the release of Apple’s new iPad – a multitouch tablet computer, my friend Scott Robbin came up with a keyboard concept that he believes is an improvement from what Apple is offering. I thought it was a great idea because made the tablet work like a blackberry almost. You used your thumbs to enter the key strokes as opposed to hunting and pecking with one hand while holding the iPad with the other. It also had the advantage of not obstructing the center with your hands or the keyboard.
However I thought that Scott’s concept, as well as the original iPad soft keyboard have limitations of their own, namely that both require you to use both hands. Scott’s post sparked my own idea of a soft keyboard that would require only one hand to operate.
You’ll have to forgive the actual design, but I whipped it together in a few minutes using the Shear filter in Photoshop. The point is that the curved keypad allows you to use your thumb to type while holding the iPad with the same hand. This way, you use the iPad on the train or bus if you had to hold onto the railing. Naturally if you were left handed, you could flip the keyboard to the other side.
I spent this afternoon putting together a quick demo for my Rich Internet Interfaces Beyond Mice and Keyboards talk I’m giving at the DePaul University iTea Seminar tomorrow. The example is a simple one that illustrates using a WiiMote as an interface to controlling a 3D model. This can be easily used as a game control and or more general UI control. I believe it is a more natural way to interface with 3D elements navigating in 3D space and can be generalized to let people more intuitively navigate through and interface with software such as the operating system. After a few hours of looking for the WiiMote, charging it and some coding, I’ve put together this simple demo:
I can’t claim credit for putting together code to display a 3D X-Wing. I’m not interested in that. I quickly found what I was looking for here: http://code.google.com/p/papervision3d/downloads/list – the source code repository for PaperVision – the open source 3D engine for Flash. At the very bottom there’s sample code for displaying a 3D X-Wing.
First things first, you will need the WiiFlashServer that interfaces the WiiMote with your Mac. That code and supporting documentation can be found here: http://wiiflash.bytearray.org/. There are 2 pieces to the puzzle here. The WiiFlashServer which listens to the WiiMote and sends messages to your app and the WiiFlash Actionscript API which can receive those messages.
Once you download the Actionscript API and put that in your project directory, include it in your application:
Initialize the Wiimote like so:
wiimote = new Wiimote();
wiimote.addEventListener( Event.CONNECT, onWiimoteConnect );
The only thing we really have to do now is to use the pitch, yaw and roll values of the WiiMote and use them to control the X-Wing, like so:
var rotX = 0;
var rotY = 0;
var rotZ = 0;
rotX = wiimote.roll/Math.PI * 180;
rotZ = wiimote.yaw/Math.PI * 180;
rotY = wiimote.pitch/Math.PI * 180;
// no easing - animation too jagged
//group.rotationX = -rotY;
////group.rotationY = rotZ;
//group.rotationZ = -rotX;
You can see from the code there’s a section I commented out. You can use the raw WiiMote values to feed to the X-Wing directly but I found that the values can be jumpy and discreet causing the X-Wing to turn and move with a lot of jitter. I added a simple easing function to make the movements smoother.
As you can see, it’s not rocket science. With a little bit more work, we can easily add other controls like button presses to control the firing of the X-Wing.
Here’s the link to the source if you’re interested. Thanks for watching.
I’ve been thinking about 3D desktops, alternative user interfaces, and user experience for a while now. I keep a sketchbook that has illustrations of some of my ideas. When I saw the Apple patent filings for their 3D desktop ideas, I was delighted to see that some other folks have very similar ideas to mine.
I hate 3D for the sake of 3D. When I think of adding a new dimension to something as vital as your desktop, many questions come up like:
What benefits does adding another dimension mean to the user?
Am I making certain tasks harder when I add another dimension?
What problem am I solving?
Watch this quick video on some of the ideas I’ve been tossing around.
The Mac OS spaces have never made sense to me. I never use it, not because i haven’t tried. I’d rather stick to my dual monitor. But when I am out and about with only the 1 screen, I still don’t use it. Here’s why – When I put something in another space, I just forget about it. It’s out of sight – and out of mind. I rather just have tons of apps on one screen and use Expose to sort it out.
Having a shoebox/diorama analogy of multiple desktops makes more sense to me since I can zoom out and see where my other spaces are. I have a sense of the spatial relationships of multiple desktops as well and that is key.
Cover Flow on the Desktop
I’m a big fan of Cover Flow even before it was integrated in all the Apple products – when it was somebody’s plugin for iTunes. That’s right – it wasn’t create by Apple. However having it confined within the iTunes application feels restrictive. Once you have a 3D desktop, why not make cover flow a full class citizen of that desktop?
Carousel to Navigate Apps
This is probably one of my weakest ideas and needs to be fleshed out. The basic premise is once you have a 3D Desktop, your apps should exist in 3D space. I’m not sure on the benefits of the Carousel myself, but I felt it was a cool UI.
Using Head Tracking to “look around”, by looking around
To me, this concept is what makes a 3D desktop really cool. The scenario is this: Say I’m downloading something, I can put that window on the desktop on the side of my main one. When I want to check on the progress of the download, instead of having to hit keys on the keyboard to find out, I just very naturally look around to see how it’s going. This is a very natural interaction that doesn’t have to be memorized or taught. This interaction therefore has a low cognitive cost. Instead of having to explicitly take action via keyboard commands by switching desktops to see the progress and switching back, possibly losing context, I quickly glance over and come back to my task.
There’s a lot more than can be explored. Right now, the “floor” of the desktop has not been considered. Also, I have some ideas around 3d folder navigation that I can dive into in the next episode of 3D Desktop concepts.
On May 8th I as at RIApalooza Two, an unconference event for Rich Internet Application enthusiasts. Adam Flater and I gave a talk titled ‘RIA Beyond the Mouse and Keyboard’. Adam and I talked about how computing user interfaces are evolving and moving away from the current status quo of mouse and keyboard.
Here’s a demo that illustrates my ideas on computer human interaction. Instead of controlling the movement of a 3D globe through the mouse and keyboard, why not move your head to look around the globe. You’ll need a webcam for this demo to work.
Side to side motion works pretty well. Up and down is a little iffy.
Moving around to see what’s on the other side is a much more natural and intuitive human behavior than say pressing some computer keys to change perspective of an object. I believe that as computers become more powerful and more and more people have to interact with them in their daily lives, computer scientists will have to design systems and software that provide a more natural user interface (nui) than what we currently have.
Although many people today deal with computers and the internet, this highly technical tool is still completely inaccessible to a large population because it is so difficult to use. By creating interfaces that are more natural to use, we make technology more approachable to the masses. That’s my 2 cents. Feel free to chime in.
Pongpaet’s expertise ranges from product design and development, and martial arts. Prior to Pinstagram, Pongpaet was VP of Product at Spoton, a loyalty and social media company. He's worked at Accenture Technology Labs in the research department coming up with next generation user interfaces. At Roundarch, a technology and strategy consulting firm, Pongpaet’s work included envisioning and designing the dashboard of the future for the Tesla Model S electric car.