Monday, October 18, 2010

The magic keyboard



As I hinted at in an earlier post, the magic trackpad hardware is very well designed and bears a strong resemblance to the Fingerworks series of input devices. Now, I've decided to extend the functionality of this device by building a wireless multitouch keyboard using two magic trackpads. Currently, this keyboard will only work in linux since it relies on my extensive modifications to the linux kernel driver for the magic trackpad.

The first image above shows my two magic trackpads with plastic overlays to indicate key placement. It turns out the magic trackpad will still detect contact through a thin insulator placed on top of its surface. The overlays are simply standard laser-printable overhead projector transparencies. I originally considered laser etching the surface of the trackpads at TechShop, but I think I will hold off until I've settled on a key arrangement that I like. Here's a picture showing how I originally planned to lay out the keys relative to my hand.



For now, I opted against the layout above in order to simplify the code that converts coordinates to keycodes. The layout I'm currently using consists simply of three concentric circles, so the code simply checks which circles contain the current touch to determine the corresponding "row" and then compares the X coordinate to a table of thresholds to determine the "column". I created the layout seen in the other pictures using Illustrator. I'll post links to PDFs for the left and right halves in case anyone feels like trying this out.

The keyboard was intended to be placed on the lap to minimize elbow flexion. To hold the halves together I used two strips of rubberized truck tarp (very strong stuff used for a windsurfing roof rack). I'm still working out how to make this attachment adjustable without being bulky.


The transparent blobs seen on the home row of the keyboard in the previous images is clear nail polish. A few coats of this allowed creating a raised ridge to facilitate finding the home row while touch typing. This is actually the principal drawback of a multitouch keyboard- the inherent lack of tactile feedback. The nail polish helps somewhat, but I'm still thinking of other ways to mitigate this issue. The image below shows the nail polish blobs close up.



I currently have preliminary mixed typing/mousing support and a few simple gestures. I also implemented certain behaviors which I think make the use of a touchpad far more ergonomic:
(1)Moving the cursor and single clicking is done with two fingers rather than one.
This helps eliminate the annoyance of inadvertent cursor movement or clicks when a single finger alights on the trackpad. I also think it's more ergonomic, since for me using a single finger requires more effort than using two.
(2)Double clicking is done with a three finger tap- this eliminates the hassle of properly timing a double click.
(3)Scrolling is done with four fingers instead of two. This allows just plopping your entire hand down on the trackpad to scroll rather than contorting it to extend only two fingers.

Todo list:

Software:
Add support for modifiers (i.e. shift,ctrl,alt,meta) using chords.

Write a GUI to rearrange key placement which generates suitable header files based on user's choices. Perhaps eventually come up with a means to allow reconfiguration on the fly.

Hardware:
Come up with better attachment and adjustment system for the straps which connect the two halves of the keyboard

To give some context for this project- here's some images of the devices the magic trackpad derives from - the iGesture and touchstream. You can see that the magic trackpad is significantly smaller than both the iGesture and touchstream which is why I think that modifiers will have to be handled only via chords (no room for separate keys). In the images below there's a weird blob on the USB cords of the iGesture and touchstream- those are my improvised hot-glue strain reliefs.





11 comments:

  1. I love this, but I reckon that (and no pun intended it lacks an arduino or something similar) purely as you could interpret the co-ordinates on a uc and output the key strokes via USB/PS2 which would eliminate the need to modify the actual software on the computer. But again I really do love this idea!

    ReplyDelete
  2. I agree that a physical hack would have been nice as far making the keyboard OS independent, but it also would have been significantly more complicated. The microcontroller you're alluding to would have to be able to act as both a bluetooth host and a bluetooth HID device. I don't know of any readily available generic bluetooth modules (not serial converters). The only way I could see implementing what you suggest would be to have a full blown computer act as an intermediary between keyboard and computer. That or an android phone with a heavily modified bluetooth stack so it can act as both host and HID device. The last and probably most difficult approach would be to open up the trackpad and see if the data being transmitted over bluetooth can be tapped off somewhere then manipulated. This would be a lot of work and potentially just impossible if all functionality is one chip and the data is never exposed.

    ReplyDelete
  3. Hey sebastian,

    When you are ready hit me up. I will laser etch your stuff for free bud.

    Check out my blog. Just toss me a comment / email.

    www.synthetos.com/blog

    Ril3y

    ReplyDelete
  4. Hey. A friend linked me to an article on your keyboard on hackaday, which also says that you're hesitant to post code that causes kernel panics.

    You should have a look at the uinput kernel module, which allows userspace-based input drivers. It's a decent protocol - when I get a few minutes, I'll email you some stuff I have that uses it from ruby.

    ReplyDelete
  5. Okay, I have some code to give you to help out with using the uinput module, but I can't seem to find a way to email you just from what you've got available here on blogger. Email me at base64 cmlkZG9jaGNAZ21haWwuY29tCg== and I'll send you some code.

    ReplyDelete
  6. Have you considered using the magic track pad as a numeric keypad? It would be the perfect complement to the wireless keyboard.

    ReplyDelete
  7. I personally don't use keyboards with numpads, I find it makes them too wide. That being said, implementing a numpad on the magic trackpad would be very easy. In fact, it's actually the first experiment I did when I only had one magic trackpad.

    ReplyDelete
  8. Thanks for responding.

    I thought of a useful application for the App Store. Would consist of a small program designed to allow different tables (number of rows and columns as needed) and set in each cell a letter, number or shortcut. Then this table could be saved in a printable file for use in the magic track pad, switching to a particular gesture (a "N" for the Numeric keypad, a "P" for Photoshop, a "C" final Cut pro ... )

    The problem is I'm not a programmer, but I would buy it in the AppStore if someone developed.

    Greetings and congratulations for your amazing work!

    ReplyDelete
  9. I have posted about The Magic Keyboard on my blog (http://people-technologyinteraction.blogspot.com/2010/11/el-magic-keyboard-otro-nuevo-teclado.html#comments).

    Greetings and congratulations.

    ReplyDelete
  10. This was also posted to the Fingerfans forum - a FingerWorks support community: http://fingerfans.dreamhosters.com/viewtopic.php?t=2824

    How well does overhead foil work as touch surface? I've been experimenting with different surfaces on my TouchStream and found that simple high-quality paper worked best for me because of minimal friction.

    As for home row "dimples", it should be pretty easy to make a few small dents from the underside of the overhead foil, just like the dimples on the TouchStream. I also experimented with crease lines to imitate key area borders; this was much better in terms of tactile feedback than the small dimples, but these crease lines were difficult to make well. Anyway, some ideas to try out!

    I commend you for the overall project. It was obvious that someone would attempt "2 x touchpad = 1 TouchStream", or = 1 Apple "iType" as I call this future product... :-) I don't think it will be long until Apple releases a real iType, but surely only for Mac.

    ReplyDelete
  11. Can you please email me. I would like to work on this. kaplan@github.com

    ReplyDelete