AT Design »

Confessions of an Engineer – insights into long-distance assistive technology design

Tom Jakobs, MS, P.E. and Ben Chater, J.D.

In the summer of 2009 I (Jakobs) received an email from a colleague requesting some help for a college student with cerebral palsy. Ben wanted to access an iPhone with a head stick. I sent Ben an email, to which he quickly responded. It was the beginning of a long-term, long-distance design project – one that both of us feel worked out well without ever meeting in person. But it wasn’t easy. It required time, clear and respectful communication, and persistence. And while lots of people say “well that seems obvious”, very few are willing to do the work necessary to be successful. Here’s how the process worked.

Hi Tom,

Thanks a bunch for your note. I can't tell you how much I appreciate it. I've used a headstick for 18 years, and capacitive touch screens have always annoyed me. But now that I've got the iPhone bug, it's reached a whole new level. Here's my deal in a nutshell:

I'm attaching some photos of me with my stick that should help. …It's a headband with an angled aluminum rod. At the end of the rod, I have a metal coupling (which I got at a hardware store) that fits over end of the rod (with a little masking tape in between to create a friction fit). In the end of the metal coupling, I have my "tip"--a 1-inch long piece of pencil with an arrowhead eraser stuck on the end. I use that exact setup to type (about 20 words/minute on a regular laptop), to move CDs around (by sticking my tip through the hole in the CD and moving it around), to operate TV remotes, to send text messages on cell phones that have physical buttons, to manipulate pages into my fax machine... It's the only "hand" I have that I can be precise with.

I would love to be able to use my iPhone and the trackpad on my Macbook independently. …There are just a few important features of my headstick that I'd need to maintain. I need the stick to be as light-weight and low-profile as possible...though it does need to be strong. (Aluminum's been pretty good.) I've also gotten used to the particular angle in my stick. Most importantly, however, is the tip. I need some friction...moving paper and typing would be much more difficult if I had a hard metal or plastic tip. Also, I like some edges around the side of the tip so I can grab and hook things. An artisan-quality arrowhead eraser has always worked the best… Let me know how I can be clearer. I know I ramble.

Thanks a lot!

~Ben

Above is an excerpt from one of our first emails. Notice how clearly Ben communicates in writing. He’s told me exactly what an engineer needs to know: (a) Here’s what I have to work with, (b) Here’s what I want to do, and (c) Here are some design parameters that are important to me. And while some consider it unimportant, also notice the kind tone. Both Ben and I understood from the beginning that we were establishing a relationship. This was important if we were going to make it through more difficult times.

Hi Ben,

Thanks for the great description. If you get a chance, it would be helpful to me to get a close-up picture of the end of your head stick.

I’ll start doing some experimenting to see if we can find something that works. I’ll get back to you. You are always welcome to check-up on me – I don’t mind at all.

More soon,

Tom

The next step is to be patient with the process. The engineer sees a project from a different perspective than the user. Both people have to be satisfied that the project is moving in the right direction for it to be successful. Ben and I have emailed back and forth 93 times in a year. We also addressed lots of ideas and requests for clarifications from parents, friends, and other engineers. Patience and persistence are required to make steady progress.

Hummmm....

Here’s what we have done. We have looked at several conductive materials under multiple conditions, and to my surprise – they all work. They work so well that I can’t figure out why Howard’s original suggestion of mounting the Pogo stylus to the head stick didn’t work. Based on what we’re seeing, I think it should have.

So here is one possibility. In order to use the Pogo stick, it is important (I think) that the stick is relatively perpendicular to the iphone surface. Do you think it was when you used it, Ben? Did the Pogo stick not work at all, or just not reliably?

I need to understand why the Pogo stylus failed in order to understand what we need to do to make a tip that will work for you, so give me as much info about the use experience as possible. If you don’t have much info, I’ll buy a couple of them so we can figure out what is going on.

We’re narrowing in on a solution! My guess is that we’ll find something that works for you. Right now, I think we’re missing some pieces of the problem.

Looking forward to your insights,

Tom

The first prototype was quick and dirty. The idea was to get something to Ben so that he could provide feedback. At this point we’re about one week into the project.

Hi Ben,

We're shipping a rough prototype to you today via 2-day Fed Ex. The idea is for you to criticize it. Tell me what you like, what you don't like, and how consistently it works for you. After we hear back from you, we'll go to round 2!

You should have it on Thursday,

Tom

And Ben’s initial response to try #1.

Hi Tom,

Got the "magic finger" today. To be blunt, it's freaking miraculous! It's one of the most extraordinary things I think I've ever seen. I'm extremely curious what's under its little blue skirt, but I don't really want to dissect it. It works real well on the iPhone, and it also works on my MacBook touchpad. I'm truly amazed!

It works quite consistently. I don't have to push hard to make it work. Sometimes when it doesn't work immediately when I touch the screen, I do give it a little pressure or rock the tip on the screen a little to get it going.

The foam tip is a bit large compared to what I'm used to, and it has a fairly broad "contact area." This makes it a bit tough to be precise on the iPhone... sorta hard to tell where exactly to aim, because the tip takes up so much space. (Even confuses the iPhone sometimes because the point of contact is so big.) Also, without a semi-pointy end it requires more force and precision to type on a regular keyboard. This may be just the way it is...we probably need a fair amount of surface area to make it work.

The texture of the black tip is better than I thought it would be. As rubbery as we can make it would be great, but it's an awesome first go of it. I will have some more suggestions about the shape of the tip.

It is, of course, heavier than a pencil eraser. I've been playing with it for about 2 hours, and my neck's starting to get tired. It is longer than it needs to be...I have it inserted into my headstick all the way, and there's about half an inch between the top of the blue skirt and the bottom of the coupling. Don't know if making it shorter might be a way of cutting down on weight.

I was on Skype tonight, and I noticed a tiny bit of interference in the audio that varied with the location of my stick. Don't know if this is anything to note. Hope you’ll forgive the detailed critique…

Thanks. A million.

~Ben

Ben’s response is encouraging and detailed. He is fun to work with and that makes a difference! At this point there is a lot of enthusiasm. We’ve been working on the project for 10 days (intermittently) and everyone is happy. We built a few units for Ben and adjusted them to his specifications. Ben made a great video of him using the tips. A few weeks later reality starts to set in.

Hey Tom!

Hope you're doing well. Wanted to check in with you to brainstorm, because all the tips I have are beginning to wear out. I think over time as the foam gets used and pressed on a lot, it sort of flattens out right over the end of the metal rod, and (I think because the "touching" surface area gets smaller because the end of the rod starts to sort of protrude) the functionality decreases. I'm back to using the tip you sent before the last 3, because the solder on the washer on that one busted, so I can physically pull the whole foam end--washer and all--down away from the end of the rod to get it to work a little better. (Don't know why the foam tip hasn't slid all the way off...probably still some solder stuck to the rod keeping the washer from sliding all the way off...)

Ideas? Maybe if we could somehow attach the foam on there so it has a little space between it and the end of the rod??? I guess the foam might always compress over time... We don't think there's any other type of more durable and dense foam out there that would be as conductive, right?

Thanks a lot! Have a good weekend.

~Ben

I’d like to say this is unusual – but it really isn’t. Fortunately, we had a good relationship started when this happened. I knew that Ben was sincere in his effort to help us, and Ben knew that my team would do all we could to help him. The project is now getting harder. The simple solution – which might work for some people – doesn’t work for someone with Ben’s physical abilities. He’s tough on the tip and it needs to be more durable. Ben continues to provide excellent insights into how he uses the tip and how he made it work for a while.

Hey Tom,

Can't say as I'm entirely surprised...those tips definitely got a lot harder for me to use over the time I had them, but I suspected the specific way I use them might be a significant factor.

I have pretty good control over the pressure I use on the iPhone. When the tips are brand new, I barely have to touch the phone...they're that functional. But over time, I have to push harder, and I also have to "hold" each tap for longer before it registers. Sometimes I actually have to sort of "roll" the tip over a button to get it to recognize it (think of giving a fingerprint...). …

One thing that does help sometimes is sort of pinching the foam tip a little bit so the foam puffs out a little more at the end. When the tips aren't working well, I can actually feel the end of the rod touching the phone through the foam (if I tap out a beat on the face of the phone, I can actually hear the rod tap). Squeezing the foam down over the end of the tip puts more padding down there...and makes a bigger surface area that touches the phone(?????)

I do put a lot of pressure on the end of the tip, just in general use. I often lean/rest on it while thinking when I'm on my computer, and with all the extraneous stuff I do with my stick, there's lots of reason for that foam to get super compressed and pushed up around the end of the rod. From my own observations, I think that's the most likely cause of this. … I know it must be difficult, not being able to see it in use. Let me know what else you need to know.

Thanks a bunch!

~Ben

The next couple of months are challenging. Yet, everyone is working together. Ben’s parents and friends send encouraging emails and try to help. We’re sending Ben replacement tips as he wears them out. We’re trying new materials that we hope are better and they are not working well. And then we get a break!

Hey Tom,

Hope you're doing well. I have good news. My Mom was on Amazon the other day, and she saw some iPhone styli on sale for like $1.87ea. She ordered 15 for me...12 of one type and 3 of another. I received the 12, and I opened them up today, and they have little rubber tips! I couldn't believe it. They still don't work on my headstick as-is but today we chopped a few up to investigate … My Mom rocks for finding the new styli!

I can send you a few of these styli if you want to check them out.

~Ben

The new tip was promising, but still not durable enough. We made the tip stiffer by filling it with silicone rubber and sent Ben a prototype.

Well Tom, Erik...y'all never thought the day would come. But I do believe you've done it! Hopefully I'm not jinxing it with these words, but this tip is PERFECT!!!!!! (...with one tiny little minor footnote, below.) Its functionality on the iPhone is perfect, at any angle. The rubber tip seems to be extremely durable...whatever little reinforcement ring you put around the outside has been completely effective at protecting it from 1 week's worth of my abuse (and I haven't held back!). The rubbery texture is, of course, exactly what I needed; it eliminates all my lingering desire for my old eraser tip (flipping pages is a breeze, e.g.). The little washer edge is, as it always was, incredibly useful and durable. The black rubber dip stuff still kinda flakes off, but I don't really need that anymore with the rubber tip. You guys have nailed it.

Thanks, Mom, for spotting those rubber-tipped iPhone styli!

The tiny little minor footnote has nothing to do with the functionality of the tip, but only with the fit into my headstick. The strips of solder don't really do the trick. You actually came way closer to nailing the fit with this tip than you have with the others; this one fit like a glove without any tape for a day or two. Then it started wiggling a little. When there's a wiggle, the strips of solder make it difficult to effectively use masking tape to beef up the diameter a smidge; each strip of solder is a slightly different size. If I were to be able to get just one more of these things from you, I might ask that you just leave the copper tubing naked, and let me fine-tune the diameter with evenly-wrapped masking tape, unless there are other ideas...

I have about 10 of those iPhone styli I could send you, if it would be helpful. I'm not doing anything with them.

You rock, guys! Thanks a billion.

~Ben

The new prototype tip lasts for about two months with daily use. This is probably the best we’re going to accomplish. The mount into the head stick isn’t perfect, but Ben seems to be able to make it work reliably. We may incorporate a set-screw into the head stick to make mounting easier. For this project, there is no “once and for all” solution. We have had others ask about the iPhone tips, and each person required a different design based on their physical abilities and head stick design. For various reasons, not all of these people received a tip that met their needs. Based on my experience as an engineer, here’s why some of these projects worked and some failed.

1. For most of the people that we work with – who typically have complex access needs – establishing a good relationship is necessary. We are rarely successful when we just send equipment. If a person is not willing to really work with us, for whatever reason, the project is not likely to result in something worthwhile.

2. The process of meeting someone’s needs is iterative. Patience and persistence are needed by both the engineer and the user. When designing assistive technology, little things make all the difference. For example, one of our biggest “hassles” with AccuPoint is figuring out how to mount it so that is accessible every day. We have been through five types of Velcro in an effort to find an adhesive that doesn’t fail after a few weeks or months. As engineers, it would be easy to just let this go – but to the user, anything that is unreliable makes the system completely unusable. We are quick to make custom mounts when they are needed.

3. Respect is key. Take your frustrations out on your design partner and the project will suffer. Try to “assume good intent” by your partner. The design process is difficult and the communication process can be ambiguous, particularly when it is a long distance relationship.

4. Be generous with your time and talents. This is a two-way street and the benefits are extraordinary for everyone when there is give and take.

Ben is an exceptional guy. He worked hard to establish a respectful relationship even though he had a lot of other responsibilities. He just finished law school (spring 2010) – so he is no stranger to patience and persistence. We expect to support Ben’s “iPhone habit” for as long as this technology is useful to him. Working together has been a creative and gratifying experience for everyone involved.

Questions? Contact us.