Software, technology, sysadmin war stories, and more. Feed
Thursday, April 12, 2012

Bit-banging with front-facing cameras

A couple of days ago, I saw something which lets you tap your keyboard with your smartphone to get access to your pictures. It apparently uses the timing of that keystroke combined with the timing of your phone being jostled as measured by its internal accelerometer. Further investigation suggests this is by the same folks who originally had a "tap two phones together" scheme which worked much the same way.

This is pretty interesting, but it got me to thinking. Something like this, where you correlate timings and probably work in geolocation for both participants and pick out the most likely pairing is not exactly common. Things like this typically wind up getting patented. At that point, then you have to watch your step if you operate in that space or you might face a lawsuit.

This then got me thinking about ways you could make this work between devices without involving the accelerometer in order to dodge a hypothetical patent. Obviously, there's Bluetooth. That's theoretically been possible for years and years, but I have yet to see it work properly. Maybe I don't run in the right circles, but it seems like that's always something which is "coming soon".

Then it came to me: phones frequently have front-facing cameras now. The important part here is that they can both see and display things with the same bigger surface. So now what happens if you have two devices like this? You could just hold them up to each other, right?

This got me thinking of a visual three-way handshake. One system puts up the equivalent of a SYN, and the other one scans it with its camera, plucks out the sequence number, generates an ACK, generates its own SYN, and pushes that out on its own display. The other system sees that, ACKs that sequence number, and puts that out on the screen, and now you're talking.

After that, it's just a matter of throwing bytes back and forth until they're done. Then they make some kind of alert noise or flash something, and you pull them apart and go back to doing whatever.

I should point out that computers with web cams could also participate in this. As long as the devices can both see and display patterns to each other at the same time, there should be some way to make it go.

So there's my half-baked idea for another way to do such things. It doesn't require an external network connection, or indeed, any network I/O. You could even do it on an airplane when they've banned intentional RF transmissions (wifi, BT, ...) since it's only using visible light.

Finally, I have one other random thing which would be fun to try. If you took two phones and actually put them together face to face touching each other, could it still work? Granted, the screens would be far too close for the cameras to focus, but I bet a camera could still tell the difference between light and darkness from the other screen. If you have that, you have ones and zeroes, and you can get data across.

The better question would be is if you could come up with multiple colors to allow for full-duplex sending. You'd also need to do the equivalent of echo cancellation, but this time it's in the video space. That is, if you're transmitting a white block at the time when your partner is sending a blue block, you might get back a block which isn't pure blue, since it's mixed with your own white light. You can say "oh, right, I sent that, so let's subtract it out", and then arrive at whatever they sent.

Following from this, maybe you could tell different colors apart, and now you don't just have ones and zeroes. You'd end up with a whole constellation of possibilities which can encode multiple bits at once.

Now you're talking multiple bits per baud (!) and suddenly, all of the high-end modem stuff of the '90s starts taking on a whole new meaning.

So hey, maybe patents aren't all bad. If not for the general concept of having to evade them, I might never have thought about this particular flavor of insanity.