Friday, October 17, 2008

Augmented Reality

My brain's been stuck on augmented reality these days. I've been working out the ways it will change the world, but something in my brain keeps saying that it's more important than I've worked out.

I've tried to write a lot of essays on the subject, but they keep getting too long as I work out details. So I'm not posting them. This time, I'm going to try to keep things shorter.

Why is augmented reality (AR) going to be as world-changing as the internet? What benefits does it offer? First, what the hell is it, really?

It'll be a long time before we wear goggles that alter how reality looks, if we ever do (it may actually be easier to replace our eyes or get a neural feed), but AR is useful long before then.

For example, there are new displays that can be transparent (or, at the worst, display video from a camera on the other side of the display). Imagine including GPS and orientation systems into the display, so that it knows exactly where it is facing. This system can't do any kind of visual analysis, but what it can do is reference a database full of things that are recorded in the world at specific points. Some of these things will be real (addresses), some will be fake (restaurant reviews), but they can all be displayed just by knowing where we are and where we're facing.

This should be enough to accomplish the vast majority of AR's real power. What's it accomplish?

As far as I can tell, the biggest benefit is deep asynchronous communities (DAC).

An example of a DAC is YouTube: people will post thousands of videos to YouTube on any given subject, and friends and total strangers will see it and respond. It's significantly "deeper" than asynchronous communities offered by forums or MySpace because it's easier to communicate on a complex and/or personal level: just fire up your camera and talk. It's impossible to overstate the effectiveness of full audiovisual communication in comparison to text.

However, YouTube is a feeble DAC compared to a group of teenagers messaging each other with texts and pictures. The teenagers have an advantage: their DAC is supported by the strong backbone of their synchronous community: they're friends that hang out, which means that they share common interests and bonds. Although the messages and pics are not very deep communications, they are building off a mountain of deep communications: the synchronous community of the teens.

Some more modern not-so-deep asynchronous communities (such as XKCD) have worked in the opposite direction, working to establish synchronous communities from their asynchronous ones. This has worked only to a small extent...

The point is that humans build communities. It's part of our instinct. Even if the "people" we're building the community out of are represented only by simple Times New Roman forum posts, we manage. We build absurdly deep communities out of things that require only the lightest community - there's not much reason behind the deep sense of community that allows Eve Online to stay Online.

AR is, I think, one of the ultimate methods to do this.

AR is limited by the fact that it's local to where you are (or are pretending to be). However, this limitation is potentially a strong congealing force because it allows us to easily get a synchronous meeting going. Chatting in person is a hundred times more powerful than chatting through AIM.

"Why do we need AR? We could just go out with our friends!"

Ah, but the network is crippled as it stands now.

You can probably picture in your head dozens of people that looked like they might be Your Sort of Folk... but you never talked to them. Certainly never became their friend.

You also probably remember a bunch of people you were friends with that you aren't really friends with now - either they moved away, or they just didn't much like hanging out, or any number of other details.

Let's imagine ourselves up a naive political theory thought experiment. Wire up a college with the kind of AR we're talking about.

How long do you think it will take before the college is wholly transformed into a thousand different environments? I bet within a week the college has a fantasy facade, a scifi facade, a goth facade, a cartoon facade, a "student life" facade... and that's not even counting the facades for makeshift games such as assassins.

The system can't interpret visual information, remember. All of these facades are constructed virtually, but in "real space". Someone measures out the parameters of all the buildings, and then people specify virtual additions. The software shrugs and pleasantly burps up the fact that there is a huge dragon on top of Kinnecut hall, and that the Ecclesiastical Studies Building has been spraypainted with rude (and animated) murals. All visible by holding up your viewplate and looking through it.

It's possible that the arnets will be private - Jimmy and Don might have their own personal data for their own personal net - but many of these nets will be pseudo-public. Imagine cycling through networks. The scifi folk spend their time on a space station, where every window looks out over the nearby nebula. For some reason the student government has imprinted a map of the world onto school property and is running some kind of weird economic sim based on who travels where... All of them sprinkled liberally with various comments, videos, items, and so forth, some generated automatically, some generated by the participants.

It's like an immersive YouTube, except that you can choose to live in their worlds rather than just watch them talk. These shared realities are more powerful than virtual realities because they are so much more invasive: you can be in (and be helped by!) an augmented reality while you're listening to a lecture or eating lunch. You can be face-to-face with other people from your reality, talking and interacting at full speed, not crippled by text-only or time lag or anything else.

Add in the fact that if someone walks by your screen, you see their name (in your arnet) and various commentary about them, this can make your community spontaneous but very, very deep.

"Is it really so important, though? It sounds a lot like just joining a club, but with added graphics!"

I think it's not just important, I think it changes the world. Putting aside the cool power to reshape reality to whatever you want, putting aside the way it can be used to build "paths" through reality, rewind and fast forward time, provide instructions on complex things in the real world, and give us bulletproof personal ratings... it allows us to live together in a world we create, rather than living vaguely near each other in a world of life's choosing.

It's hard to say what the advantages to many of the AR abilities are. Will being able to virtually rewind time ever be as critical as, say, email? Will being guided on specific physical paths ever be as important as having a cell phone?

I can make some guesses (yes and yes), but it's almost impossible to describe WHY in less than ten pages, and there's a lot of chaos between here and there. They certainly won't become important all that quickly: AR will be around for years before they become important.

But communities will start important and stay important, and it won't be long before access to our central community database is considered a sense as important as sight or hearing. It's certainly not going too far to assume that AR will radically reshape what it means to be part of a community.

I think.

What do you think?


Ellipsis said...

I agree that AR will have huge consequences.

I disagree, though, that it will be important before we're all wearing AR goggles. I just don't think people will find it compelling to carry a screen around with them everywhere...unless by screen, you mean laptop.

I can imagine the basic idea of giving virtual objects a physical location picking up before the goggles do, but when we're talking about AR, the presentation really is part of the core concept, and I think that while technologies like you describe will likely be developed and used by some of us, AR will not become a truly relevant technology until someone comes out with a pair of stylish-looking AR goggles - the ipod of AR.

In short, convenience and ease-of-access are what would let AR take off.

Craig Perko said...

I do mean "laptop", although technically I mean "lightweight display for laptop". I think that once we get some useful digital paper (or similar), we'll see an explosion of laptops with detachable screens - the sorts of screens that would be very useful for this.

The idea here is that it is the MINIMUM possible for the technology. I don't think it will take off at that level, but I do think that will drum up more support, prove the concepts, and THEN it will take off.

Ellipsis said...

I'll admit that the idea started sounding more reasonable to me even as I was in the middle of writing my comment (the "unless you mean laptop" was a legitimate consideration).

But, call me superficial, but I'm still caught up on the stylish goggles thought. I don't see a screen as being immersive enough, and any kind of surgery is possibly too invasive...I'm thinking of a device on the order of a bluetooth headset, that's easy to put on and be in the system.

Craig Perko said...

The problem is in the display: a display that close to your eyes isn't doing you any favors, especially if it's transparent: your eyes will have to either focus on it or on the world behind it.

There's some evidence we might be able to beam light into the eyes in such a way as to make it seem like it's coming from particular distances, but that technology is a lot more advanced than it might seem. I don't see it being feasible any time soon.

Christopher Weeks said...

People are already geocoding pictures. Google maps allows the linking of map locations to other resources on the net.

The problem that you two are discussing is the portability of the hardware. I think AR could become important much, much faster than you're projecting if implemented for vehicles. What if your windshield has a HUD or is a touch screen that normally plays the exterior camera feeds but can also layer data from the various reality circles and data feeds to which you subscribe?

And not only is GPS going to matter, but I expect RFID to continue to proliferate -- probably massively. Every car within your scan-range (even if that's only the ones adjacent to you) will have the equivalent of a Facebook page. What if every driver is using this tech to get somewhere and they tell an on-board system where they're going when they start and the vehicles form a mesh network sharing some abstraction of that info? Then, over every other car on the road is a dim arrow suggesting which direction they'll be heading so that you can make smarter lane-change decisions.

Does your understanding of AR *require* a pseudo-direct sensory interface?

Craig Perko said...

I was thinking about the idea of augmented reality applying not to your own senses, but to the senses of roots, cars, etc. However, I don't think that's as close as you think.

There are cars with simple HUDs - my grandfather had one - but there are a lot of problems with this. Most important is the fact that you're driving a car: adding in more distractions is ill-advised.

Another problem I have is that augmenting reality for cars is significantly less invasive: it would be hard to make games out of it, hard to do anything other than GPS and simple traffic advice.

I don't know... I personally don't see cars with significant AR any time soon.

Ellipsis said...

Driving is too purposeful to make cars a good avenue into AR, I think. In order to see its potential, you have to be able to explore a bit, and most people driving aren't interested in exploring at the same time.

You could have a screen that shows just video - what's on the other side of the screen and virtual objects, and I think that would work fine for the time being. I mean, crappy cellphone cameras can show you what they're being pointed at in real time, so a device specifically designed to do that should be able to do it with much better performance.

That said, being able to beam light directly into the user's eyes is the kind of thing I have in mind when I imagine the technology taking off.

Craig Perko said...

I can get along with a screen that shows video of the far side, but I think it's less than half as immersive: there's something about super-high-resolution many-distance real reality that adds, I think.

Still, I'd settle for it.

Actually, using this method, it may be possible to knock one together NOW out of a GPS, two electronic compasses, a camera, and a laptop.

Hmmm... that would be a fun project for someone who has the hardware. I'd write the code happily!

darkflame said...

"saying that it's more important than I've worked out."

Its simple;

A perfect AR device lets us replace any physical object we dont need to touch with a virtual one.
Thats hundreds, if not thousands, of human good replaced by one signal device.

More importantly, it can let anything look like anything else.

This is quite profound if you think about it.

What happens when a generation of humans will grow up with the ability to customise how they see the world?

Would not concepts like "style" and "fashion" become worthless?

Surely AR would lead to the complete victory of function over form?

Because while their will still be various styles people want...their will be no money attached to getting them.


Of course I speak about a "perfect" AR device. Such as the ones seen in "Denno Coil" (a realistic great anime set in the near very near future).

But it IS only the refinement of existing tech.
Smaller, lighter, better...but *existing* technology.

Thats why I think glass's are the way forward for now.
Unlike brain-implants or star-trek style eye-replacement, it can be done =today=, just crudely.

TOLED screen on a simple lens + GPS (/Galllio) + WiFiMax + GPU + Accelerators = AR specs.

Just needs to be put together!

Craig Perko said...

It's not quite that simple: your eyes can't handle focusing on something an inch away, then ten meters away, then an inch away...

Hell, remember the virtual boy? It's hard on your eyes even without a background to draw your attention.

I agree that, in the long run, all those things will matter... but, in the short run, they're not possible. Still, I think that AR is important, even in the short run.

isaac said...

Have you seen this?

It's certainly not a brilliant implementation of the GPS+gyroscope+camera+screen, but it's getting there, in terms of having a usable AR device.

Craig Perko said...

Wow, that looks almost exactly like what I was talking about. There are a lot of iffy bits about their technology, but essentially, exactly!

Thanks for the link!