Thursday, February 23, 2012

Dr Who Villain Randomizer

So you need a Doctor Who villain? Here's a handy-dandy chart! Roll once for a monster-of-the-week, twice for a tense two-parter, and three times for an arch-villain! Add one or two more each subsequent appearance!

1: Has time travel powers that don't get used much.
2: Has godlike powers that don't get used much.
3: Can travel via extremely unusual means (over televisions, through minds, etc), but doesn't use it much
4: Is a large group (5-50)
5: Is a horde (uncountable quantities)
6: Is physically monstrous (insectoid, usually)
7: Is physically implacable (robotic, usually)
8: Is not solid
9: Is actually the fever dream of (or enabled by) a relatively ordinary person
10: Challenged the time lords once upon a time
11: Owns large numbers of planets somewhere
12: Is heavily armed (and threatening people, probably)
13: Is integrated into the landscape/space station/etc
14: Has weird magical powers, or technology that is effectively magic, but doesn't really understand it very well.
15: Has built another villain (roll again for their subordinate)
16: Spawns more of itself, usually via harming or crippling innocent people
17: Eats something impossible, such as time, faces, emotions
18: Is only a villain because something went haywire. Roll again.
19: Has no emotions, roll again for another power
20: The king or president or whatever of the planet.

For example, the Daleks are 19, 7, 10, and random others depending on the episode. The weeping angels are 18, 1, 4, and later are upgraded with assorted others such as 3, 17, 10, and so on.

Wednesday, February 22, 2012

Big Data = Big Abuses

I've had a rough time defending myself on G+: I think that targeted ads are bad, except when they are explicitly asked for. I also think that corporations need to be shackled, legally, from tracking their customers.

These are pretty severe opinions, and I caught a lot of flak from people who disagree. Which is fine. The arguments can get heated, but it's not like a political argument: there are points being made and disagreed with, rather than opinions simply being stuck to. However, since my opinion is apparently pretty far from the norm, it needs a fairly large amount of defense.

Let me start by establishing that big data != small data.

The first line of defense against my opinions is that companies have had all this data forever, they're just using it more effectively now, and that's fine.

That doesn't hold water for me. Having something you can't use and having something you can use are simply different categories. If you have an old musket your great-great-grandaddy used in the revolutionary war and your neighbor has an AK47 they use to shoot crows, are you going to argue those are the same situation?

Whether you are for or against gun control, you need to acknowledge that those are different categories of gun ownership and use.

It's the same with data: having a giant stack of receipts you use to file taxes as opposed to having a giant database you use to predict exactly what each individual customer, by name, will buy this week... those are fundamentally different. Whether you are for or against the right to do such a thing, you still have to acknowledge that they are fundamentally different categories of data possession and use.

Now let me argue that big data = big abuses.

Our current economic woes come from big data.

Let's start with the sub-prime mortgage crisis. This was a situation only made possible because of big data. The massive amount of storage, networking, and communication required to analyze, package, resell, and distribute mortgages is fundamentally big data.

If they had to shuffle actual papers around, it would have been impossible to bundle and resell these mortgages in quantity. Because of this, it would have been impossible to shift and seemingly diminish risk, and fewer such mortgages would have been approved. The crisis may have still happened to some extent, but it wouldn't have been a bubble popping, just a pinprick on ordinary economic skin.

It's probably still a tender topic, so I imagine some readers may take offense to what I just wrote. Instead of spending a day on that one topic, I'll just move on to other topics with the same fundamental situation: big data causing big abuses.

In my opinion, the largest big data abuse is algorithmic trading, specifically high-frenquency microsecond transactions. Algorithmic trading has played a role in, and arguably been the cause of, several of the more recent stock market disasters.

The trading of stocks to take advantage of moment-to-moment price fluctuations is the way to make money on the market, as you'll see at those Wikipedia links. There is no law against it, and people have become great successes because of it. "Quants" are mathematicians turned billionaires.

So, why would you hate that? Why would you hate success? Are you some kind of horrible success-hating freak, you freak?

My reason is simple: it is success at the price of the market. "Zero sum" success. Their success did not come from making the world or the market better. It comes at the price of making it worse. Abusing it. On the best of days, it's a tax on people who are trading for real reasons related to the actual performance of the company in question. On the worst of days, it crashes the market entirely.

The argument can be made that this is effectively stealing. If a bully takes a geek's lunch money, we don't call that success. We call it stealing, even if all we see is the bully making a gesture and the long-suffering geek timidly handing over cash. Wow, sure looks voluntary, why are you complaining about it?

We can argue that what the bully does is illegal and what the quants do is legal, but let's leave aside the question of legal and talk about the question of benefit. The money HFT nets you... where does it come from? It comes from people who are participating in the market in the way the market was intended to work. It comes out of their pocket, without their permission. (It may also come from other HFT, but for the sake of argument, we'll consider that a closed pool of "HFT cash".)

How does that benefit the market?

It doesn't. Actual participants are made poorer for the sake of some people who have found a nice loophole. The loophole of big data.

With enough data, they can figure out what trades will happen when, at what prices, and hop on top of that. They can always be on top, skimming the cream off the milk everyone else is buying.

I hope that my point is clear: I'm considering everything from the perspective of a healthy market and economy. Rather than just hailing anyone with cash as a "success", I only hail those that actually improve things as a success. You can make as much money as you want, as long as you do so by offering better products or services. Otherwise, you're a burden on humanity and I want you gone.

I've used financial examples for big data, because that's the industry that's had big data the longest. However, there are other examples, such as the RIAA's persecution of its users based on data obtained from sifting through endless thousands of torrents and data transmission records. Once again, big data allowing for a big abuse, although the big abuse in this case was less devastating than crashing the world economy.

You can argue that not all big data automatically results in a big abuse. That's true: big data can have a positive result, especially in medical data, where it can lead to fairly serious breakthroughs as we discover the links between similar and dissimilar cases.

But the point is that you can't simply ignore big data. If there is data, it can be mined. For example, here is Target mining data. The data they are mining? Strictly the things you've purchased from them recently. Surely you can't fault them that! It's barely even "big data"!

That's right. With what I would consider the smallest, sparest kind of big data, Target has figured out how to tell when your baby is due - without you even buying pregnancy-related goods from them, or telling them you're pregnant. This isn't proof that Target is evil, or even an example of evil big data. It is an example of the sorts of things you can tell from even a relatively small amount of data.

We can't treat data as if it is unimportant. We need to consider any data to be a danger, just like any gun is a danger, just like any moving vehicle is a danger, any poison is a danger. In any given case, the data may be secured and well-behaved, just as in most cases a gun, car, and jar of rat poison is secure and well-behaved. But, on the other hand, we consider it illegal to wave a gun around while asking for more money.

Now, big data cannot be opted out of.

A lot of people want to "vote with your feet". If you don't like big data being used against you, stick to suppliers that don't use big data against you!

There's a lot of problems with that. The first is that it relies on you to know which places are going to use big data against you. Many of the big data applications are subtle enough that you won't notice.

The second is that opting out usually doesn't accomplish anything. For example, I don't use Facebook, but they still track me. Moreover, the fact that everyone I know uses Facebook extends their reach to me even if I don't use them - games released for Facebook only, posts only available to Facebook members, and so on. It is the same with Google, Steam, Microsoft, Apple - opting out of their services does nothing to actually stop them from affecting you.

The third problem with opting out is that you really can't. Big data is everywhere. What cell phone provider will you use that doesn't abuse their data? What internet provider? There are no innocent parties. The corporate culture doesn't allow for it.

You can argue that when abuses get bad enough, enough people will vote with their feet. I would say that's unlikely. Classically, corporations have been extremely good about maintaining a grip on their abused customers. Humans can get used to anything - for example, we've gotten used to high-frequency trading.

"Then what's the problem, if people aren't upset?"

... well, go back and read the essay again. The problem is that big data causes big abuses. What you don't fear can still hurt you. We weren't afraid of HFT, or sub-prime mortgages, or music distributors. They still caused us a lot of problems - even if we weren't even participating!


Oh, right, targeted ads.

I think that any targeted ads you didn't ask for are just like high frequency trading. It's a situation where the company is data mining to figure out the sort of things you might be about to do, and then pressuring you to do them in its favor. You were about to do them anyway, the market was about to function fine, this is just them stepping in to take their cut because they know you.

I won't argue that it's evil, but I will argue that it is anticompetitive. It is very difficult for a newcomer to compete with that. It'll get more difficult as established megacorporations band together to share your data, forming pan-industry teams to lock down their dominance.

There are some situations where it could theoretically benefit you - for example, if they are having a sale on something and you happen to want that something, you could get it at a lower price. However, if they are truly someplace you want to shop, you should be giving them explicit permission to send you notifications on that sort of sale.

Otherwise, it's them using their market share to pressure you to spend more and to wall out their competition. I consider that bad for the market and bad for the consumer. I consider it almost identical to HFT: taking advantage of someone participating in a useful (but predictable) manner.

It's not at HFT scale yet, but it's the same fundamental kind of thing, to me.

Sunday, February 12, 2012

Webcam Societies

There has been an increasing number of people concerned about the "filter bubble" effect. In essence, they worry that people will only experience the internet they want (or are thought to want), and therefore will become insulated away in their own little world of... whatever it is they are interested in.

At first glance, this seems like a bad thing. Exposure to different thoughts, cultures, and people makes you a wiser, more interesting person, right?

Well, what if we turned it on its head?

Let's come at this from another angle. Let's say that we really want the future, now. We want our awesome robots, for starters. How can we foster a world where robots are more common?

Robots are in a weird spot. People are waiting for corporations to build better robots, and corporations are waiting for people to care enough to want them. But the technology required for robots exists, we're just not moving with it.

Plenty of people build robots in their spare time. And plenty more would build robots, or at least participate in the culture that builds robots, if they were immersed in it.

Immersion is a factor most people don't really think of very much. But immersion is what makes it possible for hundreds of thousands of teenagers to obsess over some pointless pop star, immersion is what allows artists to hone their craft day after day, immersion is what makes people believe Google is less evil than Apple, or visa-versa. Hell, immersion is why you want to make a game similar to the game you played last night.

It's true that some people perform consistently without immersion. There are obsessive people who, even without being constantly surrounded by pressure, will do what they want to do. I would argue that they actually immerse themselves, rather than being immersed by the world around them, but either way they are exceptions.

People who are less obsessed aren't less valuable or less interesting. They have just as much potential. They just aren't as focused if left on their own.

For those people, don't you think a "webcam society" might work? A kind of "filter bubble extraordinaire" where they are constantly exposed to the experiences of people who are working on something specific, something with a specific goal?

If you woke up in the morning and your computer screen was filled with videos of a group from China that built a robot to do housework, wouldn't you be more inclined to want to participate? Even if you're not a robotics engineer, you can still get excited about it. Even if you can't understand the language, the drive is contagious.

What if we consider "filter bubble" from the opposite angle: what if we could immerse ourselves in a bubble of our own choosing, to drive us to work harder and with more endurance at something we want to get done? It'd also be handy as it would allow us to get feedback from people who are also legitimately interested in the same thing (as opposed to random internet goons who are more likely to say "that engineer chick's hot" rather than "the robot can wash clothes!")

By forming the filter barrier around the group doing the same thing, you increase the concentration of that thing, you get bombarded more by that thing, and detritus that would normally wash by is kept out by the filter. It's not some secret society - it's just a kind of wiki/forum where you have to sign up to comment.

This same technique has been used since pretty much the beginning of human existence. Religions and cultures have invented many flesh-world techniques for isolating, concentrating, and increasing immersion. They have also been at the forefront of inventing on-line versions of the same, although perhaps they have been outpaced by the finance obsessives of the world.

Either way, it seems like it is an interesting practice to adopt for more general interests. It could be adapted for anything - I used robots as an example, but literally any interest, from boating to funding third world farmers, could use the same practice.

Unfortunately, it's not just a matter of finding a forum and signing in. There are a few additional details that it needs to have implemented in order for it to work well.

The primary problem is that a standard form of internet interaction is too low-concentration. You need to radically increase the level of pressure, both in terms of time and power.

As I mentioned before, a "webcam society" might be a viable version. This is a group of people who use video to interact with the society. Watching people try out their robot, or seeing someone's face as they describe a plan for how to refit their boat, it's a much more powerful experience than just reading a post somewhere.

By trying to make everything audiovisual, you radically increase the power of the pressure. You can increase the timing of the pressure by delivering these videos more aggressively. Google's new G+, for example, is an always-on wall of pressure scrolling by. Even when you aren't using it, if you're on any Google service, you're getting hit. Something very similar could work for a webcam society. In fact, you can get pretty close if you created a very this-is-my-video-centric circle on such a social network.

The last thing to keep in mind is that the human mind only has so much space for obsessions. The filter is as important for what it keeps out as what it lets in. My interests are widespread, and therefore so is my effort. If you participate in a webcam society, it's probably the only one you'll be able to participate in, because otherwise you'll get distracted.

Tuesday, February 07, 2012

NPC State Density

I was designing some village-generation algorithms when I realized something that I don't think I've ever seen explained, although it's quite obvious in retrospect:

The complexity of your NPCs is limited by the state density of your NPCs.

As you may be aware, having loads of anonymous clones as your NPCs limits the appeal of any given NPC. Normally, they won't even have much unique content programmed into them. Even if they are shopkeepers or whatever, they will normally act to add only a tiny modicum of flavor to the world. On the other hand, having unique NPCs results in each of them typically having a lot of character and a lot of interesting things to do and want and say.

The example of this I tend to come back to is Beyond Good and Evil. While it had a number of nameless, pointless NPCs, it also had several extremely minor NPCs that stood out and were interesting despite not having any real complexity or adding anything to the game.

The obvious example is the catgirl in the resistance, who has I believe two lines, neither of which is interesting. But people still remember her. I also remember the two racers in the Akuda bar who didn't like you any. And Peepers, and the kids at the lighthouse, and the rhinos at the garage - these are NPCs which literally had as much dialog and story influence as random NPCs from open world games like Oblivion. That is to say, not very much. But because they were unique, they were a whole lot more memorable.

That is the part everyone realizes. Clone NPCs are boring, no matter how you write them. Unique NPCs are interesting, no matter how you write them.

It's important to realize that clone NPCs aren't just boring because they're written boring: it's actually really hard to remember them. If a wife tells you she's sick of her no-good husband always going to the bar, you're probably going to forget she exists by the time you get to the bar!

That's the basic, the obvious, the thing everyone knows.

What I discovered while making my village generators is that it's not simple uniqueness that makes an NPC able to hold more complicated relationships. It's more - something I call "state density".

State density is not just how an NPC looks, but all memorable aspects of them. For example, if you have a kind of dating sim game where the locations are poorly defined background images, those characters aren't going to have the same hooks into your memory as if they are in a Mario-like game and you're actually walking into their house and talking to them in their physical house. If the house's contents are unique or interesting, that's also going to make them more memorable.

Another aspect of state density is how frequently they come up. For example, an NPC who stays in a back corner of a house that you rarely visit is probably going to be forgotten. One that is always hanging out on the corner of main street will be more memorable. One that contacts you by radio every five seconds to tell you that you've entered a new region or found a new item is going to be extremely memorable, although perhaps not in a good way.

NPCs that can be altered also have an advantage. If you can move an NPC to any quarters you wish, they will be more memorable. If you can force them to dress up in various stupid Halloween costumes, they will be more memorable.

There is, of course, a "maximum active NPC count" that a player will be able to remember. However, I don't believe this is a big problem: if you want to use loads of NPCs, just arrange them in clusters so that the player doesn't run into a lot of NPCs when he's busily thinking of the ones he's just now talking to.

Anyway, just thoughts on NPCs.

Friday, February 03, 2012

World of Creating

I've been thinking about people creating things.

The first thing I thought about was that 3D printers might not have a "killer app". They may become popular for the same reason Wikipedia is popular. You don't say "what are the things I can look up that make Wikipedia worthwhile", you say "this is interesting, I'll look it up!" It's a culture of looking things up.

3D printers may be the same way. Not "what can I make that makes this worthwhile", but "this is interesting, I'll make it!" A culture of making things.

After that, I wandered back into my love of augmented reality. With the advent of more and more tech to immerse people into digital realities, we are seeing augmented reality slooooowly unfurl. But there's no "killer app". "What awesome thing can you do in AR that makes it worth wearing clumsy glasses all the time?"

Well, maybe that's the wrong approach. Instead, we might think "this is interesting! I'll create it!"

Imagine a culture where everyone can create digital and real goods as easily as they look something up on Wikipedia.

Let's say you need to keep the sunshine off your kitchen counter's candy stock. Instead of building one out of coat hangers and paper, instead of buying one, you create one yourself.

How do you do that?

Well, you "mold" it with your hands, right there on the counter. Create a plane by squishing your hands together, then grab the edge and pull it forward to create a support, automatically mirrored to the other side. Paint it and dust it with colors. Then hit print.

In your basement, the plastic sun cover gets printed out in the next hour or so. You can go down and grab it yourself, or maybe there's a house robot that will grab it and put it where you initially sculpted it.

I give this not as an example of a killer app. I give this as an example of how you might do something day to day. In the same way that someone giving an example of looking up "soy" on Wikipedia is not giving you the "killer lookup" - they are just giving you an example of a world at your fingertips.

The ability to create and mold digital objects in real space has a huge number of potential applications, all of which are a little hard to see, in the same way it was hard to see that Wikipedia was going to be the incredible resource it is.

Can you imagine getting up each morning and finding your house's ceiling is a different sky each day, depending on where your friends are and what they are doing? Can you imagine a world where your RSS/social network feeds are embodied as entities in real space, ones you can wave away or grab if you want more? A world where if you're curious about water filters, you build a few and play with them? A world you can orchestrate?

A world where creating something is often easier than looking up existing versions?