Jeff made a post about difficulty in games. He says that the game should always be just barely beatable. Then he says that the game's difficulty should continuously increase. I assume he assumes that the players will be getting more skilled, so the difficulty increase should be just fast enough to pace their improvement.
I disagree, however. First, it very much depends on the game what kind of difficulty I'm looking for. RPGs are typically very easy. This doesn't make me dislike them: there are a bunch of optional goals that scale in difficulty (buying the cool sword, beating the optional boss, collecting all the widgets).
RPGs wouldn't be better if they were "just barely beatable", at least, not usually. This is because different players have different levels of skill, and what is barely beatable to one person is likely to be a cakewalk to another. You can have adaptive difficulty or some kind of difficulty setting, sure, but that's usually considered an inelegant solution for an RPG. Instead, you have optional things that the players can do if they want a challenge or want to lower the challenge by leveling up. In essense, you give the player direct control over the difficulty they face.
RPGs are not alone in this. Nearly every game allows you to spend extra time to lower the difficulty of the game. Even games like Halo or Quake do this: take a few moments to grab a weapon or a shield recharge before running into the fray. Or take a lot more moments and run all the way out to the rocket launcher, because you suck too badly to do anything useful with any other weapon.
This ability for any player to beat the game in his or her own way is very important, says I. More important than making it "just barely beatable". How much challenge you want to offer within that spectrum is, of course, up to you.
However, above and beyond this idea of letting players take different routes to change the difficulty is the idea of changing the necessary skills a player has to use.
Because players grow more skilled at very different rates and typically start with massive skill differences, it's extremely hard to "scale up" the difficulty to match their skill improvement. I doubt I've improved one iota over the last four or five FPS games I've played.
Instead what games often do - and I think we should see more of this - is cycle through skill variations. This level, you have a pistol. That level, you have a rocket launcher. That level, you have a disk-thrower. Each requires you to change your tactical methodology - to use your skill in a different way. This happens in virtually every kind of modern game, and for good reason. It gives the players, good or bad, a sudden kick of something new and interesting.
It's not that any given level is going to be more hard or less hard, necessarily. There might be more enemies, or harder enemies, but you'll usually have cooler equipment to take them down with. The levels are designed to be the same difficulty, but using different techniques.
If you simply increase the difficulty, a lot of players are going to find themselves totally hosed. A good example of this is Psychonauts. I waltzed through that game. Until the last level, which had challenges roughly three times harder than the earlier levels. I barely cleared it, after many hours of dying. That wasn't too bad for me - I enjoyed most of it - but my roomate simply quit playing because the levels were too difficult to even beat.
Up until that point, Psychonauts had been simply giving you new powers and letting you use them in interesting ways. The levels were getting "harder", but you were getting meaner at the same rate. For the last level, they continued to get harder, but you didn't get any meaner. That alienates players.
So, that's my essay on why a game doesn't have to continuously increase the difficulty level.