An adventure RPG is interesting not because of the combat, but because of everything else. Combat is very important, and in many cases it is the cornerstone of the game... but the real joy of those games tends to be the adventure, not the individual fights. I'm always interested in getting a different feel out of my games, so I always wondered: can you make a game that focuses almost entirely on the fighting? A game that feels more like pro wrestling, or an old kung-fu movie, or a shonen fighting manga.
The core "problem" with this approach is that the fighting is highlighted to the extent that the adventure part suffers. Unlike an adventure RPG, the noncombat sections aren't really adventurous. It's rare for them to be about discovering what's over the next hill, or finding a hidden treasure, or saving a village. Instead, they use a variety of soap-opera interpersonal tactics combined with high stakes and emotional training sequences.
My question is: can you put that into your tabletop RPG instead of the adventure hooks?
One core piece is the fighting mechanics themselves. You can't use the same mechanistic approach other RPGs use, because the fights serve a much larger role. In a normal RPG, the fights are obstacles, and the character development/role playing usually happens outside of combat, during an adventure sequence. But in a combat game, much of the role playing and representation need to happen during the fight. Inside the fight.
Similarly, in an adventure RPG fighting is an obstacle is to wear the player characters down. Fights serve other purposes, too - pacing, foreshadowing, drama, plenty of other purposes... but the core mechanics are about attrition. It's pretty rare for combat to actually be life or death in an adventure RPG: most encounters are going to go the players' way, it's just a matter of how many resources they burn. And, of course, between combat they will often take actions to reduce the difficulty of the next fight - bar the doors, burn down the fortress, cast a sleep spell, whatever.
Fighting games can't do that. Or, rather, they have to do it DURING combat, somehow.
So the combat needs to be radically expanded.
The fundamental change is that combat needs to have pacing changes similar to the pacing changes you'd get from a dungeon crawl. Clashes followed by short reprieves. Times when you dominate, times when they dominate. Moments when you might be able to steal a few seconds to recover the feeling in your stinging fists - or would it be better to rush in so he doesn't have time to catch his breath?
There are a lot of possible mechanics to allow for that kind of clashing, but because simple is generally best, I think I would use a very basic "clash and away" system. Every clash lasts a certain number of rounds, then you get an "away" period where you recover, tag out, shuffle for position, or even just cut the away time short to try and get an edge in the first round of the clash.
The damage system would need to be adapted to have all the complexity of the normal RPG's full attrition system. That is, things like consumable items, powers, mana points, and health all need to be rolled into the combat system. Of course, they don't have to have the same justification or color - we just need the statistical backbone. So you would use up some resources during the clash, and try to damage the enemy not just in terms of health, but trying to deplete his resources or foil his resource-consuming attacks. It needs to be a battle of attrition, because that's the actual play of an RPG, and we're cramming it all into the fight.
There's lots of rules solutions for that, and it's largely a matter of what your fighting is. If you're doing a boxing game, you might use things like stamina, dizziness, leg-intactness... if you were doing a mystical kung-fu game, you might have things like chi and broken bones and righteousness. Of course, adding in the teamwork capabilities can make it even more complex.
However, all of that work only gets us halfway to our goal, because we still don't give the players any opportunity to role play. It's just a statistical game right now.
That's the secret behind an adventure RPG, and why they work so well. They pretend to give you all these rules, but in reality what they give you is a distraction to drive you to role play. The rules are really just there as an impetus to allow you to exist in the world. Similarly, the role play is often expressed not in theatrical monologues, but in the ordinary decisions you make between fights. To run or stick. To infiltrate or invade. To plan or wing it. To spend or hoard. Much of the role playing happens without the players even realizing they are role playing. Being "in character" is nice, sure, but there's a lot of meat to the actions you take even when you are not talking as your character.
If you look at a fighting manga or anime, you can see that there is a lot of opportunity for role play. The characters are all quite expressive. However, much of the focus is on their internal activities. Their emotions, basically. This is the opposite of an adventure RPG, where the expression is represented via interactions with the world. In the end, a combat manga's fighter character throws a punch. This isn't a different interaction than the thousands of punches he's thrown before. But it is very different because of the expressiveness and dedication that the fighter showed in the panels beforehand.
To reflect this, it's going to be necessary to make a world that the players can express themselves in. The world probably can't be the fight proper. While they can certainly express themselves in how they fight, there's not enough flexibility to match the variety of things players can freely do while on an adventure. We have to create a world where the players can freely do that level of expressive play.
Basically, we need to make the PCs' emotions into a world.
I don't mean literally. I simply mean that the team's emotional environment needs to be something that the players can interact with freely.
My thinking is a relatively simple token-based system is probably the answer. For example, you might have "friendship tokens" or similar which you can give out as you prefer. If an ally is punched by Master Wong the Iron Fist, you gain as much chi as the tokens held by that ally. You can rearrange your tokens, but not instantly - it's a slow process both in and out of combat.
However, that's still a bit too basic. There needs to be more freedom to create "engines" and "judgments", neither of which are really supported by that simple token system.
So another rule I recommend is the "double or nothing" rule. When you give someone a friendship token, it comes with a due date. If they don't do something in-game to validate your friendship by that time, both you and they lose a friendship token, representing a widening distance and feeling of alienation. But if they do validate it, both of you freely gain another token of the other person's, adding to their token total and making everything more potent.
Each token you add comes with this kind of due date, and you can't add another token until the previous one's been resolved (validated or canceled). However, each token you add has stricter and stricter validation requirements. The first token can be validated with a slight smile. But the seventh token requires a truly epic moment. Moreover, as you get into the 3-token range, you have to start to cement the kind of relationship you two have, and if the validations are of the wrong relationship category, there's no growth (but also no penalty). You can only have one of each kind of relationship...
That probably works well enough outside of combat, but it doesn't allow for in-combat shenanigans. For that, I would say that each player can use the friendship tokens on an ally to judge that ally's enemy, creating "enemy" tokens on them which will give them a bonus against that enemy. Like friendship tokens, enemy tokens react to resolution and are of a specific relationship type. Unlike friendship tokens, an enemy's "resolution" will cause continuous growth rather than a one-shot growth. If you hate an enemy's cheating sleaziness and put an enemy token on him, then every time he cheats that enemy token gets added again.
This doesn't generally get out of control, because enemy tokens must be spent (removed from play) in order to do special moves. Friendship tokens are similar for healing/support moves, although the expense is temporary and they recover at the end of combat.
I think this might be enough... but there is one more thing to discuss. Noncombat.
Between combats, combat manga and anime follow a very specific kind of formula. If you can cement this formula into a playable system, you can probably tap the players' internalized responses in the same way that encountering goblins taps their adventure instincts.
Intercharacter hijinks are very common in action manga, establishing relationships between the characters and also helping to establish the rules of the world. This is also a great way to allow player characters to have strong personality characteristics and express them freely. The nature of hijinks will vary depending on the setting - a high-school basketball game might have an arc about the school play, while a gritty monster-smashing brawler might have hijinks about the big eater having to rustle up some food. While some hijinks are simple one-offs, many of them are small, relatively unimportant arcs that hold together for long periods, giving the players a lot of stability in developing their characters and relationships.
There are a bunch of different kinds of training. Physical/mental training to improve stats, learning a secret move from a new master, suddenly getting insight as to a new technique from a hijink, researching a specific enemy to find a weakness, training the other team members, going on a training vacation, receiving a new secret weapon from engineering, training in secret... training is a good way to regulate how quickly the players can change their character, and also can serve to tie them very closely to the world if you are careful about it.
3) Casual Color
Casual color is often overlooked. This is the routine stuff. In a normal RPG, this would be deciding what inn to stay at, determining how many rations to take, trying to get a good price on the +1 sword you want to sell, mapping out a path to the next destination. You need to provide similar "bread" for this game. There are a lot of different options, but in essence most of them have personal condition at the heart of them. Being a top-flight fighter involves constant maintenance and tradeoffs for your body to keep it operating at peak... and those tradeoffs get even more complex if you want to change your style out.
4) Plot and Inspiration
Plot should be obvious, but it's a bit different in a fighting game. The overarching plot threads could be the same, but their presentation is completely different. In an adventure RPG, the plot is a reason for the players to care. In a fighting RPG, the plot is a reason for the characters to care. Therefore, plot points should be presented as inspiration to fight and train, which is a completely different focus than a standard RPG story.
Some plot elements are strongly interactive. For example, searching for a hidden master, choosing which of the four enemy kings to fight next, aligning yourself with a given faction. Things like following someone around at night could be considered plot, but are actually hijinks - a major difference between a fighting game and an adventure game.
However, many plot elements aren't very interactive, and serve solely to get the character to want to fight. Seeing your next enemy dominate someone you looked up to. Finding that monsters burned out the town. Someone getting sick and needing to earn money to pay for hospital care. A friend going rogue and turning to forbidden techniques... these are things you can't directly interact with. Instead, you train up such that when the moment for resolution comes, you're ready.
Anyway, those are my thoughts on fighting RPGs.