I realized that a lot of the things I have really negative reactions to, I hate because they are bad scifi. And, sure, there's a lot of bad scifi out there... but the ones I really hate are the ones that everyone else accepts blithely. Obvious examples include Bioshock Infinite, Serial Experiments Lain, Prometheus, and so on.
I realized that the issue is simply that I've read waaaaaaaaay more science fiction than most anybody else, and it's turned me into a snob. I've read literally thousands of science fiction stories. It was easy: back in Ye Olde Day, the best scifi was in short story and novelette form. You could read two or three a day just on the bus.
Because of that, I've become a bit of a scifi snob. So... I can't claim my opinions are "the only way to make good sci fi" or whatever, but I can say that most of the science fiction most people consume is trashy action flick scifi.
When you're done reading/watching/consuming good scifi, you can still feel its fingers around your neck for hours.
That's the kind of scifi most people haven't seen.
One thing to remember is that scifi appeals to different people at different times. There are good pieces of science fiction that do not appeal to me. And that's the problem with good scifi. It hits very hard against people who are in the right place. Trashy blockbuster scifi is much easier. It hits a much wider range of people.
But nobody is left stunned at the end. Nobody can feel it gripping their neck.
Things like Lain and Infinite fall very short as scifi goes... but people have been left gasping. The reason is because they have seen so little of substance that they have nothing to compare to. If the only photograph you've ever seen is a grainy, unlit photo of a penny, you're going to think it's amazing. That doesn't make it a good photograph.
Let's talk about the weaknesses of Lain, Infinite, and Prometheus, at least as I perceive them.
Secrets, mysteries, and gaps play a big role in science fiction. All three of our examples fail at this, misusing secrets. They use secrets like Lost uses secrets.
Let's talk about secrets.
One of the most fundamental practices in science fiction is to leave huge gaps unexplained in order to create large spaces.
For example, "Khan Noonien Singh was a general in the Eugenics Wars..." That's a powerful statement. It creates a massive block of "space" - these unexplained Eugenics Wars are interesting, and their name is extremely powerful. They are tied into the story, but left unexplained.
On a different scale, "Romulan blue ale". It serves no real purpose to the story, but it creates a vast expanse in your mind: the Romulans have a culinary culture and it's interesting and weird! A tiny detail that serves to create a world without actually cementing anything down.
It also goes the other direction. The internal life of characters is often left implied. Science Fiction rarely spells out motivations and emotions as clearly as other genres do. You're left with people who act a little inexplicably at times, stare a bit too long, covet something a bit too much. This creates a space inside their life, a space the reader can sense but not see. An easy example of this is Obi Wan Kenobi. Why does he live in the desert? Why does he let Darth kill him? How does he have all of these powers? What the hell are these powers?
Part of the draw of the original movies was that they doled out most of these answers in a trickle, so you kept watching to see more of them. But many of them were never really filled in until decades later, and fan theories were everywhere. Alternate explanations exist for virtually everything - for example, some people have proposed that the droids are actually hugely important 'shadow leaders' of the rebellion... and there are enough gaps that this explanation can weave through them fairly convincingly.
Secrets, mysteries, and gaps create life in these universes. They do so in a very powerful way because they let the audience fill those gaps at their leisure, as they see fit.
I think I'm understating that last bit, so let me harp on it:
The audience fills the gaps with whatever they can.
This is the primary tool used to make science fiction accessible to a wider range of people. When scifi has few gaps, the only people it can appeal to are the people who are in tune with the story as it is being told. If every aspect of the Eugenics Wars is explained, then only people who can accept the specific details of why the war happened and how it developed can be interested. Everyone else will say "oh, that's not very interesting" or "oh, that's not very likely".
By leaving it a mystery, each reader can presume it happened in a specific way according to their current... level? Place? The current way their brain works. The more they think about it on their own, the more it falls in tune with their own thinking, and the more interesting it becomes. Used wisely, this trick makes the best scifi twists rip you apart: they gave you space to imagine and assume. Then they kick that out from under you and show you just how feeble your imaginings were.
Now, this isn't the only thing scifi does. A lot of people would say that Star Trek and Star Wars aren't even proper scifi, just fantasy dolled up with technotrappings. Well, leaving that argument aside, this particular practice of creating gaps is commonplace and very powerful. The practice of subverting what the audience puts in those gaps is also a common technique, mostly used when you want to create "mindfuck" scifi.
Back to Lain, Prometheus, and Infinite. There will be spoilers, but who the hell cares?
All three of these attempt to use that last technique - the "create a gap, let the audience imagine what's inside, then subvert that" method.
Unfortunately, they do it clumsily.
First off, what's in the gap is rarely a mystery. They attempt to obfuscate something relatively simple, but burying something simple in a fog doesn't make it complicated. The only way that can work is if the audience is so inexperienced that they can't recognize the simple shape of a stumbling transhumanist singularity (Lain), interconnected multiverse anchor (Infinite), or villainous ancients (Prometheus). These stories are SUPER COMMON. I've read dozens of each.
Secondly, rather than create a legitimate gap, they simply drop fog everywhere to obfuscate it. For example, if I say the phrase "scie_ce F_ction", there's not much mystery as to what is underneath those underscores. Fogging them up doesn't make my writing more interesting, it just makes me annoying. And, yeah, I could reveal that it's actually "science FACTION", but it's like, whatever, okay, there were a few options and you chose one of the slightly less common ones. It's not interesting.
Compare this to Satoshi Kon's work, such as Paranoia Agent. Or the anime that blew my mind as a kid, Armitage the Third.
These create legitimate gaps. Paranoia Agent creates gaps by being batshit, leaving you grasping at straws in a maelstrom. It reveals the pattern near the end, just slowly enough that you can start to understand how it all works, and then it rushes ahead to extrapolate what that would actually mean. This overwhelming and powerful technique works because the idea is not common, and it's revealed fast enough that you aren't given time to extrapolate.
Armitage the Third seems like a typical robots and androids anime - they were common at the time. And you enjoy it on that level, but something feels just a touch off. The reveal might not surprise anyone who's seen a lot of scifi, but at the time it was not something that was written about in American science fiction. It really surprised me and left me thoughtful.
You can also talk about how these are also about real-world cultural issues. Paranoia Agent is arguably about the self-destructive hollowness of modern society, and Armitage is about racial purity. Those things matter. They matter a lot. Not only do they tie the story to your life, they also tie the story to the lives of the people in the story. They give incredible power to the characters, because the characters are moving through the same space. They give power to the scenes, because those scenes have happened in the real world.
The fact that Armitage is about racial purity is what gives many of its scenes so much more power than other campy blockbusters with the same basic premise. The sight of burning robot bodies piled high is devastating because it's not about robots.
This is something that only becomes clear near the end - same with Paranoia Agent. You could even say that the twist of these science fiction pieces is simply that they are ABOUT SOMETHING.
Lain, Prometheus, and Infinite not only have dull reveals, they aren't about anything. They just waddle through what someone thought was a cool scifi premise without breathing life into anything.
I really like science fiction.
But if it doesn't grab your neck and leave you breathless for hours afterwards, don't accept it. It's not good enough.