Penny Arcade Extra Credits did a recent piece on the concept of 'Hardboiled' elements in games. It used one of my newly favorite series of Max Payne as an example, highlighting the differences between the original two games and the latest installment produced by Rockstar.
I don't actually agree with a lot of the podcast, which starts with the premise Max Payne was self-aware and tongue-in-cheek. As a fan of the Max Payne series, I point out the first game was somewhat gonzo and bizarre but I'd argue the second game was a completely straight example of the genre.
Still, I understand where they're coming from. A lot of people mistake sex, drugs, violence, and nihilism for "maturity." Even more so, a lot of people mistake mature for better. There's nothing inherently better about the Wizard of Oz than Citizen Kane, even if the later has a more involving story.
During the mid-to-late nineties, there was a glut of comic books which attempted to subvert the traditional superhero ethos. I.e. superheroes don't kill, superheroes are role-models, and superheroes make the world a better place. These stories were mostly based on the trend set by the seminal graphic novels The Dark Knight Returns and Watchmen.
This period is called the Dark Age of Comics because it was of questionable artistic value. While The Dark Knight Returns and Watchmen had a lot of interesting things to say about a variety of subjects, a lot of the comics were just excuses to show people getting slaughtered in particularly violent ways. The Darkness, a character which I am fond of, was guilty of a lot of these excesses.
That doesn't mean dark and grim stories are bad. Quite the contrary, Film Noir is one of my favorite genres and you can tell a lot about my tastes by the staggering amount of love I feel for zombies. Darkness has every bit as big a place in fiction as the light. The Maltese Falcon wouldn't be nearly as good a movie if not for Sam Spade's ambivalent approach to morality.
I think, instead, it's important for writers to make sure there's a purpose to their story's darkness. Max Payne is a tale about a police officer whose wife and child are heinously murdered, which leaves the titular character with the choice of meandering along or going out in the blaze of glory by killing every mobster in New York.
The story is driven by Max Payne's suicidal desire to join his wife. Max, himself, isn't all that likable of a protagonist but we sympathize with his pain. If Max was simply a guy who decided, to kill a bunch of criminals just because then the story loses much of its value. It becomes an exercise in empty spectacle.
This relates to Anti-Heroes, Crapsack World, Villain Protagonists, and Downer Endings. These are three of the major elements of making a story grimmer and grittier. If you're unfamiliar with any of these terms, I direct you to Television Tropes.org, one of my favorite website. For those familiar with them, they're basically subversion of the typical heroic narrative.
Of the three, antiheroes are the easiest to sell to audiences. Everyone likes to read about flawed heroes and there's a certain illicit thrill to characters who live on the edge. My favorite character in urban fantasy is Harry Dresden who is a very heroic guy cursed with guilt over killing his stepfather. The Darkness is about mafia hitman who lost his true love due to the mistakes he made.
A Crapsack World is pretty self-explanatory. They're worlds where it seems the entirety of the planet is either destroyed or hopelessly corrupt. Some of the most interesting stories of good vs. evil can be told in environments where the later is all-pervasive. One of the reasons I think Post-Apocalypse stories are inherently hopeful is, usually, things can't get any worse.
Villain Protagonists are harder to sell. Iago, Richard the Third, the Lannisters, and Michael Corleone are all bad people. They can be sympathetic, Michael even convincing the audience he's an anti-hero at various points, but ultimately do more harm to the world than good. To sell them to the audience, they need to be the heroes of their own narrative.
Downer Endings are something that need to be carefully managed to avoid not ruining the reader's enjoyment. Chinatown wouldn't be half the movie it is if not for its infamous conclusion. 1984 is brilliant because Winston can't maintain his convictions in the face of Big Brother. However, some book endings have caused me to turn off the television in disgust.
For Downer Endings to work, I believe events have to follow themselves to a logical conclusion. They're forgivable in horror movies because, nine times out of ten, it's believable for the monsters to get the better of the heroes. Even so, usually audiences want a small ray of hope like a lone survivor or the creature getting put down for a time. Still, sometimes it's best for an ending which is bleak and uncompromising.
I think the best occasion for a Downer Ending is when the ending is making a statement about something. A book about nuclear warfare which ends with everyone dying in the end is bleak but the point is that atomic weapons are bad. A book about corruption where nothing is able to be changed is powerful only when there's something to be said about the system. When monsters kill the heroes, it should be because the heroes are flawed or the monster itself is the star.
Grim and Gritty definitely has a place in storytelling. However, it's important to keep a perspective on the matter. I believe the value of darkness is directly proportional to the value it places in the story. I am against it only when it is done for its own sake and think that's when it becomes obvious and usually pretty tasteless.
There's other ways of making stories darker and edgier. You can throw in copious amounts of violence, nihilism, swearing, or sex but none of these things makes a story inherently better. Which is more memorable, anyway, Obi Wan Kenobi allowing himself to die at Darth Vader's hands or random hooker 24# getting killed in a police procedural?
Stephen King said swearing shouldn't be avoided in novels when it is appropriate for the characters to do so (especially when confronting something horrible or terrifying). Likewise, I think swearing for the sake of swearing, is equally ridiculous. I think this applies to all stories and writers should bear this in mind.
Find out what you're story is saying and apply an appropriate amount of darkness and/or light to it in order to make it work for you.