One of my favorite descriptions of story comes from the late Gary Provost, a prolific writer and creative writing teacher. Here it is from How to Tell a Story by Peter Rubie and Gary Provost:
Once upon a time, something happened to someone, and he decided that he would pursue a goal. So he devised a plan of action, and even though there were forces trying to stop him, he moved forward because there was a lot at stake. And just as things seemed as bad as they could get, he learned an important lesson, and when offered the prize he had sought so strenuously, he had to decide whether or not to take it, and in making that decision he satisfied a need that had been created by something in his past.
Isn't that great?
There's a lot that can be discussed in that paragraph, but today I want to focus on the hidden need.
Stories in general showcase a weakness or need in the protagonist. When the protagonist (usually) solves the hidden need, he is then able to achieve (or not) the story goal which is the external thing or outcome that he is pursuing. These two branches are cleverly called the Inner Story and the Outer Story. Dominance of one or the other of these branches changes with genre: for example literary stories tend to be inner or psychologically driven; aggressive action stories are externally driven; but the best stories balance these two elements.
The protagonist's hidden need comes from a weakness within the protagonist: for example he can't trust, or he spends too much money, or he can't forgive. The best hidden needs further reach out so that the protagonist hurts other people: for example, because he can't trust he is causing his girlfriend to break up and marry her old boyfriend; because he spends too much money, his mom who cosigned a loan for him will be kicked out of her house; because he can't forgive his friend, the friend becomes cynical and starts behaving badly.
The story goal is the external thing or outcome that the protagonist is pursuing. It can often, although not always, stem from the hidden need. For example, because the guy's girlfriend is leaving him, he's got to get her back; because his mom is losing her house he must find a way to pay her back and/or a new place for her to live; because he can't forgive his friend, he misses an opportunity for his friend to help him with a business he is forming. All of these might be worked into a main story goal, but at least could easily be important sub-parts of the main story.
In the story, the protagonist starts out solving problems by using his old way of thinking. I found that the hidden need was often addressed in three stages right after the midpoint:
1. the hidden need is, once more, clearly demonstrated
2. the hidden need is solved
3. a small problem is addressed using the protagonist's new way of thinking
For example, in the movie U571 (a classically structured, very exciting, World War II era submarine story), second-in-command Tyler's hidden need is that he is unable to take complete authority, since he values the people under him as individuals when he needs to lead. At the midpoint Tyler's captain is killed and Tyler takes command.
Hidden need is demonstrated: Tyler and his crew are in a dire situation, and he admits to them that he doesn't know how they're going to make it.
Hidden need is solved: the chief petty officer takes Tyler aside and tells him to never, ever, tell his crew again that he doesn't know. He must inspire complete confidence. He must lead.
A small problem is addressed: On deck, a Nazi plane flies over them. Tyler tells his crew to wave. One of his sailors tells the gunner to shoot. The gunner doesn't shoot, the plane flies past, and Tyler punches the sailor. "This isn't a democracy!" he growls.
You get the idea. The second part of act two is the most difficult part of the story to write, but this is a helpful device to use to take up at least some of that white space.
There is also often a second hidden need that is solved towards the end of the story, right before the climax. For example, in the film Iron Will (also a highly recommended film!), Will is in a dog race to win money that will pay off his mom's home and his college tuition (valuable stakes). His main hidden need -- fear of a bully/confidence -- has already been solved in act two part two, but he has a second fear, to take his sled across a frozen river. Throughout the movie he has avoided rivers, sometimes with disastrous consequences. However, he can only win this contest if he crosses the ice. So, will he?
17 hours ago