Exploratory writing is fun and evil

When I'm first constructing a story, I have a general idea of where it will go, but the details are not initially nailed down. So I end up doing a lot of exploratory writing, where I improvise on the spur of the moment what the characters do, where they go, what kinds of things happen. This gives me ideas about how they can develop, what kinds of plot elements will occur, and so on.

The mistake I consistently make is to then assume that the exploratory elements generated by this process should remain in the final product, when what I really need to do is take the character progressions that I develop later, and apply them to the story elements to see if they make sense. If they don't, those elements need to be rewritten.

Last night I rewrote a chapter after realizing that the first half of it was totally unnecessary, and in fact nonsensical. Character K would never end up in location X; that would require character E to have made a decision that, in retrospect, makes no sense. (It seemed reasonable at the time, but I didn't spend more than five seconds coming up with it.) Now that I've done a bunch of other work, I have a much more thorough and developed understanding of the characters and story, and the plot element of K-at-X would never happen.

So I killed it. The last half of the chapter was rearranged and expanded a bit, but overall the chapter is now more than a thousand words shorter. Which is fine; there was quite a lot of irrelevant background detail and minor beats that really had no impact on K's story. Most of those beats were generated during the exploratory process, and so it makes sense that they just weren't important.

The lesson I'm hoping to internalize is that exploratory writing is fine -- it's fun to create scenes on the spur of the moment -- but it needs to be analyzed in detail at a later date, to determine if it actually makes sense. This can cause tectonic upheavals in the story, requiring a great deal of work, and to me this can be literally gut-wrenching.

(For a bit of fun detail: I first started work on this chapter on 2011 Aug 24. I finished the first draft of the chapter on Aug 29. From Nov 04 to Nov 07 I did an editing pass on it, which in retrospect was a waste of time; the core story elements had not been worked out properly as they have now.)