So we've now had a decade or so of really really close races. I mean, Bush vs. Gore came down to a SCOTUS decision. There have been tons of recounts over the past decade; races decided by the narrowest of margins; a constant back and forth for the parties that never deviates too far from balance, etc.
I think we can learn something from all of this recent history: that both of the parties are leaving a large chunk of swing voters unsatisfied. There's a large group of people who don't feel at home in either the Democratic or Republican camps, and until one of those parties changes to meet those swing voters we'll just keep bouncing back and forth.
I'm one of these swing voters. I'm speaking from my personal experience when I say neither party suits me. And I'm speaking from what I know of those around me. I know a lot of people who agree with me on this, that both parties do not adequately capture their positions, and that their failure to do so is massive. That is, Democrats fail on some really big issues and Republicans fail on some really big issues.
I would suggest that the problem comes in that fiscally liberal and socially conservative values tend to line up nicely together. They, generally speaking, reflect a Christian ethos of moral behavior and generous servitude.
Most of the Democrats I know are people for whom fiscal issues are more important than social issues. In their personal lives they live as highly moral and upright people. In their voting lives, they prioritize fiscally liberal issues that reflect their generous nature. That is, for example, they might happen to be pro-choice, but that's not really the reason they're voting Democrats. The reason they're voting Democrat is because they're pro-social justice and they believe in giving generously to the less fortunate and see that government can be a useful tool for accomplishing that result. On the other hand, most of the Republicans I know are people for whom social issues are more important than fiscal issues. In their personal lives they give generously of themselves to the less fortunate. In their voting lives, they prioritize social issues that reflect a desire to protect others from harm. That is, for example, they might happen to be pro-deregulated business, but that's not really the reason they're voting Republican. The reason they're voting Republican is because they have a philosophical belief in government's role of protecting vulnerable persons from harm (including, for example, the unborn, victims of crime, persons subject to tyranny in other countries, etc.).
These two things seem to really line up well in my mind, and in the minds of a large number of people I know. Maybe you'd disagree with me. But ask yourself these questions: if you are a Democrat faced with voting for a pro-life Democrat or a pro-choice Republican, (all other positions reflecting the party norms), which would you pick? If you are a Republican faced with voting for a pro-taxes Republican or an anti-taxes Democrat, (all other positions reflecting the party norms), which would you pick? I'm convinced that the answer, for most, would be to stick with their party. Most people voting Republican are doing it on social issues. Most people voting Democrat are doing it on fiscal issues. What's more though, the swing voters are motivated by these same issues (fiscally liberal, socially liberal), it's just that they shift in terms of which issue they're focusing on.
Someone, somewhere, has to get a party going where those two things line up.