The congressional polls represent not just a Republican victory with a vengeance, but a profound crisis of confidence, according to Tom Switzer.

On November 4, 2014, the political pendulum in American politics swung away from the left. The midterm elections delivered the centre-right Republican Party a gain of between seven and nine seats to control the Senate for the first time in eight years, its largest House majority in more than 80 years and impressive governor victories in safe Democrat states.

True, these were merely congressional elections when the incumbent president’s party usually loses seats (think Ronald Reagan 1986, Bill Clinton 1994, George W. Bush 2006 or Barack Obama 2010). And voter turnout was low even by midterm standards. But make no mistake: 2014 delivered a sharp rebuke to Barack Obama. The result was the equivalent in a parliamentary system of a vote of no confidence in the President.

They also reflect America’s very serious crisis of confidence. All the available public opinion evidence shows that around three quarters of the American people believe their nation in heading in the wrong direction.

At one level, Americans should be upbeat. After all, since Obama came to power in 2009, unemployment has fallen from 10 per cent to less than 6 per cent. The federal budget deficit has shrunk to the lowest level as a share of the economy since 2007 – before the global financial crisis. And the economy could break the 3% annual growth rate for the first time in nearly a decade.

And yet President Obama’s national approval ratings are in the low-40s and in some crucial battleground states they’re in the low-30s. So unpopular is Obama that the President was largely absent from the campaign trail. Worse, Democrats were airing ads distancing themselves from his widely maligned presidency.

So where did it all go wrong? Why has the Obama phenomenon imploded with the force it has, just two years after the President’s impressive re-election?

Start with Obama record. Certainly there have been widespread – and justified – criticisms of the President’s ability to lead, persuade and influence, both at home and abroad. There also remains a widespread sense of economic angst under his watch. Most of the new jobs since 2009 are “McJobs”: low-paid and part-time work that don’t make people feel better off. The Obama recovery has been the most sluggish in generations. Meanwhile, inequality has widened dramatically and medium-household income has declined by 4.4%.

Add to this the botched rollout of his administration’s controversial health-care programs, the rise of the so-called Islamic State terrorists in Iraq and the arrival of Ebola virus in the US, and it is no wonder Americans were in no mood to re-elect incumbent politicians. As it happened, more Democratic senators were up for re-election than Republicans and most of the competitive races were in Republican-leaning states. That meant Democrats were bound to struggle.

None of this should downplay the GOP’s success. Republicans avoided mistakes that have plagued them in the past. For one thing, the party chose more sensible and mainstream candidates, whom the Democrats could not portray as extremists – as happened last time in previous (and winnable) congressional elections in Nevada, Indiana or Delaware where the Republicans nominated Tea-Party aligned candidates. For another thing, the voters went to the polls without fresh memories of the 2013 government shutdown that many blamed on the GOP.

And yet the Republican brand is damaged in Middle America. Like American conservatism itself, the GOP lacks a national leader, is riven by factionalism and displays a lack of intellectual self-consciousness and philosophical reflection.

Moreover, many seasoned observers of Washington politics believe that history and demography are on the Democrats’ side. The election-winning coalitions – women, minorities, young people – are widely believed to reflect an enduring reality of the new era of American politics.

So, if neither Obama nor the Republicans fully explains the midterm results, what does?

A more plausible explanation for the rapid electoral mood swings, epitomised in Obama’s fall from adulation to contempt, has to do with America’s crisis of confidence. It’s a cultural malaise that precedes his presidency.

For generations the American people have believed that the United States is the most powerful, most prosperous, culturally and economically the most influential nation in the world.  In more recent times, however, they have been slowly and painfully coming to grips with the reality that the days of a Pax Americana or American Century are over.

Think about the sluggish growth, a debt larger than gross national product, diminished net wealth, subprime mortgage crisis, crumbling infrastructure, a rising China and the decline of US uni-polarity, a polarised and dysfunctional political system beholden to special interests and wars in Iraq and Afghanistan that have cost the US dearly in blood and treasure as well as prestige and credibility.

The nation is badly off-track and there is a hemorrhage of confidence in America’s future. There is also a widespread sense that Washington has become more dysfunctional than in the past. The causes are more deep-seated than any one president or party: a more polarising political culture; an unprecedented amount of money spent on campaigns; and the relentless 24-7 cable media, talk-radio and Internet environment.

To be sure, demographic trends work to America’s advantage: it has moderately high immigration and fertility levels whereas China will grow old before it grows rich. Thanks to the shale gas revolution, the US is on the cusp of being energy independent, something every president since Nixon during the OPEC oil crisis has yearned for. It’s also true the US has shown impressive capacity for change and renewal after past setbacks (think Civil War, Great Depression, Pearl Harbour, Vietnam, Watergate).

But one can appreciate the resilience of American society and believe that the famous ability to rebound from adversity will be put to a severe test in the coming decade. This is the background against which the midterm elections were being fought; and the anger and anxieties of the nation explain why neither major party is connecting with Middle America.

Tom Switzer, editor of the American Review, lectures and tutors in American diplomatic history and U.S. foreign policy at the University of Sydney’s United States Studies Centre. He will host Between The Lines with Tom Switzer on ABC’s Radio National in 2015. This is an extract from his address to the Australian Institute of International Affairs Victoria in Melbourne on November 10.