Andrew Marr: "Want to know anything about 20th-century Britain? Read a spy novel"

That, at least, is the message of the two unequivocal greats of British spy fiction, John le Carré (aka David Cornwell) and Graham Greene, whose own traumatic early years shaped their stories of disillusion.

Le Carré’s agonised relationship with his confidence trickster father featured heavily in his recent memoir; Graham Greene was obliged to turn spy as a boy at Berkhamsted School, where his father was headmaster.

Le Carré, who as a British agent witnessed the brutal erection of the Berlin Wall, was a lifelong loather of Communism. Graham Greene, who became a friend of the Soviet agent Kim Philby when they were working for British intelligence during the Second World War, was rather more ambiguous. But both of them based some of their finest writing on the moral ambiguities that infected the West during the Cold War.

For me, however, much of the fun of our BBC4 film on British spy fiction came from less well-known figures.

William Le Queux, author of Secrets of the Foreign Office: Describing the Doings of Duckworth Drew of the Secret Service, has a good claim to be the originator of the genre.

Writing before the First World War, he picked up on the growing paranoia about the threat from Imperial Germany, and then inflated it further, creating stories about German spies that spread real panic when they were serialised as a circulation-boosting wheeze by the Daily Mail.

Le Queux yarns such as the 1909 Spies of the Kaiser affected public policy directly. When Asquith became prime minister, realising that the British state had no way of telling how accurate Le Queux’s spy-mania was, he set up a subcommittee of the Imperial Defence which eventually produced our own counter-intelligence operations.

Stella Rimington, a former head of MI5 and a cracking spy writer herself, told us the story of those early days, including the original “C”, an eccentric admiral who had lost his leg in an early car accident, and won Whitehall arguments by opening a penknife and viciously stabbing himself in his (wooden) thigh for effect.

Once the First World War got going, intelligence quickly became more professional and yet another novelist, Somerset Maugham, was recruited as a continental agent, leading to Ashenden, a sequence of spy stories, half of which were censored by Winston Churchill as being far too accurate for public consumption.

Recalling that Ian Fleming as well as Greene, le Carré and Maugham – and the wickedly humorous Scottish novelist Compton Mackenzie – had secret intelligence experience before they started writing, my producer suggested to me that we should stop thinking about MI6 as primarily an organ of state security, and, rather “a finishing school for novelists”.

This seems to me about right; and yet some of the most interesting spy writers did come from outside. Most of the early espionage novelists were men of the patriotic right – John Buchan being the most famous example.

But Eric Ambler, who produced six spy novels between 1935 and 1940, was an advertising copywriter. For him, the real enemy was international capitalism – his agents were more often on the same side as Moscow’s men than against them. His protagonists tended to be victims as much as gung-ho heroes and Ambler’s world reflects the insecurities and national self-doubt caused by the rise of fascism.

After the defeat of the Nazis, when Britain was victorious but readjusting to the loss of the Empire, spy fiction required a very different tone and Ian Fleming, with James Bond, did more to perk up national morale than any other writer.

Foreign Office boobies might mutter reassuringly, “We punched above our weight, old boy”, but the womanising, hard-drinking Bond did so satisfyingly literally.

Then came the next backlash – Graham Greene’s The Human Factor and the wracked moral dilemmas of le Carré’s George Smiley. The real-life treachery of the Cambridge spies could no longer be evaded.

And quite right too. There’s nothing like visiting the chilly hell of the old Stasi prison Hohenschönhausen in East Berlin to remind one of the torture and deaths on the consciences of those jolly old public school boys who turned to Moscow.

Len Deighton, creator of The Ipcress File’s Harry Palmer, said frankly that he had been used as “a blunt instrument” that the critics deployed to bash Ian Fleming (and his urbane snobbery) over the head.

But spy fiction, always bashing, never seems to die. It simply changes with changing politics. So we get Gerald Seymour and Harry’s Game telling the dirty story of the fight against the IRA, while today writers like Charles Cumming are creating new spies with Arabic backgrounds.

If you knew nothing about Britain’s 20th-century history, and simply start to read spy novels, then – as an amateur historian it pains me to say it – you wouldn’t go very far wrong.

Andrew Marr’s Paperback Heroes is on tonight, 9.00pm, BBC4