Book review by Jay Richardson

There's a strong argument to be made for stand-up being a quintessentially American art form. Yet it's never enjoyed the same level of academic scrutiny as say, jazz or the blues. Or even, perhaps, as much retrospective assessment as here in the UK.

Challenging this is Kliph Nesteroff, whose compelling, low-key blog Classic Television Showbiz began with interviews with Steve Martin and Tom Smothers and has grown to become the go-to source for contemporary anecdotes about Lenny Bruce and the Mafia's grip on the golden age of US showbusiness. Rescuing forgotten comics from obscurity, it's packed with scurrilous eyewitness tales.

Now Nesteroff has written The Comedians: Drunks, Thieves, Scoundrels and the History of American Comedy, a comprehensive 400+ page tome charting a through line from vaudeville to Robin Williams' suicide and Bill Cosby's fall from grace.

A former stand-up but a Canadian émigré, Nesteroff is an obsessive nerd but retains a steady analytical distance, even though he can call on the likes of Mel Brooks as an eager primary source. He appreciates that comics are often open and indiscreet, letting them speak for themselves, applying only the lightest authorial touch in foregrounding the tensions and conflicts behind varying accounts of pivotal incidents.

The Comedians is less quirky and more scholarly than his blog. Yet Nesteroff retains an entertaining knack for pinpointing the personal traits and failings that, alongside historical circumstances and luck, prompted seismic shifts in comedy's development. He dedicates the book to Marc Maron, whose WTF podcast has been so influential and did so much to boost his own profile.

Straightforwardly chronological in the main, The Comedians tracks the arrival of television, Vietnam, the emergence of comedy clubs etc. But it occasionally waits to introduce a ground-breaking performer like Moms Mabley, citing her as an act for whom the African-American breakthrough arrived too late for the mainstream success she warranted. One of the most fascinating sections dwells on the popularisation of the comedy record by black acts like Redd Foxx, bypassing mainstream bigotry.

Drawing on more than 200 original interviews, for the first half of the 20th Century Nesteroff also leans heavily on reviews from industry bible Variety. Time and again, the press showed themselves to be short-sighted in recognising the potential of subversive acts like George Carlin and Dick Gregory. Yet awareness of comedy's evolution often eluded the comedians themselves, with the old guards consistently swept away by new styles of humour they couldn't always comprehend.

By contrast, the likes of Hugh Hefner and Cosa Nostra emerge as stalwart and even progressive supporters. The account of a Mob attack on Joe E. Lewis became the stuff of folklore. And the sheer number of assassination attempts on Jackie Mason are staggering. But as Nesteroff asserts if you were a comedian working between 1930 and 1950, you effectively worked for the Mafia, so complete was their stranglehold on the nightclub business. He finds several acts willing to Testify that the mobsters were good bosses, provided you didn't crack wise at the wrong heckler. The ascent of insult comic Don Rickles – a lovely, insecure man by all accounts – is just one example of someone who benefited enormously from their patronage.

Nesteroff even advances a plausible argument for 'stand-up' being a Mafia coinage, with the Mob's influence on comedy for better and ill comparable to that of the Church, McCarthyism and powerful corporate sponsors. The persecution and marginalisation of acts like Gregory and Ray Bourbon, a gay comic who had a sex change and was busted for impersonating a woman, provide some of the book's most affecting tales.

If there's one martyr who recurs, it's Bruce. Although his story is a familiar, even semi-mythical one, Nesteroff ably reiterates his importance by threading him understatedly through his account, while retaining the sense of him as a man. I was tickled to learn that Bruce almost appeared in Doctor Strangelove and was arrested so often he acquired an understudy who travelled with him to gigs.

Although the dedication and grit needed to make it as a comedian in America is shown to be perennial, as are the jealousy, accusations of gag theft and enormous egos – the hubristic demise of The Jerry Lewis Show is simply breathtaking – some characters are very much of their time: vaudevillian impresario Benjamin Franklin Keith, whose career began exhibiting 'prematurely born Negro babies'; The Karate Kid's Pat Morita performing as The Hip Nip; comic actor Lulu Roman, who was convicted for drug possession yet let in and out of prison between tapings of the popular country and western comedy show Hee Haw.

The more modern chapters don't quite carry the same anecdotal heft and read more like straight reportage. Nevertheless, Larry David's personality leaps off the page as vividly as Jonathan Winters, Phil Silvers or the irascibly dirty B.S. Pully.

UK readers may be intrigued by an alternative origin story for the expression 'working blue' to that of Max Miller's. And most will appreciate such an exhaustive, readable primer on US stand-up, as well as, to a lesser extent, sketch and broadcast comedy, plugging the gaps in most people's knowledge between the icons and the also-rans. However, for anyone with an interest in the history of comedy, The Comedians is indispensable.

• The Comedians: Drunks, Thieves, Scoundrels, and the History of American Comedy by Kliph Nesteroff is published by Grove Press/Atlantic Monthly Press, priced £18.45. Click here to order.

Published: 9 Nov 2015