Jonathan Becker is remembering an occasion when he was working with the late creative director of Vogue, André Leon Talley. They were supposed to be taking a picture of “some jewellery lady and her daughter on a pony”. If you can’t already tell, Becker wasn’t very excited. But Talley was – and he was annoyed at Becker for not being more so. “He was twice my size,” says Becker, “and he was slapping me upside the head, ‘Get the girl. Take the picture now.’”

It was not, however, his sort of shoot. “André loved fashion and brands,” he says, “and that just didn’t intrigue me in the least.” Despite being a photographer who has spent five decades taking pictures of, well, everyone, Becker is not himself a household name. But many of the people he has photographed, for the likes of Vanity Fair and Vogue, are – and more. Becker’s latest book, Lost Time, is bursting with them.

Harvey Weinstein looms behind actor Léa Seydoux at fancy London club Annabel’s in 2015. Madonna appears lit by laughter in 1990. Author Fran Lebowitz smirks as she lets a giant cream armchair overrun her at the Vanity Fair Oscars party in 2000. David Bowie looks mildly disgruntled at the Tribeca film festival in 2003.

Then there are the art-world greats: Jean-Michel Basquiat, Cindy Sherman and Roy Lichtenstein; as well as those from the literary realm, including William Burroughs, Arthur Miller and Tom Wolfe. Becker could make a dangerously eclectic cabinet with the politicians, and their significant others, that he has captured, from Jackie Kennedy to Nancy Reagan, John F Kennedy Jr and Carolyn Bessette-Kennedy to several of the Trumps, as well as international names such as Mikhail Gorbachev and Aung San Suu Kyi. And royals, too, lots of royals.

“These are figures from a vanished world,” the book’s editor, Mark Holborn, writes in its intro; and, speaking to Becker by video from his cabin on an island in Maine, it does feel like a surreal catalogue of late 20th and early 21st-century grandees.

But back to Talley. It might not sound like it, but the pair were great friends. “I loved André,” says Becker. “It brings tears to my eyes when I think about him.” They first connected at a shoot of the former Vogue editor-in-chief Diana Vreeland, in the living room of her Park Avenue house in New York in 1979. When Becker confessed to Vreeland where he recognised her from – taking her home in the taxi he began driving in 1978 to subsidise his freelance income – Talley apparently shrieked. “He was really the most dramatic person I ever met.”

Becker’s speech is peppered with phrases that feel very old New York, helped by his regular pauses to relight his cigar. “I wouldn’t have had a mothball’s chance in a public urinal,” he says at one point, referring to how his academic father, William Becker, regarded his chances if he didn’t go to Harvard. When the great Hungarian-French photographer Brassaï responded to something Becker wrote about his work with the line, “You have well understood and expressed the spirit in which I made my photographs,” Becker thought, “Well, I’m out of the urinal.” It gave him licence, he says.

His backstory rattles through moves to Paris – Brassaï became an early mentor – and California, to avoid a $500 spaghetti bill from the famous Elaine’s restaurant in New York, where he had used the kitchen to take portraits of Paul Simon and Andy Warhol. While on the west coast, he did “sordid work for magazines with unmentionable titles”, and was an extra on Grease II – his mother, Patricia Birch, was the director.

The pictures in Lost Time perfectly evoke an era. Take his 1988 photograph of Robert Mapplethorpe, shot a year before he died of Aids. It was the opening of his retrospective at the Whitney in New York. It’s an unassuming picture. Mapplethorpe is seated, in profile, leaning on a stick. He doesn’t take up much of the frame but his thin fingers have centre stage, dramatically underlining the gauntness of his face just above.

“The plague of Aids had cast its pall – the elephant in the room. I sought to articulate it visually,” Becker writes in the book. “It was terrifying, this plague, in New York – and it was taking people down,” he says now. “There was no cure and it hadn’t really been photographed before.” Becker’s picture was widely reproduced.

Although his subjects have always surprised him in some way, he was perhaps most taken aback by Dr Kevorkian, the outspoken advocate of assisted suicide. Becker shot him at home in Detroit in 1994. He was a controversial figure and had already stood trial for assisting suicide – but no one had ever seen his art. “Here he was, underneath it all, revealing he had necrophiliac tendencies,” says Becker. “It was one of these pictures that really made an impact – the equivalent would be that it broke the internet.”

Becker would be sent off on location, to Aspen, the Adirondacks, South America and Palm Springs, and left to get on with the story. Budgets, or “the budget thing”, as he jokes, “never existed. If someone had said, ‘There’s no budget’, I would have said, ‘Of course there’s no budget’ – and I would have meant there’s no limit on the money you can spend.”

It was on the hoof, rather than through more staged scenarios, that Becker took many of his greatest compositions. In one thrilling snapshot, Nicole Kidman takes a luxuriant drag from a cigarette at the Vanity Fair Oscars party in 2000. Tom Cruise, her husband at the time for another year, looks on from the shadows. It appears like a moment of disapproval. “I think he was annoyed with Nicole that night,” says Becker. Cruise is, he says, conservative, “well-mannered and very aware of his movie stardom”. Which Becker thinks explains the annoyance: “She’d had a few drinks and was not playing the role.”

He is refreshingly open, often expressing whether or not he liked a subject. Myanmar leader Aung San Suu Kyi? Didn’t like her. “I had a funny feeling about her,” he says. He seems similarly unimpressed by gonzo legend Hunter S Thompson, too. “He was a freak. He liked to fire guns and carry on and he was definitively outrageous.” Becker does concede, however, that Thompson was “lots of fun”.

Melania Trump, who he photographed not long after her wedding, he likes. She was “quiet and she had a bit of a sense of humour”, while Donald Trump “has his charm”. Becker has photographed the ex-president a number of times but the shot that makes it into the book shows him swivelling round in a brassy golden throne in a brassy golden room in Trump tower. What is Trump’s charm exactly? “I’m not sure,” says Becker. “It’s a narcissistic charm. He’s sort of sharing his great power – he considers himself very powerful. But he’s also very gullible.” Becker doesn’t seem squeamish about who he photographs and claims not to really care about politics, going “more to character”. But he won’t “take assignments to whitewash people. My job isn’t to make people beautiful necessarily, or to make them more palatable. What I like to do is to extract character and let them do their own theatre.”

Becker is endlessly curious and has a bloodhound’s nose for humour, but he seems drawn to peace and quiet – at one point, he shows me the tranquil island where he spends some of the summer, made even more tranquil by the fact that it is 6am on a Saturday. Two or three times a week, he visits his studio in New York. “It’s on the east side, so if I drive in I don’t have to penetrate the city too much.”

Although he has made a career photographing fabulous New Yorkers at fancy dos, he finds the city cacophonous, always has. “I’m a New Yorker first, American second,” he says, but he does talk of a sense of loss. “When I was a kid, there was no doubt it was the greatest city in the world. Frank Sinatra was singing about it. It was really the capital.” He doesn’t get that feeling any more, he says.

The more Becker looks at the book, “the more I realise this is a lost time”. He points to a shot of Jackie Kennedy Onassis, in which she seems “beguiled” by the writer Bernard Malamud. “This was a time when intellectuals had more status than movie stars or socialites,” he says. “It wasn’t all about the Kardashians. They would have been laughed off the planet.”

• Jonathan Becker: Lost Time by Jonathan Becker (Phaidon Press Ltd, £79.95). To support the Guardian and Observer, order your copy at guardianbookshop.com. Delivery charges may apply.