In a remarkably candid essay in the journal Nature this March, one of the world’s top climate scientists posited the alarming possibility that global heating may be moving beyond the ability of experts to predict what happens next.

“The 2023 temperature anomaly has come out of the blue, revealing an unprecedented knowledge gap perhaps for the first time since about 40 years ago, when satellite data began offering modellers an unparalleled, real-time view of Earth’s climate system,” wrote Gavin Schmidt, a British scientist and the director of the Nasa Goddard Institute for Space Studies in New York.

If this anomaly does not stabilise by August, he said, it could imply “that a warming planet is already fundamentally altering how the climate system operates, much sooner than scientists had anticipated”.

Many in the science and environment community read these words with alarm. Was the leap in temperatures over the past 13 months, which has exceeded the global heating forecasts of experts, a sign of a systemic shift, or just a temporary anomaly? If the world was warming even faster than scientists thought it would, seemingly jumping years ahead of predictions, would that mean even more crucial decades of action had been lost?

With August now here, Schmidt is a fraction less disturbed. He said the situation remains unclear, but the broader global heating trends are starting to move back in the direction of forecasts. “What I am thinking now is we aren’t that far off from expectations. If we maintain this for the next couple of months then we can say what happened in late 2023 was more ‘blippish’ than systematic. But it is still too early to call it,” he said. “I am slightly less worried, but still humbled that we can’t explain it.”

In an exclusive interview with the Guardian, Schmidt, said records were beaten last year by a surprising margin and predicts 2024 is also likely to set a new peak, though the trend may nudge closer towards expectations.

Looking back at the most extreme months of heat in the second half of 2023 and early 2024 when the previous records were beaten at times by more than 0.2C, an enormous anomaly, he said scientists were still baffled: “We don’t have a quantitative explanation for even half of it. That is pretty humbling.”

He added: “We should have better answers by now. Climate modelling as an enterprise is not set out to be super reactive. It is a slow, long process in which people around the world are volunteering their time. We haven’t got our act together on this question yet.”

This is not to doubt the underlying science of global heating, which more than 99.9% of climatologists agree is caused by human burning of gas, oil, coal and forests.

That alone is creating alarming new temperature records every year, as the world experienced last month with two consecutive days of heat in excess of anything in human records, and probably also anything in more than 120,000 years.

This is wreaking havoc over an even wider swath of the world by intensifying forest fires, droughts, floods, sea-ice loss and other manifestations of extreme weather.

The worsening trend will continue until fossil fuels are stopped. “As climate change continues, every decade it gets warmer, the impact is larger and the consequences are greater,” Schmidt said. “So in that sense, we are already in uncharted territory with respect to climate and with every decade we go more further out on a limb.”

The recent El Ninõ added to global heat pressures. Scientists have also pointed to the fallout from the January 2022 Hunga Tonga-Hunga Ha’apai volcanic eruption in Tonga, the ramping up of solar activity in the run-up to a predicted solar maximum, and pollution controls that reduced cooling sulphur dioxide particles. But Schmidt said none of these possible causes was sufficient to account for the spike in temperatures.

Schmidt said he hoped a clearer picture would emerge by the time of the American Geophysical Union meeting in December, when many of the world’s top Earth system scientists will gather in New Orleans, Louisiana.

One of the most worrying theories to emerge is that the Earth is losing its albedo, which is the ability of the planet to reflect heat back into space. This is mainly because there is less white ice in the Arctic, Antarctic and mountain glaciers. Peter Cox, a professor at Exeter University, noted on X that this is “contributing hugely to the acceleration of global warming”. It would also suggest the recent records are not just a freak conjunction of factors.

On 29 July, the total extent of sea ice was at a record low for the date and nearly 4m sq km – an area bigger than India – below the 1981-2010 average, according to Zackary Labe, a climate scientist at the US National Oceanic and Atmospheric Administration.

It continues to melt rapidly because temperatures in some parts of Antarctica recently hit 24C above the average for the time of year in the middle of the austral winter.

António Guterres, the secretary-general of the United Nations, warned recently that “Earth is becoming hotter and more dangerous for everyone, everywhere”.

He pointed out that scorching conditions killed 1,300 pilgrims during the hajj in Saudi Arabia, shut down tourist attractions in Europe’s sweatbox cities and closed schools across Asia and Africa.

Temperatures above 50C used to be a rarity confined to two or three global hotspots, but the World Meteorological Organization noted that at least 10 countries have reported this level of searing heat in the past year: the US, Mexico, Morocco, Algeria, Saudi Arabia, Kuwait, Iran, Pakistan, India and China.

In Iran, the heat index – a measure that also includes humidity – has come perilously close to 60C, far above the level considered safe for humans.

Heatwaves are now commonplace elsewhere, killing the most vulnerable, worsening inequality and threatening the wellbeing of future generations. Unicef calculates a quarter of the world’s children are already exposed to frequent heatwaves, and this will rise to almost 100% by mid-century.

The pace of change is disorienting. Schmidt says there is a 72% chance that 2024 will beat last year’s heat record. The likelihood will rise still higher if there is no cooling La Niña by December.

While some argue that the world will soon pass the lower Paris agreement guardrail of 1.5C of heating above the preindustrial average, Schmidt says the more important goal should be to phase out carbon emissions as quickly as possible: “What should be motivating people is that with every tenth of a degree of warming, the impacts will increase. That is the fundamental equation. It doesn’t matter where we are now, but we have to get to net zero. The faster that happens, then the happier we will be.”

At times, he acknowledged that his work puts him in a bind because as a scientist he wants his forecasts about global heating to be accurate, but as a human he would rather they proved an overestimate.

“We would all rather be wrong than right on this,” he says. “That is the one thing that sceptics don’t understand.”