When the twin towers collapsed that day, they left behind two giant holes in the world's most famous skyline, like missing milk teeth in a child's smile. Once the dust cloud – visible 20 miles away – had settled, all that remained was a cathedral-sized mound of rubble and over 3m cubic metres of air.

In the subsequent decade, New Yorkers have instinctively turned to the twin towers to orientate themselves, only to find nothing there.

But over the last few months, and with the 10th anniversary of the 21st century's most notorious event imminent, a new structure has started to stretch upwards, piercing the Manhattan skyline, its roof sprouting cranes like the leaves of a young plant. The building already stands boldly above the others around it and it is only two-thirds complete.

With every week that passes, it is claiming its status as the pre-eminent Manhattan landmark. By early next year it will supersede the Empire State Building in height. Even now it draws the eye, particularly at night when its construction lights twinkle like a brooch from Tiffany.

This week the tower stands at 80 floors and counting. It is going up at the rate of a floor a week, the product of 24/7 activity by a team of 1,100 workers. By the time it opens, it will be 104 floors; a beacon will take it to the historically resonant height of 1,776ft (541 metres).

Up on the 55th floor of the emerging skyscraper – once called Freedom Tower but now known by the more temperate name 1 World Trade Centre – the stunning views are ample evidence of the building's potential.

You can see way across the water, beyond Ellis Island and Lady Liberty to New Jersey, while below us, the memorial gardens are being assembled in honour of those who died.

"This is the worst disaster in our country's history and we're up 55 storeys and climbing," says Lee Ielpi, a New York firefighter who knows more about Ground Zero than most. For the past 10 years he has spent most of his waking life here. He arrived at the site soon after the towers fell and for the next nine months he worked 12-hour days atop the rubble searching for bodies. After the site was cleared, he founded a tribute centre that houses a 9/11 exhibition and works with bereaved families.

Ielpi's son Jonathan, also a firefighter, was called out with the Queens-based Squad 288 after the first plane struck and he was in the south tower helping people escape when it came down. His body was discovered three months to the day after the disaster; Ielpi carried his son out with his own hands.

Ielpi rattles off the key statistics, by now as familiar to him as his son's name. "On this site 2,749 people were murdered in a matter of 102 minutes. There are still 1,125 people missing, 10 years later. Only 174 whole bodies were found. One of them was Jonathan."

Later, he says: "People ask me, 'It's been 10 years, what's it like?' Well, it's like I haven't seen my son for 10 years. Nothing more than that."

Ielpi takes us on a tour of the 16-acre site, which, on 12 September will be opened to the public for the first time. We walk around the footprint of the south tower and come to a stop in front of an oak tree, one of 415 planted at Ground Zero.

"This is the first time I've seen that guy blooming," Ielpi says. "I see it and I think of my son. Twenty nine. Married with two little boys. Loved helping people. Knew where he was going on the 11th, knew what he had to do. So yeah, I look at this tree, and this is nice."

Jonathan's name is etched in a bronze panel that runs around the four sides of the footprint of the south tower. Also named here are the other victims – those on the aircraft, the emergency workers, and the victims of the Pentagon and Pennsylvania crashes. A second tribute at the north tower bears the names of those who died there as well as those killed in the first terror attack on the World Trade Centre, on 26 February 1993. From Sunday both footprints will be transformed into massive reflective pools with water falling 30 feet from their edges.

"I can't tell you how powerful it is," Ielpi says. "I've stood by that edge and listened as the water cascades down. If you want to hear somebody, there's a good likelihood you're going to hear somebody."

Standing in the middle of Ground Zero, you can now start to appreciate the fusion of commemoration and rebirth that is gathering pace. Next year the 9/11 Memorial Museum will open, running underneath the plaza and housing many artefacts from the twin towers, including two enormous tridents from the north tower that are already in place. In 2013 the fourth tower on the site, a 60-floor skyscraper designed by Fumihiko Maki that is now going up, is scheduled to open, followed in January 2014 by 1 World Trade Centre.

Towers two and three, by the British architects Lord (Norman) Foster and Lord (Richard) Rogers respectively, have yet to have firm completion dates.

The relief that there is something finally happening at the site is palpable. To have had nothing to show a decade after the attacks would have been an enormous embarrassment for New York and for America.

Part of the reason for the painfully slow progress has been physical – the World Trade Centre was built on more than 20 metres of landfill on the Hudson river which, added to the maze of commuter and subway lines that runs beneath it, makes for a fiendishly complicated infrastructure.

But a less acceptable cause of delay has been the messy and at times ugly interaction between the countless entities that have vied to control aspects of the project, from the Port Authority of New York and New Jersey that owns the site, to the private developer Larry Silverstein who leases much of it, through New York city, the railway and subway authorities, insurance companies, local businesses and residents of Lower Manhattan and, of course, the relatives of those who died.

Paul Goldberger, the New Yorker's architecture critic, who has tracked the project over the last decade, says the rebuilding has been beset by "enormous battles between all the participants who could not agree on what to do, how to do it and who should pay for it". At its lowest point, the paralysis that took hold after 9/11 made a mockery of New York's reputation as the can-do city where anything is possible.

"That's a myth left over from another age. As New York has become older and more mature, it has lost the we'll-do-it-no-matter-what attitude that it pioneered but that now exists in places like Dubai or Singapore."

Goldberger has mixed feelings about the redevelopment as it takes shape. He praises the memorial features of the project, with the footprints of the twin towers as their centrepiece, which he thinks will honour the victims in a moving and important way. But he harbours doubts about the buildings that will surround the memorial, particularly 1 World Trade Centre. Designed by the American architect David Childs, it has gone through several incarnations, each one more conventional than the last.

"It's going to be a good building, but no more than that," Goldberger says. "We lost the opportunity to build a great building by being overly conservative in the design, overly concerned with security. It saddens me that we didn't take this opportunity to reassert American leadership. America is where the skyscraper began. We could have taken this piece of land, in the city of skyscrapers, and built the greenest, most exciting and innovative skyscraper that will show a whole new direction. We didn't do that."

Despite the reservations of the city's pre-eminent architecture critic, commercially 1 World Trade Centre looks set on a path towards success. Tara Stacom, vice chairman of Cushman & Wakefield, the property firm handling the leasing of the office space, predicts it will be the "most important building in the western hemisphere – it's going to be the coolest, hippest place to work and live."

She says a range of media and law firms, entertainment companies and financial businesses have expressed interest in taking up space, including some from Europe and the UK. In the biggest deal to date, the magazine publisher Conde Nast has signed up for one million square feet – more than a third of the building.

When the idea of another vast tower, which on completion will be the tallest in the Americas, was first suggested, sceptics argued that it would never be built because nobody would ever want to sit in an office high in the New York sky for fear of a repeat attack. But Stacom insists that has not been a worry for prospective clients. "When we tour the site it's all about what an incredible place this is going to be."

Back at Ground Zero, overlooking the pool at the south tower where his son died, Lee Ielpi says it's time to move on. "There are some family members – and you can understand it – who can't get beyond 9/11. Their loved ones were taken away in a horrible way. But you have to get on. You never forget, but if you don't get on you're going to be trapped."

Ielpi believes that when the rebuilding is finally finished, it will radiate its message far beyond the limits of Manhattan, of New York or even of the US. "We're putting a memorial here, a museum here, we are going to remember our loved ones, and then we are going to surround it with beautiful buildings to replace those that were destroyed through hatred. We're going to show the terrorists that through our resilience – ours, as a world's – we can do it. We will overcome the obstructions that come along."