Through the rubble of the streets, they have been filing back on bicycles, donkey carts and by foot, looking for their homes or what trace is left of them.
“I am going to my house, even though I know that it is destroyed. I am going to remove the rubble to get a shirt out,” Mohammed Abou Diab said.
The smell of death is in the air, residents say, with bodies still lying beneath the ruins. The scale of the devastation has shocked them.
“The destruction is huge… it all needs to be rebuilt. It’s not suitable for human beings to live in – not even for animals”, Abu Saif Abu Mustafa told the BBC.
“It’s as if an earthquake hit the city,” Rashad Khamis al-Najjar from the wider Khan Younis region said as he surveyed the scene.
“The houses are not liveable, the mosques are not suitable for worship, and the roads and the infrastructure, even the electricity, have all been completely destroyed.”
Another resident had a similar sense of horror at what he saw on his return: “We see total destruction everywhere as if it were an earthquake or a natural catastrophe.
“The houses that haven’t been destroyed are burned or looted by thieves… We are dying slowly. There are no homes to live in and we live like the dead.”
Gaza’s second city was the focus of a sustained onslaught by the Israeli army from December, believing that Hamas leaders and fighters had been driven there from the north and established a new stronghold in tunnels and hospitals.
Neighbourhood by neighbourhood, Israel ordered Palestinians to leave. The Israeli army said it was doing everything it could to protect civilians.
A city of some 400,000 people was deserted bit by bit as Israeli forces mounted ferocious bombardments.
Almost from the moment that Israel announced that its mission in Khan Younis was completed and its troops had been pulled out, Palestinians began to move from their overcrowded shelters further south towards their city.
But many have found that with their homes rendered unliveable, they have little option but to try to gather what possessions are left and return to their temporary shelters.
One woman, Nour Ayyash, said she couldn’t reach her apartment because the stairs were gone. Her brother managed to climb up and retrieve some clothes for her children.
Another returning resident told the BBC: “We came to get some of our things, we wanted to see if anything was left of our homes and to take any of our belongings, like clothes for example.”
But some would rather stay, even if there is nothing left of what was once their home.
Mohammed Abu Rizzeq lost more than his house, his wife was killed in an Israeli bombardment after they were displaced.
But he told the BBC: “Our biggest request is that Israel withdraw from our land – there’s been enough killing and destruction. It’s better for us to have a tent on the rubble of our homes than being displaced and in exile.”