The group has now been given permission for a full judicial review, ahead of the census on 21 March.
The court also ordered the ONS to pay Fair Play for Women's legal costs.
The campaigners had submitted that the previous guidance on answering the ONS 'What is your sex?' question allowed for self-identification which conflates biological sex with “the idea of a feeling, called gender identity”.
Jason Coppel QC argued in court that the guidance “conflates and confuses” sex with gender identity.
He told the court that a person's sex on a passport, or other legal document such as a driving licence, can be altered without a formal legal process.
The QC argued this could potentially have the effect of “distorting” the data the ONS gathers through the census.
'Strongly arguable case'
Mr Justice Swift said he was satisfied that the campaign group had a “strongly arguable case”.
Dr Nicola Williams, director of Fair Play For Women, said: “It is also wholly unnecessary. because a new question has been added to this census specifically about gender identity.”
“Being male or female is a biological reality that affects all our lives,” she added.
Dr Williams added: “The public needs to trust that our national statistician will always prioritise good data over political correctness."
“Wider questions must now be asked about how this was allowed to happen. This is not a problem confined to the ONS," she said.
“The idea that sex isn't just a matter of being born female or male, and that we all have a gender identity, has become embedded within organisations.
“As a result, sex is being overridden by gender identity in a whole range of public policies without proper scrutiny. It's women and girls who pay the price,” she said.
The ONS guidance previously read: 'If you are considering how to answer, use the sex recorded on one of your legal documents such as a birth certificate, gender recognition certificate, or passport.'
Concerns had been expressed that census data could be corrupted without stringent rules on gender self-declaration.
It is a relatively simple matter to have gender altered on a British passport, with a letter from a GP confirming that the changed identity is “likely to be permanent”.
A British passport identity can also be altered by submitting an acquired gender recognition certificate or a new birth or adoption certificate, if the person is over 18, and has been diagnosed with gender dysphoria, which is also known as gender identity disorder, gender incongruence or transsexualism.
In February, the ONS said it was continuing to ask a binary choice male or female sex question on the census.
However, the new voluntary question on gender identity was introduced following work with “organisations and individuals from a wide range of representative groups over five years to ensure the Census questions we ask provide the best information possible.”