David Miliband has condemned “repulsive outbreaks of modern anti-Semitism” and insisted the fight against the scourge requires people to “cross party loyalties and religious faith lines”.

The former foreign secretary was speaking to the Board of Deputies’ president’s dinner during a brief return to the UK from New York where he is heading the International Rescue Committee.

While he didn’t mention the string of anti-Semitism scandals surrounding Labour, he described hatred against Jews as “assault on all humanity”. Miliband told the more than 400 guests: “Some of the most painful conversations I’ve had over the past two years are when New Yorkers say to me, ‘Should I be worried about anti-Semitism in Europe?’ And of course the answer has to be yes. The outbreaks of modern anti-Semitism are repulsive. And the response cannot be to turn away. It must be to engage, not ignore.

“Because it is an assault on all humanity, it is for all humanity to withstand. In doing so, we have a responsibility to cross party loyalties , cross religious faith lines and cross ethnic backgrounds. Just as the fight against anti-Semitism is not just for the Jewish community, so the Jewish community has something to contribute to the fight against all forms of discrimination.” He reserved special praise for former parliamentary colleague John Mann, chair of the parliamentary Committee against anti-Semitism, who was “ahead of his time” in pursuing the issue.

With just two weeks until Pesach, Miliband focused on the “modern exodus” which he said has seen one in every 120 people on the planet displaced by conflict. He said he was “moved” by the Chief Rabbi Mirvis’ visit to a refugee camp in Greece when he became “a voice of shared humanity for those of all faiths and none”.

Though “none have experienced the trauma of fleeing quite like the Jewish people”, the community also provides an example for renewal and resettlement, Miliband said.

“Wherever Jews have fled they have set about restoring life – building schools as well as synagogues, earning citizenship, building businesses, running for office, creating charities. Everywhere they have gone they have put down new roots, without forgetting their own history. They have become their own light unto the nations.” The Board, he said, was itself constantly renewing and boasted a “conscience that never forgets the rest of the world”.

The former cabinet minister, who received a standing ovation, also spoke about how his family’s experience of fleeing the Nazis provided part of the impetus for taking on his current role at the helm of an organisation, set up by Einstein, to aid refugees. “I felt that in a small way my joining the IRC could close a personal circle, repaying a small part of the debt my family owed to those who helped us by offering support to those who are fleeing and rebuilding today.”

Of Einstein, he added: “It is no coincidence that Albert Einstein, a man renowned as a champion of Jewish freedoms, was also the founder of an organisation which aims to save lives wherever they need saving, irrespective of religion.”