The Government joined Holocaust survivors this week in restating the need for a new memorial and learning centre in Westminster, after the Imperial War Museum [IWM] called for the project to be “reconsidered”.

Two years ago then-prime minister David Cameron announced the building of a new Holocaust Memorial and underground learning centre in Victoria Tower Gardens near Parliament.

This week, in an extraordinary intervention, IWM directors urged those behind the UK Holocaust Memorial Foundation to think again – just weeks before a winning design is chosen.

The IWM, on Lambeth Road, is preparing its own new and redesigned Holocaust exhibit, due to open in 2020. It says 600,000 people already visit the IWM Holocaust exhibit annually, and the new Victoria Tower Gardens centre will “divide the public”.

New and expanded galleries will include survivor testimonies, objects and original material and will present the Holocaust narrative within the wider context of the Second World War.

IWM director Diane Lees told The Guardian: “Our groundbreaking new galleries and digitally enabled learning suite will allow us to transform the way we present the Second World War and the Holocaust and, through our new narrative, enable visitors to engage with events that may be less well understood and known to them.”

She added: “We urge the reconsideration of the creation of the UK Holocaust Memorial Foundation’s learning centre at Victoria Tower Gardens, [which is] less than a mile away from us, as it will very much divide the public offer on learning about the Holocaust.”

Communities Secretary Sajid Javid jumped to the Foundation’s defence, saying the two could live peacefully side-by-side. “The Memorial’s accompanying learning centre will be a new national asset, challenging us to explore these themes and working with education initiatives up and down the country,” he said.

“The IWM’s new galleries and the new learning centre will have complementary aims and can collaborate to offer visitors expert and engaging experiences, helping us all to consider the lessons of the Holocaust for future generations.”

Sir Peter Bazalgette, Foundation chair, said the new memorial and learning centre would be “an internationally-recognised symbol against hatred” and seemed surprised by the IWM’s intervention.

He said: “As the IWM has previously agreed, the Learning Centre and IWM galleries have different remits, with distinct yet complementary objectives. We hope they will reconsider a collaborative approach.”

Ben Helfgott, President, 45 Aid Society added: “Together, next to Parliament, the UK Holocaust Memorial and Learning Centre will make a powerful national commitment to stand against hatred and learn the lessons of the Holocaust for generations to come. Turning away now from this committed location next to Parliament is simply not right . I have worked in partnership with the Imperial War Museum for many years to support their Holocaust galleries which are a vital part of educating people about what happened in the Holocaust. The Memorial and Learning Centre will complement and support the IWM’s galleries, not duplicate them. We can all work together to effectively educate and to remember.”

Gabriel Herman, chairman of the Jewish Historical Society of England, agreed. He said: “The IWM will help those who desire more knowledge about the Shoah, but a place of contemplation also needs its own space where people can learn and reflect.”

Karen Pollock, chief executive of the Holocaust Educational Trust, said: “Sitting next to Parliament, the heart of our democracy, a new national memorial and learning centre could not be more fitting in today’s fragile world. It will stand for generations to come as a symbol of this country’s commitment to remembering the Holocaust.”

However, Ben Barkow, director of the Wiener Library, said: “The UK Holocaust Memorial Foundation has not been as constructive or communicative as it could be in accepting the input of the UK’s experts from libraries, museums, memorial institutions or survivor organisations, historians and education specialists.

“My concern is that they prefer an approach that privileges triumphal survivor and rescue stories over historical context and nuance.”