A peer has resigned from the Labour Party days before he was due to face a hearing over allegations that he made anti-Semitic comments in a TV interview.

Lord Ahmed was due to appear before Labour’s ruling National Executive Committee on Wednesday to answer accusations that he blamed a Jewish conspiracy for his dangerous driving jail term.

He insists he does not recall making the alleged comments, and his solicitor Stephen Smith said he would not be able to receive a “fair trial” from the Labour panel.

A Labour Party spokesman said: “I can confirm that he has resigned from the party.”

Lord Ahmed was suspended from the Labour Party in March after The Times reported that he blamed his 2009 prison sentence – for sending text messages shortly before his car was involved in a fatal crash – on pressure placed on the courts by Jews ”who own newspapers and TV channels”.

The Muslim peer allegedly told an Urdu-language broadcaster in Pakistan that the judge who jailed him for 12 weeks was appointed to the High Court after helping a ”Jewish colleague” of Tony Blair during an important case.

Mr Smith questioned the reliability of the evidence against Lord Ahmed, saying: “The evidence is flawed, in my view it is unreliable and yet they seem to be accepting it as right. In those circumstances I don’t think he can get a fair trial.”

Lord Ahmed said he was “very disappointed” with the way he had been treated by Labour.

He said: “I don’t want to appear before any kangaroo court where the rules of justice have been denied. All I want is any evidence in front of me so I can look at it and defend myself. I can’t defend myself if there is no evidence produced.”

Lord Ahmed said he had “never been anti-Semitic” and had “unreservedly apologised” for any offence caused to the Jewish community.

He added that his “heart” is still with the Labour Party. “I don’t want my friends in the Labour to think that I have walked away but the rules of justice demand it. It’s very unfortunate.”

In his resignation letter Lord Ahmed said: “I do not recall when this interview was held and nor the person who carried out this interview. All I know is what has been reported in The Times. I reject the core story that emerges out of the alleged interview.”

Lord Ahmed and Mr Smith have called for The Times to release the interview footage for it to be forensically examined.

“I believe that justice of the case demands that the film of the interview should be subjected to forensic test in order to search for the truth of the matter,” Lord Ahmed wrote. “This has been denied to me.”

He continued: ” I am most concerned that the Party which has freedom and justice as it core value … is content to proceed against me on the basis of incredible and untested evidence. That indeed leads me to believe that the decision might have already been made.”