Cameron is missing point (of course) about the use of 'Yids' by Spurs fans. Most Spurs fans are not Jews, so they are not 'reclaiming' the word in the true sense of that term. As it is Spurs fans chanting 'We are Yids' this then allows non-Spurs fans to chant 'Fuck off Yids' 'go back to Auschwitz' etc and justify it in terms of 'Spurs fans say "Yids"'. So it's not that saying 'Yids' is in itself a 'race crime' (yes, Cameron) but that constantly legitimating it through the false use of 'reclaiming' makes it possible for fans everywhere to have 'fun' yelling a hate term ('racial slur' as the US anti-racists call it). No, you can't 'ban' it as such and I wouldn't advocate locking people up for it. But you'd hope that a prime minister would be able to pick himself through this and produce something more nuanced than he has. This is about the 'climate' that surrounds words and what is legitimated when people in authority appear to sanction their use.
As for whether Jews use the word. My father used a Yiddish word 'Yiddlech'. I guess he thought that as it was a kind of self-defining term in a language that was mostly understood only by Jews from eastern Europe (or their descendants) , then he could say it with affection. It wasn't the word as chanted at him by anti-semites, which was of course 'Yids'. Some Jewish people reading this may well say that they or their parents used the word. However, the nuance is that it would be Jews themselves using the word. That is of course a 'hostage to fortune' just as it is for African-Americans when they use the 'n-word'.
Dealing with this sort of stuff doesn't solve racism. However, politicians and people in the public eye have to indicate whether they think it's OK to bandy about racial slurs or whether it makes fighting racism harder. If Tottenham Hotspur Football Club listened to the FA, (or indeed listened to David Baddiel who is saying some very reasonable things about this), they could put out strong statements explaining all this.