The musician said the immigration policies of the last Labour government had a disastrous effect as British workers found themselves undercut by Europeans prepared to accept lower wages.
Labour "left the British working man screwed like he'd never been screwed before by cheap labour coming in from Europe", Daltrey said.
He added: "We do need immigration, but surely it should be a level playing field where they can't undercut every working class bloke in England for their jobs."
Daltrey, 67, grew up in working class Shepherd's Bush, west London, and is fiercely proud of his roots. Too poor to afford a musical instrument, he fashioned his first guitar out of a block of wood and some wire.
He passed the 11-plus and attended a local grammar school but was expelled at 15 and worked as an electrician's mate, tea-boy and apprentice sheet metal worker before finding fame with The Who.
In an interview with The Daily Telegraph, Daltrey also criticised the Government for failing to help rural areas and described the current crop of British politicians as "tragic". He now lives in east Sussex, where he runs a trout fishery, and was a member of the Countryside Alliance until he quit in protest at the organisation's focus on fox hunting.
"I live 50 miles from London and we've got some of the highest levels of teenage and childhood poverty in the country. It's disgusting. Just because it's a rural area, it gets forgotten," Daltrey complained.
"I'm not anti-fox hunting because, to me, shooting foxues is even worse and the results are horrendous. So don't tell me the fox hunting debate was about animal welfare, because I can't take that hypocrisy. It was class war, and that's what I hated."
Despite the advancing years, the man who once sang "Hope I die before I get old" is about to begin a tour of the UK and North America, performing songs from The Who's rock opera, Tommy.