Rights groups say refugees have been scapegoated by the UK government for years, and are at the core of key parties' manifestos this general election.