The new Prime Minister had previously hinted at raising the personal allowance to tackle the cost of living crisis