Children, teenagers and women appeared to be worst affected by any swap to plant-based milks, yoghurt and cheese