The Bench said both Justice Patel, his wife and also daughter, who are in Mumbai, should be adequately protected