The building occupies an entire block and has its own ZIP code. It’s owned by Rudin, one of the oldest real estate firms in New York City.