Big, spooky gothic mansion, filled with gruesome looking artifacts. Live tigers as pets. Go full Adams.
Really, the easiest way to do this is to create an "Owner" to be the real rich loner who lives in the house, and have everyone pretend to be his family, guards, and staff. The "owner" can be the party face, a popular contact, or no one at all -- there can be more awkward questions if no one ever meets "the new neighbor", but at least you can skip all the neighborhood social gatherings.