AIUI, it's where the photographer stands, not the subject.
Good eye! And yes, there's a big difference. You can photograph a subject on private property, as long as
you are on public property. However, there is a gray area here, so tread lightly...especially when it comes to individual persons and not "things" such as buildings. For example:
If you approach a private business and ask them to take photos of their building and/or property, you are likely going to be told "no" for a variety of reasons. However, there is no law prohibiting you from taking photos from the sidewalk or street.
But...I wouldn't advise taking photos of someone in their house through the front window. They have taken reasonable precautions to protect their privacy, and you can get in trouble, even if taking photos from the sidewalk or street. The same thing with someone at an ATM machine. Don't do that either.