I am uncomfortable with the thought that it is possible to observe an animal in the wild and not have any impact whatsoever on it's behavior. This is a well know "observer effect" - http://en.wikipedia.org/wiki/Observer_effect.
In addition, I would point out that humans are animals too, and we are part of the life experience interaction of the animals we observe. Don't be too quick to put yourself outside the worldview of the animal, who may not see you any different than some other predator or prey it encounters. Occasionally, news stories here in California regarding unfortunate interactions in our ever-expanding suburbs, indicate that the highly stressed Cougar can find human presence to be...an opportunity.

Lastly, I haven't seen anyone comment on what I find most obvious - animals, and especially birds in my experience, are very sensitive to having eyes pointed in their direction....I never look directly at birds when moving closer, and usually wait 10 or 20 seconds after repositioning before looking at them....but what else than a huge 5 or 6" eye is a supertelephoto lens?

I am very comfortable in my judgment regarding the boundary between gentle "phishing" and extreme behavior designed to startle and provoke wildlife.