Is it really the way it is in the US? Kinda strange. So only one person in a public place needs to say that he doesn't want to be filmed and that's it? So you can't film in a public place if people are against that? Isn't it against the law to prohibit that? :/
It's a complicated legal mess because people don't understand that the U.S. does not actually have personal rights - but constitutional restrictions upon government. So, legal policies have arisen to try and grant people 'rights' that they can leverage against each other.
Which has the predictable effect of people trying to leverage themselves against those they don't like to get them thrown in jail.
In a public space, you are free to record whomever and whatever you want.
There are, however, a few caveats. If you are recording someone and making false claims that are damaging or disparaging against them (or unsubstantiated claims) - then the person can file a suit against you for slander or libel, respectively.
Recording someone and then profiting from the footage can, also, end up with a lawsuit against you because you were profiting from their image without their consent. In this sense - that persons' image can be thought of as a sort of copyright or trademark that must be effectively licensed via contract. Large companies avoid this, while most citizens who post youtube videos pay no attention to it... and no one really thinks to try and pursue the revenues people make from youtube videos (and even if they did, the cost to prosecute would likely exceed any potential compensation mandated by the court).
The other issue is that when you pair a person's photo with a name or address, it can constitute an issue in terms of Personal Identification - increasing a person's risk of identity theft or other such things. In such a case, the person or the guardian of that person can ask to have the image removed, and can contact law enforcement to bring legal action against those who fail to remove an image or video that couples these (within some constraints).
Of course - most of these involve a court system, which involves a plaintiff and a defendant who must give full IDs to the judge. These groups do -not- want to have to submit themselves to the public record. Not that the media would cover the fact that half of them are 35 years old and don't live in Missouri, much less attend a university.