I really find this right up there with being afraid of spiders (look at the scale of a spider compared to a human). If someone is yelling they are making a lot of noise, are usually highly agitated and emotionally involved with anger and/or frustration.
Ultimately, yelling is just noise. It really is. It is usually unreasonable due to the emotion. Yelling almost always means the whole issue needs to be put aside until the steam has been let off and reasonable, conscious thinking can be pursued.
Assuming the yelling is not followed up with physical violence, why are people so afraid of being yelled at? Why don't we just walk away when that starts? Even when the yeller is your boss or other authority figure, why do we get so distraught at what amounts to noise?