It might be based on a point where plastic became possible to look nice and not be huge. Glasses have really only been fashion-possible in the last ... 10, 15 years, so before that you were really stuck with several variations of Ugly. So anyone before that point would be more likely to have a bias against anything that looks like the stuff they were stuck having before decent glasses became possible.

(That and I look TERRIBLE in thick frames, so I just automatically impose my personal fashion rules on everyone else on the planet.)