Not at all, you will see people dressed very nicely and more casual. I am probably old school and wear jacket tie and slacks for most shows. Sometimes I will not wear tie especially in summer months. I would also advise maybe bringing a sweater in case you get cold in the theater.
Like NYadgal, I like that you asked. So many people nowadays, especially men, don't give any thought at all to how they look. I'm an old-timer and while I think it's fine to wear whatever is comfortable, I admit that people going so far as to show up in hoodies or shorts get my silent disapproval--I can't help it.
But I really only wanted to add that if, like me, you actually feel MORE comfortable dressing a little bit (as a mark of respect to the occasion and the performers, and also because it looks nice and feels good!), then you might also like to know that in summertime you'll probably still feel comfortable wearing just "nice casual." I usually bring a jacket anyway, but that's mostly because of the A/C that others have commented on.
This is such a great question with a lot of great answers. At risk of beating a dead horse, I'll add my experience. Typically, when I take my wife to a show we like to dress up. Of course since we don't live in NYC and given the cost of face-value show tickets, it's usually a special night out so we're going to be more dressed up anyway. That said, I've gone to shows in a full jacket, tie and slacks outfit and felt totally over dressed compared to most. Seems a (for lack of a better term) business casual is more the norm (khakis and a polo or button down on guys, nice pants or a skirt and simple blouse on the women). There are also plenty of folks that show up in jeans and t-shirts and as noted the occasional group in clothes that are even more casual (workout or running gear for instance). I may be more old school than my age suggests, but I just feel the audience shows a certain level of respect and appreciation for the show, it's cast, it's director, etc. through the way you dress.
That said, as I've posted in another thread, my wife and I are going to Kinky Boots on our anniversary and are both planning to wear a pair of red patent leather thigh high boots so not sure what that tells you :)