Here's what I do.
I look at each element of the creative and production and score them.
For a musical the book can get up to 20 points, the score can get 20 as well. (For non-musical the script gets up to 40 points) Then the concept and execution, meaning the idea and how well it achieves it goals, are evaluated to a maximum of 10 points. So, for the creative the maximum is 50 points.
Then I took at the performances by the leads, the supporting cast, ensemble and orchestra (20 points); the direction and staging (10 points); the design elements: lighting, costumes, set, sound (10 points) and the ultimate entertainment value (10 points.) The production maximum is 10 points.
If a newspaper requires a 5-star rating it is based on the following:
90+ points * * * * *
80-89 pts * * * *
70-79 pts * * *
60-69 pts * *
50-59 pts *
1/2 stars are given for being near the top of each segment: like 66 out of 100 gets * * 1/2. Anything less than 50 receives no stars
I have been using this, with some refinements for the past 20 years and it seems to work well. A poorly written show with strong production values like PHANTOM OF THE OPERA gets 66 points which translates to = * * 1/2 while a well written and performed show like RAGTIME would get * * * * 1/2. (VERY few ever get 5 stars!)
I usually have a general idea where a show should land before I do my score sheet and most of the time that estimate turns out to be correct.
Cast albums are NOT "soundtracks."
Live theatre does not use a "soundtrack." If it did, it wouldn't be live theatre!
I host a weekly one-hour radio program featuring cast album selections as well as songs by cabaret, jazz and theatre artists. The program, FRONT ROW CENTRE is heard Sundays 9 to 10 am and also Saturdays from 8 to 9 am (eastern times) on www.proudfm.com