Good question. It has been discussed a bit here and there, but you're right that the conversation has mostly been dominated by Broadway.
As Hogan said, the likelihood of Off-Broadway theatres opening sooner is much higher than Broadway re-opening. Partly for the reasons Hogan said.
But it definitely definitely depends on each individual theatre. The Off-Broadway commercial houses may still struggle to make a profit with reduced capacity. Can a 200 seat house make money with only 60-or-so people in the audience? It would be difficult, but easier than Broadway. And some shows like Perfect Crime seemed perfectly content to scrape by for decades with tiny audiences, partly because of the situation with the theatre manager. But most of their audiences consisted of (unsuspecting) tourists, which brings it back to Hogan's point.
Non-profit Off-Broadway houses are a totally different question: they have more local audiences. Plus their model is already designed in such a way that they don't absolutely have to make a profit with every show they do. They have donors, sponsors, subscribers, etc. However, that doesn't mean they're in the clear. Who knows what kind of financial situation they may be in after this past year. Each theatre will be different, depending on whether they've managed to maintain loyalty in their subscriber base, whether they've been able to secure enough donations to keep them afloat, and what kind of expenses they've had to continue paying throughout the pandemic (are they still paying rent on their facilities? Are they still paying to maintain them? How many staff members are they still paying?) So after all that, the question is - will they be able to mount productions, and run them with only 60 people per audience? As Hogan said, we may see a few here and there, but it's not as simple as "will Off-Broadway re-open or not?"