This is useful, from a law professor. The short answer seems to be: yes, unless you have a claim for accomodation (i.e., being permitted to work remotely) under the Americans with Disabilities Act:
For those who have not previously requested accommodations, they first must prove they are eligible. The Equal Employment Opportunity Commission has suggested that if employees have conditions that make them more susceptible to Covid-19, employers should take steps to eliminate the direct threat posed by Covid-19.
Faculty members who seek accommodations under the ADA must show that that they will be able to perform the essential functions of their job at home. When most colleges thrust faculty members into online teaching in March and continued to pay them, the colleges essentially agreed that faculty members are able to perform the essential functions of their jobs from home, at least during this pandemic. Colleges would be hard pressed to now argue that working from home is not a reasonable accommodation. Moreover, they have resisted tuition refunds for students this spring on the grounds that online education is not inferior to in-person classes....
Faculty members who do not qualify as disabled may still be entitled or even required to work from home. Under the ADA, a person is protected if they are “associated” with a person with a disability. Further, if they or a member of their household falls into one of the categories identified by the CDC as high risk for serious complications from Covid-19, they may be considered a direct threat under the ADA, and local public-health authorities would advise them to stay at home.
Faculty members with pre-existing medical conditions, including age-related conditions, also may be eligible to work from home. Since exposure to the virus may result in an ADA-required ”substantial limitation” in major life activities such as breathing, faculty who have a compromised immune system, chronic health condition, or other medical condition, should inquire whether they may be permitted to work from home as a reasonable accommodation in accordance with the EEOC guidance.