I think we should use this tag only for questions about musicians in general.
For questions about specific musicians, tags about the involved musicians should be added.
Following the musicians tag doesn’t make much sense if there are all these questions about specific musicians (that you might not even know) included. There can be no expert in all specific musicians.
A tag about a specific musician would include questions about this musician and questions about the works of this musician, both of which are related fields: an expert in one of these areas is likely also an expert (or versed) in the other one; i.e., a fan often knows both, the musician and the music, and so it makes sense to follow this tag.
The same is currently the case for genre, which should not be used for questions about specific genres.
So I think the excerpt should be (wording improvements welcome!):
For general questions about musicians. Do not use for questions about specific musicians.
If we agree, I think album, bands, soundtrack, and cover should be handled in the same way.