What subjects are "off limits" that you would not consider publishing, no matter how well written?This is going to vary from publisher to publisher. However, in the LDS industry, there are some basic standards--for example, most are not going to publish books that celebrate or glorify lifestyle choices contrary to the doctrine of the Church. Most will not publish books that bash Church leaders or policies. Most will not publish novels with graphic sex or violence. Most will not consider books on the occult.
After that, you're looking at individual preferences. Some won't touch novels with polygamy in any form; others don't mind it in historical novels. Many won't publish "contemporary" topics (addiction, unwed pregnancy, homosexuality) in any form; others will, if it's done tastefully and shows the consequences of poor choices.
I won't accept anything that I think will upset or tick off the average LDS reader, even if I think it's well written or it's something that I personally like. For example, I received a submission a few years ago about an addict who turned their life around. I thought it was well written, had a great message, and that some LDS people would be touched by it. But I rejected it because too many people would be upset by its grittiness and I cannot afford to offend my readers. Other taboo topics at my company include homosexuality, child abuse, incest, the occult, gratuitous violence, descriptive intimacy, murder of children or real-time description of the death of children. Topics that would raise a flag, but might not be an automatic rejection are addiction, spouse abuse, infidelity, unwed pregnancy, loss of testimony.