So very generally speaking, what behaviors do you think are treatable, manageable, or are such that you believe a dog with X behaviors deserves a chance to be owned, or rehomed, or placed through a shelter?
I think you and DaySleepers covered it pretty well. It varies a lot.
What you said reminded me of my previous dog, Fiona the American Bulldog. She was a hard, sharp, guard dog. Surprisingly dog social, but hated most human strangers. It's how I was able to have a revolving door of fosters while I had her. She wouldn't hesitate to escalate if things got dicey, with both dogs and people. She did, multiple times actually, and those people deserved it, but those are stories for another day. I was able foster fail her before she hit the adoption floor after talking about it with the shelter director, because I knew the more she bonded with me, she wasn't going to bond with others the same way. She was cordial but never really bonded with my SO; if he hurt me, she wouldn't hesitate to protect me. Owning her wasn't really hard or mentally taxing for me, despite her being a liability, because I come from a country where most dogs, both bought and adopted, are actually closer to that temperament then, say, the temperament of a more social breed. My SO didn't really have any hard feelings about that either, because he has a similar background (he's from Turkey and grew up around LGDs). I also just don't really care if my dogs like strangers or not, just don't act a fool in public.
So, all in all, if someone wants to step up and take initiative for a dog like that, I think rescues and shelters should be open to that. Of course, if they feel euth is a better option for a dog like that, I'm not against that either.
And what behaviors do you think make euthanasia the most humane, ethical, or responsible outcome for a dog?
In general, I would say any behavioral problem that is not coupled with a medical problem. Of course, like Daysleepers said, there's a lot of factors coupled with that.
Outside of that, I think it will depend on the shelter and rescue for the most part.
About "long termers"...If a rescue or shelter is able to adopt out a good number of dogs, and work to lower its return rate, while keeping the "long termers" humanely, I'm not against that. If a rescue or shelter is unable to at the moment, but is working to lower its euth rate through other means like retention programs and transport, I'm not against them euthing dogs they deem appropriate for that. Slight tangent, it would depend why the dog is a "long termer". If it is because the rescue or shelter is keeping the dog alive, when euth is more appropriate, then that's not right. If it is because the dog is not getting enough attention despite being pretty adoptable, then I would find it hard to say they should euth. The shelter across town has a "longer termer" right now actually; it's a 40 ish pound, wooly coat, natural bobtail (or just bad dock job), black mutt thing who splits his time as an office dog and with his foster dad. That's not a bad life. He's been at that shelter for a little over a year. If I had to pick his problem behavior, he's a little reactive, but that's common with dogs in general. Even the generally slow to go dogs like pits at that shelter were getting adopted before him. In that scenario, I think the best course course of action would be to give dogs like that more publicity.