Unfortunately, this dog sounds like he needs to be under constant supervision in the back yard. You will not be able to leave your house and leave him in the backyard. When you can't supervise, your pup needs to be in the house, crated if need be, or in a completely secure outdoor kennel with shelter and water that he cannot dig out of or climb over. If the weather tends to be extremely hot or extremely cold, this probably won't be an option all the time.
Also, dogs don't generally get any exercise when left alone in a yard. They're not great at entertaining themselves out there. They get into trouble, as you've experienced! If you are unable to walk him due to physical restrictions, you might try playing games like fetch, or get a ball shooter like Patricia mentioned. You can teach him tricks, maybe do obstacles. You might also try hiring someone to take your dog for walks. Getting your pup mentally and physically tired will reduce his desire to escape. Pitties are very, very active dogs, and they need to be exercised so they don't get into trouble!
Likely, you being out there with him will eliminate his desire to escape, but if he still tries to get out you might invest in coyote rollers so he can't gain purchase on the top of the fence and pull himself over. Make sure he can't dig out, and if he tries you will need to bury chicken wire along the base of the fence. Remove things he can use as a "ladder" or "step stool" near the fence, like trees or piles of rocks or things like that.The least expensive option would be to keep him on a leash at all times when outside, but it sounds like he won't try to escape if you're out there with him, so hopefully you don't have to.