THISThe dog runs away from you when it has a prize because it is afraid you would take it away. Instead, teach the dog to 'trade'. You would simply offer a more valuable prize in return for the inappropriate item.
You want to work on both of these cues at home in controlled scenarios. Start with a reasonably low-value 'prize' with your high value reward at first, and then increase the value of the prize. The reward for leaving the prize alone or trading the prize should always be more valuable than the prize. Until these cues are learned, the dog will have to be on a leash to prevent him from getting into garbage on the street.
Your dog is not "ready" to be off leash (many never are).
You want the dog to walk next to you like glue. Here is a question for you, Why should he? "Because you said so" is not an answer that is acceptable. What reaso do you give the dog for this behavior? Why is it rewarding to the dog.. so rewarding in fact that he would give up a bone to stick to you? Why would a dog prioritize staying next to your leg all the time on a walk? It is highly unnatural so how can you train it and make it rewarding?
Fact is, most dogs would not do this. Dependent dogs will (afraid to leave your side as you are their security) and dogs with huge, high pack drive will (because it is their nature) but that is not most dogs.