Baby teeth falling out. Dogs typically lose their baby teeth at this age. Their gums are sore while this is going on. In IPO training we don't do bite work, even on a rag, while puppies are teething. We might play a little with them when they are 12-16 weeks old, then we stop until they are 8 months old. Even then, because there is no rush, we often wait a bit.
Give him raw frozen meaty bones to chew on. The frozen will make his gums feel better and the chewing will get rid of the baby teeth. If the loose baby tooth does not come out in another month, go to the vet (sometimes they can get stuck but that is pretty rare).