No problem as long as the milk doesn't give him diarrhea. Young puppies take more naturally to a soft, squishy, milky textured food, but just watch his bowels and if he does get loose you should either cut back dramatically on the milk, or altogether. If you stop the milk, you might want to add warm water to his kibble to soften it, and hopefully he will eat that up as well.
What food are you currently feeding? Is it the same as from the breeder? We feed our lab Costco's Kirkland dog food - she loves it (but she's not picky anyway). Many lab owners like Kirkland, Diamond brand, or Chicken Lover's. You want a better/top quality food - not the cheap grocery foods like Purina, Ol Roy, etc.
Look for one that has more meat based rather then all grains.