Unfortunately, whether he'll try to swallow the gravel - or chew on it hard enough to break his teeth - is going to be completely dependent on the individual dog. I would be extra careful, as Labs in particular are known for swallowing foreign objects. Puppies do explore the world with their mouths, so it's fairly normal behavior, but that doesn't mean it's safe in this instance.
You could try either raking up the gravel to make a potty space, or putting down something like a square of sod to help prevent the behavior (take him out on leash so he can't leave the area, that makes it super easy to reward when he potties outside anyway). Ideally you do need to prevent it, because every time he does it, there's a risk of him making a bad decision (like swallowing the rock), and it creates a habit that may become harder to break with time. Additionally, he may find you trying to get it out of his mouth a fun 'game' that gets him lots of attention, which will accidentally reinforce the behavior. With luck, he'll be less interested as he matures and start ignoring the gravel.
In the meantime, work on trading games with his toys and low value chews - offer him a higher value treat, take the toy/chew while he eats it, then give the toy/chew back. Practice often and work up to higher value items, so that he learns trading is fun and that 95-99% of the time, he gets the original item back. Then it will be much easier to convince him to trade in those incidents when you can't give him back whatever thing he's picked up that might be dangerous/expensive/gross/etc. But prevention is still key for the gravel thing to keep it from becoming a long-term problem.
Incidentally, I made the mistake of making it very fun for my younger dog to pick up gravel. He's two and will still occasionally bring a piece inside when he feels he isn't getting enough attention. We know by now that he'll just carry it around, stare at us, and drop it repeatedly and loudly to try to get us to engage, so I'm not too worried he'll actually chew/swallow a piece. But if I'd focused more on preventing the behavior rather than trading for treats/attention, he likely would never have created the 'game' in the first place, and I wouldn't have to worry at all.