I’ve seen this a few times in shelter dogs. Prior to adoption, the dog may have been confined to an empty yard with little environmental stimulation. Since a puppy uses his mouth as an instrument to explore his world, if the only thing he had to chew on were rocks then that might explain a compulsion for them.
The first thing you need to do is to get a clean bill of health from your vet and then we need to focus on changing his behavior and his motivation for rock eating. The first order of business is to limit his access to rocks. Fencing off a small grassy area and combing the area with a garden rake should eliminate most if not all of the rocks. You can then use the grassy area as a safe training zone to help change his motivation.
Now the fun part! Take a variety of toys and simply play with him so he learns there are alternatives to rocks. Simple environmental enrichment can sometimes make them forget all about their old habits.
After a couple weeks, we need to see if it will be safe to allow him to have access to places where rocks are available. To do this, add a rock to the grassy area that is too large to pick up or swallow but still has sides where he can mouth it. If he makes a beeline for the rock, make sure he knows this is not acceptable behavior, but reward and praise him whenever he picks up a favorite toy instead.
He will gradually lose interest in the rock. If he needs extra encouragement, then spraying the rock with bitter apple or a similar noxious but non-toxic product can also help discourage the behavior during this period.