What can I do about those sprouts under my oak tree?

There are almost never seedlings growing from acorns. If there were, you would be able to pull them up easily. What you see are sprouts from the roots of the existing tree; therefore, you do not want to spray a herbicide on them in an attempt to eliminate the sprouts for you will hurt the “mother” tree along with the sprouts.

Only a small percentage of oaks send up suckers from the roots. It is a genetic trait, like freckles, except I like freckles. But like freckles and sunshine, some trees have the ability to sucker, but do not unless stimulated to do so. Oaks having a slight tendency to sprout suckers will often do so when roots hit a barrier, such as trees confined to a parking lot planter, or between a sidewalk and driveway. Also, when roots are disturbed and damaged by rototilling, they are more likely to sprout suckers. But some trees never will make suckers. When choosing an oak in a garden center, if there are sprouts coming up at the inside edges of the container, I would avoid that tree. 

You may choose to mow them along with the grass, if grass still exists. Or if the grass has thinned too much, you might plant Asiatic Jasmine groundcover, and use hedge trimmers to trim the jasmine and oak sprouts to a uniform height. You can cover the area of sprouts with a heavy gauge woven geotextile, and then either mulch or spread large gravel or decomposed granite over the top of the geotextile. My favorite solution, when appropriate, is to cover the ground with geotextile and then build a wood deck. 

Or if you prefer a thick green lawn, you may remove the oak tree, and all of the tree roots with a backhoe. If you just cut down the tree, grind down the stump and all the large roots you can see, there will still be thousands of oak sprouts emerging from the remaining roots in your new lawn or bed area for a few years afterwards.  The area will need to be continually sprayed with an herbicide.

James