It's actually pretty common to not bond to a new baby puppy right away! They need a lot of time and energy and can be a big change to the household, but also are still in the process of developing their individual personality and figuring out the world. It can take a while before things 'click' and you really start to understand each other. I know a lot of dog people who don't like puppies at all and always try to bring home an older pup/adult dog, or even hire a puppy raiser.
That being said, if you really feel like this pup isn't right for you or is negatively impacting your mental health, there's no shame in returning him to the breeder. Sometimes things don't work out, and it's better for everyone that the puppy find a home that's a better fit instead of trying to hang on to the point where you resent the dog or are so stressed by him that it's impacting both of your quality of life
Personally, I'd give it a little more time and try not to put a lot of pressure on immediately feeling that bond right now, because very often that bond does come, but you know the situation best.