Your son’s PC more expensive than mine, and I do quite a bit of gaming. Get a cheap one you can upgrade; it’ll give him small goals as he saves toward an upgrade, and he’ll most likely look after it better. After spending time together working on it and him saving.
Just spend the money on motherboard and power supply. Throw an older GPU in, and a low spec CPU, to get him by. Can give some pointers I’m learning about it, but I am some sort of enthusiast. Overall, it’ll cost you slightly more this way. But knowledge is invaluable: for both financial and skill set. It’ll also be much cheaper than $4,000.
Find out what games he likes and you can build something toward that goal. That way he can still play, or pay to upgrade, and enjoy the improvements he makes whilst becoming tech savvy.
If not and you just want to do it once, and still be upgradable for the near future do your research on CPUs and GPUs. Find out what you’lll need to achieve what your son wants off the bat. $2,000–$2,500 should cover it. Unless you want to buy a laptop there’s plenty of options.