Why does increasing graphic settings also increase fps?

I bought a week ago a used PC with FX8320/RX580 4GB/16GB RAM. Don't comment on me buying an FX in 2024,it was a great deal and it can easily run what I want it to run.

At medium settings, and 1600p, I was getting around 30fps on some of my more demanding games. Also on some, quite a few stutters. Just messing around yesterday, I decided to max out all settings on a game, just to see what would happen. Low and behold, the game started running with no stutters at 60FPS (my cap). I tested it out on the other games as well, on all of them same case, maxing out all graphics delivered better performance.

How come this happens? From my understanding, my CPU may be bottlenecking my GPU, and increasing graphical demands shifts the load to the GPU. But why in the first place is the CPU loaded with graphical demands? Shouldn't the CPU handle computing demands, and the GPU all graphical?