This guide covers rather basic user-level proxy setup.
But maybe you can also suggest some software like OpenVPN, which I can install on my own CentOS 7 VPS, which can later be used as a proxy, but not for all traffic from my Windows machine, but for browser only?
Preferably by enabling/disabling some browser extension or doing something similar.
I know about OpenVPN and similar tools, but they are usually designed to have all the traffic sent through them, while I need that only for a few sites, which can't be opened directly.