For the longest time, I had that problem myself, so much that I decided to go grayscale (like this blog) and focus only and exclusively on brightness and contrast. Contrast is the basic building block of UI design. Two pieces of information with the same contrast against their background will be perceived at the same importance level (all other things being equal), while more contrast identifies more importance and less contrast identifies less importance.

RGB is the result of the discovery that a lot of the colors that humans are capable of perceiving can be generated out of the combination of three light sources, one red, one blue and one green, at different intensities. This is how pretty much all screens and projectors work. Unfortunately, this is not how your brain works, it’s a technological artifact and it’s mostly due to the physiology of the color receptors in the human eye (and not to the physics of light itself!).

The Munsell color system is a color space that specifies colors based on three color dimensions, hue, value (lightness), and chroma (color purity or colorfulness).

https://colorusage.arc.nasa.gov/