The brain is divided into two halves (hemispheres)

Usually, the left half of the brain controls the right side of the body.

The right half of the brain controls the left side of the body.

your right brain tries to say the color and your left brain insist to say the word, thus, causes you to slow down the process.