Fix your eyes on the black dot in the middle of the infrared image until the animated gif switches to black and white. The castle will immediately show its true colors.
So what is happening here to create the false color? You may have noticed that if you stare at a bright light and then look away, it will create a dark spot in your vision for a few seconds. Similarly, if you stare at a dark object on a bright white wall for several seconds and then close your eyes, you will see the reverse – the dark object will show up white. The image above is doing this same kind of thing with the color cells on your retina, oversaturating them with one color so you see the reverse once the color is gone. See How your eyes work for details.