Colorful Image Colorization

The paper attacks the problem of hallucinating a plausible color version of a grayscale photograph, an inherently underconstrained task where earlier approaches relied on significant user interaction or produced desaturated colorizations. The authors propose a fully automatic method that produces vibrant, realistic results by embracing the problem's uncertainty: they pose colorization as a classification task and use class rebalancing at training time to increase the diversity of colors in the output. The system runs as a single feed-forward pass through a CNN at test time and is trained on over a million color images.

The method is evaluated with a colorization Turing test in which human participants choose between a generated and a ground-truth image; it successfully fools humans on 32% of trials, significantly higher than previous methods. Beyond generation quality, the authors show colorization is a powerful pretext task for self-supervised feature learning, acting as a cross-channel encoder, and this yields state-of-the-art performance on several feature-learning benchmarks. The dual contribution of realistic automatic colorization and effective self-supervision made it influential in both graphics and representation learning.