SACC

Insufficient lighting poses challenges to both human and machine visual analytics. While existing low-light enhancement methods prioritize human visual perception, they often neglect machine vision and high-level semantics. In this paper, we make pioneering efforts to build an illumination enhancement model for high-level vision. Drawing inspiration from camera response functions, our model could enhance images from the machine vision perspective despite being lightweight in architecture and simple in formulation. We also introduce two approaches that leverage knowledge from base enhancement curves and self-supervised pretext tasks to train for different downstream normal-to-low-light adaptation scenarios. Our proposed framework overcomes the limitations of existing algorithms without requiring access to labeled data in low-light conditions. It facilitates more effective illumination restoration and feature alignment, significantly improving the performance of downstream tasks in a plug-and-play manner. This research advances the field of low-light machine analytics and broadly applies to various high-level vision tasks, including classification, face detection, optical flow estimation, and video action recognition.

Left: Comparison with the baseline model (trained with normal light data only) and previous state-of-the-art on multiple downstream tasks.

Right: Example nighttime face detection results. Our approach better enhances faces hidden in darkness, resulting in more accurate detection.

Highlights:

We are the first to propose an illumination enhancement model for low-light high-level vision. Our model could enhance images from the machine vision perspective despite being lightweight in architecture and simple in formulation.
We train the enhancement model with base enhancement curves or pretext tasks to satisfy different downstream scenarios. Our training strategy can narrow the normal/low-light domain gap and improve the model's performance without annotated data. Besides, our framework serves as a plug-and-play remedy for multiple downstream tasks.
We evaluate our method on various high-level vision tasks, including classification, face detection, optical flow estimation, and video action recognition. Extensive experiments across multiple benchmarks demonstrate the superiority of our approach over state-of-the-art low-light enhancement and domain adaptation methods.

The architecture of the proposed deep concave curve, which intends to enhance the illumination of the input low-light image. We first predict the minus second-order derivative -∇²c and then integrate and normalize it into a concave curve g. Finally, we apply g to the input image I_L to obtain the enhanced image.

The training framework of self-aligned concave curve (SACC). When task information is available, we generate high-quality pseudo labels by assembling knowledge from a pre-defined curve family. When task information is unavailable, we first train a pretext head on normal-light data and then learn the deep concave curve on dark data with a fixed pretext head.

Experimental Results

Our proposed framework applies to various models and vision tasks. To justify its effectiveness, we evaluate it on several representative low-light vision tasks, including classification, face detection, optical flow estimation, and video action recognition. Please refer to our paper for full results.

BibTeX

@article{SACC_Journal,
  author       = {Wenjing Wang and
                  Rundong Luo and
                  Wenhan Yang and
                  Jiaying Liu},
  title        = {Unsupervised Illumination Adaptation for Low-Light Vision},
  journal      = {{IEEE} Transactions on Pattern Analysis and Machine Intelligence},
  year         = {2024},
}

If you have any questions, please contact Wenjing Wang (daooshee@pku.edu.cn).

Unsupervised Illumination Adaptation for Low-Light Vision

Abstract

Experimental Results

BibTeX