Style prompt tuning for bridging visual gaps in autonomous driving

초록

Artificial intelligence models for semantic segmentation and image classification in autonomous driving must maintain reliability across adverse conditions such as rain, fog, snow, and nighttime. However, models trained on only clear daytime images often fail to generalize under such domain shifts. Existing unsupervised domain adaptation (UDA) methods employ image-level style transfer using generative adversarial networks (GANs) or diffusion models, which necessitate paired data and risk altering content. Therefore, this study proposes Style Prompt Tuning, a novel UDA framework that utilizes image–text models to automatically generate and optimize textual prompts representing target-domain styles. These prompts guide a U-Net-based style network to synthesize source images in the target style while preserving their semantic content. Our approach employs clustering within the Contrastive Language—Image Pretraining (CLIP) embedding space and a composite loss function, including content, style, transfer, patch, and total variation terms to optimize prompt quality. The generated stylized images augment the source dataset and are used to train more robust task models. Experiments on semantic segmentation benchmarks (Cityscapes-to-Adverse Conditions Dataset with Correspondences (ACDC), DarkZurich, BDD100k-night, and Nighttime Driving) and image classification (Visual Domain Adaptation 2017) reveal our approach to achieve improvements of +3.4 mean intersection-over-union (mIoU) and +0.8% accuracy over prior UDA methods. These results highlight our method's practical effectiveness for real-world autonomous driving applications under visually challenging scenarios.

키워드