ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP),

Lighting Image/Video Style Transfer Methods by Iterative Channel Pruning

K. Wu, F. Tang, N. Liu, O. Deussen, T. Le, W. Dong, T. Lee

Material

Abstract

Deploying style transfer methods on resource-constrained devices is challenging, which limits their real-world applicability. To tackle this issue, we propose using pruning techniques to accelerate various visual style transfer methods. We argue that typical pruning methods may not be well-suited for style transfer methods and present an iterative correlation-based channel pruning (ICCP) strategy for encoder-transform-decoder-based image/video style transfer models. The correlation-based channel regularization preserves the feature distributions for content and style references, and the iterative pruning strategy prevents layer collapse when pruning on the encoder-decoder structure. Experiments demonstrate that the proposed ICCP can generate visual competitive results compared to SOTA style transfer methods and significantly reduces the number of parameters (at least 70K) and inference time. Model is available at https://github.com/wukx-wukx/ICCP

BibTeX

@inproceedings{Wu2024LightingImageVideoStyle,
  author    = {K. Wu, F. Tang, N. Liu, O. Deussen, T. Le, W. Dong, T. Lee},
  booktitle = {ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  doi       = {10.1109/ICASSP48485.2024.10446950},
  keywords  = {Training;Visualization;Correlation;Computational modeling;Lighting;Signal processing;Acoustics;visual style transfer;model pruning},
  pages     = {3800-3804},
  title     = {Lighting Image/Video Style Transfer Methods by Iterative Channel Pruning},
  year      = {2024}
}