Video super-resolution differs from single-image super-resolution (SISR) because each output frame must be reconstructed from several related but spatially misaligned inputs. Optical-flow methods can introduce artefacts when motion is estimated incorrectly, while three-dimensional convolution and recurrent convolutional neural networks (RCNNs) can require heavy computation or struggle to maintain long-term dependencies. Deformable alignment provides a more flexible alternative, but many existing systems use only one or a few standard convolutions to predict motion offsets. This can restrict the receptive field andweaken alignment accuracy when objects undergo rapid motion or complex trajectories across neighboring frames. These challenges motivate further investigation into effective alignment strategies that can improve the estimation of motion offsets while avoiding excessive model complexity and training difficulty in video super-resolution (VSR) networks.
This study was published (DOI: 10.1049/cit2.70151) online on May 30, 2026, in CAAI Transactions on Intelligence Technology. The research was conducted by authors affiliated with Konka Group Co., Ltd.; Tsinghua Shenzhen International Graduate School, Tsinghua University; the School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen; the Department of Computer Science, City University of Hong Kong; the College of Artificial Intelligence, Chongqing University of Technology; and the Shenzhen Institute of Future Media Technology. The team proposed effective spatio-temporal alignment network (ESTA-Net) to improve inter-frame alignment, feature fusion, and ×4 video super-resolution reconstruction.
The researchers trained ESTA-Net on 64,612 seven-frame sequences from Vimeo-90K and evaluated its performance on three benchmark datasets, including Vid4, Vimeo-90K-T (the Vimeo-90K testing set), andREDS4. Each sequence is processed around a middle reference frame. A group convolution-driven bi-scale alignment module (GCBAM) first estimates motion offsets at both the original and half feature resolutions. Specifically, it employs six cascaded group convolutions with channel shuffle to enlarge the receptive field while limiting parameter growth, followed by deformable convolution to align neighboring features with the reference feature. Then, the aligned outputs from both scales are dynamicallyfused. After that, an attention-based feature enhancement module (AFEM), containing 10 attention-based feature enhancement blocks, uses efficient channel attention (ECA) to highlightkey channels carrying useful textures and structural information. Finally, pixel-shuffle layers finally enlarge the reconstructed frame by a factor of four.Benefiting from these designs, ESTA-Net achieved PSNR (peak signal-to-noise ratio)/SSIM (structural similarity index) scores of 26.83 dB/0.8073 on Vid4, 36.69 dB/0.9407 on Vimeo-90K-T, and 29.12 dB/0.8365 on REDS4. The ablation studies further showed that jointly incorporating GCBAM and AFEM improved the PSNR on Vid4 from 26.52 dB to 26.83 dB, thereby validating the effectiveness of the key components in ESTA-Net.
The authors said the central aim was to improve motion alignment without relying on a much larger parameter budget. By using grouped operations, they said, the offset-prediction network could look across a wider spatial area and better capture complex motion patterns. The bi-scale design was intended to strengthen alignment when motion becomes large, while channel attention helped retain fine information after features from several frames were combined. They added that the clearest gains appeared in challenging visual regions, including small facial features, road signs, building textures, foliage, aircraft markings, and compression-damaged lines.
The results indicate that better temporal alignment can improve video clarity without requiring an extremely large model. ESTA-Net may support video enhancement in safety monitoring, high-definition imaging devices, compressed-media restoration, and other systems where fine structures must remain recognisable. Tests on the real-world VideoLQ dataset also showed clearer aircraft markings and fewer compression artefacts in selected scenes, although this evaluation was qualitative rather than a broad quantitative assessment. The model requires 907.43 billion floating-point operations (FLOPs) for a 1280 × 720 high-resolution frame, suggesting that further optimizationis still needed before it can be considered suitable for lightweight, real-time, or edge deployment. Future studies could test more varied real-world degradations and develop faster implementations while preserving alignment accuracy.
###
References
DOI
10.1049/cit2.70151
Original Source URL
https://doi.org/10.1049/cit2.70151
Funding Information
This study was supported by the Establishment of Key Laboratory of Shenzhen Science and Technology Innovation Committee under Grant No. ZDSYS20190902093015527 and the Shenzhen Science and Technology Innovation Committee under Grant No. JSGG20220831104402004.
About CAAI Transactions on Intelligence Technology
CAAI Transactions on Intelligence Technology is a leading venue for original research on the theoretical and experimental aspects of artificial intelligence technology. We are a fully open access journal co-published by the Institution of Engineering and Technology (IET) and the Chinese Association for Artificial Intelligence (CAAI) providing research which is openly accessible to read and share worldwide.