Efficient video summarization with hydra attentive vision transformer

Abstract

A video summary’s objective is to produce succinct and condensed synopses that accurately depict the content of the original video while sacrificing none of its vital features. Effective deep summarization models have arisen in the field of video summarizing, made feasible by the advancement of gated recursive unit (GRU) and long and short-term memory (LSTM) technologies. However, if the video is very long, GRU and LSTM models are unlikely to capture long-term dependencies as well as they could otherwise. In recent years, significant progress in the field of supervised video summarization has been accomplished through the use of techniques involving sequence-to-sequence learning. However, it is important to remember that traditional recurrent neural networks (RNNs) have some limitations when it comes to modeling long sequences and that using transformers to represent sequences necessitates a large number of input parameters, resulting in increased computational complexity. We present a new video summarizing methodology that addresses the aforementioned issues by utilizing a Hydra Attention-based Vision Transformer framework. The suggested method captures long-term dependencies and extracts significant characteristics from video sequences well. The proposed architecture improves the accuracy and reduces the processing time of video summaries by using the capabilities of Hydra Attention and transformer-based sequence modeling. Our solution outperforms state-of-the-art approaches in terms of performance and computing economy in empirical evaluations of the SumMe and TVSum datasets.