Apparatus and method for video representation learning

초록

An apparatus for video representation learning according to an embodiment may extract video features from video data to generate a video embedding, extract image features from image data extracted from the video data to generate an image embedding, and extract audio features from audio data extracted from the video data to generate an audio embedding. Further, contrastive learning may be performed by generating a first compositional embedding based on the video embedding and the audio embedding, generating a second compositional embedding based on the video embedding and the audio embedding, generating a positive sample and a negative sample based on a correlation between the image embedding and the audio embedding, and then using the data.