UTAR Institutional Repository

A semantic-aware video auto-captioning method

Tin, Ley Ter (2020) A semantic-aware video auto-captioning method. Final Year Project, UTAR.

[img]
Preview
PDF
Download (3026Kb) | Preview

    Abstract

    Video Description is a hot field of research nowadays. However, most of the approach lacks of semantic context to formulate their description. To be able to achieve such fine level of detail, most deep learning methods require astonishing amount of data on top of a good neural network design. This project explores and introduces a method to extract semantical information from video, and then generates description elevate on the semantic context. Rather an end-to-end network, this project aims at designing a pipeline of entity resolution for video captioning. The pipeline explores at the possibility of combining different state-of-the-art models which specialize in different field of information extraction. Then the semantically rich information will then be orchestrated into sentence that describes the video. On top of that, this project will design an entity trimming method based on linguistic semantics to generate a less ambiguous and more meaningful description. The method will be evaluated on popular benchmarking standards to compare its performance against another existing model. The project achieved BLEU score of 48.01 and METEOR score of 32.80 on MSVD dataset.

    Item Type: Final Year Project / Dissertation / Thesis (Final Year Project)
    Subjects: Q Science > Q Science (General)
    Divisions: Faculty of Information and Communication Technology > Bachelor of Computer Science (Hons)
    Depositing User: ML Main Library
    Date Deposited: 07 Jan 2021 15:50
    Last Modified: 07 Jan 2021 15:50
    URI: http://eprints.utar.edu.my/id/eprint/3938

    Actions (login required)

    View Item