Real-Time Multimodal Abstractive Summarization

A real-time multimodal abstractive summarisation system.

A real-time multimodal abstractive summarisation system. It is offered as a reference project: a complete, worked example of how a system like this is designed, built and evaluated, so that a developer can study it, run it and learn from it.

It suits learners who want to work with language models and text or speech: preparing text data, using or fine-tuning a model, evaluating it and building an interface around it.

The main parts are real-time processing of incoming data, fusion of more than one type of input, text or speech data preparation, model training, fine-tuning or prompting, and evaluation with task-appropriate metrics. The reference build uses Apache Kafka, Python, Hugging Face Transformers, spaCy, and PyTorch, with a clear project structure, setup instructions and sample data so that it can be run and extended.