Integration of Natural Language Processing and Computer Vision for Advanced Multimodal Intelligent Systems: A Systematic Review of Architectures, Benchmarks, and Applications

Arnab K. Dutta, Sreyashi P. Bhattacharya, Rajdeep S. Biswas, Mousumi D. Halder

Abstract


ABSTRACT
Natural Language Processing (NLP) and Computer Vision (CV) have long evolved as separate disciplines, yet most real-world intelligent systems must interpret images and language together. Recent advances in Transformer architectures, contrastive pre-training, and large language models have enabled a rapid convergence of the two fields into multimodal intelligent systems capable of visual question answering, captioning, retrieval, grounding, and instruction-following dialogue. This review-based study synthesises 30 peer-reviewed and widely cited publications spanning 2015–2024 and organises them through a four-dimensional analytical framework: fusion strategy, pre-training objective, data scale, and downstream performance. A taxonomy of five fusion paradigms is proposed, and benchmark results reported by the original authors are compared on a like-for-like basis. The synthesis shows that VQAv2 test-dev accuracy rose from 65.32% for region-based attention models to 82.0% for few-shot Flamingo-80B, a gain of 16.68 percentage points, while contrastive models reach roughly 76% zero-shot ImageNet accuracy without task-specific labels. Parameter-efficient bridging of frozen encoders and language models emerges as the dominant design trend, whereas object hallucination and weak college-level reasoning remain unresolved reliability barriers. Application mapping across healthcare, assistive technology, autonomous mobility, e-commerce, and education identifies deployment constraints, and a research agenda for low-resource, multilingual, and edge-deployable multimodal systems is outlined.

KEYWORDS: Multimodal learning, Natural language processing, Computer vision, Vision-language models, Transformers, Contrastive pre-training, Visual question answering, Large language models, Image captioning, Hallucination

Full Text:

PDF 174-188