Prediction of Lung Diseases Using Multimodal Deep Learning on Chest X-Rays, CT Scans and Patient Reports

A multimodal deep learning system that predicts lung diseases from chest X-rays, CT scans and patient reports.

A multimodal deep learning system that predicts lung diseases from chest X-rays, CT scans and patient reports. This reference project walks through the whole job, from the problem and the design to the build and the evaluation, so that a developer can study how it works, run it and extend it.

It suits learners who want to work with neural networks end to end: preparing datasets, designing and training a model, tuning it, evaluating it properly and explaining the outcome.

The main parts are fusion of more than one type of input, dataset preparation and augmentation, model design and training, evaluation with standard metrics, plus error analysis, and a simple interface or API to try the model. The reference build uses Python, PyTorch, TensorFlow, and NumPy, with a clear project structure, setup instructions and sample data so that it can be run and extended.