2025 / 06 / 13

A Hybrid Vision Transformer-CNN Network for Multi-Class Severity Classification of Ulcerative Colitis

1100657 Yi-Hsuan Lee, 1100719 Hsiang-Lin Chen

Background

Ulcerative colitis (UC) is a chronic, relapsing inflammatory bowel disease that affects the colon and significantly impacts patients' quality of life.

Disease severity is typically graded using the Mayo Endoscopic Score (MES), based on colonoscopy images; however, manual grading is subjective and prone to variability.
Deep learning-based methods offer the potential to provide more objective, consistent, and efficient assessment of UC severity in clinical practice.

Related Work

Original references retained from the source presentation.

Stidham et al. (2019) applied Inception-v3 for MES classification, achieving expert-level accuracy.
Sutton et al. (2022) evaluated various CNNs and found DenseNet121 effective for UC classification.
Polat et al. (2022) introduced Class Distance Weighted Cross-Entropy (CDW-CE) loss, improving UC classification performance.
Maurício et al. (2024) demonstrated that Visual Transformer(ViT) models offer improved interpretability in UC severity assessment.
Shah et al. (2024) proposed a hybrid CNN-ViT architecture, combining local and global feature extraction.

Dataset

Training - 70% / Validation - 20% / Testing - 10%

70%Training

20%Validation

10%Testing

Dataset

Training - 70% / Validation - 20% / Testing - 10%

Original PPT image

Visual Transformer (ViT)

Original PPT image

ConvNeXt

Original PPT image

DenseNet

Dense Block

DenseNet reference material is preserved from the original slide.

Image source: https://arxiv.org/pdf/1608.06993 (Paper: Densely Connected Convolutional Networks)

Original PPT image

Methodology

Original PPT image

Methodology

Original PPT image

Methodology

Original PPT image

Evaluation Metrices

Original PPT image

Evaluation Metrices

Original PPT image

Evaluation Metrices

Original PPT image

Experiment Setup

Batch size - 16 / Epoch - 80 / Patience - 7 / Seed - 456

Batch size
16

Epoch
80

Patience
7

Seed
456

Learning rate
5e-5

Loss
Cross Entropy

Optimizer
AdamW

LR Scheduler
ReduceLROnPlateau (patience = 10, factor = 0.2)

Pre-processing
Resize (224*224), Normalize (Training + Validation + Testing)

Setup
RandomHorizontalFlip, RandomRoatation, ColorJitter (Training)

Result

Original PPT image

Result

Original PPT image

Result

Original PPT image

Result

Original PPT image

Result

Original PPT image

Result

Original PPT image

Result

Original PPT image

Result

Original PPT image

2025/06/13

Conclusion

Thank you for your listening!