Colonial Korea

• Colonial Korea (1910-1945)
• 1910s no free press
• March 1st Protests incited change
• More liberal press policies in the colony

The 1920s

• During the 1920s, choice of 100-200 different print shops.¹
• Choice influenced outcome.
• A good example are poems from Kim So-wŏl’s Chindallaekkot 진달래꽃 collected works.

(a) Hansong Toso 漢城圖書 issue of Chindallaekkot (collected works)

(b) Chungang Sorim 中央書林 issue of Chindallaekkot (collected works)

(c) Haksaenggye 學生界 (July 1920) issue of the poem.

(d) Kaebyok 開闢 (August 1922) issue of the poem.

Figure 1: Some Day Long From Now 먼 후일 poem from various print runs

1. depending on the year sampled. See De Fremery, “How Poetry Mattered in 1920s Korea,” 102 for more details.

The 1920s

• During the 1920s, choice of 100-200 different print shops¹
• Choice influenced outcome
• A good example are poems from Kim So-wŏl’s Chindallaekkot 진달래꽃 collected works

(a) Hansong Toso 漢城圖書 issue of Chindallaekkot (collected works)

(b) Chungang Sorim 中央書林 issue of Chindallaekkot* (collected works)

(c) Haksaenggye 學生界 (July 1920) issue of the poem.

(d) Kaebyok 開闢 (August 1922) issue of the poem.

Figure 2: Some Day Long From Now 먼 후일 poem from various print runs.

1. depending on the year sampled. See De Fremery, 102 for more details.

Characteristics?

• Work done by De Fremery¹

(a) Hansong Toso Chusik Hoeisa printed Tang 당

(b) Taedong Inswaeso printed Tang 당

(c) Hansong Toso Chusik Hoeisa printed Chŏnyŏk 저녁

(d) Taedong Inswaeso printed Chŏnyŏk 저녁

Figure 3: Examples of subtle typographic differences. Notice the angle of the stroke on Tigŭt ㄷ and the differences in the Chiŭt ㅈ

1. De Fremery, 131, p.595.

Research

Can neural networks be used to classify colonial Korean print shops and identify the specific visual features that distinguish their typographic styles?

Interpretability

• All studies aim to detect, but how a model detects is often neglected
  • For CNNs and Vision Transformers, several interpretability methods have proven successful
  1. GradCAM and derivates¹
  2. SHAP (SHapley Additive exPlanations)²

(a) Example of GradCAM

(b) Example of SHAP

Figure 4: Interpretability methods visualized

1. Selvaraju et al., “Grad-CAM”.

2. Lundberg and Lee, “A Unified Approach to Interpreting Model Predictions”.

Dataset

• The dataset contains 66,277 pages from four major printshops.
  
• Distribution: Taedong Inswaeso (42%), Hansŏng Toso Chusik Hoeisa (29%), Sinmungwan (20%), and Chosŏn Inswae Chusik Hoeisa (9%).
• Class imbalance reflects real-world production volumes; no rebalancing was applied.
  

Results Approach 1

ConvNext Base architecture - 98% Accuracy (F1=0.98)

Can neural networks be used to classify historical printshops and identify the specific visual features that distinguish their typographic styles?

• Tested various models (Resnet18/152, ConvNext Base, Swin S3 Base)

• 30 Epochs

• Focus on ‘margins’

• Finding in itself

• Curious to more

Approach 2

Following idea of Seuret et al.¹ a page is cut into 4 random cutouts, while reducing overlap to max 30%

1. “Dataset of Pages from Early Printed Books with Multiple Font Groups”.

Approach 2 Results

99.8% Accuracy (F1=0.99) Swin S3 Base-224

Can neural networks be used to classify historical printshops and identify the specific visual features that distinguish their typographic styles?

• Tested various models (Resnet18/152, ConvNext Base, Swin S3 Base)

• 30 Epochs

• Focus on ‘near random’

• Likely overfit?

• Significant augmentation or lower epoch did not result more interpretable features.

MIL

• Multi Instance Learning.¹
• Used in the field of medical imagery.²
• Similar issues faced by humanists:
  • High classification score
  • Explainable classification
• We follow the AttriMIL implementation of Cai et al.³

Figure 5: MIL visualization.⁴

MIL:

1. Image > Bag of Instances
2. Instance > Feature Extractor (Resnet, or Vision Transformer)
3. Features > Attention based aggregator
4. Aggregator output > Classifier

Can extract attention per patch

ViT:

1. Input > Patches
2. Patches > Linear Embedding (adding [CLS] Token Embedding)
3. Positional Encodings added
4. Through Transformer encoders
5. Classificiation head

ViT’s attention: Learn internal representations by understanding relationships between all parts of a single image. It’s for feature aggregation within one image.

MIL’s attention: Solve the weak supervision problem by explicitly weighting the contribution of distinct instances (which are individual patches) to infer a bag-level label, knowing that only a few instances might be responsible for a positive bag label.

1. Maron and Lozano-Pérez, “A Framework for Multiple-Instance Learning”.

2. Papadopoulos, Topouzis, and Delopoulos, “An Interpretable Multiple-Instance Approach for the Detection of Referable Diabetic Retinopathy from Fundus Images”; Javed et al., “Additive MIL”; Yang et al., “HAMIL”; Deng et al., “Cross-Scale Multi-Instance Learning for Pathological Image Diagnosis”; Waqas et al., “Exploring Multiple Instance Learning (MIL)”; Gadermayr and Tschuchnig, “Multiple Instance Learning for Digital Pathology”.

3. “Rethinking Attention-Based Multiple Instance Learning for Whole-Slide Pathological Image Classification”.

4. Cai et al.

MIL Figure

Figure 6

MIL Figure

Figure 7

Embeddings Space

Figure 8: UMAP Visualisation of MIL Embeddings

Embeddings Space

Figure 9: UMAP Visualisation of MIL Embeddings

Sampling clusters

(a) Cluster3

(b) Cluster 6

(c) Cluster 8

Figure 10

Compared

(a) Chosŏn Inswae Chusik Hoeisa

(b) Hansong Toso Chusik Hoeisa

(c) Taedong Inswaeso

(d) Sinmungwan

Figure 11: Sharp Stroke of ㄷ Tigŭt as feature for Taedong Inswaeso / Sinmungwan. Also prevalent in ㄴ Nieun and ㄹ Lieul

Shifts in Features

Figure 12: Cluster Shifts

Further

• Bags of Patches as feature, not singular patch
• Improvements on Clustering
• Move to typology of printshop

Thank you

Please Reach out! a.m.van.de.pol@hum.leidenuniv.nl

Referenced Works

Cai, Linghan, Shenjin Huang, Ye Zhang, Jinpeng Lu, and Yongbing Zhang. “Rethinking Attention-Based Multiple Instance Learning for Whole-Slide Pathological Image Classification: An Instance Attribute Viewpoint.” arXiv, March 2024. https://arxiv.org/abs/2404.00351.

De Fremery, Peter Wayne. “How Poetry Mattered in 1920s Korea.” PhD thesis, Harvard University, 2011.

Deng, Ruining, Can Cui, Lucas W. Remedios, Shunxing Bao, R. Michael Womick, Sophie Chiron, Jia Li, et al. “Cross-Scale Multi-Instance Learning for Pathological Image Diagnosis.” Medical Image Analysis 94 (May 2024): 103124. https://doi.org/10.1016/j.media.2024.103124.

Gadermayr, Michael, and Maximilian Tschuchnig. “Multiple Instance Learning for Digital Pathology: A Review of the State-of-the-Art, Limitations & Future Potential.” Computerized Medical Imaging and Graphics: The Official Journal of the Computerized Medical Imaging Society 112 (March 2024): 102337. https://doi.org/10.1016/j.compmedimag.2024.102337.

Hyundam Mun’go Foundation. “Hyundam Mun’go Collection.” Archive, 2021.

Javed, Syed Ashar, Dinkar Juyal, Harshith Padigela, Amaro Taylor-Weiner, Limin Yu, and Aaditya Prakash. “Additive MIL: Intrinsically Interpretable Multiple Instance Learning for Pathology.” arXiv, October 2022. https://doi.org/10.48550/arXiv.2206.01794.

Lundberg, Scott M, and Su-In Lee. “A Unified Approach to Interpreting Model Predictions.” In Advances in Neural Information Processing Systems 30, edited by I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, 4765–74. Curran Associates, Inc., 2017.

Maron, Oded, and Tomás Lozano-Pérez. “A Framework for Multiple-Instance Learning.” Advances in Neural Information Processing Systems 10 (1997).

Papadopoulos, Alexandros, Fotis Topouzis, and Anastasios Delopoulos. “An Interpretable Multiple-Instance Approach for the Detection of Referable Diabetic Retinopathy from Fundus Images.” Scientific Reports 11, no. 1 (July 2021): 14326. https://doi.org/10.1038/s41598-021-93632-8.

Selvaraju, Ramprasaath R., Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. “Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization.” International Journal of Computer Vision 128, no. 2 (February 2020): 336–59. https://doi.org/10.1007/s11263-019-01228-7.

Seuret, Mathias, Saskia Limbach, Nikolaus Weichselbaumer, Andreas Maier, and Vincent Christlein. “Dataset of Pages from Early Printed Books with Multiple Font Groups.” In Proceedings of the 5th International Workshop on Historical Document Imaging and Processing, 1–6. HIP ’19. New York, NY, USA: Association for Computing Machinery, 2019. https://doi.org/10.1145/3352631.3352640.

Waqas, Muhammad, Syed Umaid Ahmed, Muhammad Atif Tahir, Jia Wu, and Rizwan Qureshi. “Exploring Multiple Instance Learning (MIL): A Brief Survey.” Expert Systems with Applications 250 (September 2024): 123893. https://doi.org/10.1016/j.eswa.2024.123893.

Yang, Yang, Yanlun Tu, Houchao Lei, and Wei Long. “HAMIL: Hierarchical Aggregation-Based Multi-Instance Learning for Microscopy Image Classification.” Pattern Recognition 136 (April 2023): 109245. https://doi.org/10.1016/j.patcog.2022.109245.

Extras

Dataset

• We scraped the Hyundam Mun’go for magazines & paperbacks dating between 1900-1950.¹
• 177.101 Images of Pages.
• 14.597 Publications.
• Contributions of 202 unique print shops.
• 2552 Publishers.
• 787 Distribution outlets.

Note: These are uniquely named entities. The Publishers/Distribution Outlets need more cleaning for a definitive answer.

1. Hyundam Mun’go Foundation, “Hyundam Mun’go Collection”.