Evaluating Document Intelligence for Accessibility Remediation Tasks

Supervisors: Maximiliano Jeanneret Medina (HES-SO and UNIFR), Denis Lalanne

Student: Sinthuja Vijayananthan (UNIBE)

Project status: Ongoing

Year: 2026

Abstract

As a cornerstone of education, materials in the form of PDF textbooks, PowerPoint slides, or scanned document images, relying predominantly on the visual modality, remain inaccessible to many students [1]. Recent advances in AI/ML have significantly improved the detection of document elements, character recognition, and document parsing, enabling the preservation of layout information with substantially fewer errors than in previous years [3, 4]. While automated tools have been created for scientific publications [6, 7], or comics [2, 5] little has been done for educational documents.

Focusing on accessibility in an educational context, this master project will focus on evaluating capabilities of AI/ML for performing automated document accessibility remediation tasks. The following activities are envisioned:

  • Identify relevant technical approaches to perform automated document remediation tasks (e.g., pipeline-based, prompt-engineering-based);
  • Evaluate technically selected models of a specific approach;
  • Assess the remediated documents considering best-practices in accessibility evaluation.

Keywords: accessibility; document intelligence; generative AI; Human-Computer Interaction

References

  1. Maximiliano Jeanneret Medina, Cédric Baudet, Maud Njitchouang, and Denis Lalanne. 2025. Shaping Document Accessibility Remediation with Artificial Intelligence: A Multilevel Understanding Study. Communications of the Association for Information Systems 58: 430–454. https://doi.org/10.17705/1CAIS.05818

  2. Yun Jung Lee, Hwayeon Joh, Suhyeon Yoo, and Uran Oh. 2023. AccessComics2: Understanding the User Experience of an Accessible Comic Book Reader for Blind People with Textual Sound Effects. ACM Transactions on Accessible Computing 16, 1: 1–25. https://doi.org/10.1145/3555720

  3. Nikolaos Livathinos, Christoph Auer, Ahmed Nassar, Rafael Teixeira de Lima, Maksym Lysak, Brown Ebouky, Cesar Berrospi, Michele Dolfi, Panagiotis Vagenas, Matteo Omenetti, Kasper Dinkla, Yusik Kim, Valery Weber, Lucas Morin, Ingmar Meijer, Viktor Kuropiatnyk, Tim Strohmeyer, A Said Gurbuz, and Peter W J Staar. 2025. Advanced Layout Analysis Models for Docling. Retrieved from https://arxiv.org/abs/2509.11720

  4. Linke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu, Rui Zhang, Qunshu Lin, Bin Wang, Zhiyuan Zhao, Man Jiang, Xiaomeng Zhao, Jin Shi, Fan Wu, Pei Chu, Minghao Liu, Zhenxiang Li, Chao Xu, Bo Zhang, Botian Shi, Zhongying Tu, and Conghui He. 2025. OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 24838–24848.

  5. Frédéric Rayar, Bernard Oriola, and Christophe Jouffrais. 2020. ALCOVE: An Accessible Comic Reader for People with Low Vision. In Proceedings of the 25th International Conference on Intelligent User Interfaces, 410–418. https://doi.org/10.1145/3377325.3377510

  6. Lucy Lu Wang, Isabel Cachola, Jonathan Bragg, Evie Yu Yen Cheng, Chelsea Haupt, Matt Latzke, Bailey Kuehl, Madeleine van Zuylen, Linda Wagner, and Daniel Weld. 2021. SciA11y: Converting Scientific Papers to Accessible HTML. In ASSETS 2021 - 23rd International ACM SIGACCESS Conference on Computers and Accessibility. https://doi.org/10.1145/3441852.3476545

  7. Lucy Lu Wang, Isabel Cachola, Jonathan Bragg, Evie Yu Yen Cheng, Chelsea Haupt, Matt Latzke, Bailey Kuehl, Madeleine van Zuylen, Linda Wagner, and Daniel Weld. 2021. Improving the Accessibility of Scientific Documents: Current State, User Needs, and a System Solution to Enhance Scientific PDF Accessibility for Blind and Low Vision Users. Retrieved from http://arxiv.org/abs/2105.00076