EXTENDING STORY READING FOR DEAF AND HARD-OF-HEARING CHILDREN: ARTIFICIAL INTELLIGENCE AS A MEDIATOR OF BILINGUAL LITERACY IN DONNA JO NAPOLI’S HANDS & HEARTS

Autori

  • Ioanna Kaliakatsou PhD in Children’s Literature, Teacher Independent Researcher/Teacher, Ministry of Education, Religious Affairs and Sports, Greece

DOI:

https://doi.org/10.46827/ejse.v12i6.6928

Parole chiave:

deaf and hard-of-hearing children, multimodal picture books, artificial intelligence, inclusive literacy, shared story reading

Abstract

Shared story reading represents a fundamental context for children's literacy development; however, deaf and hard-of-hearing children may experience barriers when literacy practices are predominantly organized around spoken language. Contemporary perspectives in Deaf education emphasize bilingual approaches that recognize sign language as a linguistic foundation and written language as an additional pathway for participation and learning. This chapter explores the potential of artificial intelligence (AI) to mediate bilingual literacy through Donna Jo Napoli’s multimodal picture book Hands & Hearts. Drawing on theories of multimodality, Deaf bilingual education, and inclusive literacy, the chapter examines how AI-supported features can extend connections among American Sign Language (ASL), written language, visual representation, and shared story reading. It argues that AI should be understood not as a replacement for human communication or picture books, but as a mediating tool that expands opportunities for accessible, meaningful, and inclusive bilingual literacy experiences for deaf and hard-of-hearing children.

Biografia autore

  • Ioanna Kaliakatsou, PhD in Children’s Literature, Teacher Independent Researcher/Teacher, Ministry of Education, Religious Affairs and Sports, Greece

    Dr. Kaliakatsou Ioanna is an educator and researcher with extensive experience in primary education, early childhood literacy, and inclusive pedagogy. She holds a PhD and a Master's degree in Education and has been actively involved in research and professional development in the fields of literacy, multimodal learning, and children's literature. Her academic interests focus on picture books as multimodal texts, critical visual literacy, inclusive literacy practices, and the role of materiality and sensory engagement in early childhood education. Her work explores how contemporary picture books can create accessible and meaningful learning experiences for diverse groups of children, including autistic learners and children with different communication profiles. Her research also examines the intersections of multimodality, embodied cognition, Universal Design for Learning (UDL), and social inclusion, with particular emphasis on developing pedagogical approaches that promote participation, agency, and equity in early literacy classrooms. She has presented her work at national and international conferences and has contributed to academic publications on children's literature, literacy education, and educational innovation. Alongside her academic work, Dr. Kaliakatsou Ioanna has extensive experience in teacher education and curriculum development and is actively engaged in initiatives that promote inclusive education and community participation. Her current research continues to investigate multisensory picture books, visual meaning-making, and the educational potential of multimodal storytelling in fostering literacy, communication, and social inclusion for all children.

Riferimenti bibliografici

Berke, M. (2013). Reading books with young deaf children: Strategies for mediating between American Sign Language and English. The Journal of Deaf Studies and Deaf Education, 18(3), 299–311. https://doi.org/10.1093/deafed/ent001

Bragg, D., Koller, O., Bellard, M., Berke, L., Boudreault, P., Braffort, A., Kearns, J., Palmer, M., Ploetz, T., & Vogler, C. (2019). Sign language recognition, generation, and translation: An interdisciplinary perspective. In Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) (pp. 651–653). Association for Computing Machinery. https://doi.org/10.1145/3308561.3353774

Bus, A. G., van IJzendoorn, M. H., & Pellegrini, A. D. (1995). Joint book reading makes for success in learning to read: A meta-analysis on intergenerational transmission of literacy. Review of Educational Research, 65(1), 1–21. https://doi.org/10.3102/00346543065001001

CAST. (2018). Universal Design for Learning Guidelines version 2.2. CAST. Retrieved from https://lincs.ed.gov/professional-development/resource-collections/profile-1050

Chamberlain, C., & Mayberry, R. I. (2008). American Sign Language syntactic and narrative comprehension in skilled and less skilled deaf readers: A test of the linguistic interdependence hypothesis. Applied Psycholinguistics, 29(3), 367–388. https://doi.org/10.1017/S014271640808017X

Dickinson, D. K., & Smith, M. W. (1994). Long-term effects of preschool teachers’ book readings on low-income children’s vocabulary and story comprehension. Reading Research Quarterly, 29(2), 104–122. https://doi.org/10.2307/747807

Grosjean, F. (2010). Bilingual: Life and reality. Harvard University Press. Retrieved from https://books.google.ro/books/about/Bilingual.html?id=XgRum7AWOoUC&redir_esc=y

Hall, W. C. (2017). What you don’t know can hurt you: The risk of language deprivation by impairing sign language development in deaf children. Maternal and Child Health Journal, 21, 961–965. https://doi.org/10.1007/s10995-017-2287-y

Hoffmeister, R. J., & Caldwell-Harris, C. L. (2014). Acquiring English as a second language via print: The task for deaf children. Cognition, 132(2), 229–242. https://doi.org/10.1016/j.cognition.2014.03.014

Humphries, T., Kushalnagar, P., Mathur, G., Napoli, D. J., Padden, C., Rathmann, C., & Smith, S. (2012). Language acquisition for deaf children: Reducing the harms of zero tolerance to the use of alternative approaches. Harm Reduction Journal, 9, Article 16. https://doi.org/10.1186/1477-7517-9-16

Jewitt, C., Bezemer, J., & O’Halloran, K. (2016). Introducing multimodality. Routledge. https://doi.org/10.4324/9781315638027

Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. Routledge. Retrieved from https://books.google.ro/books/about/Multimodality.html?id=ihTm_cI58JQC&redir_esc=y

Kusters, A., De Meulder, M., & O’Brien, D. (2017). Innovations in Deaf studies: Critically mapping the field. American Annals of the Deaf, 162(4), 285–297. https://doi.org/10.1353/aad.2017.0026

Ladd, P. (2003). Understanding Deaf culture: In search of Deafhood. Multilingual Matters. https://doi.org/10.21832/9781853595479

Lane, H. (1992). The mask of benevolence: Disabling the Deaf community. Alfred A. Knopf. Retrieved from https://books.google.ro/books/about/The_Mask_of_Benevolence.html?id=11FHAAAAMAAJ&redir_esc=y

Marschark, M., & Hauser, P. C. (2012). How Deaf children learn: What parents and teachers need to know. Oxford University Press. Retrieved from https://psycnet.apa.org/record/2011-25233-000

Mayberry, R. I. (2007). When timing is everything: Age of acquisition effects on second-language learning. In A. M. B. de Groot & J. F. Kroll (Eds.), Tutorials in bilingualism: Psycholinguistic perspectives (pp. 89–117). Lawrence Erlbaum Associates. Retrieved from https://doi.org/10.1017/S0142716407070294

Mayberry, R. I., & Kluender, R. (2018). Rethinking the critical period for language: New insights into an old question from American Sign Language. Bilingualism: Language and Cognition, 21(5), 938–944. https://doi.org/10.1017/S1366728917000724

Mayer, C. (2007). What really matters in the early literacy development of deaf children. Journal of Deaf Studies and Deaf Education, 12(4), 411–431. https://doi.org/10.1093/deafed/enm020

Mol, S. E., Bus, A. G., de Jong, M. T., & Smeets, D. J. H. (2008). Added value of dialogic parent–child book readings: A meta-analysis. Early Education and Development, 19*(1), 7–26. https://doi.org/10.1080/10409280701838603

Napoli, D. J., & Bates, A. J. (2014). Hands & Hearts: With 15 words in American Sign Language. Abrams Books for Young Readers. Retrieved from https://books.google.ro/books/about/Hands_Hearts.html?id=TXRmAgAAQBAJ&redir_esc=y

New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60–92. Retrieved from https://doi.org/10.17763/haer.66.1.17370n67v22j160u

Nikolajeva, M. (2013). Picturebooks and emotional literacy. The Reading Teacher, 67*(4), 249–254. https://doi.org/10.1002/TRTR.1229

Nikolajeva, M., & Scott, C. (2006). How picturebooks work. Routledge. Retrieved from https://books.google.ro/books/about/How_Picturebooks_Work.html?id=EbCfCLUOZD0C&redir_esc=y

Nikolaraizi, M., Vekiri, I., & Easterbrooks, S. R. (2013). Investigating deaf students’ use of visual multimedia resources in reading comprehension. American Annals of the Deaf, 157(5), 458–473. https://doi.org/10.1353/aad.2013.0007

Oatley, K. (2016). Fiction: Simulation of social worlds. Trends in Cognitive Sciences, 20(8), 618–628. https://doi.org/10.1016/j.tics.2016.06.002

Padden, C., & Humphries, T. (2005). Inside Deaf culture. Harvard University Press. Retrieved from https://books.google.ro/books/about/Inside_Deaf_Culture.html?id=2B4XWIFPgowC&redir_esc=y

Sandler, W., & Lillo-Martin, D. (2006). Sign language and linguistic universals. Cambridge University Press. Retrieved from https://books.google.ro/books/about/Sign_Language_and_Linguistic_Universals.html?id=F2yMNh3x5DsC&redir_esc=y

Serafini, F. (2014). Reading the visual: An introduction to teaching multimodal literacy. Teachers College Press. Retrieved from https://books.google.ro/books/about/Reading_the_Visual.html?id=kdjMAQAAQBAJ&redir_esc=y

Swanwick, R. (2016). Deaf children’s bilingual learning: A critical review. Deafness & Education International, 18(1), 3–20. https://doi.org/10.1080/14643154.2016.1155610

Swanwick, R. (2017). Translanguaging, learning and teaching in deaf education. International Journal of Bilingual Education and Bilingualism, 21(5), 539–553. https://doi.org/10.1080/13670050.2017.1281219

Dowloads

Pubblicato

2026-08-08

Fascicolo

V. 12 N. 6 (2026)

Sezione

Articles

Licenza

Authors who publish with this journal agree to the following terms:

 

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

 

  1. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

 

  1. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).