Fantastic Futures 2024

PETER LEONARD, LINDSAY KING

Transformers-based models have shown that many complicated problems in culture beyond digital text – a human voice, a handwritten word, even a scene from a motion picture – are now tractable to computation. Added to this is the simultaneous development of 'conversational interfaces' as a way of interacting with cultural material. Peter Leonard and Lindsay King draw on their experience working with Stanford's collections to explore several non-commercial multi-modal large language models and what conversational modalities can offer GLAM patrons.

MIA RIDGE

Ingesting enhanced data appropriately and responsibly into collections management and discovery platforms is a key factor in operationalising AI to improve collections documentation and discovery, but is often difficult in practice. Mia Ridge presents original research into the extent to which projects can integrate enriched data into collections platforms, with success and failure results gathered from over 60 projects.

JOSHUA NG

In 2022, Archives New Zealand spearheaded an innovative proof-of-concept project to explore the potential of machine learning and hyperscale cloud computing in auto-classifying digital public records and surfacing information of interest to Māori. Joshua Ng provides an overview of the process, the solutions that were developed and what to consider in the design and usage of these tools.

PETER BROADWELL (PRESENTED BY PETER LEONARD)

The ability to recognise and digitally encode human movements and actions, both in individual and interpersonal contexts, holds great potential for elevating the accessibility and understanding of digitised audiovisual materials. As part of the Machine Intelligence for Motion Exegesis (MIME) project, collaborators worked to apply new transformer AI-based video analysis models to archival recordings of theatre performances, political speeches and other events, exemplifying the methods’ potential relevance to any video collection containing human movement.

TERHI NURMIKKO-FULLER (PRESENTED BY JANET MCDOUGALL)

In the age of AI and large datasets, smarter data that has been represented in ways that capture various perspectives and nuances is needed. This has the power to simultaneously reduce historical and social bias and maximises the potential of bringing mapped data and powerful algorithms together. Janet McDougall reports on a linked data project in motion at the ANU, identifying all content relating to Indigenous research across its myriad repositories.

BENJAMIN LEE, KATH BODE, ANDREW DEAN

From the application of AI-generated poetry to understand one of the earliest examples of generative language from the 1960s to using AI in the investigation of Irishness in Australian newspaper fiction, Benjamin Lee, Andrew Dean and Kath Bode present case studies on how academic AI projects can further scholarly inquiry and reinterpret digital collections.

JAMES SMITHIES, KARAITIANA TAIURU

Despite the growing use of large language models as expert chatbots and research assistants, there have been no obvious or accepted ways to evaluate their quality as research tools. The AI as Infrastructure (AIINFRA) project aims to develop resources to resolve that situation, including a test framework and prototype AI research tool, with a focus on transnational collaboration, Slow AI, and Indigenous design principles.

BETH SHULMAN, CHARLOTTE BRADLEY, RIPLEY RUBENS

The National Institute of Dramatic Arts, ANU School of Cybernetics and the NFSA embarked on a collaborative project to demystify AI and conceptualise ways emerging technology could be used on archival media. The end result is Imaginative Restoration: an interactive installation that uses damaged media that has been restored with AI and breathes new meaning into it through drawings added by a user.

EMMANUELLE BERMÈS

The Musée des Arts Décoratifs in Paris (MAD) partnered with Ecole des chartes (ENC) and the national library of France (BnF) in order to bring together expertise with artificial intelligence and to prototype the automated annotation of large paper collections. Emmanuelle Bermès shares their experiments using the YOLO (You Only Look Once) model on the Royère collection of architecture and furniture drawings to perform object detection using a specific ontology.

JEFF WILLIAMS

In January 2024, ACMI launched an AI Ambassadors program with the goal of enhancing staff understanding through practical experimentation. Led by ACMI's Technology Team and primarily involving non-technical staff, the program focuses on demystifying machine learning, exploring generative AI fundamentals, debating ethics and usage, and providing assignments to expand on these topics through action. Jeff Williams discusses practical insights on how to approach knowledge-sharing and foster an environment of experimentation, enabling the integration of AI more effectively into organisations and removing the barriers to adoption.

JON DUNN

Beginning in 2015, Indiana University (IU) undertook a massive project to digitise rare and unique audio, video and film assets from across IU’s collections which resulted in over 350,000 digitised items. With accessibility needs in mind, IU Libraries is exploring the feasibility of using the open source Whisper automated speech recognition tool in conjunction with IU’s HPE Cray EX supercomputer to generate captions or transcripts for this collection. Jon Dunn shares successes and failures, experiences processing files using a shared research-oriented high-performance computing environment and future plans for using the output to support accessibility and discovery.

EMILY PUGH

PhotoTech is a four-year project headed by the Getty Research Institute, which focused on using AI for both art-historical research and metadata generation in relation to digital image collections. Based on the outcomes of this project, Emily Pugh provides an overview of the current approaches to image searching as they exist in the field of cultural heritage, showing the disconnects between these approaches and those required by an environment of AI-enhanced search.

MORGAN STRONG

The Queensland Art Gallery | Gallery of Modern Art (QAGOMA) embarked on an ambitious project to leverage AI to make their digital content easily accessible on personal devices. By pointing a mobile device at an artwork – 2D or 3D – the artwork is identified rapidly to return related digital content, such as descriptions, colour and shape analyses, interactive components and interactive questions. Morgan Strong explores the selection process, challenges, onboarding visitors, model training and design considerations for the app.

MIGUEL ESCOBAR VARELA

Miguel Escobar Varela offers an overview of the early stages of a collaborative project that brings historians, digital humanists and computer scientists together to explore the interconnectedness of Singapore's history with the broader Malay-speaking world during the late colonial period. The study takes a look at Malay-language periodicals published in Singapore, utilising a unique methodology that combines optical character recognition (OCR) and large language models to convert newspapers and magazines written in Jawi to Rumi, enabling the processing and analysis of a vast corpus of historical periodicals at an unprecedented scale.

KATRINA GRANT, MARNI WILLIAMS

Humanities research data often remain hidden behind traditional forms of publishing or paywalls, creating a disconnect from this rich data being easily accessed, reused, aggregated and linked. To address this, a group of publishers, art historians and digital humanists at the Power Institute for Art and Visual Culture at the University of Sydney are developing new approaches to research that allow scholars, artists, community knowledge holders and museum partners to produce rich data sets, annotated objects and data visualisations, and to communicate them beyond the codex as part of an open publishing infrastructure.

SVETLANA KOROTEEVA

Traditional music cataloguing in libraries involves a lot of manual work, where librarians manually categorise each piece of music by genre. This process is time-consuming and sometimes lacks consistency. Svetlana Koroteva presents their study on developing a tool that can correctly identify genres from records and audio features: beginning from selecting a model and preparing the data to training the model.