GLAMs (Galleries, Libraries, Archives and Museums) fastai study group

Deep Learning
101

Chris Albon (director of ML at Wikimedia) is hosting a weekly livestreamed office hours about machine learning at the Wikimedia Foundation on Twitch. Likely to be of interest to people in this thread :slight_smile: https://twitter.com/chrisalbon/status/1386715389975883777?s=20

102

Iā€™ve just shared a tool that might be of interest to people on this thread. It might be particularly useful for creating training sets of historic images (newspapers). I made nnanno to help work with the newspaper navigator dataset from the Library of Congress Labs. I used nbdev to generate the code from notebooks so itā€™s also easy enough to adapt the notebook to your own needs if you want to work with this dataset.

1 Like
103

Ground Truths: Surveying GLAM Data for AI looks like a great initiative:

Cultural heritage organizations are developing collections as data for a variety of purposes, including for use in artificial intelligence (AI) systems. The potential for community-wide benefit is significant, yet it is challenging to move forward in part because data are difficult to find.

In an effort to make finding and reusing cultural heritage data for AI easier, Thomas Padilla (Center for Research Libraries), Elizabeth Lorang (University of Nebraskaā€“Lincoln), and Harish Maringanti (University of Utah) are surveying the cultural heritage community. This survey is part of Ground Truths, a new project which seeks to (1) advance conversations about machine learning in cultural heritage organizations by centralizing and sharing information about data development efforts, (2) pose questions about data and process that support responsible operationalization of machine learning in cultural heritage organizations, and (3) encourage reuse of cultural heritage data in AI contexts.

You can fill out the Google form to point to any datasets created by you/your institutions.

104

This thread is probably a bit dormant but just in case anyone stumbles across this thread in the future, I wanted to share a resource Iā€™ve been working on. The Living with Machines Project and the British Library have been working on using machine learning to classify genre of books.

As part of this process weā€™ve been documenting the steps we took along the way in a Jupyterbook. The goal of this ā€˜bookā€™ is to try and cover more of the steps involved in practically using machine learning in a GLAM context i.e. not just how to train a model but what comes before and after this. Itā€™s still a work in progress so weā€™d appreciate any feedback on the content/our approach :slight_smile:

1 Like
105

Seems like I found my clique here! I knew it was the place when read IIIF in the first post :slight_smile:

Wondering if anyone has worked with indexing of historical photography? Iā€™m looking into making a classifier to detect genres/subjects such as ā€œFamily portrait, individual portrait, urban landscape, natural landscapeā€, that sort of category/attribute. If someone knows of datasets or other previous work in this area please let me know! Will share my work of course, if I get anywhere interesting.

1 Like
106

Sorry for the slow reply!

It depends a bit on the approach you want to take. Two broad options are:

  • image classification
  • image labelling

For the former every image should go into one category only. This works well if you have well defined mutually exclusive categories.

The later can be a better approach when you want to allow images to have multiple labels or none.

This programming historian lesson covers these approaches a little bit on a not completely dissimilar type of task.

What sort of images are you hoping to work with?

1 Like
107

Hi @Danielvs thanks for the link, Iā€™ll take a look!

As for the images, the institution I work at has a large 19th/early 20th century photography collection, so thatā€™s my main ā€˜workhorseā€™

2 Likes
108

No problem; if you werenā€™t already familiar with it, you may also want to join the ai4lam slack (ai4lam). You may also be interested in GitHub - bigscience-workshop/lam: Libraries, Archives and Museums (LAM), which aims to make more datasets related to GLAM available easily.

One option you may also consider exploring is using existing metadata as training data. For example, this repo (GitHub - europeana/rd-img-classification-pilot: R&D image classification pilot) may give you an initial set of labelled data from other institutions, which may help train a baseline model without having to annotate all the data from scratch.

Excited to hear how you get on with this work!

109

Hello World! I am Vinayak, I currently work at Okkular as a Machine Learning Engineer. I completed the v3 of the fastai course and enjoyed it wholly. Was a part of the fastai study group hosted by Aman Arora last year on weights and biases platform and got to learn a lot from there.

I am really keen to work on data centric deep learning! Estimating model uncertainty, calibrating the outputs to represent true estimates of probability, quantifying the error and coming up with metrics to ensure data quality like cohenā€™s kappa etc. are things that interest me and would love to work on some of these if anyoneā€™s interested!

Thanks and look forward to being an active member of this forum :slight_smile:

110

Iā€™m trying to join the Slack channel but it seems restricted to some email domains. Would you be able to send me an invite link?

To get my feet wet, I went on to train an urban/natural landscape classifier with metadata from our DAMS, it works rather well for a first exercise. This Europeana repo looks promising, but the BigScience one is focused on NLP if I understand it correctly.

112

Did you ever manage to get access? (sorry for slow response)

113

Not sure if anyone still looks here, but just in case, the below might be useful!

Label Studio, an open-source annotation tool, has recently made it easier to set up their tool without needing to install your own server etc., by using Hugging Face Spaces. This could be very useful if you want to create a project with multiple annotators where a local install wonā€™t work and setting up a cloud server might be too much work or not allowed by your IT department.

2 Likes
114

Iā€™m afraid notā€¦

Also thanks for Label Studio, Iā€™ve been struggling with large images on CVATā€¦

1 Like
115

If anyone wants an invite to the ai4lam slack channel, please DM and I can invite you directly. Slack invite link that should work.