Category: News

JGI Seed Corn Funding Project Blog 2023/24: James Pope

Introduction

Open-source skin cancer datasets contain predominantly lighter skin tones potentially leading to biased artificial intelligence (AI) models. This study aimed to analyse these datasets for skin tone bias. 

What were the aims of the seed corn project? 

The project’s aims were to perform an exploratory data analysis of open-source skin cancer datasets and evaluate potential skin tone bias resulting from the models developed with these datasets.  Assuming biases were found and time permitting, a secondary goal was to mitigate the bias using data pre-processing and modelling techniques. 

What was achieved? 

Dataset collection

The project focused on the International Skin Imaging Collaboration (https://www.isic-archive.com/) archive that contains over 20 datasets totalling over 100,000 images.  The analysis required that the images provide some indication of skin tone.  We found that only 3,623 recorded the Fitzpatrick Skin Type on a scale from 1 (lighter) to 6 (darker).  For each image, we mapped the Fitzpatrick Skin Type to light or dark skin tone.  As future work, the project began exploring tone classification techniques to expand the images considered. 

Artificial Intelligence Modelling

We then developed a typical artificial intelligence model, specifically a deep convolutional neural network, to classify whether the images are malignant (i.e. cancerous) or benign. The model was trained from 2/3 of the images and evaluated in the remaining 1/3.  Due to computational limits, the model was only trained for 50 epochs. The model’s accuracy (how many correct classifications it made of either benign or malignant tumours out of all the tumours it was evaluated on) was comparatively poor with only 82%. 

Bias Analysis

The model was then evaluated relative to light and dark skin tones.  We found that the model was better at identifying cancer in light versus dark skin tone images.  The recall/true positive rate for dark skin tones was 0.26 while for light skin tones it was 0.45.  The resulting disparate impact (a measure used to indicate if a test is biased for certain groups) was found to be 0.58, which indicates the model is potentially biased.

Future plans for the project 

The project results were limited due to the subset of images with skin tone and constrained computational resources.  Future work is to further develop the tone classifier to expand the number of labelled images. Converting colour values from images into values more closely related to skin tone and then comparing with the tone labels of the image, might help train an AI model to exclude the tumour itself when classifying skin tone of the whole image. This is important as we know that the tone of tumours themselves is often different to that of the surrounding skin.

More powerful computational resources will be acquired and used to sufficiently train the model.   Future work will also employ explainable AI techniques to identify the source of the bias. 

Contact details and links 

James Pope: https://research-information.bris.ac.uk/en/persons/james-pope,

Ayush Joshi https://research-information.bris.ac.uk/en/persons/ayush-joshi,  

JGI Seed Corn Funding Project Blog 2023/24: Naomi Scott

In the second century A.D., Julius Pollux, Professor of Rhetoric at the Academy in Athens, wrote the Onomasticon (‘Book of Words’), and dedicated it to the Emperor Commodus. The work sits somewhere between an encyclopaedia and a lexicon. Chapters are organised by topic, and Pollux lists appropriate words on diverse themes such as ‘The Gods’, ‘Bakery Equipment’, ‘Diseases of Dogs’, and ‘Objects Found On Top Of Tables’. Throughout his work, Pollux quotes canonical authors such as Homer, Aeschylus, and Sappho in support of what he considers correct and elegant linguistic usage. This means that in addition to providing a wealth of information on everyday life in the ancient world, the Onomasticon is also one of our best sources of quotations from otherwise lost works of ancient Greek literature.   

Despite Pollux’s obvious importance, his work has not been translated into any modern language. The vast size of the Onomasticon (10 books in total, each comprised of around 250 chapters) means that it is unwieldy even for researchers able to study the original ancient Greek text. With seed-corn funding from the Jean Golding Institute, my project ‘Crowd Sourcing Julius Pollux’s Onomasticon’ has set to work on filling this gap. Eventually, my aim is to use crowd-sourcing to produce not only a translation of the Onomasticon, thereby making it accessible to researchers in a wide variety of disciplines, but an edition of the work which is fully data-tagged, so that researchers can better navigate the text, and produce key data about it: Which ancient authors and genres are most frequently cited as sources and in what contexts; what topics are granted the most or least coverage within the text; and how are different lexical categories distributed within the encyclopaedia? Without the answers to questions such as these, any individual chapter or citation within the Onomasticon cannot be placed in the wider context of the work as a whole.  

Creating a New Digital Edition 

While a digitised version of the ancient Greek text of the Onomasticon exists, it is based on the work of Erich Bethe, whose early twentieth-century edition of Pollux removed all the chapter titles which have been used to organise the text since it was first published as a printed book in 1502. Bethe did this because he did not consider the chapter titles to be Pollux’s own. Both for the purpose of splitting the text up into manageable short chunks for translation, and for the purpose of data-tagging, I decided it was essential to reinstate the titles. Additionally, my own examination of manuscripts of the Onomasticon dating as far back as the 10^th century has revealed that the chapter titles are in fact much older than first thought, and that the text as we currently have it (abridged from Pollux’s even longer original!) may even have been conceived with the chapter titles. 

The first step in producing a digital edition suitable for crowd-sourcing and data-tagging is therefore to reinsert the titles into the text. This would be an enormous undertaking if done manually. Working with a brilliant team from Bristol’s Research IT department, led by Serena Cooper, Keiran Pitts, and Mike Jones, we have set about automating this process. Ancient Greek OCR (Optical Character Recognition) software designed by Professor Bruce Robertson at the University of Mount Allison in Canada, two editions of the text were scanned — one Bethe’s chapterless version, and the other by Karl Wilhelm Dindorf, whose 1824 edition of the text includes the titles.  The next step is to use digital mapping software to combine the two texts, inserting the titles from Dindorf into the otherwise superior version of the text produced by Bethe.  

Next Steps 

Once the issue of the chapter titles has been resolved, the next step will be to create a prototype of around 20 chapters, which can then be made available to the scholarly community to begin translating and data-tagging the text. A prototype would allow us to get feedback from researchers around the world working with Pollux, and to better understand what kinds of data would be most useful to those seeking to understand the text. This feedback can then be integrated into an eventual complete edition of the text which can then be translated and data-tagged as a whole.  

Eventually, this project will not only make the Onomasticon more accessible to researchers, and help to revolutionise our understanding of this important work. A complete translation and data-tagged edition complete with chapter titles will also allow the Onomasticon to have an impact beyond the academic community. The eventual plan is to train arts professionals engaging with the ancient Greek world to use the digital edition and translation. The Onomasticon’s remarkably detailed picture of ordinary life and ordinary stuff in antiquity makes it a vital resource for anyone trying to recreate the ancient Greek world on stage, on screen, or in novels. The hope is that this project will therefore not only change the way that scholars understand the Onomasticon and its place in the history of the encyclopaedia. It can also offer artists a window onto antiquity, and through its impact on art, shape the public understanding of the ancient world.