The University of Bristol's central hub for data science and data-intensive research, connecting a multidisciplinary community of experts across the University and beyond.
JGI Seed Corn Funding Project Blog 2023/24: James Armstrong
The human brain is a highly complex structure and an inaccessible organ to study, which has hampered our understanding of how the brain grows, becomes diseased, and responds to drugs. In the last ten years, a new method has been developed that uses stem cells to grow miniature brain tissues in the lab. These “brain organoids” have proven to be an incredibly useful tool for scientists studying the human brain.
However, a well-known limitation of this tissue model is their unpredictable growth: within the same batch, some organoids will undergo typical neural development with large cortical buds (Figure 1A) while others will fail to produce these important structural features (Figure 1B). This Jean Golding Institute funded project sought to answer the question – can do seemingly identical stem cell cultures undergo such different growth? To this end, we aimed to track the growth of ~600 brain organoids over 20 days, then to use computer vision / machine learning methods to pick out key structural features that could be used to predict the tissue growth.
Figure 1. (A-B) Examples of two brain organoids, grown using the same methods, that were identical at day 3 but undergo very different growth. (C) An example of the images acquired during this project.
This work was led by Dr James Armstrong, a Senior Research Fellow who runs a tissue engineering research group at Bristol Medical School (www.TheArmstrongGroup.co.uk). Members of his team (Martha Lavelle, Dr Aya Elghajiji, with help from Carolina Gaudenzi) have so far grown ~200 organoids, with another member of his team (Sammy Shorthouse) collecting microscopy images at ten intervals throughout the growth (Figure 1C). As expected, we saw tremendous variation in the growth of the brain organoids, in terms of their size, shape, and budding. Sammy has developed a program that takes these images and automatically processes them (indexing, identifying correct focal plane, centring, and cropping). He is now developing this script into a user-friendly “app”. For the next stages, Dr Qiang Liu in the Faculty of Engineering has been working with Sammy to develop computer vision methods that can pick out the key structural features of the organoids at the different stages of their growth. We are now growing the next batch of organoids and hope to reach the ~600 mark by the end of the summer. This should provide us with our target dataset, which should be large enough to start drawing links and making predictions of tissue growth.
Open-source skin cancer datasets contain predominantly lighter skin tones potentially leading to biased artificial intelligence (AI) models. This study aimed to analyse these datasets for skin tone bias.
What were the aims of the seed corn project?
The project’s aims were to perform an exploratory data analysis of open-source skin cancer datasets and evaluate potential skin tone bias resulting from the models developed with these datasets. Assuming biases were found and time permitting, a secondary goal was to mitigate the bias using data pre-processing and modelling techniques.
What was achieved?
Dataset collection
The project focused on the International Skin Imaging Collaboration (https://www.isic-archive.com/) archive that contains over 20 datasets totalling over 100,000 images. The analysis required that the images provide some indication of skin tone. We found that only 3,623 recorded the Fitzpatrick Skin Type on a scale from 1 (lighter) to 6 (darker). For each image, we mapped the Fitzpatrick Skin Type to light or dark skin tone. As future work, the project began exploring tone classification techniques to expand the images considered.
Artificial Intelligence Modelling
We then developed a typical artificial intelligence model, specifically a deep convolutional neural network, to classify whether the images are malignant (i.e. cancerous) or benign. The model was trained from 2/3 of the images and evaluated in the remaining 1/3. Due to computational limits, the model was only trained for 50 epochs. The model’s accuracy (how many correct classifications it made of either benign or malignant tumours out of all the tumours it was evaluated on) was comparatively poor with only 82%.
Bias Analysis
The model was then evaluated relative to light and dark skin tones. We found that the model was better at identifying cancer in light versus dark skin tone images. The recall/true positive rate for dark skin tones was 0.26 while for light skin tones it was 0.45. The resulting disparate impact (a measure used to indicate if a test is biased for certain groups) was found to be 0.58, which indicates the model is potentially biased.
Future plans for the project
The project results were limited due to the subset of images with skin tone and constrained computational resources. Future work is to further develop the tone classifier to expand the number of labelled images. Converting colour values from images into values more closely related to skin tone and then comparing with the tone labels of the image, might help train an AI model to exclude the tumour itself when classifying skin tone of the whole image. This is important as we know that the tone of tumours themselves is often different to that of the surrounding skin.
An example image from ISIC which had its Fitzpatrick Skin Type labelled. The light green indicates where individual pixels correspond with expected colours associated with the labelled skin type. Notice that the centre of the image, where the tumour is, does not match.
More powerful computational resources will be acquired and used to sufficiently train the model. Future work will also employ explainable AI techniques to identify the source of the bias.
JGI Seed Corn Funding Project Blog 2023/24: Naomi Scott
A page from a 10th century manuscript of Julius Pollux’s Onomasticon
In the second century A.D., Julius Pollux, Professor of Rhetoric at the Academy in Athens, wrote the Onomasticon (‘Book of Words’), and dedicated it to the Emperor Commodus. The work sits somewhere between an encyclopaedia and a lexicon. Chapters are organised by topic, and Pollux lists appropriate words on diverse themes such as ‘The Gods’, ‘Bakery Equipment’, ‘Diseases of Dogs’, and ‘Objects Found On Top Of Tables’. Throughout his work, Pollux quotes canonical authors such as Homer, Aeschylus, and Sappho in support of what he considers correct and elegant linguistic usage. This means that in addition to providing a wealth of information on everyday life in the ancient world, the Onomasticon is also one of our best sources of quotations from otherwise lost works of ancient Greek literature.
Despite Pollux’s obvious importance, his work has not been translated into any modern language. The vast size of the Onomasticon (10 books in total, each comprised of around 250 chapters) means that it is unwieldy even for researchers able to study the original ancient Greek text. With seed-corn funding from the Jean Golding Institute, my project ‘Crowd Sourcing Julius Pollux’s Onomasticon’ has set to work on filling this gap. Eventually, my aim is to use crowd-sourcing to produce not only a translation of the Onomasticon, thereby making it accessible to researchers in a wide variety of disciplines, but an edition of the work which is fully data-tagged, so that researchers can better navigate the text, and produce key data about it: Which ancient authors and genres are most frequently cited as sources and in what contexts; what topics are granted the most or least coverage within the text; and how are different lexical categories distributed within the encyclopaedia? Without the answers to questions such as these, any individual chapter or citation within the Onomasticon cannot be placed in the wider context of the work as a whole.
Creating a New Digital Edition
While a digitised version of the ancient Greek text of the Onomasticon exists, it is based on the work of Erich Bethe, whose early twentieth-century edition of Pollux removed all the chapter titles which have been used to organise the text since it was first published as a printed book in 1502. Bethe did this because he did not consider the chapter titles to be Pollux’s own. Both for the purpose of splitting the text up into manageable short chunks for translation, and for the purpose of data-tagging, I decided it was essential to reinstate the titles. Additionally, my own examination of manuscripts of the Onomasticon dating as far back as the 10th century has revealed that the chapter titles are in fact much older than first thought, and that the text as we currently have it (abridged from Pollux’s even longer original!) may even have been conceived with the chapter titles.
The first step in producing a digital edition suitable for crowd-sourcing and data-tagging is therefore to reinsert the titles into the text. This would be an enormous undertaking if done manually. Working with a brilliant team from Bristol’s Research IT department, led by Serena Cooper, Keiran Pitts, and Mike Jones, we have set about automating this process. Ancient Greek OCR (Optical Character Recognition) software designed by Professor Bruce Robertson at the University of Mount Allison in Canada, two editions of the text were scanned — one Bethe’s chapterless version, and the other by Karl Wilhelm Dindorf, whose 1824 edition of the text includes the titles. The next step is to use digital mapping software to combine the two texts, inserting the titles from Dindorf into the otherwise superior version of the text produced by Bethe.
Next Steps
Once the issue of the chapter titles has been resolved, the next step will be to create a prototype of around 20 chapters, which can then be made available to the scholarly community to begin translating and data-tagging the text. A prototype would allow us to get feedback from researchers around the world working with Pollux, and to better understand what kinds of data would be most useful to those seeking to understand the text. This feedback can then be integrated into an eventual complete edition of the text which can then be translated and data-tagged as a whole.
Eventually, this project will not only make the Onomasticon more accessible to researchers, and help to revolutionise our understanding of this important work. A complete translation and data-tagged edition complete with chapter titles will also allow the Onomasticon to have an impact beyond the academic community. The eventual plan is to train arts professionals engaging with the ancient Greek world to use the digital edition and translation. The Onomasticon’s remarkably detailed picture of ordinary life and ordinary stuff in antiquity makes it a vital resource for anyone trying to recreate the ancient Greek world on stage, on screen, or in novels. The hope is that this project will therefore not only change the way that scholars understand the Onomasticon and its place in the history of the encyclopaedia. It can also offer artists a window onto antiquity, and through its impact on art, shape the public understanding of the ancient world.
JGI Seed Corn Funding Project Blog 2023/24:Miranda Armstrong
Introduction
Type 1 diabetes (T1D) requires consistent self-management, which places a large burden on those who live with it. We explored the role smartwatches could play in reducing that burden.
Figure 1: Theoretical closed-loop system that uses smartwatch data in its algorithm. Closed-loop systems without smartwatch integration are the current state of the art of T1D technology. They begin to automate the T1D management process by using data from the eco-system of devices to predict future changes in blood glucose and change insulin dosage to counteract these changes.
Aims
The project aimed to collect and build a dataset that would allow for exploration into the potential of smartwatches in T1D management. This would include both data from the smartwatch and T1D technology the participants used, and user experience of using the smartwatch alongside their typical T1D management. To meet the aim, the following goals were set:
Collect data from participants, including from smartwatches and T1D devices, and in interviews and focus groups.
Clean, anonymise, and combine data from different sensors into a consistent format, and transcribe the interviews and focus groups.
Hold an online data challenge using a sample of the collected data to promote the dataset and highlight potential uses for it.
Release the dataset publicly to allow other researchers to use it as part of their work and therefore increase the value of the dataset.
What was achieved
Figure 2: An example day of data from one participant, with some of the data available. The upper axis highlights the data available to current commercial closed-loop systems and the lower axis shows some smartwatch data from the same period.
Data Collection
The project recruited 24 participants, and each were given a smartwatch or could use their own. Over six months, participants donated data from their smartwatches and type 1 diabetes (T1D) devices to create a dataset aimed at exploring the integration of smartwatch data into a closed-loop algorithm. This dataset reflects real-life conditions and participants used a range of T1D technology. Over 2000 days, the data that was donated had a high coverage from all the devices the participant used. During this time participants were involved in interviews and focus groups to discuss their opinions of the smartwatches and potential roles they could see in T1D management. A total of 62 interviews and 11 focus groups were completed across the study period.
Data Processing
We processed a large amount of data to prepare it for public use. The smartwatch and T1D data were cleaned and anonymised (so no one involved in the study could be identified) and then organised into two formats. One was an easy-to-use dataset for researchers to test their algorithms, and the other kept the data in its original form for deeper exploration. We also transcribed and anonymised the interviews and focus groups so other researchers could analyse them to understand the participants’ experience of using the smartwatch.
Initial Findings
Initial engagement with the interviews and focus group data has highlighted several potential uses for smartwatches in T1D management. These include as a device to display data quickly and discretely to the user, as an interface with T1D technology for easier access, and as a data source to inform management decision making around activity. There are also design implications highlighted in this analysis. These include utilising automation to provide benefit without increasing user burden, allowing customisation to accommodate the wide range of user preferences and usage patterns to promote uptake, and flexibility to allow these systems to adapt to changing user needs and ensure use of the device into the future.
Future Plans
The data challenge and the public release of the dataset are scheduled for later this year. We plan to run the competition from mid-September to the end of November 2024, with £1600 in prize vouchers available across entries. If you would like to hear more details about the competition, please leave your details in this form. The whole dataset will then be published after the competition of the data competition.
Additionally, we will conduct our own analysis on the data that has been collected. This will expand our initial findings that highlight where and how a smartwatch could be used to improve T1D management. It will also test if adding smartwatch data can improve the prediction of blood glucose, by factoring in information on activity. This could be utilised in closed-loop systems (Figure 1) and would allow them to factor activity into their algorithms. For example, if the user was to go for a walk, this system could detect that activity and then predict the drop in blood glucose levels it would cause and so reduce insulin delivery to counteract this drop. Such a system would improve T1D management and reduce the burden placed on those managing it.
JGI Seed Corn Funding Project Blog 2023/24:Mark Mumme, Eleanor Walsh, Dan Smith, Huw Day and Debbie Johnson
What is Children of the 90s?
Children of the 90s (Co90s) is a multi-generational population-based study following the health and development of nearly 15,000 families living around Bristol, whose children were born in 1991 and 1992.
Co90s initially recruited its participants during the early stages of the mum’s pregnancy and captures information prospectively, at key time points, using self-reported questionnaires, interviews, clinics and electronic health records (EHR).
The Co90s supports about 20 project teams using NHS data at any one time.
What is Synthetic Data?
At its most basic, synthetic data is information generated artificially rather than recorded directly from real-world events. It is essentially a computer-generated version of the data that doesn’t contain any real data and preserving privacy and confidentiality.
Privacy vs Fidelity
Generating synthetic data is frequently a balancing act between fidelity and privacy (Figure 1).
“Fidelity”: how well does the synthetic data represent the real-world data?
“Privacy”: can personal information be deduced from the synthetic data?
Figure 1: Privacy versus fidelity
Why synthetic NHS data:
EHR data are incredibly valuable and rich data sources, yet there are significant difficulties to accessing this data, including financial costs and the time taken to complete multiple application forms and have these approved.
Because the authentic NHS data is so difficult to access, it is also not unusual for researchers to have never worked with, or possibly even seen, this type of data before. They often face a learning curve to understand how the data is structured, what variables are present in the data and how those variables relate to each other.
The journey for a project to travel (Figure 2) just to get NHS data typically goes through the following stages:
Figure 2: The stages a project goes through to get NHS data
Each of these stages can take several months and are usually sequential. It not unheard of for projects to run out of time and/or money due to these lengthy timescales.
Current synthetic NHS data:
Recently, the NHS has released synthetic Hospital Episode Statistics (HES) data (available here; https://digital.nhs.uk/services/artificial-data) which is, unfortunately, quite limited for practical purposes. This is because a very simple approach was adopted; each variable is randomly generated independently from all others. While it is possible to infer broadly accurate descriptive statistics for single variables (e.g., age or sex), it is impossible to infer relations between variables (e.g., how the number of cancer diagnoses increases with age). In the terms introduced above, it has high privacy but low fidelity. As shown in the heatmap, Figure 3, we observe practically no association between diagnosis and treatment because synthetic NHS data is randomly generated variable-by-variable.
Figure 3: Heatmap displaying the relations between disease groupings (right side) and treatment (bottom) from the synthetic NHS data. The colour shadings represent the number of patients (e.g., the darker the shading, the higher the number). The similarity in shading within each diagnosis row shows that treatment and diagnosis were largely independent in this synthetic dataset.
What do researchers want from synthetic data?
We developed an anonymous survey and asked 230 researchers experienced with EHR data, what would be important to them when considering using synthetic EHR data. Out of the 24 responding most were epidemiologists at fellow or professor level. Researchers were then invited to an online discussion group to expand on insights from the survey. Seven researchers attended.
Most researchers had a more than 3 years of experience using EHRs both within and outside of cohort studies. Although few had much knowledge of synthetic EHR data, many had heard of synthetic EHR data and were interested in its application, particularly as a tool for training and learning about EHRs generally.
The most important issues to researchers (Figure 4) were consistent patient details and having all the additional diagnosis & treatment codes rather than just the main ones:
Figure 4: What researchers look for in synthetic EHRs
The most important utility for these researchers was to test/develop code and understand broad structure of the data, as shown below (Figure 5):
Figure 5: Priorities of researchers when using synthetic data
This was reflected in their main concerns about maintaining the utility of the data in the synthetic version by producing high level of accuracy and attention to detail.
During the discussion it was recognised that EHRs are “messy” and synthetic data should emulate this, providing an opportunity to prepare for real EHRs.
Emulate “messy” real data discussion visual
Being able to prepare for the use of real EHRs was the main use case for synthetic data. No one suggested using the synthetic data as the analysis dataset in place of the real data.
Preparation for using real EHR data visual
It was suggested, in both survey responses and the discussion group, that any synthetic data should be bespoke to the requirements of each project. Further, it was observed that each research project only ever used a portion of the complete dataset, therefore synthetic data should be minimized also.
“I think any synthetic data set based on any of the electronic health records should be stripped back to the key things that people use, because then the task of making it a synthetic copy [is] less.” (online participant)
Summary
Following the survey and discussion with some researchers familiar with EHRs a few key points came through:
Training – using synthetic data to understand how EHRs work, and to develop code.
Fidelity is important – using synthetic data as way for researchers to experience using EHRs (e.g. the real data flaws, linkage errors, duplicates).
Cost – the synthetic data set, and associated training, must be low cost and easily accessible.
Next Steps
There is a demand for a synthetic data set with a higher level of fidelity than is currently available, and particularly there is a need for data which is much more consistent over time.
The Co90s is well placed to respond to this demand, and will look to:
Obtain approximately 10 years’ worth of NHS data – record level but pseudonymised.
