The University of Bristol's central hub for data science and data-intensive research, connecting a multidisciplinary community of experts across the University and beyond.
Emilio Romero (Ask-JGI Data Science Support 2023-24)
Emilio Romero, 2nd year PhD Student in Translational Health Sciences
Over the past year, my experience helping with the Ask-JGI service has been really rewarding. I was keen to apply as I wanted to get more exposure to the research world in Bristol, meet different researchers and explore with them different ways of working and approaching data.
From a technical perspective, I had the opportunity to work on projects related to psychometric data, biological matrices, proteins, chemometrics and mapping. I also worked mainly with R and in some cases SPSS, which offered different alternatives for data analysis and presentation.
One of the most challenging projects was working with chemometric concentrations of different residues of chemical compounds extracted from vessels used in human settlements in the past. This challenge allowed me to talk to specialists in the field and to work in a multidisciplinary way in developing data matrices, extracting coordinates and creating maps in R. The most rewarding part was being able to use a colour scale to represent the variation in concentration of specific compounds across settlements. This was undoubtedly a great experience and a technique that I had never had the opportunity to practice.
ASK-JGI also promoted many events, especially Bristol Data Week, which allowed many interested people to attend courses at different levels specialising in the use of data analysis software such as Python and R.
The Ask-JGI team have made this year an enjoyable experience. As a cohort, we have come together to provide interdisciplinary advice to support various projects. I would highly recommend anyone with an interest in data science and statistics to apply. It is an incredible opportunity for development and networking and allows you to immerse yourself in the wider Bristol community, as well as learning new techniques that you can use during your time at the University of Bristol.
ELLIS Summer School on Machine Learning in Healthcare and Biology – 11 to 13 June 2024
Huw Day, Richard Lane, Christianne Fernée and Will Chapman, data scientists from the Jean Golding Institute, attended the European Laboratory for Learning and Intelligent Systems (ELLIS) Summer School on Machine Learning for Healthcare and Biology at The University of Manchester. Across three days they learned about cutting edge developments in machine learning for biology and healthcare from academic and industry leaders.
A major theme of the event was how researchers create real knowledge about biological processes at the convergence of deep learning and causal inference techniques. Through this machine learning can be used to bridge the gap between well-controlled lab experiments and the messy real world of clinical treatments.
Advancing Medical Science using Machine Learning
Huw’s favorite talk was “From Data to Discovery: Machine Learning’s Role in Advancing (Medical) Science” by Mihaela van der Schaar who is a Professor of ML, AI and Medicine at the University of Cambridge and Director for the Cambridge Centre for AI in Medicine.
Currently, machine learning models are excellent at deducing the association between variables. However, Mihaela argued that we need to go beyond this to understand the variables and their relationships so that we can discover so-called “governing equations”. In the past, human experts have discovered governing equations with domain knowledge, intuition and insight to extract equations from underlying data.
The speaker’s research group have been working to deduce different types of underlying governing equations from black box models. They have developed techniques to extract explicit functions as well as more involved functional equations and various types of ordinary and partial differential equations.
Slide 39 from Mihaela van der Schaar’s talk, showing observed data of the effects of chemotherapy on tumour volume over time and then two examples of derived governing equations in plots on the left with the actual equations written out on the right
The implications for healthcare applications are immense if these methods are able to be reliability integrated into our existing machine learning analysis. On a more philosophical angle, it begs interesting questions about how many systems in life sciences (and beyond) have governing equations and what proportion of these equations are possible to discover.
Gaussian processes and simulation of clinical trials
A highlight for Will and Christianne was the informative talk from Ti John which was a practical introduction to Gaussian Processes (GP) which furthered our understanding of how GPs learn non-linear functions from a dataset. By assuming that your data are a collection of realisations of some unknown random function (or combination of functions), and judicious choice of kernel, Gaussian Process modelling can allow the estimation of both long-term trends from short-term fluctuations from noisy data. The presentation was enhanced with this interactive visualisation of GPs, alongside an analysis of how the blood glucose response to meals changes after bariatric surgery. Another highlight was Alejandro Frangi’s talk on in silico clinical trials in which he described how mechanistic modelling (like fluid dynamic simulations of medical devices) can be combined with generative modelling (to synthesise a virtual patient cohort) to explore how medical treatments may perform in patients who would never have qualified for inclusion in a real randomised controlled trial.
Causality
Richard’s favourite talk was by Cheng Zhang from Microsoft Research on causal models, titled “Causality: From Statistics to Foundation Models”. Cheng highlighted that an understanding of causality is vital for the intelligent systems that have a role in decision-making. This area is on the cutting edge of research in AI for biology and healthcare – understanding consequences is necessary for a model that should propose interventions. While association (statistics) is still the main use case for AI, such models have no model of the “true” nature of the world that they are reflecting which leads to hallucinations such as images with too many fingers or nonsensical text generation. One recipe proposed by Cheng was to build a causally-aware model to:
Apply an attention mechanism/transformer to data so that the model focuses only on the most important parts
Use a penalised hinge loss- the model should learn from its mistakes, and should account for some mistakes being worse than others
Read off optimal balancing weights + the causal effect of actions – after training, we need to investigate the model to understand the impact of different actions.
In essence, this is a blueprint to build a smart system that can look at a lot of complex data, decide what’s important, learn from its mistakes efficiently and can help us understand the impact of different actions. As an example, we might be interested in how the amount of time spent studying affects students’ grades. At the outset, it’s hard to say if studying more causes better grades because these students might also have other good habits, have access to more resources, etc. Such a model would try to balance these factors and give a clearer picture of what causes what- effectively, it would try to find the true effect of studying more on a student’s grade, after removing the influence of other habits.
This behaviour is desired for the complex models being developed for healthcare and biology; for example, we may be interested in engineering CRISPR interventions to make crops more resilient to climate change or developing brain stimulation protocols to help with rehabilitation. A model proposing interventions for these should have a causal understanding of which genes impact a trait, or how different patterns of brain activity affect cognitive function.
A public event organised by The Alan Turing Institute – 20 June 2024 Blog post by Léo Gorman, Data Scientist, Jean Golding Institute
Let’s say you are a researcher approaching a new dataset. Often it seems that there is a virtually infinite number of legitimate paths you could take between loading your data for the first time and building a model that is useful for prediction or inference. Even if we follow statistical best practice, it can feel that even more established methods still don’t allow us to communicate our uncertainty in an intuitive way, to say where our results are relevant and where they are not, or to understand whether our models can be used to infer causality. These are not trivial issues. The Alan Turing Institute (the Turing) hosted a theory and methods challenge fortnight (TMCF), where leading researchers got together to discuss these issues.
JGI team members Patty Holley, James Thomas and Léo Gorman (left to right) at the Turing
Members of the Jean Golding Institute (Patty Holley, James Thomas, and Léo Gorman) went to London to participate in this event, and to meet with staff at the Turing to discuss opportunities for more collaboration between the Turing and the University of Bristol.
In this post, I aim to provide a brief summary of my take-home messages that I hope you will find useful. At the end of this post, I recommend materials from all three speakers which will cover these topics in much more depth.
Andrew Gelman – Beyond “push a button, take a pill” data science
Andrew Gelman presenting
Gelman mainly discussed how are statistics used to assess the impact of ‘interventions’ in modern science. Randomised controlled trials (RCTs) are considered the gold-standard option, but according to Gelman, the common interpretation of these studies could be improved. First, the trials need to be taken in context, and it needs to be understood that these findings might be different in another scenario.
We need to move beyond the binary “it worked” or “it didn’t” outcomes. There are intermediate outcomes which help us understand how well a treatment worked. For example, let’s take cancer treatment trial. Rather than just looking at if a treatment worked for a group, we could look at how the size of the tumour changed, and whether this changed for different people. As Gelman says in his blog: “Real-world effects vary among people and over time”.
Jessica Hullman – What do we miss with average model effects? How can simulation and data visualisation help?
Jessica Hullman presenting
Hullman’s talk expanded on some of the themes in Gelman’s talk, Let’s continue with the example of an RCT for cancer treatment. If we saw an average effect of 0.1 between treatment and control, how would that vary for different characteristics (represented by the x-axis in the quartet of graphs below). Hullman demonstrated how simulation and visualisation can help us understand how different scenarios can lead to the same conclusion.
Causal quartets, as shown in Gelman, Hullman, and Kennedy’s paper. All four plots show an average effect of 0.1, but these effects vary as a function of an explanatory variable (x-axis)
Hadley Wickham – Challenges with putting data science into production
Hadley Wickham presenting
Wickham’s talk focused on some of the main issues with conducting reproducible, collaborative, and robust data science. Wickham framed these challenges under three broad themes:
Not just once: an analysis likely needs to be runnable more than once, for example you may want to run the same code on new data as it is collected.
Not just on my computer: You may need to run some code on your own laptop, but also another system, such as the University’s HPC.
Not just me: Someone else may need to use your code in their workflow.
According to Wickham, for people in an organisation to be able to work on the same codebase, they have the following needs (in order of priority), they need to be able to:
find the code
run the code
understand the code
edit the code.
These challenges exist at all types of organisation, and there are surprisingly few cases where organisations fulfil all criteria.
Panel discussion – Reflections on data science
Cagatay Turkay, Roger Beecham, Hadley Wickham, Andrew Gelman, Jessica Hullman (left to right) at the Turing
Following each of their individual talks, the panellists reflected more generally. Here are a few key points:
Causality and complex relationships: When asked about the biggest surprises in data science over the past 10 years both Gelman and Hullman seemed surprised at the uptake of ‘blackbox’ machine learning methods. More work needs to be done to understand how these models work and to try and communicate uncertainty. The causal quartet visualisations, presented in the talk, only addressed simple/ideal cases for causal inference. Gelman and Hullman both said that figuring out how to understand complex causal relationships for high-dimensional data was at the ‘bleeding edge’ of data science.
People problems not methods/tools problems: All three panellists agreed that most of the issues we face in data science are people problems rather than methods/tools problems. Much of the tools/methods exist already, but we need to think more careful.
Léo’s takeaway
The whole trip reminded me of the importance of continual learning, and I will definitely be spending more time going through some of Gelman’s books (see below).
Gelman and Hullman’s talk in general encouraged people to think: At each point in my analysis, were there alternative choices that could have been made that would have been equally reasonable, and if so, how different would my results have looked had I made these choices? This made me want to think more about multiverse analyses (see analysis package and article).
Further Reading
Theory and Methods Challenge Fortnight – Garden of Forking Paths
The speakers were there as part of the Turing’s Theory and Methods Challenge Fortnight (TMCF), more information can be found below:
TMCF website (worth checking for updates and write-ups)
For people who have not heard of Andrew Gelman before, he is known to be an entertaining communicator (you can search for some of his talks online or look at the Columbia statistics blog). He also has several great books:
Again, check the Columbia statistics blog, where Hullman also contributes. The home page of Hullman’s website also includes selected publications which cover causal quartets, but also reliability and interpretability for more complex models.
Hadley Wickham
Wickham has made many contributions for R and data science. He is chief scientist at Posit and is lead of the tidyverse team. His book R for Data Science is a particularly useful resource. Other work can be found on his website.
Researchers across the University benefit from our JGI Seedcorn Funding. Funding is great when you have someone to do the work – but what if you don’t have the right data science expertise in house? For that, this summer we are trialling a new JGI Data Scientist Support service. This provides an alternative support mechanism for researchers who need expertise and time, but not funding.
The Jean Golding Institute’s team of data scientists and research software engineers are here to support researchers across the University of Bristol fostering a collaborative research environment spanning multiple disciplines. Over the past seven years, our team has expanded thanks to various funding sources, reflecting the increasing importance of data science support in facilitating research outcomes and impact.
Get in touch with our team to find out how they can help you with:
Data analysis – recommendations or support with tools and methods for statistics, modelling, machine learning, natural language processing, computer vision, geospatial datasets and reproducible data analysis.
Software development – technical support, coding (for example: Python, R, MATLAB, SQL, bash scripts), code review and best practices.
Data communication – data visualisation, dashboards and websites.
Research planning – experimental design, data management plans, data governance, data hazards and ethics.
Our aim is to support researchers and groups that may not have in-house expertise but have project ideas that can be developed into applications for funding. We’re seeking projects that can take place over the summer until early autumn (July – October 2024).
The JGI team will get back to you within one week, to discuss your request.
If demand exceeds our current resource levels, we’ll meet with applicants to help prioritise projects. As with seedcorn funding, priority will go to applications that match JGI strategic goals and have clear pathways to benefit, such as an identified funding call or impact case.
Examples of data science projects
Social mobility analysis project – using local and national level data to investigate how different people in Bristol and other UK cities feel about life in their local environment. The JGI data scientist worked as part of a multidisciplinary team including University of Bristol researchers and external stakeholders, for around 2 days per week for 3 months. They analysed survey and geospatial data using Python, presented findings to the group. The output of the project was a grant application in which a data scientist was costed longer-term.
Antimicrobial resistance project – examining patterns in observed levels of antimicrobial resistance during the COVID pandemic. The JGI data scientist worked with a University of Bristol researcher and collaborated with a public sector stakeholder, for around 4 days per week for 4 months. They performed statistical modelling using R, producing data visualisations of the trends found. The project has led to an Impact Acceleration Funding application to develop a tool used to support local health planning.
Transport research-ready dataset grant – linking administrative datasets to support research into car and van use in the UK. The JGI data scientist developed data pipelines and provided methodological and data governance input into a successful ESRC funding application in a collaboration between researchers at the universities of Bristol and Leeds. The data scientist was a named researcher on the application and went on to perform data analysis as part of the project team.
