Reproducibility of statistical tests in pharmaceutical products development

EPSRC-CASE PhD project at Durham University in collaboration with AstraZeneca (Cambridge)

Prof. Frank Coolen, Department of Mathematical Sciences, Durham University (frank.coolen@durham.ac.uk)

Project description

Statistical inference about reproducibility of tests is a non-trivial topic about which there has been considerable confusion in the literature. Recently, it has started to receive increasing attention, but it remains a topic for which classical statistics approaches are not well suited. Prof. Frank Coolen has recently, in collaboration with PhD student Sulafah Bin Himd, developed nonparametric predictive inference (NPI) for reproducibility of tests. NPI is a frequentist statistical method with explicit focus on prediction, so considering events in terms of future observations. NPI is based on relatively few modelling assumptions, enabled by the use of imprecise probabilities to quantify uncertainty. It seems logical to consider reproducibility of tests as a predictive problem. The use of NPI to derive lower and upper probabilities of reproducibility, that is the event that a future repeat of the test would lead to the same overall test conclusion (typically rejecting a null-hypothesis or not), provides an attractive alternative to classical statistical methods.

In this research project, the student will first study the initial results on NPI for reproducibility, in which some very basic statistical tests were considered. The student will also gain an insight into practical tests during the product development processes at AstraZeneca, and a choice of these tests will be made to focus on first for the development of NPI methods to investigate the test reproducibility. It is expected that the student can study reproducibility of a variety of tests during this project, leading to valuable insights at AstraZeneca and to novel statistical methodology supported by algorithms in the statistical software R to implement the new methods.

The following are examples of possible test scenarios at AstraZeneca that could be considered during this project:

• Assessment of growth inhibition of tumour cell xenografts implanted into rodent species to assess compound potency. Tumour cells are implanted into rodents and allowed to develop. Rodents will be treated with (increasing) concentrations of potential drug compounds. Growth of the tumour (either change in tumour size or final/start size) is monitored, potentially alongside a number of biochemical measurements describing the within tumour. The analyses typically involve ANOVA and comparison of multiple groups against a control. The goal of a reproducibility study is to understand if repeating the study under the same conditions would lead to the same results in terms of significant differences in growth inhibition or other biomarkers.
• Experiments similar to the above description, but analysis undertaken by fitting a dose response curve to inhibition data to derive an IC50 estimate. This can be used to rank potential compounds according to efficacy, or to compare them, i.e. are IC50 values from two different compounds significantly different from each other? Would repeating such an experiment lead to a similar IC50 value, or with similar confidence intervals?
• It is also of interest to consider how inference on reproducibility might influence experimental design, and to compare the NPI methods for reproducibility of tests to alternative methods, for example a priori (or posteriori) power calculations based on the variability within the data. The methods can also be compared to alternative approaches for investigating reproducibility, e.g. Bayesian or bootstrap-based methods.

Supervision Team

Practicalities

Stipend

Further Information and Applying

Information about AstraZeneca is available here: AstraZeneca, and about the Durham University Mathematical Sciences Department here: Maths Department.

If you are interested in this opportunity, or want further information, please contact Prof. Frank Coolen (frank.coolen@durham.ac.uk) by Friday 24 March.

Frank Coolen