Dino Oglic

Senior Director & Head of Research, Center for AI

Biomedical Campus

AstraZeneca, Cambridge

I work at AstraZeneca Cambridge (UK) as the Head of Research in the recently formed Center for AI within Data Sciences & AI, BioPharmaceuticals R&D. Our mission is to devise innovative products and solutions using machine learning algorithms that will make the drug discovery pipeline more efficient and aid in a better understanding of biology and medicinal chemistry.

I obtained my doctorate in machine learning at the University of Bonn in 2018, with the focus on foundational machine learning and kernel methods. After holding research scientist and post-doctoral positions at the University of Nottingham and King’s College London, I joined the Late Respiratory and Immunology machine learning group at AstraZeneca Cambridge. In 2022, I achieved a significant career milestone by advancing to the role of Senior Director in Machine Learning and AI. Initially, I assumed dual responsibilities as the Head of Audio and Signal Processing team and served as the interim Head of Research. In March 2023, I was appointed as the Head of Research at the Center for AI and a member of the Leadership Team. This progression underscores my commitment to driving innovation and excellence in the dynamic field of machine learning and AI.

research

I am interested in theoretical aspects of machine learning and techniques for the design and analysis of computationally efficient learning algorithms. In addition to this, I am interested in artificial intelligence and its applications such as speech recognition and the design of materials and molecules. I also maintain an active interest in numerical and functional analysis, causality, randomized algorithms, non-convex optimization, graph theory, natural language processing, and signal processing. My primary research interests include:

1. Learning with structured data

   Real-world applications of machine learning often involve data that has no natural representation in Euclidean spaces (i.e., as fixed dimensional real-valued vectors). For instance, this is the case with molecular property prediction (e.g., binding affinity, solubility, toxicity, etc.) or generative models involving protein sequences and structures. I am interested in designing efficient deep learning algorithms and developing novel architectures for learning with structured data. The main focus of my research in the post-2018 period is on unsupervised and supervised representation learning involving graphs and sequences, as well as generative models and transfer learning for improved prediction on downstream small-sample tasks (e.g., molecular properties). For instance, I have recently been working (published and unpublished) on large language models for antibodies and proteins, zero-shot models for lead optimization of antibodies and nanobodies, supervised representation learning with graph neural networks, and effective attention-based models for learning on graphs.

2. Robust representation learning

   The state-of-the-art deep learning models have demonstrated remarkable effectiveness on some learning tasks involving signals (e.g., speech recognition in clean conditions). However, these models fail to generalize to unseen conditions that could arise as a result of an intervention (e.g., a change in the marginal distribution of the data generating process). I am interested in deriving mathematical structures and representations that could provide means to support standard distribution shifts/interventions and other notions of robustness. The main focus of my work here is on devising noise robust representations and deep learning algorithms for automatic speech recognition in the waveform domain. I have so far investigated the effectiveness of techniques from signal processing based on bandpass filtering of speech signals, implicit convolutional blocks, deep convolutional networks, and means to incorporate regularization mechanisms into deep learning models (e.g., using variational dropout and Bayesian deep learning).

3. Kernel methods

   In addition to information in labeled training data, supervised learning algorithms need a notion of similarity on the available instances in order to generalize from training examples to unseen observations. This information can be provided through a similarity function that quantifies the relationship between any pair of instances. A kernel function is a special type of similarity function that is symmetric and positive definite. Kernel methods give us a flexible framework to model complex functional relationships in a principled way and also come with well-established statistical properties and theoretical guarantees. I have worked on the development and analysis of methods for alleviating some of the shortcomings characteristic to this class of learning algorithms.

   • Scalability

     The computational and space complexities of learning algorithms from this class are at least quadratic in the number of instances and this is too expensive on modern datasets. In collaboration with colleagues from Bonn/Nottingham and Oxford, I have investigated theoretical properties of approaches for low-rank approximation of kernel matrices based on the Nyström method (via kernel k-means++ landmarks) and random Fourier features (allowing for scalable learning with positive definite kernels).

   • Indefinite kernels

     In learning problems with structured data, it is relatively easy to devise a kernel based on intuition of domain experts. Such functions are typically not positive definite and it is often the case that verifying this condition is well beyond the expertise of practitioners. My prior work provides an extension of standard kernel methods to indefinite kernels, thus, alleviating the requirement for positive definiteness (allowing scalable learning with any notion of similarity between instances).

   • Inductive bias

     A fundamental limitation of learning procedures with an a priori specified hypothesis space (e.g., kernel methods with a preselected kernel) is that they can learn good concept descriptions only if the hypothesis space selected beforehand is large enough to contain a good solution to the considered problem and small enough to allow good generalization from a small number of training examples. To tackle this problem, we proposed an approach for supervised feature construction that iteratively extends the feature representation by performing functional gradient descent over a dictionary of features (capable of learning any stationary positive definite kernel).

4. Active learning / search with a focus on the design of molecules and materials

   I have worked on the drug-design problems with small-molecules through my research on active search and Bayesian optimization in structured spaces. Active search deals with the discovery of items from structurally different partitions of a rare and valuable (fixed, but unknown) target class. It can be a challenging setting to study theoretically because the data is neither independent nor identically distributed. In collaboration with colleagues from the Department of Chemistry (Nottingham) and GlaxoSmithKline, we have demonstrated the effectiveness of our active search algorithm on the design of molecules effective against idiopathic pulmonary fibrosis.

I am also following the developments in the following machine learning research areas:

1. Reliable and robust machine learning
2. Active learning
3. Federated learning
4. Self-supervised learning