Show Notes
Quan Zhou, Texas A and M University, gives an OxCSML Seminar on Friday 25th June 2021. Abstract:
In a model selection problem, the size of the state space typically grows exponentially (or even faster) with p (the number of variables). But MCMC methods for model selection usually rely on local moves which only look at a neighborhood of size polynomial in p. Naturally one may wonder how efficient these sampling methods are at exploring the posterior distribution. Consider variable selection first. Yang, Wainwright and Jordan (2016) proved that the random-walk add-delete-swap sampler is rapidly mixing under mild high-dimensional assumptions. By using an informed proposal scheme, we obtain a new MCMC sampler which achieves a much faster mixing time that is independent of p, under the same assumptions. The mixing time proof relies on a novel approach called "two-stage drift condition", which can be useful for obtaining tight complexity bounds. This result shows that the mixing rate of locally informed MCMC methods can be fast enough to offset the computational cost of local posterior evaluation, and thus such methods scale well to high-dimensional data. Next, we generalize this result to other model selection problems. It turns out that locally informed samplers attain a dimension-free mixing time if the posterior distribution satisfies a unimodal condition. We show that this condition can be established for the high-dimensional structure learning problem even when the ordering of variables is unknown.
This talk is based on joint works with H. Chang, J. Yang, D. Vats, G. Roberts and J. Rosenthal.
Bio: Quan Zhou is an assistant professor of the Department of Statistics at Texas A&M University (TAMU). Before joining TAMU, he was a postdoctoral research fellow at Rice University. He did his PhD at Baylor College of Medicine.
More from Department of Statistics
A lecture exploring alternatives to using labeled training data. Labeled training data is often scarce, unavailable, or can be very costly to obtain. To circumvent this problem, there is a growing interest in developing methods that can exploit sources of information other than labeled data, such as weak-supervision and zero-shot learning. While these techniques obtained impressive accuracy in practice, both for vision and language domains, they come with no theoretical characterization of their accuracy. In a sequence of recent works, we develop a rigorous mathematical framework for constructing and analyzing algorithms that combine multiple sources of related data to solve a new learning task. Our learning algorithms provably converge to models that have minimum empirical risk with respect to an adversarial choice over feasible labelings for a set of unlabeled data, where the feasibility of a labeling is computed through constraints defined by estimated statistics of the sources. Notably, these methods do not require the related sources to have the same labeling space as the multiclass classification task. We demonstrate the effectiveness of our approach with experimentations on various image classification tasks.
A high-level overview of key areas of AI ethics and not-ethics, exploring the challenges of algorithmic decision-making, kinds of bias, and interpretability, linking these issues to problems of human-system interaction. Much attention is now being focused on AI Ethics and Safety, with the EU AI Act and other emerging legislation being proposed to identify and curb "AI risks" worldwide. Are such ethical concerns unique to AI systems - and not just digital systems in general?
A brief introduction to various legal and procedural ethical concepts and their applications within and beyond academia. It's all very well to talk about truth, beauty and justice for academic research ethics. But how do you do these things at a practical level? If you have a big idea, or stumble across something with important implications, what do you do with it? How do you make sure you've got appropriate safeguards without drowning in bureaucracy?
David Steinsaltz gives a lecture on the ethical issues in statistics using historical examples.
This seminar explains and illustrates the approach of Markov melding for joint analysis. Integrating multiple sources of data into a joint analysis provides more precise estimates and reduces the risk of biases introduced by using only partial data. However, it can be difficult to conduct a joint analysis in practice. Instead each data source is typically modelled separately, but this results in uncertainty not being fully propagated. We propose to address this problem using a simple, general method, which requires only small changes to existing models and software. We first form a joint Bayesian model based upon the original submodels using a generic approach we call "Markov melding". We show that this model can be fitted in submodel-specific stages, rather than as a single, monolithic model. We also show the concept can be extended to "chains of submodels", in which submodels relate to neighbouring submodels via common quantities. The relationship to the "cut distribution" will also be discussed. We illustrate the approach using examples from an A/H1N1 influenza severity evidence synthesis; integrated population models in ecology; and modelling uncertain-time-to-event data in hospital intensive care units.