ICMS

Mar 3, 2010 - Mar 5, 2010

15 South College Street, Edinburgh EH8 9AA

Organisers

Name	Institution
Mengersen, Kerrie	Queensland University of Technology
Robert, Christian P	University of Paris-Dauphine
Titterington, Mike	University of Glasgow

 

Statistical mixture distributions are used to model scenarios in which certain variables are measured but a categorical variable is missing. For example, although clinical data on a patient may be available their disease category may not be, and this adds significant degrees of complication to the statistical analysis. The above situation characterises the simplest mixture-type scenario; variations include, among others, hidden Markov models, in which the missing variable follows a Markov chain model, and latent structure models, in which the missing variable or variables represent model-enriching devices rather than real physical entities. In the title of the workshop the term `mixture' is taken to include these and other variations along with the simple mixture. The motivating factors for this three-day workshop are that research on inference and computational techniques for mixture-type models is currently experiencing major advances and that simultaneously the application of mixture modelling to many fields in science and elsewhere has never been so rich. We thus assembling top players, from statistics and computer science, in both methodological research and applied inference at this fertile interface. The methodological component will involve both Bayesian and non-Bayesian contributions and biology and economics will feature strongly among the application areas to be covered.

Funded by
EPSRC
LMS
Edinburgh Mathematical Society
Glasgow Mathematical Journal Trust
Royal Statistical Society

 

Arrangements

Participation is by invitation only.  This workshop is now full and applications are closed.

UK Visas
If you are travelling from overseas you may require an entry visa. A European visa does not guarantee entry to the UK. Please use this link to the UK Visas site to find out if you need a visa and if so how to apply for one.

Venue
The workshop will be held at 15 South College Street, Edinburgh. All lectures will be held in the Newhaven Lecture Theatre. To view this room and a list of the visual equipment available click here. In addition, two blackboards have recently been installed. Follow this link for a map showing the location of 15 South College Street, or this map also prove useful.  

Wireless Access
The workshop venue, 15 South College Street, has wireless access throughout. On arrival at Registration you will be given instructions and a code for accessing the wireless network.  For those without laptops, there will also be a couple of computers available for you to check your emails.

Travel
Information about travel to the UK and Edinburgh is available here.

A taxi directly from the airport will cost approximately 15.00 to 20.00 GBP to the city centre for a one-way journey.

Lothian buses charge £1.20 for a single, £3.00 for a day ticket. Please note that the exact fare is required and no change is given.

If travelling by train, please note that Edinburgh has two railway stations - Waverley Railway Station being the main station and closest to the workshop venue at 15 South College Street. If you alight at Edinburgh Waverley, the workshop venue is an easy 10 minute walk over North and South Bridge map. The second railway station is called Haymarket and is at the West End of the city centre.

Accommodation
Please refer to you invitation email to check if ICMS is offering to reserve accommodation for you, or if you are expected to arrange and pay for your own accommodation.   For those arranging their own accommodation, a list of Edinburgh of various hotels and guest houses and prices is available here . Sections 4 is particularly relevant.  You may also find the following suggestions helpful:

Jurys Inn Edinburgh, 43 Jeffrey Street, Edinburgh EH1 1DH (close to workshop venue and city centre)
+44 (0)131 200 3300 - www.jurys-edinburgh-hotels.com 
If booking, please inform the hotel that you are attending an ICMS event and that you'd like to be offered the ICMS corporate rate of 73.00 GBP per night for bed and breakfast.  

The Kenneth Mackenzie Suite, 7 Richmond Place, Edinburgh, EH8 9ST (University accommodation)
+44 (0)131 651 2007 - http://www.edinburghfirst.com/accommodation/kennethmackenzie.asp

Hotel Ibis, 6 Hunter Square, Off Royal Mile, Edinburgh EH1 1QW
+44 (0)131 240 7000 - www.hotels-europe.com/ibishotels/england/edinburgh.htm

Smiths Guest House, 77 Mayfield Road, Edinburgh EH9 3AA
+44 (0)131 667 2524 - www.smithsgh.com - mail@smithsgh.com

Scottish Youth Hostel Association
0870 155 3255 (UK only) - Scottish Youth Hostel Association

Edinburgh Tourist Board - www.edinburgh.org/accom

Hotel Direct - www.hoteldirect.co.uk/edinburgh

Registration
Registration will take place 08.45 - 09.35 on Wednesday 3 March at 15 South College Street, Edinburgh.

Catering
Morning and afternoon tea/coffee/biscuits as well as a buffet lunch will be provided on each day of the workshop.

A workshop dinner will take place on Thursday 4 March at 19.30 at Blonde Restaurant, 75 St Leonards Street, Edinburgh, EH8 9QR.

Poster Session
There will be a poster session from 16.30 to 17.30 on Wednesday 3 March.  Anyone wishing to present a poster is invited to put up the poster during the lunch break or afternoon coffee on that day in readiness for the session.  Ideally posters should be A1 size, however, it will not present a problem if posters are larger.    

Public Lecture by Kerrie Mengersen (Queensland University of Technology)
Doors will open at 17.30 on Wednesday 3 March for a Public Lecture entitled, "Where are they and what do they look like?  Discovering patterns in data using statistical mixture models"  pdf of abstract  The lecture will commence at 18.00 prompt.  Please note that latecomers will not be permitted into the lecture theatre.  Seats have already been reserved for all participants in the Mixture Estimation workshop.     

Distinguished Lecture by Michael Jordan
This lecture will be held in Room G07, the Informatics Forum, 10 Crichton Street, Edinburgh on Thursday 4 March at 16.30as part of the Informatics Distinguished Lecture series.   It is entitled, "Applied Bayesian nonparametrics"  pdf of abstract

Financial Arrangements
Your individual financial arrangements will be laid out in your invitation email and the email received shortly before the workshop.

If we have agreed to pay some of your travel costs, you will have been informed by email. Reimbursement will take place after the workshop and will involve payment directly into your bank account. At Registration you will be given an expenses claim form and this should be submitted to ICMS, with original receipts. It would be helpful if you could bring your bank details to the workshop. In addition to the bank account number, participants from the USA and Canada will require their bank’s routing number, those from the UK will be asked for the bank sort code, and those from Europe and the rest of the world, their IBAN and SWIFT/BIC code. We cannot reimburse any item without a signed claim form and original receipts.

 

Programme

Please continue to check this Programme as it may be subject to change:

Wednesday 3 March

08.45 - 09.35	Registration and coffee
09.35 - 09.45	Introduction and welcome
	Bayesian debate on mixture inference
09.45 - 10.30	John Geweke (University of Technology, Sydney) Interpretation and Inference in mixture models  pdf of presentation
10.30 - 11.00	Coffee/tea provided in the Chapterhouse (ground floor)
11.00 - 11.45	Sylvia Fruehwirth-Schnatter (Johannes Kepler University Linz) Dealing with label switching under model uncertainty  pdf of presentation
11.45 - 12.30	Gilles Celeux, Agostino Nobile, Christian P Robert, Kerrie Mengersen (discussants) pdf of presentation - celeux        pdf of presentation - Nobile pdf of presentation - Robert
12.30 - 13.00	General discussion
13.00 - 14.00	Lunch provided in the Chapterhouse (ground floor)
14.00 - 15.00	Methodology (25 minute talks plus 10 minutes discussion)
	Murray Aitkin (University of Melbourne) How many normal components in the galaxy velocity data? Posterior deviance  distributions for the number of components, and their interpretation  pdf of presentation
	Geoff McLachlan (University of Queensland) The modelling of high-dimensional data via normal mixture models  pdf of presentation
15.00 - 15.30	Coffee/tea provided in the Chapterhouse (ground floor)
15.30 - 16.30	Asymptotic theory (25 minute talks plus 10 minutes discussion)
	Jiahua Chen (University of British Columbia) Testing the order of finite mixture models by EM-test   pdf of presentation
	Bruce Lindsay (Pennsylvania State University) Mixture related  analysis in many dimensions  pdf of presentation
16.30 - 17.30	Poster Session in the Atrium (Level 3).  Coffee/tea are available from the Chapterhouse during this poster session.
18.00 - 19.00	Public Lecture by Kerrie Mengersen (Queensland University of Technology) Where are they and what do they look like?  Discovering patterns in data using statistical mixture models pdf of abstract pdf of presentation
19.00 - 20.00	Informal wine reception in the Chapterhouse, ground floor, 15 South College Street.  Please note that the building must be vacated by 20.15.

 

Thursday 4 March

09.15 - 10.15	Michael Jordan (University of California, Berkeley) Completely random measures, hierarchy, and nesting in Bayesian nonparametrics pdf of presentation
10.15 - 10.45	Coffee/tea provided in the Chapterhouse (ground floor)
10.45 - 12.15	Machine learning etc (25 minute talks plus discussion)
	Yee Whye Teh (University College London) On hierarchical clustering, partitions, and mixture models  pdf of presentation
	Iain Murray (University of Toronto & University of Edinburgh) Sampling latent variable models  pdf of presentation
	Katherine Heller (University of Cambridge) The IBP compound Dirichlet process and its application to focused topic modelling
12.15 - 13.30	Lunch provided in the Chapterhouse (ground floor)
13.30 - 14.30	Machine learning etc (25 minute talks plus 10 minutes discussion)
	Chris Williams (University of Edinburgh) Greedy learning of binary latent trees  pdf of presentation
	Mark Girolami (University of Glasgow) Inferring spectral mixture components in multiplexed Surface Enhanced Raman Resonance Spectroscopy   pdf of presentation
14.30 - 15.00	Coffee/tea provided in the Chapterhouse (ground floor)
15.00 - 16.00	Bayesian computational methodology (25 minute talks plus 10 minutes discussion)
	Christophe Andrieu (University of Bristol) Exact approximations of MCMC algorithms
	Paul Fearnhead (Lancaster University) Sequential Monte Carlo methods and perfect sampling for mixture models  pdf of presentation
16.30	Distinguished Lecture by Michael Jordan (University of California, Berkeley) To be held in Room G07, the Informatics Forum, 10 Crichton Street, Edinburgh Applied Bayesian nonparametrics  pdf of abstract
19.30	Workshop Dinner at Blonde Restaurant, 75 St Leonard's Street, Edinburgh

 

Friday 5 March

09.15 - 10.45	(25 minute talks plus discussion)
	Clare Alston (Queensland University of Technology) Bayesian mixture models: a blood free dissection of a sheep  pdf of presentation
	Olivier Cappé (Telecom ParisTech & CNRS) Online EM algorithms for mixtures, HMMs and beyond  pdf of presentation
	Brendan Murphy (University College Dublin) A mixture of experts latent position cluster model for social network data  pdf of presentation
10.45 - 11.15	Coffee/tea provided in the Chapterhouse (ground floor)
11.15 - 12.15	Financial applications (25 minute talks plus discussion)
	Robert Kohn (University of New South Wales) Bayesian mixtures of autoregressive models   pdf of presentation
	Richard Gerlach (University of Sydney) Smooth transition mixture GARCH models for forecasting risk measures in financial markets   pdf of presentation
12.15 - 13.30	Lunch provided in the Chapterhouse (ground floor)
13.30 - 15.30	Biological sciences (25 minute talks plus discussion)
	Kim-Anh Do (University of Texas M.D. Anderson Cancer Centre) Bayesian mixture modelling with applications to translational cancer research  pdf of presentation
	Michael Newton (University of Wisconsin, Madison) Gamma-based clustering via ordered means with application to gene-expression analysis
	Peter Müller (University of Texas M.D. Anderson Cancer Centre) Bayesian semiparametric mixture models with covariate-dependent weights  pdf of presentation
	Chris Holmes (University of Oxford) Investigations in variable selection for Bayesian mixture models   pdf of presentation
15.30 - 16.00	Coffee/tea provided in the Chapterhouse (ground floor).   Close of workshop.

 

Presentations:

Presentation Details
Aitkin, Murray
How many normal components in the galaxy velocity data? Posterior deviance distributions for the number of components, and their interpretation
View Abstract Hide Abstract
This talk revisits the extensive discussion of likelihood and Bayesian methods for finite mixtures, and their application to the galaxy data, in Aitkin (2001 Statistical Modelling 1). Posterior distributions are given for the deviances for 1-7 components (with different means and variances), and are interpreted through the posterior distributions for the corresponding likelihood ratios and posterior odds. The conclusion from this approach is very clear: no more than three normal components are supported by the data. A full discussion of this approach, and the application to the galaxy data, can be found in my new book "Statistical Inference: an Integrated Bayes/Likelihood Approach" to be published by Chapman and Hall/CRC in 2010.
Alston, Clair
Bayesian mixture models: a blood free dissection of a sheep
View Abstract Hide Abstract
In this talk we will present a Bayesian Mixture Model analysis of CT scan data measured at regular intervals on a live sheep carcass. The aim of this analysis is to determine the proportion / volume of the three major tissues, fat, muscle and bone present in the animal. We will outline a spatial Normal mixture model which allows the neighbouring pixels within a scan to influence the allocation of a pixel to a component group. Priors and starting values for this model will be summarised. The model will be extended to account for binning of the data and spatial and / or temporal neighbouring scans. The resulting estimates for a series of scans from a single sheep will then be used to calculate an estimate of volume using the Cavelleri approach. These results will then be assessed and compared with the results from the simplistic boundary techniques which are more commonly employed. Computational and implementation issues will also be discussed.
Andrieu, Christophe
Exact approximations of MCMC algorithms
View Abstract Hide Abstract
The notion of exact approximations of MCMC algorithms has recently arisen in various areas of statistics where latent variables are present. The talk will review this general framework and current applications and discuss the current understanding of their theoretical properties.
Cappé, Olivier
Online EM algorithms for mixtures, HMMs and beyond
View Abstract Hide Abstract
With the generalization of sources of information that generate sustained high volumes of data, there has been recently a renewed interest in online estimation for various statistical models. In this talk, I consider versions of the Expectation-Maximization (EM) algorithm that can be used for online estimation in latent data models. Starting from the case of finite mixtures, I discuss generalizations to hidden Markov models (HMMs) as well to more general state-space models. Keywords: Online Estimation; Expectation-Maximization Algorithm; Exponential Family Models; Stochastic Approximation; Hidden Markov Models; Sequential Monte Carlo.
Chen, Jiahua
Testing the order of finite mixture models by EM-test
View Abstract Hide Abstract
Designing a valid and easy-to-use statistical test for the order of the finite mixture model is challenging. Most results on hypothesis tests to date have focused on a special case of homogeneity. Some general results are limited to theoretical interest and difficult to implement. In this talk, we apply an EM-test for the null hypothesis of order $m_0$ versus an alternative hypothesis of order larger than $m_0$. The null limiting distribution of the proposed EM-test is found to be a mixture of $chi^2$ distributions. The weights in this mixture limiting distribution when $m_0 = 1, 2, 3$ have simple forms and can be conveniently estimated. Simulation study indicates the limiting distribution is a good approximation to the finite sample distributions and the test has good efficiency.
Do, Kim-Anh
Bayesian mixture modelling with applications to translational cancer research
View Abstract Hide Abstract
Early detection is critical in disease control and prevention. Biomarkers provide valuable information about the status of a cell at any given time point. Biomarker research has benefited from recent advances in technologies such as gene expression microarrays, and more recently, proteomics. The long term translational research goal is that if drugs can be targeted to specific tissues in the body, then dosage can be altered to achieve the desired effect while minimizing side effects such as toxicity. Motivated by specific problems involving high throughput data in the form of phage peptides, we have developed nonparametric and semiparametric mixture models for real-time analysis in the context of biomarker discovery. Novel statistical methodology development will be highlighted with direct applications to cancer research challenges that address our long term translational goal. (This is joint work with Peter Mueller, Luis Novelo, Wadih Arap, Renata Pasqualini).
Fearnhead, Paul
Sequential Monte Carlo methods and perfect sampling for mixture models
View Abstract Hide Abstract
We will show how sequential Monte Carlo methods can be used to perform inference for mixture models, include models with an unknown number of components. For certain mixture models for discrete data, we will show how we can adapt this algorithm to enable exact (iid) sampling from the posterior. We will discuss how the performance of these and similar algorithms compares to that of MCMC alternatives; including empirical study based on models for inferring population structure from population genetic data.
Frühwirth-Schnatter, Sylvia
Dealing with label switching under model uncertainty
View Abstract Hide Abstract
Many useful methods have been developed to deal with label switching for finite mixtures where the number of components is known. In the first part of the talk, some of these methods will be reviewed. It is demonstrated for simulated as well as real data that post-processing of the MCMC draws by k-means clustering in the point process representation as suggested in the monograph of Frühwirth-Schnatter (2006) is a simple, yet useful tool of dealing with label switching. In the second part of the talk, the role of the prior distribution in this context is investigated. It is shown that certain shrinkage priors help to discriminate between coefficients in the component specific parameter which are more or less homogenous and coefficients which are heterogeneous. This increases the chance of identifying a unique labelling, in particular, if only a few coefficients are relevant for heterogeneity. The final part of the talk deals with label switching when the number of components is unknown. In particular, it is discussed how to handle the labelling problem for overfitting mixtures where it does not make sense to identify the components through a permutation of the component labels. It turns out that k-means clustering in the point process representation help to diagnose overfitting mixtures. Furthermore it provides information which components should be collapsed in order to identify the mixture.
Gerlach, Richard
Smooth transition mixture GARCH models for forecasting risk measures in financial markets
View Abstract Hide Abstract
A continuous mixture of Generalised Autoregressive Conditionally Heteroskedastic (GARCH) models is proposed for application to volatility modelling and risk measurement and management, via quantile or Value at Risk (VaR) forecasting, in financial markets. A Bayesian framework is employed for estimation, inference and forecasting via an adaptive Markov chain Monte Carlo scheme. A mixture prior on the mean parameters is proposed, that alleviates the usual identifiability problem as the speed of transition or continuous mixing parameter tends to zero. An informative prior for this parameter is suggested that allows for reliable inference and a proper posterior, despite the non-integrability of the likelihood function. The proposed methods are illustrated using both simulated and international stock market return series. A VaR forecasting study of six international market return series compares the proposed model to a range of popular alternatives, including the two limiting cases: the double threshold GARCH and standard GARCH models. Results generally favour the proposed continuous mixture model and highlight explosive and smooth nonlinear behaviour in financial markets.
Geweke, John
Interpretation and inference in mixture models
View Abstract Hide Abstract
The mixture model likelihood function is invariant with respect to permutation of the components of the mixture. If functions of interest are permutation sensitive, as in classification applications, then interpretation of the likelihood function requires valid inequality constraints and a very large sample may be required to resolve ambiguities. If functions of interest are permutation invariant, as in prediction applications, then there are no such problems of interpretation. Contrary to assessments in some recent publications, simple and widely used Markov chain Monte Carlo (MCMC) algorithms with data augmentation reliably recover the entire posterior distribution.
Girolami, Mark
Inferring spectral mixture components in multiplexed Surface Enhanced Raman Resonance Spectroscopy
View Abstract Hide Abstract
Surface Enhanced Raman Resonance Spectroscopy (SERRS) is a developing technology which provides the ability to measure the presence and activity of DNA species in cells and organs with the use of nano-particle binding. Multiplexed readouts of this technology have reached a stage of development where formal inferential methodology is required to provide the necessary deconvolution. Ongoing work with analytical chemists will be reported where a A Reversible jump Markov chain Monte Carlo has been developed for the deconvolution task and results on spectral inference from readouts obtained from mouse liver and spleen will be presented.
Heller, Katherine
The IBP compound Dirichlet process and its application to focused topic modelling
View Abstract Hide Abstract
The hierarchical Dirichlet process (HDP) is a Bayesian nonparametric mixed-membership model. While powerful, it assumes that the probability of a mixture component contributing to a data point is positively correlated with the amount that it contributes. In many settings, this is an undesirable prior assumption. For example, in topic modelling a topic (component) might be rare throughout the corpus but dominant within those documents (data points) where it occurs. We develop the IBP compound Dirichlet process (ICD), a Bayesian nonparametric prior that decouples across-data prevalence and within-data proportion in a mixed-membership model. The ICD integrates features from both the HDP and the Indian buffet process (IBP). It assigns a subset of the shared mixture components to each data point. This subset, the data point’s "focus", is determined independently from the amount that each component in the subset contributes to it. We use an ICD mixture model, the focused topic model (FTM), to analyze text corpora. We demonstrate superior performance over the HDP-based topic model. (Joint work with Sinead Williamson, Chong Wang, and David Blei)
Holmes, Chris
Investigations in variable selection for Bayesian mixture models
View Abstract Hide Abstract
Bayesian mixture modelling provides a useful tool for cluster analysis as it allows for a full characterisation of the uncertainty in the principle dependence structures within a dataset. Clustering is often used as an exploratory method to uncover hidden structure in data or to recover suspected structure. In this talk we will report on on-going investigations into the use of priors which induce sparsity in the posterior model and help characterise the relative influence of explanatory variables to cluster data within mixture models. Such priors are appropriate when some, or perhaps many, of the recorded variables are a priori suspected to be irrelevant to the clustering task. We demonstrate a hierarchical approach that provides accurate indication of irrelevant variables and is able to quantify the relative relevance of those variables useful for clustering. We pay particular attention to efficient MCMC sampling schemes for inference within the model while allowing for an unknown number of mixture components and an unknown number of relevant variables.
Jordan, Michael
Completely random measures, hierarchy, and nesting in Bayesian nonparametrics
View Abstract Hide Abstract
Bayesian nonparametric modelling and inference are based on using general stochastic processes as prior distributions. Despite the great generality of this definition, the great majority of the work in Bayesian nonparametrics is based on only two stochastic processes: the Gaussian process and the Dirichlet process. Motivated by the needs of applications, I present a broader approach to Bayesian nonparametrics in which priors are obtained from a class of stochastic processes known as "completely random measures" (Kingman, 1967). In particular I will present models based on the beta process, the Bernoulli process, the gamma process and the Dirichlet process, and on hierarchical and nesting constructions that use these basic stochastic processes as building blocks. I will discuss applications of these models to several problem domains, including protein structural modelling, computational vision, natural language processing and statistical genetics.
Kohn, Robert
Bayesian mixtures of autoregressive models
View Abstract Hide Abstract
In this paper we propose a class of time-domain models for analyzing possibly nonstationary time series. This class of models is formed as a mixture of time series models, whose mixing weights are a function of time. We consider specifically mixtures of autoregressive models with a common but unknown lag. The model parameters, including the number of mixture components, are estimated via Markov chain Monte Carlo methods. The methodology is illustrated with simulated and real data.
Lindsay, Bruce
Mixture related analysis in many dimensions
View Abstract Hide Abstract
A number of my current projects are devoted to understanding both mixtures and the modes of mixtures in many dimensions. Computational considerations in these cases often mean making some compromises between statistical efficiency and practical efficacy. Are halfway solutions okay? And which halfway solutions are best?
McLachlan, Geoff
The modelling of high-dimensional data via normal mixture models
View Abstract Hide Abstract
There has been a proliferation of applications in which the number of experimental units n is comparatively small but the underlying dimension p is extremely large as, for example, in microarray-based genomics and otherhigh- throughput experimental approaches. Hence there has been increasing attention given not only in bioinformatics and machine learning, but also in mainstream statistics, to the analysis of complex data in this situation where n is small relative to p. In this talk, we focus on the clustering of high-dimensional (continuous) data, using normal mixture models. Their use in this context is not straightforward, as the normal mixture model is a highly parameterized one with each component-covariance matrix consisting of p(p+1)/2 distinct parameters in the unrestricted case. Hence some restrictions must be imposed and/or a variable selection method applied beforehand. We shall briefly review the existing literature and consider some new approaches that have been proposed recently.
Müller, Peter
Bayesian semiparametric mixture models with covariate-dependent weights
View Abstract Hide Abstract
We construct semiparametric mixtures with covariate-dependent weights by defining a probability model for random partitions. The probability of a random partition is allowed to depend on covariates specific to the experimental units that are clustered together. Conditional on a partition we assume a sampling model for the response variable that is independent across clusters and exchangeable within clusters. The resulting posterior predictive model takes the form of a mixture model with covariate-dependent weights. The motivating application is predicting time to progression for patients in a breast cancer trial. We proceed by reporting a weighted average of the responses of clusters of earlier patients. The weights should be determined by the similarity of the new patient's covariate with the covariates of patients in each cluster. We achieve the desired inference by defining a random partition model that includes a regression on covariates. Patients with similar covariates are a priori more likely to be clustered together. Posterior predictive inference in this model formalizes the desired prediction. We build on product partition models (PPM). We define an extension of the PPM to include a regression on covariates by including in the cohesion function a new factor that increases the probability of experimental units with similar covariates to be included in the same cluster. We discuss implementations suitable for any combination of continuous, categorical, count and ordinal covariates.
Murphy, Brendan
A mixture of experts latent position cluster model for social network data
View Abstract Hide Abstract
Social network data represent the interactions between a group of social actors. Interactions between colleagues and friendship networks are typical examples of such data. The latent space model for social network data locates each actor in a network in a latent (social) space and models the probability of an interaction between two actors as a function of their locations. The latent position cluster model extends the latent space model to deal with network data in which clusters of actors exist - actor locations are drawn from a finite mixture model, each component of which represents a cluster of actors. A mixture of experts model builds on the structure of a mixture model by taking account of both observations and associated covariates when modelling a heterogeneous population. Herein, a mixture of experts extension of the latent position cluster model is developed. The mixture of experts framework allows covariates to enter the latent position cluster model in a number of ways, yielding different model interpretations. Estimates of the model parameters are derived in a Bayesian framework using a Markov Chain Monte Carlo algorithm. The algorithm is generally computationally expensive - surrogate proposal distributions which shadow the target distributions are derived, reducing the computational burden. The methodology is demonstrated through an illustrative example detailing relations between a group of lawyers in the USA. (Co-author: Isobel Claire Gormley)
Murray, Iain
Sampling latent variable models
View Abstract Hide Abstract
Simple mixture models assume that a dataset has a single underlying catagorical variable. More sophisticated latent variable modelling is often called for. One might consider sampling (trying out) a variety of models from the rich family of statistical models available. I suggest that undirected models from machine learning should be better known in statistical circles. I discuss their relative merits and how, via sampling (Monte Carlo), to compare them to the directed models that are more commonly used in statistics.
Newton, Michael
Gamma-based clustering via ordered means with application to gene-expression analysis
View Abstract Hide Abstract
Discrete mixture models provide a well-known basis for effective clustering algorithms, although technical challenges have limited their scope. In the context of gene-expression data analysis, a model is presented that mixes over a finite catalog of structures, each one representing equality and inequality constraints among latent expected values. Computations depend on the probability that independent gamma-distributed variables attain each of their possible orderings. Each ordering event is equivalent to an event in independent negative-binomial random variables, and this finding guides a dynamic-programming calculation. The structuring of mixture-model components according to constraints among latent means leads to strict convcavity of the mixture log likelihood. In addition to its beneficial numerical properties, the clustering method shows promising results in an empirical study.
Teh, Yee Whye
On hierarchical clustering, partitions, and mixture models
View Abstract Hide Abstract
We explore a Bayesian hierarchical clustering algorithm that can produce non-binary trees (also known as rose trees). We interpret a rose tree as a mixture over partitions of a data set, and use a greedy agglomerative algorithm to find rose trees with high marginal likelihood given the data. We show in experiments that rose trees are better models of data than the typical binary trees returned by other hierarchical clustering algorithms. (Joint with Charles Blundell and Katherine Heller).
Williams, Chris
Greedy learning of binary latent trees
View Abstract Hide Abstract
Inferring latent structures from observations helps to model and possibly also understand underlying data generating processes. A rich class of latent structures are the latent trees, i.e. tree-structured distributions involving latent variables where the visible variables are leaves. These are also called hierarchical latent class (HLC) models. Zhang (2004) proposed a search algorithm for learning such models in the spirit of Bayesian network structure learning. While such an approach can find good solutions it can be computationally expensive. As an alternative we propose a greedy, bottom-up procedure. We show that even with restricting ourselves to binary trees we obtain HLC models of comparable quality to Zhang's solutions (in terms of cross- validated log-likelihood), while being generally faster to compute. This claim is validated by a comprehensive comparison on several datasets. Furthermore, we demonstrate that our method is able to estimate interpretable latent structures on real-world data with a large number of variables. By applying our method to a restricted version of the 20 newsgroups data (from Sam Roweis) these models turn out to be related to topic models, and on data from the PASCAL Visual Object Classes (VOC) 2007 challenge we show how such tree-structured models help us understand how objects co-occur in images. Authors: Stefan Harmeling and Christopher K. I. Williams.

Participants

Name	Institution
Aitkin, Murray	University of Melbourne
Alfó, Marco	"Sapienza" Universita' di Roma
Alston, Clair	Queensland University of Technology
Anagnostopoulos, Christoforos	The Institute for Mathematical Sciences, Imperial College London
Anaya-Izquierdo, Karim	The Open University
Andrieu, Christophe	University of Bristol
Archambeau, Cedric	Xerox Research Centre Europe
Bakra, Eleni	MRC Biostatistics Unit
Benaglia, Tatiana	MRC Biostatistics Unit
Bochkina, Natalia	University of Edinburgh
Böhning, Dankmar	University of Reading
Cappé, Olivier	Telecom ParisTech & CNRS
Celeux, Gilles	INRIA
Chen, Jiahua	University of British Columbia
Chopin, Nicolas	ENSAE-CREST
Clark, Daniel	Heriot-Watt University
Critchley, Frank	The Open University
Dean, Nema	University of Glasgow
Do, Kim-Anh	University of Texas M.D. Anderson Cancer Center
Eslami, Seyed Mohammadali	University of Edinburgh
Fearnhead, Paul	Lancaster University
Filippone, Maurizio	University of Glasgow
Forbes, Florence	INRIA
Frühwirth-Schnatter, Sylvia	Johannes Kepler University Linz
Gerlach, Richard	University of Sydney
Geweke, John	University of Technology Sydney
Girolami, Mark	University of Glasgow
Gormley, Claire	University College Dublin
Green, Peter	University of Bristol
Heller, Katherine	University of Cambridge
Hinde, John	National University of Ireland Galway
Holmes, Chris	University of Oxford
Hurn, Merrilee	University of Bath
Jordan, Michael	University of California, Berkeley
Juan, Alfons	Universitat Politècnica de València
Kivinen, Jyri	University of Edinburgh
Kohn, Robert	University of New South Wales
Kornbrot, Diana	University of Hertfordshire
Lindsay, Bruce	Pennsylvania State University
Matias, Catherine	CNRS Laboratoire Statistique et Génome
McLachlan, Geoff	University of Queensland
McParland, Damien	University College Dublin
Mengersen, Kerrie	Queensland University of Technology
Müller, Peter	University of Texas M.D. Anderson Cancer Center
Muniz Terrera, Graciela	MRC Biostatistics Unit
Murphy, Brendan	University College Dublin
Murray, Iain	Unversity of Toronto & University of Edinburgh
Newton, Michael	University of Wisconsin, Madison
Nobile, Agostino	University of Glasgow
Orbanz, Peter	University of Cambridge
Ray, Surajit	Boston University
Robert, Christian P	University of Paris-Dauphine
Savage, Rich	University of Warwick
Sever, Krunoslav	Helmut-Schmidt Universität Hamburg
Shi, Jian Qing	Newcastle University
Städler, Nicolas	Seminar für Statistik, ETH Zürich
Stathopoulos, Vassilios	University of Glasgow
Storkey, Amos	University of Edinburgh
Sutton, Charles	University of Edinburgh
Teh, Yee Whye	University College London
Titterington, Mike	University of Glasgow
Torsney, Ben	University of Glasgow
Van Steen, Kristel	University of Liege
West, Robert M	University of Leeds
Whiteley, Nick	University of Bristol
Williams, Chris	University of Edinburgh
Worton, Bruce	University of Edinburgh
Wraith, Darren	INRIA
Yau, Christopher	University of Oxford