we would be too simple-minded to understand it." Anonymous |
The ability of humans to speak and to understand speech requires an enormous amount of brain resources. These resources have to manage information about many thousands of words and many syntactic constructions and their interconnections, not just to one another but to meanings and to the structures that allow us to recognize the sounds of speech and to move the muscles of our mouths to produce speech. This complex combination of brain structures can be called the brain's linguistic system. It allows a person not only to talk and to understand speech but also to read and write. It also gives us the power to think as well as the power to acquire new knowledge and abilities and to learn how to speak in the first place. The Langbrain website is about this system.
Synopsis:
A simple method for inferring the network of (linear) dependencies among a set of variables is to compute all pairwise correlations and subsequently to draw the corresponding graph (for some specified threshold). While popular and often used on many types of genomic data (e.g. gene expression, metabolite concentrations etc.)the naive correlation approach does not allow to infer the dependency network. Instead, graphical Gaussians models (GGMs) should be used. These allow to correctly identify direct influences, have close connections with causal graphical models, are straightforward to interpret, and yet are essentially as easy to compute as naive correlation models. This page lists pointers to learning GGMs from data, including procedures suitable for "small n, large p" data sets (category iii).
Introduction:
Graphical Gaussian Models (GGMs), also known as "covariance selection" or " concentration graph" models, have recently become a popular tool to study gene association networks. The key idea behind GGMs is to use partial correlations as a measure of independence of any two genes. This makes it straightforward to distinguish direct from indirect interactions. Note that partial correlations are related to the inverse of the correlation matrix. Also note that in GGMs missing edges indicate conditional independence.
A related but completely different concept are the so-called gene relevance networks which are based on the "covariance graph" model. In the latter interactions are defined through standard correlation coefficients so that missing edges denote marginal independence only.
There is a simple reason why GGMs should be preferred over relevance networks for identification of gene networks: the correlation coefficient is weak criterion for measuring dependence, as marginally, i.e. directly and indirectly, more or less all genes will be correlated. This implies that zero correlation is in fact a strong indicator for independence, i.e. the case of no edge in a network - but this is of course not what one usually wants to find out by building a relevance network... On the other hand, partial correlation coefficients do provide a strong measure of dependence and, correspondingly, offer only a weak criterion of independence (as most partial correlations coefficients usually vanish).
The best starting place to learn about GGMs is the classic paper that introduced this concept in the early 1970s. (A.P. Dempster. 1972. Covariance Selection. Biometrics 28:157-175). Further details can be found in the GGM books by J. Whittaker (1990) and by D. Edwards (1995).
Application of GGMs to Genomic Data:
Application of GGMs to genomic data is quite challenging, as the number of genes (p) is usually much larger than the number of available samples (n), and classical GGM theory is not valid in a small sample setting. With this page I'd like to provide a commented list of some recent work dealing with GGM gene expression analysis (there are only very few so far). In my understanding, all of these paper fit in one of three categories:
For small n, large p data it seems that methods from section iii. are most suited (see below for references and software).
I. Classic GGM Analysis:
The following papers simply apply classical GGM theory (i.e. with not further modification) to analyze gene expression data. It turns out that such an analysis is necessarily restricted to very small numbers of genes or gene clusters as to satisfy n > p.
II. Limited Order Partial Correlations:
One way to circumvent the problem of computing full partial correlation coefficients when the sample size is small compared to the number of genes is to use partial correlation coefficients of limited order. This results in something inbetween a full GGM model (with correlation conditioned on all p-2 remaining genes) and a relevance network model (with unconditioned correlation). This is the strategy employed in the following papers:
III. Regularized GGMs:
Another possibility (and in my opinion the statistically most sound way) to marry GGMs with small sample modeling is to introduce regularization and moderation. This essentially boils down to finding suitable estimates for the covariance matrix and its inverse when n < p. This can either be done in a full Bayesian manner, or in an empirical Bayes way via variance reduction, shrinkage estimates etc. Once regularized estimates of partial correlation are available then heuristic searches can subsequently to be employed to find an optimal graphical model (or set of models).
Outside a genomic context using regularized GGMs was first proposed by F. Wong, C.K. Carter, and R. Kohn. (2003. Efficient estimation of covariance selection models. Biometrika 90:809-830). For gene expression data this strategy is pursued in the following papers:
VERSION=1.0UTTERANCE=testf1.mfclmname=wdnetlmscale=20.00 wdpenalty=-30.00vocab=dictN=31L=56I=0t=0.00I=1t=0.36I=2t=0.75I=3t=0.81... etcI=30t=2.48J=0S=0E=1W=SILENCEJ=1S=1E=2W=FOUR... etcJ=55S=29E=30W=SILENCEv=0 a=-3239.01 l=0.00v=0 a=-3820.77 l=0.00v=0 a=-246.99 l=-1.20
This paper explored the CD phone recognition on TIMIT using hybrid NN/HMM system.
In this paper, the authors propose to using Dirichlet Mixture models instead of Gaussian Mixture models for the hybrid NN/HMM system. In the conventional NN/HMM system, the NN's posteriors are Gaussianized to feed into the HMM framework. However, as the posterior probabilities are lying on probability simplex and their distribution could be modeled by Dirichlet distributions. Thus Dirichlet Mixture models would be more preferable to Gaussian Mixture model.
The concept is not that new, is similar to Conditional Neural Network or Neural Conditional Random Fields. And the results on TIMIT is far from DBN based systems.
