PCA whitening

I have encountered this phrase "PCA whitening" many times, all I thought about it was just a simple PCA. However, it is not!

The Whitening is actually PCA + scaling!!!

Also there is a good implementation explanation here : http://ufldl.stanford.edu/wiki/index.php/Implementing_PCA/Whitening

Implementing PCA_Whitening - Ufldl.pdf Download this file

Posted via email from Troy's posterous

PCA again

PCA is a simple but rather popular techniques for feature processing and is used widely as some initial processing steps involved in many applications. However, every time when I need to do PCA, I have to find some references :<. 

Thus again, to put some information for my own reference. The figure is from the attached slides I encountered today, quite clear.

3 - Feature extraction.pdf Download this file

Posted via email from Troy's posterous

Unity3D Resources

The original post is from: http://www.virtualgamelab.com/2/post/2010/04/unity-resources.html
This is just for my own reference!

As promised, this blog entry is going to give an overview of existing Unity resources. I have tried to cover a wide variety of video and text-based tutorials, extensions for Unity and links to most example projects and documentation on the web. I have tried to categorize the resources and also present them in an order that should get you started in no time. Furthermore, I have spent a substantial amount of hours either absorbing these tutorials/docs completely or at least scanning through them (I haven't bought any paid tutorials except Will Goldstone's book). Thus, I believe all of these links should lead you to some form of valuable information about Unity. Thanks to all the authors for doing such an incredible job. Without further ado, my take on summarizing resources for Unity.

Unity website: http://unity3d.com/

Which license do you need? http://unity3d.com/unity/licenses

Download the free version of Unity to get started: http://unity3d.com/unity/download/

If you have no previous experience with Unity, start with these six video tutorials which give a quick overview of the Unity interface and some important features
http://unity3d.com/support/documentation/video/ 

Continue with a more in-depth text-based walk through of very basic Unity functionality and work flow http://unity3d.com/support/documentation/Manual/Unity%20Basics.html

To get you started with scripting, have a look at the following PDF document. It was written for an older version of Unity, but still covers relevant aspects of scripting with JavaScript. (2 hours to complete, no previous JavaScript knowledge required)
http://docwiki.unity3d.com/uploads/Main/Scripting%20Tutorial.pdf 

Unity features three scripting languages; JavaScript, C#, and Boo. Depending on your scripting language of choice, you might want to choose different tutorials to get started with Unity. Each tutorial link will also mention which scripting language is used. Most Unity tutorials available on the web are using JavaScript. Notice that JavaScript is commonly used for web applications. Unforunately, books which cover JavaScript are normally of little use for Unity's implementation of JavaScript. But worry not, a vast amount of tutorials will follow...

If you want to familiarize yourself with Unity's functionality more, browse through Unity's manual. You can skip the “Basics” section as we already went through this (see above).
http://unity3d.com/support/documentation/Manual/index.html 

For a printable version of the 600+ pages manual, seehttp://unity3d.com/support/documentation/printable.html

If you rather prefer to jump into the action, skip the manual altogether and have a look at the three books which have been published on Game Development with Unity (one recently added on 06/10/11). 
1) Author Will Goldstone guides you through a complete project and introduces most of Unity's functionality (using JavaScript). The book is available as print and digital version and all needed assets and resources can be downloaded with the book. It's been a helpful investment from my point of view.
http://www.packtpub.com/unity-game-development-essentials/book 
This forum thread discusses the book and its content. 
http://forum.unity3d.com/viewtopic.php?t=29041

2) Ryan Henson Creighton more recently published "Unity 3D Game Development by Example Beginner's Guide". Find out more about the book here: https://www.packtpub.com/unity-3d-game-development-by-example-beginners-guide/book
and the forum thread - http://forum.unity3d.com/threads/61349-Unity-3D-Game-Development-by-Example

3) Craig Stevenson and Simon Quiq (publisher Deep Pixel) released "Unity 3 Blueprints: A practical guide to Indie games development". Their website provides all the art assests and code to create four classic games from scratch (Match the Pairs, Top-Down Shooter, Tower Defense, Marble Madness).
Amazon link

To get up-to-date on Unity 3 functionality, check out the following official documentation:

Unity 3 - What's new? http://unity3d.com/unity/whats-new/unity-3

- Beast lightmapping Basics - http://unity3d.com/support/documentation/Manual/Lightmapping.html
- Beast Lightmapping In-Depth - http://unity3d.com/support/documentation/Components/class-LightMapping.html
- Tree Creator - http://unity3d.com/support/documentation/Components/class-Tree.html
- Umbra Occlusion Culling - http://unity3d.com/support/documentation/Manual/Occlusion%20Culling.html

You are now faced with the choice of tackling example projects provided by Unity Technologies or jump straight into user-generated tutorials. As the example projects are quite complex, I would suggest working through some video tutorials first. Nonetheless, here are the links for Unity's official example projects. They can be taken completely apart, reverse-engineered and reused for own projects.

3D Platform Game
http://unity3d.com/support/resources/tutorials/3d-platform-game
2D Platform Game
http://unity3d.com/support/resources/tutorials/2d-gameplay-tutorial
Iphone tutorials and more example projects
http://unity3d.com/support/resources/tutorials/

More Unity Example projects
http://unity3d.com/support/resources/example-projects/ 

Brand New Car Tutorial by Unity Technologies
http://unity3d.com/support/resources/tutorials/car-tutorial

Now, let's head on to TUTORIALS. This list starts with mostly basic and general tutorials at the top and lists more specific tutorials at the end.

Will Goldstone , author of the Unity Game Development book, worked on a series of helpful video tutorials here (using JavaScript): http://learnunity3d.com/ (also http://learnmesilly.com/).
More recently by Will Goldstone: http://www.unity3dstudent.com/

TornadoTwins Video Tutorials 
They show step by step how to create a simple game using Javascript.
http://www.youtube.com/user/TornadoTwins 

Walker Boys Studio - Unity Tutorials from the Guildhall at SMU (an extensive list of tutorials with more on the way)
http://walkerboystudio.com/html/unity_training___free__.html

CannedMushroom Video Tutorials (Unity and other software)
This is a series of projects intended for 2-hour self-instructed lessons using JavaScript
http://www.youtube.com/user/cannedmushrooms 

Unity Jumpstart 
Proof of Concept Game to learn creating your own game from the ground up (JavaScript)
http://forum.unity3d.com/viewtopic.php?t=28433 
http://www.unityjumpstart.com/ProofOfConcept_1/ 

3dbuzz.com 
Series of Unity tutorials in C# (among many other tutorials)
registration required (free)
website - http://www.3dbuzz.com/vbforum/sv_home.php
forum thread - http://forum.unity3d.com/threads/37093-Over-7-hours-of-Unity-Training-Videos-Download-them-now!

BurgZergArcade - Unity Tutorials
Hack&Slash Tutorial using C# and plenty of other information and useful tutorials
website - http://www.burgzergarcade.com/

UnityScript Basics (Scripting Basics for Noobs)
If you're new to scripting, read up on this good introduction to scripting in Unity. Lots of analogies are provided which makes it really easy to understand. The details are explained for C#, but the introduction is great for any scripting language.
website - http://www.unityscript.com/lessons1/basics.php

UnityLabs 
Text-based tutorials with screenshots covering a wide range of topics incl. scripting (using JavaScript), basic introduction to unity, character controllers, and user interfaces
http://www.unitylabs.net/tutorials/unity-beginners 

InfiniteAmmo Tutorial 
General Introduction to Unity (3 parts so far)
Some scripting covered using Javascript (e.g. movement controls)
http://infiniteammo.ca/blog/impromptu-unity3d-intro-tutorial/ 

Technicat 
Introduction to Game Development with Unity including Workflow, Scripting, GUI, Version Control, etc.
http://technicat.com/games/unity.html 

Advanced Media Lab at North Carolina State University 
http://vimeo.com/channels/ncsuunity 

GearTech Games 
4 Videos on how to work through a project (and more videos)

workflow between Modo and Unity
lightmapping

http://www.geartechgames.com/GearTech_Games/Beginning_a_Project/Beginning_a_Project.html

Making Sense of Unity 
These video tutorials provide more indepth coverage of unity scripting using Javascript (not meant to be introductory material)

lecture style explanations of concepts
more entertaining and not like most other screen-capture tutorials :)

http://www.youtube.com/etphonemyhome

IO Development Diary
This series of video tutorials (some paid, some free) follows the development of a Space Shooter and covers a Modo-Unity workflow and C# scripting.
forum thread - http://forum.unity3d.com/viewtopic.php?t=55762
website - http://fallingbrickwork.fearedfuturegames.com/

Virtual Autonomy 
Guide for working with Google SketchUp and Unity
(also shaders and multi-user environments)
text-based with screenshots
http://virtualautonomy.com/blog/index.php/tutorials/

Robotduck - Blog 
The blog of this Unity user provides tips on Unity scripting and functionality and showcases some of his projects
http://robotduck.wordpress.com/ 

Ethical Games 
Unity Tutorials for Flash Developers
http://ethicalgames.wordpress.com/2009/01/14/unity-for-flash-developers-tutorial-1/ 

Car Tutorial 
Physics Setup for a car, AI for driving around racetrack
http://www.gotow.net/andrew/blog/?page_id=78 

Terrain Tutorial 
seven videos on how to create terrains in Unity from heightmaps to finish
http://vimeo.com/album/156452 

In-Depth Terrain Tutorial 
text-based with screenshots
http://www.unifycommunity.com/wiki/index.php?title=Terrain_tutorial 
forum thread
http://forum.unity3d.com/viewtopic.php?t=16941 

Paul Bourke -Unity for stereoscopic display
text-based with screenshots
http://local.wasp.uwa.edu.au/~pbourke/miscellaneous/stereographics/Unitystereo/ 

Paul Bourke – Unity and Idome
text-based with screenshots
http://local.wasp.uwa.edu.au/~pbourke/miscellaneous/domemirror/UnityiDome/ 

RENCI – Unity for Dome projections
http://unc.renci.org/rencinews/the-dome-corner-006 

Official Unity Tech. Tutorial for the Animation View (NEW) 3-part-series of video tutorials 
http://unity3d.com/support/resources/tutorials/video-animation-view

PAID TUTORIALS

VTC - Unity tutorial using JavaScript (subscription based content on vtc.com)
http://www.vtc.com/products/Introduction-to-Game-Development-Using-Unity-3D-Tutorials.htm
Unity-Tutorials (mostly paid and some free tutorials)
http://www.unity-tutorials.com/ 

Design3.com (subscription based tutorials, Unity-related content among other software like Maya, 3DS Max, Photoshop)
http://www.design3.com/by-software/unity  

GENERAL RESOURCES

Once you went through some or all of these tutorials, you should be creating your own content in no time. Next, I'll provide an overview of general resources for Unity. Whenever you need to find some information about Unity, scripting, projects, collaborations or anything Unity-related, check these links out:

Searching for resources - The All-In-One Unity Reference Search 
http://www.google.com/cse/home?cx=001712401338047450041:csfhqk-trfa 
credit goes to Robotduck for providing this link to the public; tremendous time saver (seehttp://robotduck.wordpress.com/)

Unity Scripting Reference 
http://unity3d.com/support/documentation/ScriptReference/index.html 

Unity Component Reference 
each available Component described in detail
http://unity3d.com/support/documentation/Components/index.html 

Unity Wiki (UnifyCommunity) 
http://www.unifycommunity.com/wiki/index.php?title=Main_Page 

Unity Answers 
Invaluable when you have specific questions about Unity or Scripting
http://answers.unity3d.com/ 
This link should also get you started on learning Unity
http://answers.unity3d.com/search?q=learning+tutorial 

Unity Forum 
http://forum.unity3d.com/ 
http://forum.unity3d.com/viewtopic.php?t=30633 

Unity Feedback 
feature requests go here
http://feedback.unity3d.com 

Unity IRC 
Point your favorite IRC client to irc.freenode.net and join #unity3d to chat in real time. 

Overview of Unity Resources 
http://www.reddit.com/r/unity_tutorials/ 

Overview of Unity blogs 
http://www.reddit.com/r/unity_blogs/ 

TOOLS AND EXTENSIONS

Lastly, I want to list extensions and tools which can make your life as a Unity developer easier.

Unity Extensions

Terrain Toolkit
External Lightmapping Tool
Locomotion System
Explosion Framework
Head Look Controller

http://unity3d.com/support/resources/unity-extensions/
overview of some extensions – Unity youtube and vimeo channels 
http://www.youtube.com/user/Unity3D 
http://www.vimeo.com/unity3d/videos 

Visual C# Express-  free IDE for your C# development
http://www.microsoft.com/express/Downloads/#2008-Visual-CS

3DAttack - Tools and Home of Unity Creative Magazine
First Person Shooter Developer Kit
forum thread - http://forum.unity3d.com/viewtopic.php?t=53136
website - http://www.3dattack.us/3DAttack/News.html

AngryAnt 
One of Unity's developers provides projects to extend Unity's functionality (e.g. pathfinding and AI)
http://angryant.com/ 
forum thread Path - http://forum.unity3d.com/threads/19341-Pathfinding-in-unity-First-Path-1.0-tutorial-now-available!

sturestone's A* Pathfinding (currently version 2.9)
forum thread - http://forum.unity3d.com/threads/67417-A*-Pathfinding-2.9-Is-Released-(Unity-3-Compatible)

Weltenbauer 
Augmented Reality / Webcam Input
http://produktion.weltenbauer.com/#/5 
forum threads:
http://forum.unity3d.com/viewtopic.php?t=32613&highlight=toolkit (Webcam Toolkit)
http://forum.unity3d.com/viewtopic.php?t=11889&highlight=toolkit (ARToolkit Extension)
http://forum.unity3d.com/viewtopic.php?t=32669&highlight=toolkit (UnityAR)

SeeingMachines FaceAPI / VisionBlaster – Head Tracking in Unity (purchase required)
http://visionblaster.com/cms/index.php?option=com_content&view=article&id=48&Itemid=62&lang=en 
and http://www.seeingmachines.com/product/faceapi/downloads/

Mostly Tigerproof – Using Google Analytics and Unity to track game stats
This is a blog entry about Google Analytics and Unity
http://blog.mostlytigerproof.com/2009/10/06/gathering-statistics-using-google-analytics-and-unity-3d/ 

Antares Project - Extensive Set of Tools to extend the Unity Editor - Open Source
Also available: Antares.dll (free for non-commercial work)
forum thread - http://forum.unity3d.com/viewtopic.php?t=50843

Antares Deformator - Deform your meshes (Beta Version)
forum thread - http://forum.unity3d.com/viewtopic.php?t=54290

UniWii – WiiMote implementation 
http://www.unifycommunity.com/wiki/index.php?title=UniWii 
forum thread
http://forum.unity3d.com/viewtopic.php?t=7284 

Unity Terrain Tools - EasyRoads (purchase required)
http://unityterraintools.com/downloads.php
forum thread
http://forum.unity3d.com/viewtopic.php?t=23519

Six Times Nothing - Road/Path Tool and River Tool http://www.sixtimesnothing.com/

Dastardly Banana - FPS Weapon Tool, Radar examplehttp://dastardlybanana.com/Downloads.htm

Starscene Software - Tools, Games and Utilities for Unity (purchase required for utilities) 
http://www.starscenesoftware.com
e.g. Vectrosity - Line Drawing Tool
http://starscenesoftware.com/vectrosity.html
Fractscape - Terrain Tool
Stitchscape - Stitch multiple terrains together

GUIX - visual Menu/GUI builder (purchase required)http://www.ennanzus.com/developer/GUIX/buy.htm

EZ Game Saver - saving tool (purchase required)
http://www.anbsoft.com/middleware/ezs/index.htm
Note that I will cover saving to text file in a later blog

Fire Tool - Realistically spreading fire (on hold for the moment)
http://forum.unity3d.com/viewtopic.php?t=49588

Decal Framework - Easily place decals in your scene
forum thread - http://forum.unity3d.com/viewtopic.php?t=54613

Visual Logic Editor by NeoDrop (Antares VIZIO, Work in Progress)
forum thread - http://forum.unity3d.com/viewtopic.php?t=31132

Nimbus Volumetric Clouds
forum thread - http://forum.unity3d.com/viewtopic.php?t=51972

RapidUnity Vehicle Editor Resource Pack
forum thread - http://forum.unity3d.com/viewtopic.php?t=56257

Ocean 3D - Ocean Simulation
website: http://www.nikko3d.com/index.php/downloads-2/

ShaderFusion - Node-Based Shader Editor (Requires Unity3)
forum thread - http://forum.unity3d.com/viewtopic.php?t=60611

Strumpy Shader Editor - Node-Based Shader Editor
forum thread - http://forum.unity3d.com/threads/56180-Strumpy-Shader-Editor-Beta-3.1-released-(performance-usability-upgrade)

Overview of extensions on UnifyWiki 
http://www.unifycommunity.com/wiki/index.php?title=Extensions 

LightUp (purchase required)
Extension of Google Sketchup (Lighting Solution) which works nicely for exporting lightmaps to Unity
http://www.light-up.co.uk/ 

Community Project - GTAIV Vehicle Replica (Pledge of > USD 50 required)
forum thread - http://forum.unity3d.com/viewtopic.php?t=50493

Stereoscopic Solutions
3D Anaglyph System (purchase required)
Stereoskopix3D

Plugin for Kinect's Primesense Camera
forum thread - http://forum.unity3d.com/threads/67982-Kinect-plugin

Unity Web Suite - tutorials and examples in C# to create online content
forum thread - http://forum.unity3d.com/threads/17674-Unity-Web-Suite-(Examples-Tutorials)
website - http://www.nickbreslin.com/2009/04/phphandler-20-released/

Tools for Visual Programming:
Antares Universe - Vizio (forum thread)
cost (as of 05/22/11) Euro 142.50 
Visual programming tool similar to the approaches of Quest3D and Virtools.

Playmaker by Hutong Games
cost (as of 05/22/11) Euro 95.00
Visual State Machine Editor (website and forum thread)

uScript by Detox Studios (Beta Version)
website and forum thread
Visual Scripting Tool based on UDK's Kismet

Posted via email from Troy's posterous

Resolving nvcc & gcc conflict ion for Theano

There is a compatibility issue affecting some Ubuntu 9.10 users, and probably anyone using CUDA 2.3 with gcc-4.4. Symptom: errors about “__sync_fetch_and_add” being undefined. Solution 1: make gcc-4.3 the default gcc (http://pascalg.wordpress.com/2010/01/14/cuda-on-ubuntu-9-10linux-mint-helena/) Solution 2: make another gcc (e.g. gcc-4.3) the default just for nvcc. Do this by making a directory (e.g. $HOME/.theano/nvcc-bindir) and installing two symlinks in it: one called gcc pointing to gcc-4.3 (or lower) and one called g++ pointing to g++-4.3 (or lower). Then add compiler_bindir = /path/to/nvcc-bindir to the [nvcc] section of your .theanorc (libdoc_config).

.theanorc file should be under the folder $HOME. If there is no such file, then create a new one.

From: http://deeplearning.net/software/theano/tutorial/using_gpu.html

Posted via email from Troy's posterous

Papers on learning sparse Gaussian Precision Matrix

The basic idea of learning a sparse full covariance for the Gaussian using Graphical models was proposed in the paper "Covaraince Selection".

In the paper "Sparse Gaussian Graphical Models for Speech Recognition", the authors adopted the sparse full covariance learning to speech recognition.

In the third paper, "Projected Subgradient Methods for Learning Sparse Gaussians", a new approach for estimating the covaraince is proposed.

covariance selection.pdf Download this file

Sparse Gaussian Graphical Models for speech recognition_is2007.pdf Download this file

projected subgradient methods for learning sparse gaussians.pdf Download this file

Posted via email from Troy's posterous

LangBrain Website

http://www.ruf.rice.edu/~lngbrain/main.htm

"If the human mind were simple enough for us to understand, we would be too simple-minded to understand it." Anonymous

The ability of humans to speak and to understand speech requires an enormous amount of brain resources. These resources have to manage information about many thousands of words and many syntactic constructions and their interconnections, not just to one another but to meanings and to the structures that allow us to recognize the sounds of speech and to move the muscles of our mouths to produce speech. This complex combination of brain structures can be called the brain's linguistic system. It allows a person not only to talk and to understand speech but also to read and write. It also gives us the power to think as well as the power to acquire new knowledge and abilities and to learn how to speak in the first place. The Langbrain website is about this system.

Posted via email from Troy's posterous

Graphical Gaussian Models for Genome Data

From: http://strimmerlab.org/notes/ggm.html

Synopsis:

For software to efficiently identify GGM networks from data visit theGeneNet page.

A simple method for inferring the network of (linear) dependencies among a set of variables is to compute all pairwise correlations and subsequently to draw the corresponding graph (for some specified threshold). While popular and often used on many types of genomic data (e.g. gene expression, metabolite concentrations etc.)the naive correlation approach does not allow to infer the dependency network. Instead, graphical Gaussians models (GGMs) should be used. These allow to correctly identify direct influences, have close connections with causal graphical models, are straightforward to interpret, and yet are essentially as easy to compute as naive correlation models. This page lists pointers to learning GGMs from data, including procedures suitable for "small n, large p" data sets (category iii).

 

Introduction:

Graphical Gaussian Models (GGMs), also known as "covariance selection" or " concentration graph" models, have recently become a popular tool to study gene association networks. The key idea behind GGMs is to use partial correlations as a measure of independence of any two genes. This makes it straightforward to distinguish direct from indirect interactions. Note that partial correlations are related to the inverse of the correlation matrix. Also note that in GGMs missing edges indicate conditional independence.

A related but completely different concept are the so-called gene relevance networks which are based on the "covariance graph" model. In the latter interactions are defined through standard correlation coefficients so that missing edges denote marginal independence only.

There is a simple reason why GGMs should be preferred over relevance networks for identification of gene networks: the correlation coefficient is weak criterion for measuring dependence, as marginally, i.e. directly and indirectly, more or less all genes will be correlated. This implies that zero correlation is in fact a strong indicator for independence, i.e. the case of no edge in a network - but this is of course not what one usually wants to find out by building a relevance network... On the other hand, partial correlation coefficients do provide a strong measure of dependence and, correspondingly, offer only a weak criterion of independence (as most partial correlations coefficients usually vanish).

The best starting place to learn about GGMs is the classic paper that introduced this concept in the early 1970s. (A.P. Dempster. 1972. Covariance Selection. Biometrics 28:157-175). Further details can be found in the GGM books by J. Whittaker (1990) and by D. Edwards (1995).

 

Application of GGMs to Genomic Data:

Application of GGMs to genomic data is quite challenging, as the number of genes (p) is usually much larger than the number of available samples (n), and classical GGM theory is not valid in a small sample setting. With this page I'd like to provide a commented list of some recent work dealing with GGM gene expression analysis (there are only very few so far). In my understanding, all of these paper fit in one of three categories:

1. analysis with classic GGM theory,
2. using limited order partial correlations, and
3. application of regularized GGMs.

For small n, large p data it seems that methods from section iii. are most suited (see below for references and software).

 

I. Classic GGM Analysis:

The following papers simply apply classical GGM theory (i.e. with not further modification) to analyze gene expression data. It turns out that such an analysis is necessarily restricted to very small numbers of genes or gene clusters as to satisfy n > p.

1. P. J. Waddell and H. Kishino. 2000. Correspondence analysis of genes and tissue types and finding genetics links from microarray data. Genome Informatics 11:83-95
2. P. J. Waddell and H. Kishino. 2000. Cluster inferences methods and graphical models evaluated on NCI60 microarray gene expression data. Genome Informatics 11:129--140
3. H. Toh and K. Horimoto. 2002. Inference of a genetic network by a combined approach of cluster analysis and graphical Gaussian modeling. Bioinformatics 18:287--297
4. H. Toh and K. Horimoto. 2002. System for automatically inferring a genetic network from expression profiles. J. Biol. Physics 28:449--464
5. X. Wu, Y. Ye and K. R. Subramanian. 2003. Interactive analysis of gene interactions using graphical Gaussian model. ACM SIGKDD Workshop on Data Mining in Bioinformatics 3:63-69

 

II. Limited Order Partial Correlations:

One way to circumvent the problem of computing full partial correlation coefficients when the sample size is small compared to the number of genes is to use partial correlation coefficients of limited order. This results in something inbetween a full GGM model (with correlation conditioned on all p-2 remaining genes) and a relevance network model (with unconditioned correlation). This is the strategy employed in the following papers:

1. A. de la Fuente, N. Bing, I. Hoeschele, and P. Mendes. 2004. Discovery of meaningful associations in genomic data using partial correlation coefficients. Bioinformatics 20:3565-3574.
2. A. Wille, P. Zimmermann et al. 2004. Sparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana. Genome Biology 5:R92
3. P. M. Magwene and J. Kim. 2004. Estimating genomic coexpression networks using first-order conditional independence. Genome Biology 5:R100
4. A. Wille and P. Bühlmann. 2006. Low-order conditional independence graphs for inferring genetic networks. Statist. Appl. Genet. Mol. Biol. 4: 32.
5. R. Castelo and A. Roverato. A robust procedure for Gaussian graphical model search from microarray data with p larger than n. Preprint.

 

III. Regularized GGMs:

Another possibility (and in my opinion the statistically most sound way) to marry GGMs with small sample modeling is to introduce regularization and moderation. This essentially boils down to finding suitable estimates for the covariance matrix and its inverse when n < p. This can either be done in a full Bayesian manner, or in an empirical Bayes way via variance reduction, shrinkage estimates etc. Once regularized estimates of partial correlation are available then heuristic searches can subsequently to be employed to find an optimal graphical model (or set of models).

Outside a genomic context using regularized GGMs was first proposed by F. Wong, C.K. Carter, and R. Kohn. (2003. Efficient estimation of covariance selection models. Biometrika 90:809-830). For gene expression data this strategy is pursued in the following papers:

1. A. Dobra, C. Hans, B. Jones, J.R. Nevins, and M. West. 2004. Sparse graphical models for exploring gene expression data. J. Multiv. Analysis 90:196-212.
   See the web page of M. West for various other related articles.
2. In these papers a regularized estimate of the correlation matrix is obtained, either by Stein-type shrinkage (3) or by bootstrap variance reduction (2). This estimate is subsequently employed for computing partial correlation. Network selection is based on false discovery rate multiple testing. This method is implemented in GeneNet.
   J. Schäfer and K. Strimmer. 2005. An empirical Bayes approach to inferring large-scale gene association networks. Bioinformatics 21: 754-764.
3. J. Schäfer and K. Strimmer. 2005. A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statist. Appl. Genet. Mol. Biol. 4: 32.
4. J. Schäfer and K. Strimmer. 2005. Learning large-scale graphical Gaussian models from genomic data. In: J. F. Mendes. (Ed.). Proceedings of "Science of Complex Networks: from Biology to the Internet and WWW" (CNET 2004), Aveiro, PT, August 2004. (Publisher: The American Institute of Physics).
5. N. Mainshausen and P. Bühlmann 2006. High-dimensional graphs and variable selection with the lasso. Annals of Statistics 34 (3)
   This approach uses lasso regression to induce sparsity on a node level among the partial correlations.
6. H. Li and J. Gui. 2006. Gradient directed regularization for sparse Gaussian concentration graphs, with applications to inference of genetic networks. Biostatistics 2006 7:302-317.
   These authors regularize the concentration matrix rather than the covariance matrix.

Posted via email from Troy's posterous