Wednesday, May 19, 2010

Loss functions for RBMs

Click here to download:
assignment1.tar.gz (126 KB)

Label: DBN, RBM, loss functions

One is to use the mean squared error criterion, which is to minimize the squared error between the original input values and the reconstructed visible values.

Another way is to minimize the negative log likelihood of the reconstruction, given the hidden vector. From the hidden vector, we could compute the probabilities of each visible units given the hidden vector, thus the loss function would be:

-log P(x|h) = - sum_i ( x_i * log p_i(h) + (1-x_i) * log ( 1 - p_i(h)))

Posted via email from Troy's posterous

No comments:

Post a Comment