a***r
发帖数: 420 | 1 弱弱地问一个关于EM algorithm的基础问题，
大家都知道：(theta 为新一轮估计值，theta’为上一轮估计值)
L(theta) = Q(theta, theta')- H(theta,theta')
where H(theta，theta')=E(log[f(x|y,theta)]|y,theta')
为啥H(theta,theta’)总是小于等于H(theta',theta') 呢？
书上简单说了下，by Jensen‘s Inequality，
但是愚鲁的我没有想明白。。。
望版上大虾指点！ | n*****n
发帖数: 3123 | 2 E(-log[f(x|y,theta)/f(x|y,theta')]|y,theta')>=-log[E(f(x|y,theta)/f(x|y,
theta')|y,theta')] by jensen's
the right hand side is 0 after integral since the expectation is w.r.t
f(x|y,
theta')
baozi please | a***r
发帖数: 420 | 3 多谢，请吃
【在 n*****n 的大作中提到】
: E(-log[f(x|y,theta)/f(x|y,theta')]|y,theta')>=-log[E(f(x|y,theta)/f(x|y,
: theta')|y,theta')] by jensen's
: the right hand side is 0 after integral since the expectation is w.r.t
: f(x|y,
: theta')
: baozi please
| a***r
发帖数: 420 | 4 我还是有个更基本的问题，就是
g(x)=-log(x)是convex function，
故 E(g(x)) >= g(E(x))
这个imply E(g(x)|y) >= g(E(x|y))) 么？
conditional expectation又一次成功地把我搞晕了。。。
【在 n*****n 的大作中提到】
: E(-log[f(x|y,theta)/f(x|y,theta')]|y,theta')>=-log[E(f(x|y,theta)/f(x|y,
: theta')|y,theta')] by jensen's
: the right hand side is 0 after integral since the expectation is w.r.t
: f(x|y,
: theta')
: baozi please
| t****r
发帖数: 702 | 5 if g is convex, then this is true by the defination of conditional expectati
on. In other words, Jensen's inequality holds for conditional expectation.
【在 a***r 的大作中提到】
: 我还是有个更基本的问题，就是
: g(x)=-log(x)是convex function，
: 故 E(g(x)) >= g(E(x))
: 这个imply E(g(x)|y) >= g(E(x|y))) 么？
: conditional expectation又一次成功地把我搞晕了。。。
| n*****n
发帖数: 3123 | 6 Yes, jensen's holds for conditional expectation.
actually, many theorems have conditional versions, like MCT, DCT, Fatou's
lemma, Jensen's, Cauchy-Schwarz, Holder
【在 a***r 的大作中提到】
: 我还是有个更基本的问题，就是
: g(x)=-log(x)是convex function，
: 故 E(g(x)) >= g(E(x))
: 这个imply E(g(x)|y) >= g(E(x|y))) 么？
: conditional expectation又一次成功地把我搞晕了。。。
| a***r
发帖数: 420 | 7 嗯，想了确实是这样，谢谢！
expectati
【在 t****r 的大作中提到】
: if g is convex, then this is true by the defination of conditional expectati
: on. In other words, Jensen's inequality holds for conditional expectation.
| a***r
发帖数: 420 | 8 thx
【在 n*****n 的大作中提到】
: Yes, jensen's holds for conditional expectation.
: actually, many theorems have conditional versions, like MCT, DCT, Fatou's
: lemma, Jensen's, Cauchy-Schwarz, Holder
|
|
