a***r 发帖数: 420 | 1 弱弱地问一个关于EM algorithm的基础问题,
大家都知道:(theta 为新一轮估计值,theta’为上一轮估计值)
L(theta) = Q(theta, theta')- H(theta,theta')
where H(theta,theta')=E(log[f(x|y,theta)]|y,theta')
为啥H(theta,theta’)总是小于等于H(theta',theta') 呢?
书上简单说了下,by Jensen‘s Inequality,
但是愚鲁的我没有想明白。。。
望版上大虾指点! | n*****n 发帖数: 3123 | 2 E(-log[f(x|y,theta)/f(x|y,theta')]|y,theta')>=-log[E(f(x|y,theta)/f(x|y,
theta')|y,theta')] by jensen's
the right hand side is 0 after integral since the expectation is w.r.t
f(x|y,
theta')
baozi please | a***r 发帖数: 420 | 3 多谢,请吃
【在 n*****n 的大作中提到】![](/moin_static193/solenoid/img/up.png) : E(-log[f(x|y,theta)/f(x|y,theta')]|y,theta')>=-log[E(f(x|y,theta)/f(x|y, : theta')|y,theta')] by jensen's : the right hand side is 0 after integral since the expectation is w.r.t : f(x|y, : theta') : baozi please
| a***r 发帖数: 420 | 4 我还是有个更基本的问题,就是
g(x)=-log(x)是convex function,
故 E(g(x)) >= g(E(x))
这个imply E(g(x)|y) >= g(E(x|y))) 么?
conditional expectation又一次成功地把我搞晕了。。。
【在 n*****n 的大作中提到】![](/moin_static193/solenoid/img/up.png) : E(-log[f(x|y,theta)/f(x|y,theta')]|y,theta')>=-log[E(f(x|y,theta)/f(x|y, : theta')|y,theta')] by jensen's : the right hand side is 0 after integral since the expectation is w.r.t : f(x|y, : theta') : baozi please
| t****r 发帖数: 702 | 5 if g is convex, then this is true by the defination of conditional expectati
on. In other words, Jensen's inequality holds for conditional expectation.
【在 a***r 的大作中提到】![](/moin_static193/solenoid/img/up.png) : 我还是有个更基本的问题,就是 : g(x)=-log(x)是convex function, : 故 E(g(x)) >= g(E(x)) : 这个imply E(g(x)|y) >= g(E(x|y))) 么? : conditional expectation又一次成功地把我搞晕了。。。
| n*****n 发帖数: 3123 | 6 Yes, jensen's holds for conditional expectation.
actually, many theorems have conditional versions, like MCT, DCT, Fatou's
lemma, Jensen's, Cauchy-Schwarz, Holder
【在 a***r 的大作中提到】![](/moin_static193/solenoid/img/up.png) : 我还是有个更基本的问题,就是 : g(x)=-log(x)是convex function, : 故 E(g(x)) >= g(E(x)) : 这个imply E(g(x)|y) >= g(E(x|y))) 么? : conditional expectation又一次成功地把我搞晕了。。。
| a***r 发帖数: 420 | 7 嗯,想了确实是这样,谢谢!
expectati
【在 t****r 的大作中提到】![](/moin_static193/solenoid/img/up.png) : if g is convex, then this is true by the defination of conditional expectati : on. In other words, Jensen's inequality holds for conditional expectation.
| a***r 发帖数: 420 | 8 thx
【在 n*****n 的大作中提到】![](/moin_static193/solenoid/img/up.png) : Yes, jensen's holds for conditional expectation. : actually, many theorems have conditional versions, like MCT, DCT, Fatou's : lemma, Jensen's, Cauchy-Schwarz, Holder
|
|