q**j
发帖数: 10612 | 1 简单regress 的解是
b =（X'X)^(-1) X'y。
如果把X和Y都分成对应的两块就有
b_1 = (X1'X1)^(-1) X1'Y1
b_2 = (X2'X2)^(-1) X2'Y2
请问这个b1,b2和b是怎么联系到一起的？ |
A*******s
发帖数: 3942 | 2 is this relevant to ur question?
http://www.jstor.org/pss/2684974
【在 q**j 的大作中提到】
: 简单regress 的解是
: b =（X'X)^(-1) X'y。
: 如果把X和Y都分成对应的两块就有
: b_1 = (X1'X1)^(-1) X1'Y1
: b_2 = (X2'X2)^(-1) X2'Y2
: 请问这个b1,b2和b是怎么联系到一起的？
|
n*****n
发帖数: 3123 | 3 关系比较复杂，不过应该可以做出来。
可以先考虑一个简单情况，就是X和Y都拿掉一个observation, 这个许多书上都有。 |
T****s
发帖数: 915 | 4 (X1'X1 + X2'X2) b = X1'X1 b1 + X2'X2 b2
【在 q**j 的大作中提到】
: 简单regress 的解是
: b =（X'X)^(-1) X'y。
: 如果把X和Y都分成对应的两块就有
: b_1 = (X1'X1)^(-1) X1'Y1
: b_2 = (X2'X2)^(-1) X2'Y2
: 请问这个b1,b2和b是怎么联系到一起的？
|
s*****n
发帖数: 3416 | 5 what do you mean by "把X和Y都分成对应的两块"?
I don't see how you can do it, unless this is not simple regression. instead
it is multivariate regression.
【在 q**j 的大作中提到】
: 简单regress 的解是
: b =（X'X)^(-1) X'y。
: 如果把X和Y都分成对应的两块就有
: b_1 = (X1'X1)^(-1) X1'Y1
: b_2 = (X2'X2)^(-1) X2'Y2
: 请问这个b1,b2和b是怎么联系到一起的？
|
s*****n
发帖数: 3416 | 6 that is what I thought, x1 orthogonal to x2.
but it is unclear what lz means. the notation is very vague. |
A*******s
发帖数: 3942 | 7 is it equivalent to the following problem?
say we have old data and we already have OLS regression estimates. Now new
data (more observations) come in but we don't want to do SVD or get inverse
based on the whole X'X matrix. Instead we would like to use some fast matrix
updating algorithm to update the regression coefficient estimates.
Am I correct? |
s*********2
发帖数: 106 | 8 我先前理解错LZ的意思了，他是不是说把一个dataset分成两部分做regression。这两
部分分别做regression和用整个dataset做regression，coefficient estimates之间的
关系? |
q**j
发帖数: 10612 | 9 you are more than correct!
inverse
matrix
【在 A*******s 的大作中提到】
: is it equivalent to the following problem?
: say we have old data and we already have OLS regression estimates. Now new
: data (more observations) come in but we don't want to do SVD or get inverse
: based on the whole X'X matrix. Instead we would like to use some fast matrix
: updating algorithm to update the regression coefficient estimates.
: Am I correct?
|
s*****r
发帖数: 790 | 10 then there will be no relation. theoretically, you can have completely
different results, without any fixed relation. because the two parts are
independent.
【在 s*********2 的大作中提到】
: 我先前理解错LZ的意思了，他是不是说把一个dataset分成两部分做regression。这两
: 部分分别做regression和用整个dataset做regression，coefficient estimates之间的
: 关系?
|
F******n
发帖数: 160 | 11 In that case, it is not a trivial problem, but it could be done by applying
the incremental SVD algorithm.
This efficient/adaptive algorithm was originally given by a guy (Matthew)
from Mitsubishi research lab. You can easily google that paper.
The main point relevant to the LS regression:
b = (X'X)^(-1) (X'y) (1)
is that, matrix inversion (X'X)^(-1) is essentially an SVD problem. As your
new data come in, you should sort of have an updated X and y, and
consequently an updated X'X. In above equation (1), two parts can be
adaptively updated (one trivial + one incremental SVD):
#1: it's trivial to buffer and adaptively update (X'y) part, for which idea
should be straight: old-data-sum-buffer + new-data-sum => updated-result.
#2: you can apply incremental SVD algorithm to get the solution of (X'X)^(-1
), by buffering old SVD solution, and adding new data information. In a
nutshell, the idea is that, new data "vectors" can be decomposed into: i)
the part falling into the old SVD subspace and, ii) the part orthogonal to
that subspace span. Only the orthogonal/"new" component needs to be re-
computed for updating the SVD subspace. This is much more efficient than re-
computing the whole SVD problem, namely, old + new data and re-solving
equation (1) from scratch. This incremental algorithm is specifically done
by employing a QR decomposition, followed by a small (or relatively cheap)
SVD in "new" subspace.
Your two parts of data could be in any of following three scenarios:
1) they have identical distribution
2) they have partially overlapping distribution
3) they have totally independent distribution
for any of which, the above adaptive/incremental algorithm should be generic
enough to work well.
Hope this helps.
inverse
matrix
【在 q**j 的大作中提到】
: you are more than correct!
:
: inverse
: matrix
|
F******n
发帖数: 160 | 12 By the way, this is just my personal quick idea/suggestion. You may want to
elaborate and double-check it for technical details.
In case you got other ideas/solutions, it would be nice to share it, or your
final closure here, too.
applying
your
【在 F******n 的大作中提到】
: In that case, it is not a trivial problem, but it could be done by applying
: the incremental SVD algorithm.
: This efficient/adaptive algorithm was originally given by a guy (Matthew)
: from Mitsubishi research lab. You can easily google that paper.
: The main point relevant to the LS regression:
: b = (X'X)^(-1) (X'y) (1)
: is that, matrix inversion (X'X)^(-1) is essentially an SVD problem. As your
: new data come in, you should sort of have an updated X and y, and
: consequently an updated X'X. In above equation (1), two parts can be
: adaptively updated (one trivial + one incremental SVD):
|
T*******I
发帖数: 5138 | 13 这个问题看起来像是分段回归，但不确定，因为楼主没有说清楚是如何分的。在总样本
量足够大的条件下，不论怎样分，部分回归的统计量与全体回归的统计量之间的关系可
以用一个Chi-square统计量来判断其差异显著性。请参见我的分段回归论文中所阐述的
相关内容。
【在 q**j 的大作中提到】
: 简单regress 的解是
: b =（X'X)^(-1) X'y。
: 如果把X和Y都分成对应的两块就有
: b_1 = (X1'X1)^(-1) X1'Y1
: b_2 = (X2'X2)^(-1) X2'Y2
: 请问这个b1,b2和b是怎么联系到一起的？
|
F******n
发帖数: 160 | 14 分段回归、部分回归、全体回归
挺好，挺好，呵呵
【在 T*******I 的大作中提到】
: 这个问题看起来像是分段回归，但不确定，因为楼主没有说清楚是如何分的。在总样本
: 量足够大的条件下，不论怎样分，部分回归的统计量与全体回归的统计量之间的关系可
: 以用一个Chi-square统计量来判断其差异显著性。请参见我的分段回归论文中所阐述的
: 相关内容。
|
