G*****h
发帖数: 320 | 1 透露点个大家也许有点兴趣的信息。
不久前的参加一个 study section 电话会议（正式评审前的），主要讲今年 NIH 给各
个评审小组 recalibration 分数的机会。就是说，今年开始参加评审的的得分会从头
开始计算 percentile，不再和以前的2次和起来算 percentile。
主要是因为有不少小组积累一点时间以后，打得分数开始朝某些分数段集中，偏离
percentile 的正常分布。这样重新校正，等于从头开始，让分数分布更合理些。因为
打分的时候，大家不必考虑以前的分数。
这里是今年到目前为止，NIH 评审了将近 2千5百份 R01 的分布：
10-19，6.1%
20-29，13.9%
30-39，15.2%
40-49，11.4％
50-59，4.1％
60-69，0.9%
70-79，0.1%
80-89，0%
Unscored：45.8%
这样看来，今年评审的 R01，分数20分以内，都很有希望的样子。 |
l***s
发帖数: 841 | |
A******y
发帖数: 2041 | 3 That's because some study sections don't use all the 10 scale points...some
only use 1, 2, and 3. So in the end, you have to have all 1s to get funded,
and in the end, there are too many 1s and not enough money, haha, so who do
you give the money to? |
G*****h
发帖数: 320 | 4 Exactly! I have experienced at least three recalibration events. One before
and one after the implementation of new scoring system.
The new scoring system is very bad too. At least no one I know likes it...
some
funded,
do
【在 A******y 的大作中提到】
: That's because some study sections don't use all the 10 scale points...some
: only use 1, 2, and 3. So in the end, you have to have all 1s to get funded,
: and in the end, there are too many 1s and not enough money, haha, so who do
: you give the money to?
|
s******y
发帖数: 28562 | 5 关于这个，我有一个问题，就是有的study section 给分特别慷慨，有的则
给分非常苛刻，那么在最后分钱的时候这个是怎么处理的？比方说有的section
给一大堆人评了10几分的高分，而有的section 除了一两份申请能拿到10分以内，其他
都是20几30几。那么最后算这个percentile 的时候，到底是按照那个具体的section
里面的所有申请来算的，还是整个NIH 所有的申请一起算？
我以前问过一个资深发考题，但是她也没有告诉我这个问题该怎么处理。
【在 G*****h 的大作中提到】
: 透露点个大家也许有点兴趣的信息。
: 不久前的参加一个 study section 电话会议（正式评审前的），主要讲今年 NIH 给各
: 个评审小组 recalibration 分数的机会。就是说，今年开始参加评审的的得分会从头
: 开始计算 percentile，不再和以前的2次和起来算 percentile。
: 主要是因为有不少小组积累一点时间以后，打得分数开始朝某些分数段集中，偏离
: percentile 的正常分布。这样重新校正，等于从头开始，让分数分布更合理些。因为
: 打分的时候，大家不必考虑以前的分数。
: 这里是今年到目前为止，NIH 评审了将近 2千5百份 R01 的分布：
: 10-19，6.1%
: 20-29，13.9%
|
G*****h
发帖数: 320 | 6 Percentile 是按照每个 study section 算的（不是把不同的 study sections 合起来
算）。所以同样的分数在不同的 study sections 就完全可能得到不同的 percentile。
正常情况下，是把这个 study section 这次的，和以前 2 次的所有申请合起来（总共
3次的），按照分数排队。然后根据这个排序来算 percentile。这次 recalibrate 分
数就是从头来，不考虑前两次的了。这样把分数刻意拉开，就不会因为和以前的相比而
吃亏了。以后的也就基本按照这次的 scale 来打分。
在每个 study section，三个主要 reviewers 先给个初步分数，讨论结束后，3个
reviewers 再给出建议分数（也就是他、她们自己要打的分数）。其他成员再根据前面
的讨论，在建议分数范围之内打分。如果超出这个范围的，就需要举手和说出理由。
比如3个主要reviewers的分数是2、2、3分，其他人要么给2分，要么3分。也许给2分的
多，但如果给 3分 reviewer 影响力很大，听他的人多，那么给 3分的会多些。最后的
分数，是这些分数的平均。
【在 s******y 的大作中提到】
: 关于这个，我有一个问题，就是有的study section 给分特别慷慨，有的则
: 给分非常苛刻，那么在最后分钱的时候这个是怎么处理的？比方说有的section
: 给一大堆人评了10几分的高分，而有的section 除了一两份申请能拿到10分以内，其他
: 都是20几30几。那么最后算这个percentile 的时候，到底是按照那个具体的section
: 里面的所有申请来算的，还是整个NIH 所有的申请一起算？
: 我以前问过一个资深发考题，但是她也没有告诉我这个问题该怎么处理。
|
H*****e
发帖数: 120 | 7 I am not sure if this new approach will be better. But I somehow like it.
In my section, most people are nice. The scores are very close. This
produces a strange problem. Many applications are about the same on paper
but one or two score could make them get the funding or not. A while ago,
two simliar applications that I reviewed ended with that the one I like did
not get funding but the other got it. With new system, I hope it would
enlarge the difference and one or two reviewers's score do not change it
overall outcome. However, we are all human. My personal feeling is that
the program directors and higher level adminstrators in NIH would have more
power to make the decision. |
S******9
发帖数: 2837 | 8 这样很容易在最后meeting的时候有1-2个大佬力挺的话，很容易就改变所有的形式。以
后grants很容易就被瓜分了。
did
more
【在 H*****e 的大作中提到】
: I am not sure if this new approach will be better. But I somehow like it.
: In my section, most people are nice. The scores are very close. This
: produces a strange problem. Many applications are about the same on paper
: but one or two score could make them get the funding or not. A while ago,
: two simliar applications that I reviewed ended with that the one I like did
: not get funding but the other got it. With new system, I hope it would
: enlarge the difference and one or two reviewers's score do not change it
: overall outcome. However, we are all human. My personal feeling is that
: the program directors and higher level adminstrators in NIH would have more
: power to make the decision.
|
y******8
发帖数: 1764 | 9 This was the intention of the new scoring system, which was on NIH webpage.
"My personal feeling is that the program directors and higher level
adminstrators in NIH would have more power to make the decision."
did
more
【在 H*****e 的大作中提到】
: I am not sure if this new approach will be better. But I somehow like it.
: In my section, most people are nice. The scores are very close. This
: produces a strange problem. Many applications are about the same on paper
: but one or two score could make them get the funding or not. A while ago,
: two simliar applications that I reviewed ended with that the one I like did
: not get funding but the other got it. With new system, I hope it would
: enlarge the difference and one or two reviewers's score do not change it
: overall outcome. However, we are all human. My personal feeling is that
: the program directors and higher level adminstrators in NIH would have more
: power to make the decision.
|
