If we want to predict the final performance at the end of the season of a bunch of hitters, given information 1/2 the way through, we can do better than suggest that their batting average at the end of the season is likely to be the same as 1/2 way through. The James-Stein estimator often improves substantially improves your capacity to predict. For this example see Persi Diaconis in the Princeton Companion to Mathematics, using an example due to Bradley Efron.
Much the same would apply to third year review of probationary faculty. We might count the number of publications or some such for each faculty member, perhaps in all fields, divided by the years (3). You might want to adjust those numbers by some factor to take into account the different publication rates or article lengths in different fields, or maybe just count working papers or pages of a published book. In any case you could predict the amount published at tenure time by multiplying this rate by say 6. Surely people's performance is often speeded up as they come up for tenure--maybe fear, maybe the long time it takes to get an article accepted, maybe maturity of thought. (Baseball players have slumps, wear down, find their groove ....) So you might want to put in a nonlinear adjustment--although I am not sure this will work. In any case, using the James-Stein estimator for all your probationary faculty will provide a much better measure of their likely performance at the end of the probationary period ("the season") than would just their average annual performance so far. [I realize fully that a baseball player is not fully appreciated in terms of their batting average, and the same is the case for scholars and their publication performance in terms of numbers.]
Say you have p faculty, and a p-length vector of their article-numbers/year: A
Then the improved estimate in terms of the current rate, Aimproved= A times {1-((p-2)/|A|)}max of {} or zero
where |A| is length of the vector A. And the bracketed expression is a max of its value or 0 (to avoid negative values).
You could also do this at tenure time, to better predict whether your candidates will actually do well in the future.
Of course, you need to have reasons to believe that your information at the third year review or at tenure time is likely to be indicative of future performance. Such reason might be how they otherwise act, or your past assessment of other candidates. The assumption about hitters is perhaps more reliable than the ones about scholars.
----------------------------------------------------------------
I am revising a chapter in my Doing Mathematics and so I am rediscovering this stuff. In reading Diaconis (who is a great expositor, distinguished mathematician, and it seems a former magician), I came upon the stuff on batters, and thought of probationary faculty. I consider this a deformation professionel.
