When someone does an analysis like this it always shows a strong correlation between recruiting stars and later success in the NFL. Thus, we can be fairly certain that the overall pool of 3-star players contains a higher percentage of very good players than the overall pool of 2-star/unrated players, with 4 and 5-star groups even better. The problem is that it doesn't tell us anything about individual players, because there are unrated players who turn out to be great, such as Carrie.

Furthermore, one other flaw is that the set of players recruited by any individual school is not a random sample, it's one selected by the coaches to meet certain specific criteria. Thus, depending on the criteria, that subset could be better or worse than the overall pool. For example, are the 2-star players recruited by Ohio representative of the overall pool of 2-star players? Are the unrated players recruited by Ohio representative of the overall pool of unrated players? Are they better, on average, or worse, and if so, by how much?

Over the last decade I have tried multiple times to answer the "stars" question from the reverse perspective. My questions have been, first, "are the 3-star players recruited by Ohio better than the 2-star players", and second, "is one rating service more accurate than the others?" Every time I have done this I have ended up with a dead end.

If one looks solely at the subset of players recruited by Ohio, I consistently find no correlation between the "stars" and how they do on the field once they get there. There actually may be a "U-shaped" correlation where highly rated players are more likely to do better, and unrated players are more apt to do better, while players that are rated, but given low ratings, are less apt to turn out to be stars.

I also find that no rating service is consistently better than any of the others. Sometimes one will do better than the others, but then the next year it is reversed. Therefore, starting in 2012 I started viewing all the film myself and rating the players myself. Was I more accurate or less accurate than the services? Well, in 2012 I did pretty well, but in 2013 I was not good at all (though I did give Poling 4 stars), and 2014 was a mixed bag. On the whole, nope, I doubt I beat the services, but I'm probably not any worse, either.

At the end of all this, my conclusion is this. When Ohio does end up with a 3 or 4 star player, it most likely is a lower 3 or 4 star player. When they do recruit a 2-star or unrated player, it is more than likely a better than average 2-star or a much better than average unrated player. The net result is that the three pools (lower 3-star, higher 2-star, much higher unrated) all end up about the same.