Law of Small Numbers Again in Play on February 2013 Florida Bar Exam; UF Law Ranks 1st for Results in 2012-13 Year
Statement by Dean Robert Jerry
April 16, 2013
The oft-cited statistical principle that excessively small samples can produce wide variances (sometimes referred to as the “law of small numbers”) is again demonstrated in yesterday’s release by the Florida Board of Bar Examiners of the results of the February 2013 Florida Bar Examination. Yet we are pleased that when the small number of UF Law graduates who took the February 2013 bar exam (22) is taken into account, including the small but nevertheless disappointing number who did not pass (6), UF Law’s number one ranking on the July 2012 exam (taken by 284 UF Law graduates, 91.2% of whom passed) remains a number one ranking for all results for the 2012-13 bar examination cycle.
As I said two years ago when the Levin College of Law’s graduates had the highest pass rate in the state on the February 2011 test, I am disappointed when the February numbers receive publicity without any analysis of the number of test takers or without including the results of the much larger July exam. Only 7% of UF Law’s test takers in 2012-13 took the February 2013 examination. Using the February results as a benchmark is as sensible as evaluating how well a baseball player hits in a 162-game season by only counting his at-bats in 11 games. That’s essentially what it is like to look at the February bar results in isolation.
The UF Law pass rate for the 2012-13 bar examination cycle is 89.9%, which exceeds the overall pass rate for all takers by nearly 10 percentage points. I congratulate the one other law school in Florida that tied us with an 89.9% pass rate for the cycle.
As I have said before, the extremely small number of February test-takers makes the results statistically invalid for predicting the results of future exams, for the purpose of comparing bar performance of law school graduates from different law schools, and certainly for comparing law schools. In 2011 and again last year I urged anyone interested in examining a law school’s performance on the Florida bar exam to look at annual performance by combining the much larger July exam with the results of the February exam. This reveals how an entire graduating class performs on the exam. It also avoids the large variances in results that occur year to year when only the very small sample of February test takers is used to calculate a pass rate.
The Florida Bar Board of Examiners, which is an administrative arm of the state Supreme Court, conducts exams every February and July for law school graduates seeking to practice law in Florida. This year, 819 law graduates, only 22 of whom were UF Law graduates, took the February 2013 exam, which is much less than the 3,034 who took the July 2012 test (284 of whom were UF Law graduates). This disparity holds true every year and for every school. To ensure consistency in evaluating bar exam performance, July and February exam results in the year of the graduation cycle should be combined to improve the accuracy of class-by-class comparisons.