Longbranch Research Associates

Statistical Analysis In Litigation

1. A Short History

In 1970, Stephan Michelson was on the faculty of Harvard University, in a special division of the Graduate School of Education that eventually produced the book Inequality by Christopher Jencks et al.  Yes, Michelson is one of the “al.”  A call came from a pro bono attorney at Hogan & Hartson in Washington, D.C., describing his case and an article by two economists, published in the Washington Post, claiming that, if only he understood economics, he would know that his client was wrong on the facts.  This is a world of accidental connections.  The attorney did not know who at Harvard might help him.  Michelson was simply the closest economist to that telephone.

The case, one of several titled Hobson v. Hansen, followed from a finding that the Washington, DC schools were segregated.  However, the population was so residentially segregated there was no effective remedy.  Julius Hobson then sued to equalize expenditures between schools dominated by white students and schools dominated by black students.  June O’Neill and Arlene Holen argued in the Post that blacks attended larger schools, which were more efficient due to economies of scale.  Michelson replied:

Economy of scale does not mean that larger schools spend less per pupil.  It would mean that larger schools spend less per pupil and achieve the same results.  Unless O’Neill and Holen have adjusted for the quality of schooling, their argument is meaningless.

The attorney, Peter Rousselot, asked “Can you do that?  Can you adjust expenditures for school quality?”  Michelson said he could try, using test scores and controlling for factors other than schooling that would influence those scores.  Thus, in the great tradition of liberal education, a graduate course was created, in which Michelson and several students took on this task.

In what is probably the first use of regression analysis in federal courts, Michelson reported that O’Neill and Holen’s instincts were good: There were economies of scale.  However, their analysis was bad.  Economies of scale were not sufficient to explain all of the per-pupil expenditure difference that was related to the race of students.  Judge J. Skelly Wright accepted Michelson’s analysis, found for plaintiffs, and issued an order to equalize per pupil expenditures among schools (plus or minus 5%), Hobson v. Hansen, 327 F.Supp. 844 (1971).

Michelson stayed in Cambridge for several more years, where he accepted work in other cases.  In March, 1978 he took a one year appointment as Senior Fellow at The Urban Institute in Washington, D.C.  He was remembered in that city, from the Hobson case, and was asked to take more cases.  At the end of his Urban Institute appointment he co-founded and became the plurality owner of Econometric Research, Inc. (ERI).  The purpose of that institution was to perform statistical analyses in litigation.

Over the next several years Michelson bought out the other owners of ERI, and changed its working name to Longbranch Research Associates (LRA), to disassociate what the firm did from anyone’s notion of what econometrics is.  At its peak, LRA had 39 employees.  It followed its Hobson v. Hansen origins by specializing in methodologies that more closely modeled the litigation issue than did the analysis of the expert on the other side.  LRA worked for both plaintiffs and defendants, but always told a prospective client that it pursued the truth.  LRA would not tailor an analysis to support its client.

Although LRA “won” several large cases, and satisfied law firms returned with additional cases, they never paid attention to LRA’s claims of neutrality.  Thus, one by one, each client eventually would bring a case in which their client was wrong,.  LRA stopped work, and informed them that they should, and probably would lose.  For example, Fox and Grove, a large law firm in Chicago, for which LRA had “won” several cases, brought 48 age discrimination cases to LRA, all against their same client, AAA of Michigan.  LRA told them not to proceed with the first case, that AAA had not terminated the worst salesmen, but the oldest, an action that could not be defended.

Fox and Grove took that case to trial, and lost.  Indeed, every time LRA has notified its client that it would lose, but the client has proceeded anyway, they lost.  LRA continues to think it provides the best service to clients by telling them what the data show.  Most clients, however, do not want such advice.  Once they realized that LRA would not “fight” for a deservedly losing cause, Fox and Grove, like other law firms, would not engage LRA’s services.  The remaining 47 AAA cases were taken away, and that client was never heard from again.

By the late 1980s, LRA was becoming  smaller.  By the 1990s, personal computers had become more powerful, and had better software, than ERI and LRA had access to on its $500,000 mainframe computer.  In 1995 LRA sold that mainframe (for $250!) and returned to its roots, in which Stephan Michelson was a sole practitioner.  LRA continued to get and “win” cases, including a huge jury discrimination case in Connecticut, and an even larger death sentence case in that state.  In both cases, LRA worked for the state, the respondent, where a defendant was the “petitioner.”  In other cases it worked for assistant United States attorneys, and attorneys general of other states.  It has not had a private sector client in over three decades.  It is not true that all public sector attorneys seek the truth whatever it is, but it is apparently true that only public sector attorneys do so.  A private sector client told us:

You do not understand.  You think the point is to win the case.  No, the point is to keep our client.

He wanted LRA to put up a good fight, even though we were on the wrong side.  LRA will not do that.  We would not even know how to.

In the Cases box on the right you can get summaries of some of LRA’s cases.  You can download those summaries and case decisions as pdf documents.  Stephan Michelson’s resume is downloadable from a separate box on the right.  There is no list of all cases with which we have been associated.  When we find that the facts do not support our client, we suggest that we be fired from that case.  (We did not mean “Fire us from all cases,” although that is what our clients did.)   If there is no public record of our involvement with that client on that case, we have kept our association with it secret.  When that case did not generate a final report, we have not even kept records of what work we did and what we found.

LRA has always had a national scope.  We have testified across the United States, in the state of Washington, in Georgia, in New Hampshire, as well as in other states.  Our office is in Hendersonville, North Carolina, a half-hour from the Asheville-Hendersonville airport (AVL), an hour from Greenville-Spartanburg (GSP), and two hours from Charlotte’s airport (CLT), the sixth largest in the country.

2. Why LRA is so successful

As discussed above, one reason we are almost always associated with the “winning” side in litigation is that, when we have been approached by what will become the “losing” side, and we determine that we cannot support their case, we become disassociated from it.  You might say we pick and choose winners.

It is not that simple.  Even in Hobson v. Hansen we did not know that a) We would find economies of scale (as defendants had argued), and b) They would be insufficient explain the disparity in school funding associated with race of students.  We simply went about asking the right questions.  We joined the defendant company’s side during its appeal in EEOC v. Chicago Miniature Lamp, because we had seen the work presented by EEOC’s “expert” and Lamp’s expert, and determined that the trial court got it wrong when it found for the plaintiff, 622 F. Supp. 1281 (N.D. Ill. 1985).  We would not have taken on that work but for our conclusion that the District Court’s opinion was erroneous.  The Seventh Circuit agreed with the appeal brief, which we helped write, reversing in 947 F2d 292 (7th Cir., 1991).

The hallmark of our success is that we are expert analysts.  We do not have formal legal training (we do not compete with our clients), but we pay attention to legal literature—cases and law review articles—as well as to statistics literature.  We understand the institutions involved in a case, so that our models are relevant, understandable, correct.  Our analyses speak to judges, because they speak to the issues the judge has to decide upon.  Summaries of cases, making these points, can be found in the Cases box on the right.

Being better analysts means that our toolkit is not limited by methods we learned in school.  Indeed, in several cases we found no tool available to perform the required analysis correctly.  So we invented it.  In the 1980s, we devised the Multiple Pools Exact Test, which we titled “MULPOOLS.”  We then devised a multivariate form of that test, which we titled “MULQUALS.”  We wrote those programs in the PL1 language, in which SAS was written at that time.  We did not rewrite them when SAS converted its entire system to the C++ language.  Later, Cytel Corporation saw the same gap in the PC literature that we had in the mainframe literature, and developed Stat Xact and Log Xact, the very same tests we had used, in litigation, over several years.

When it was viable, we sold our software to other analysts.  Thus some “old-timers” do know about multiple pools tests, but few younger analysts know these methods in either mainframe or PC form.  Yet there are situations in which, without defining multiple pools and analyzing selections therefrom, one gets the wrong answer.  The mathematics of multiple pools models had not been formulated until, at first, the 1950s, and then, in multivariate form, the 1960s.  We thought that once we had shown the usefulness of these tools for problems in litigation, everyone would know them and use them.  However, new litigation analysts seem to have limited toolkits, and even more limited curiosity.  Few know what we mean when we say “That is a multiple pools issue,” and fewer still know what to do about it.

Similarly, many analysts seem to be oblivious to the notion that discrimination is an event, not an outcome.  See the description of Coker v. Charleston School Committee, in the Cases section, as an example.  Or, equally compelling, see Connecticut v. Gibbs, 1998 WL 351903 (Conn. Super, 1998) (Spada, J) affd. 254 Conn. 578, 758 A.2d 327 (2000).  In that case the “expert” on the other side was the chairman of the statistics department at Yale University.  A fine theoretical statistician he was.  But he did not understand that showing fewer Hispanics on juries than their proportion in the population was not “proof” of discrimination.  He had to find some action within the jury selection operation that excluded Hispanics.  There was no such action.  Ultimately, at the end of the trial, he agreed that his analysis did not meet the requirements of the law.  Highly credentialed “experts” are not necessarily competent to perform relevant analyses in litigation.  When that is so, they are not actually experts.

LRA is successful because we find the best statistical approximation to real world events, and can determine, within that model, if biased decisions are being made.  We have found data when attorneys thought none existed.  In 2012 we performed an analysis in which we did locate biased decisions in jury selection.  The same modeling approach can find either way—it depends on the data, the actual decisions made.  We cannot reveal the parties or venue of that case.  We informed our clients that they were wrong, we could not help them, and that ended it.

Ultimately, then, LRA is successful only if our client’s client is correct.  Our mission statement is “We determine fact from data.”  We are supremely competent, rigorous and honest.  If the facts support our client, we will find that is so, and demonstrate it to the court.

3. Credentials and skills

All material available for download is listed in the DOWNLOAD box.  Stephan Michelson’s resume lists his education, and contains all the cases in which he has appeared as a witness, or entered a report as evidence.  This list contains a minority of cases LRA has done.  Our reports or depositions have forced many cases into settlement.  Nonetheless, it is a list of over 45 years of work in litigation.  We cannot imagine a case, with a statistical issue, in which Dr. Michelson would not be accepted as an expert.

Some attorneys have argued against Dr. Michelson’s appearance on the basis of subject matter.  Especially, the question arises whether Dr. Michelson has published in a certain “field.”  One has to take such challenges seriously, not because they are serious, but because judges look for rationales for their decisions, and subject matter might seem to be one.

For example, in the Harris Trust case (summarized under CASES), defendant’s attorneys challenged Dr. Michelson’s lack of experience in banking.  The case was not about banking, it was about employment in a bank.  The bank’s expert, Harry Roberts, had no more experience in banking than did Michelson.  Roberts was determined to defend the bank, inventing “reverse regression” to do so.  Thus Dr. Michelson became the first statistical expert confronted with this novel and, by the way, useless statistical method.  He and another Senior Analyst at LRA, Dr. Gail Blattenberger, explained to the judge that reverse regression was a contrivance, not a sound method.  The judge agreed, finding for LRA’s client, the Treasury Department .

In Holden v. Burlington Northern, Inc. we were challenged for lack of expertise in railroads.  LRA expert testimony succeeded in establishing a class.  Further testimony and depositions drove the case to a settlement in which millions of dollars were paid to our client’s clients.  In a death sentence case in Connecticut, attorneys tried the same tactic.  Attorneys will do what they can to have LRA excluded from a case, because we are well-known: If the data support our client, we will convincingly prove its case.  Judges will let us do so.

Publications are a way in which an academic establishes credentials in a field.  But Dr. Michelson is not an academic.  Disappointed at the level of intellectual inquiry at Harvard, Dr. Michelson took several research positions.  He eventually started his own firm, and found private practice more stimulating than any college campus.  He had no prior experience in death sentencing, but after six years in which he worked on that single case, he is surely the leading expert in that field today.  He does on occasion publish, but that is incidental to his work in litigation.  On the other hand, his many reports can be considered publications, and perhaps could even be offered into evidence as credentials.

That is—and, indeed, it is one of the reasons private practice has turned out to be far more interesting than the pigeon-holes of academic titles—one can learn.  LRA has demonstrated time and again that it can and will learn the nuances of a “field,” and in most cases end up contributing to that field.   One can argue, and we are willing to, that most fields need fresh blood, fresh thinking.  Having been in a “field” for decades may be an anti-credential.

What counts is skill.  Very few statistical experts in litigation have it.  We have been on the other side from one “expert” (Andrew Beveridge) who has developed methods to analyze jury selection by Zip code.  His clients have prevailed in several cases.  We think his methods are incorrect—following the error of not distinguishing outcomes from events.  In the two cases in which we were experts on the other side, our client prevailed.  Apparently not many other experts have figured out the fallacies of this professor’s methods.  That he is wrong is insufficient if the opposing expert does not know why he is wrong, and cannot show what a correct analysis would find.  We have demonstrated that skill time and again, starting with our first jury selection case, when we were neophytes in that “field.”  What is important is having true expertise, and being able to formulate the presentation that demonstrates it.  A judge will use inexperience in a field only as an excuse to discharge an expert he finds weak. Few judges will reject a truly expert analysis directed at the issues calling for expert help, presented with a clear articulation of the question, the method, the data, and the statistical result.