Wisconsin Cases July 20, 2022: Erdman v. City of Madison

Case Description

1

OPINION AND ORDER

WILLIAMM.CONLEY DISTRICT JUDGE

On October 15 and 16, 2018, the court held a trial to the bench on plaintiff Catherine Erdman's claim that the City of Madison, and more specifically its Fire Department, adopted a physical abilities test (“PAT”) that has a disparate impact on women in violation of Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. § 2000e, et seq . For the reasons explained below, the court now concludes that: (1) plaintiff met her burden of proving that the Fire Department's PAT has an adverse impact on female applicants; (2) defendant met its burden of proving that the PAT is job-related and consistent with business necessity; and (3) plaintiff did not meet her burden of proving that the alternative physical abilities test she identifies, the Candidate Physical Abilities Test (“CPAT”), will serve the Fire Department's legitimate needs. Accordingly, the court will find in defendant's favor.

ADDITIONAL FINDINGS OF FACTS

A. Disparate Impact of the PAT on Female Applicants

1. A total of 1887 applicants participated in the 2014 recruitment. Of these, 1723

2

were men, 146 were women, and 18 were not clearly identified by gender.

2. Four hundred and ninety-nine applicants appeared to take the PAT -- 471 men and 28 women. Of these, 404 applicants -- 395 men, four women, and five not clearly identified -- successfully completed the PAT.

3. Excluding those not self-identifying with either gender, the following chart illustrates the results of the 2014 PAT by sex:

2014 PAT by Sex
Category	Males	Females
	Percentage Against Males Who Appeared	Percentage Against Males Who Appeared and Did Not Quit	Number	Percentage Against Females Who Appeared	Percentage Against Females Who Appeared and Did Not Quit	Number
Appeared to Take Test			471			28
Quit During Test	2.76%		13	10.71%		3
Did Not Quit During Test	97.24%		458	89.29%		25
Disqualified for Failing to Meet Minimally Acceptable Score	10.40%	10.69%	49	71.43%	80.0%	20
Failed Test	2.97%	3.06%	14	3.57%	4.00%	1
Passed Test	83.86%	86.24%	395	14.29%	16.00%	4

3

4. As reflected in the chart, the overall pass rate for women who appeared to take the test (4/28 or 14.29%) was about 17% of the pass rate for men who appeared to take the test (395/471 or 83.86%).

5. Conversely -- the women's failure rate -- defined as applicants who met the minimally acceptable score for each of the seven events, but failed to meet the cut-score for at least five of the seven events -- of 1 out of 28 (3.57%), for the test was roughly 120% that of men's failure rate of 14 out of 471 (2.97%).

6. Finally, the women's disqualification rate -- defined as those who appeared to take the test and did not quit -- of 20 out of 25 (80%) was 748% that of men's disqualification rate of 49 out of 458 (10.69%).

B. Job-Relatedness and Business Necessity of PAT

7. Debra Amesqua became the Madison Fire Department Chief in 1996. Following her appointment, Chief Amesqua engaged Landy, Jacobs and Associates (“LJA”), to develop the Department's PAT in 1997.

8. Directed by Amesqua to develop a test that correlated with the tasks on the job, LJA developed the PAT under the Uniform Guidelines on Employee Selection Procedures (1978), 29 C.F.R. § 1607, et seq .

9. In particular, Rick R. Jacobs, Ph.D., an industrial psychologist, was one of the individuals who developed the PAT in conjunction with exercise physiologists. Jacobs had developed physical ability tests for a little more than a decade before taking on the task of developing the PAT at issue here.

4

10. In developing a PAT, LJA focuses on job simulation activities, rather than traditional exercise-based activities. Chief Amesqua and other fire department personnel were involved in studying the specific jobs as performed by Madison firefighters.

11. LJA completed content validity reports in 1997 and 1999, finding the PAT valid as required under 29 C.F.R. § 1607.14(B)(4).

12. LJA relied on incumbents to establish the cut and disqualifications scores. For the 1997 PAT in particular, LJA drew on PAT scores from 94 incumbent Madison firefighters randomly selected but controlled to reflect the Madison Fire Department's diversity as to race and gender. At that time, roughly 17% of the department were women.

13. In 1999, LJA modified some of the events and developed new minimally acceptable performance standards and cut-scores by drawing on PAT scores from 102 incumbent firefighters.

14. The cut scores roughly eliminated the bottom 16% of incumbents, which Jacobs opined offered a good method for managing the two errors, passing applicants who are not able to perform the job and eliminating applications who could perform the job. The disqualification score reflected the lowest performing incumbent.

15. In both Jacobs' and Chief Amesqua's experience, applicants perform better than incumbents on the test because they are more motivated. Moreover, Amesqua wanted to use incumbents to set cut scores because of “washout” (attrition) concerns during the subsequent training academy. Specifically, for costs and other reasons, Amesqua wanted

5

to insure that the applicants entering the academy could perform the physical requirements of the job.

16. Following Chief Steven Davis's appointment to replace Chief Amesqua in late 2012 or early 2013, Ergometrics & Applied Personnel Research, Inc. (“EAPRI”) was retained by the Madison Fire Department to validate the PAT again. EAPRI's President, Carl Swander, Ph.D., and his team then conducted a “content validation study.” Like the earlier studies by LJA, this study was conducted under the Uniform Guidelines, among other professional publications.

17. As part of the study, EAPRI used Madison Fire Department “subject matter experts” (“SMEs”) to determine what the required speed should be of firefighter candidates taking the examination. As was the case in LJA's studies, EAPRI again used incumbents to set the minimum standards requirements. In addition, Ergometrics also provided physiological measurements of incumbents to gain information as to the overall energy required to perform the PAT's discrete tasks and the PAT overall as a whole.

18. Working with the SMEs, EAPRI selected tasks or test events representative of the variety of physical demands Madison firefighter job applicants would be expected to perform on the job.

19. EAPRI next studied 19 incumbents to set the minimum level of performance requirements. In determining the cut off, EAPRI set the scores for the selected task or event at one standard deviation below the mean, which means that: (1) 82% of participants should do this well or better; and (2) the required max time for passing the test is often about 1.4 times as long as the average time for incumbents.

6

20. EAPRI's method for setting employment screening standards (or cutoffs) is a common practice for ergometric testing and is more objective than setting the time requirements based on observed speeds. EAPRI checked the cut score by considering observations from experienced command staff.

21. Focusing on the specific elements of the PAT, the “ladder test” was developed and changed to reflect the equipment used at the time of the test. In particular, the 20-foot ladder was used when at least some of the Department's rigs still used a 24-foot extension ladder. The choice of ladder, as well as the required manipulation of the ladder, reflected the job requirements at that time.

22. As for the “pike pole test,” the PAT used a box or a line to require applicants to stand at least 18 inches back from the ceiling being pulled down with a pike pole to account for safety concerns with standing directly under the ceiling. Further, the pole was long enough to allow applicants of different heights to adjust the hold to account for that differential.

23. While the PAT does not include a practice requirement, in contrast to the CPAT suggested as an alternative by plaintiff, there are other methods for promoting familiarity with the test, including a manual, video and an exercise guide.

24. The content validity reports also did not include a test-retest for reliability, although again, the PAT offered other methods of testing for reliability, including an “interrater agreement,” where different administrators of the test agree on the score for the same candidate. LJA confirmed that this test for reliability was completed in 1997 and 1999 while EAPRI, relied on the execution and administration of the PAT to ensure

7

reliability in 2013.

25. The posting for the 2014 recruitment identified the following physical requirements for firefighter positions:

While not an exclusive list, the following examples are meant to illustrate some of the extreme physical demands and working conditions inherent in the role of a firefighter.

Physical Demands

1. Pick up and advance charged fire hoses.*

2. Force entry with axe/battering ram.*

3. Rescue/extricate victim(s).*

4. Perform CPR; apply bandages.

5. Climb stairs carrying heavy equipment, while wearing firefighter protective clothing that weighs in excess of 50 pounds.*

6. Strip and vent roofs, breach walls, overhaul burned buildings.*

7. Lift and climb/descend ladders (with victims).*

8. Visually determine fire status/hazards; assess patient conditions.

9. Hear calls for help; identify fire noise, etc.

10. Walk on roof tops under adverse conditions.

11. Operate power tools and extrication equipment; tie knots.

12. Stoop, crawl, crouch, and kneel in confined spaces.*

13. Reach, twist, balance, grapple, bend and lift under emergency conditions.

14. Run, dodge, jump and maneuver with equipment.*

15. All of the above may be performed wearing heavy and restrictive protective clothing/gear in excess of 50 pounds.*

Each task marked by an asterisk was assessed in the 2014 PAT.

C. Adequacy of CPAT as Alternative Test

26. The CPAT was developed in joint collaboration between the International

8

Association of Fire Chiefs and the International Association of Firefighters, in conjunction with ten fire departments in North America, including Austin, Texas, Los Angeles County, California, and New York City.

27. According to a validation study conducted by an exercise physiologist at the University of Texas-Austin, examining the Austin Fire Department, approximately 48% of female participants passed the CPAT.

28. In 2013, the Department considered whether to adopt the CPAT, although the consideration was not because of, or solely because of, concerns about female applicants' performance on the PAT.

29. While there is significant overlap between the tasks in the CPAT and that in the PAT, plaintiff identified four differences between the CPAT and the PAT that her expert maintains are material: (1) the CPAT has an overall time requirement, rather than time requirements for the discrete tasks as is the case for the PAT; (2) the CPAT has two, required practice runs without coaching help, each of which applicants can opt to “run hot” and count as a passed test if the applicant successfully completes it; (3) the cut scores under the CPAT were developed differently using applicants' times, rather than incumbents'; and (4) the pike pole and ladder tests have different elements.

30. The CPAT has not been locally validated.

OPINION

Title VII prohibits hiring practices that have a disproportionately adverse impact, also referred to as a “disparate impact,” on employees with protected characteristics, such as sex, even if there is no intent to discriminate. Ernst v. City of Chi. , 837 F.3d 788, 794 (7th Cir. 2016).

9

To prove her claim here, therefore, plaintiff must show that a particular hiring practice had a disparate impact on female applicants. 42 U.S.C. § 2000e-2(k)(1)(A)(i); Ernst , 837 F.3d at 796. As part of her prima facie case, the plaintiff must also show causation, typically by “offer[ing] statistical evidence of a kind and degree sufficient to show that the practice in question has caused the exclusion of applicants for jobs or promotions because of their membership in a protected group.” Watson v. Fort Worth Bank & Tr. , 487 U.S. 977, 994 (1988). In turn, an employer can defend against a plaintiff's prima facie case by: refuting proof that the challenged practice is a cause of the disparate impact, 42 U.S.C. § 2000e-2(k)(B)(ii); or by showing that the practice is job-related for the position and consistent with business necessity, 42 U.S.C. § 2000e-2(k)(1)(A)(i); see Griggs v. Duke Power Co. , 401 U.S. 424, 432 (1971).

Although employers have the burden of showing the relationship between a requirement and the employment in question, “employers are not required, even when defending standardized or objective tests, to introduce formal ‘validation studies' showing that particular criteria predict actual on-the-job performance.” Watson , 487 U.S. at 998. Rather, the burden then shifts back to the applicant to prove the employer refuses to adopt an alternative hiring practice that results in a less disparate impact while still serving the employer's legitimate needs. 42 U.S.C. §§ 2000e-2(k)(1)(A)(ii), (C); Ernst , 837 F.3d at 794; see also 29 C.F.R. § 1607.3B. (“[T]he user should use the procedure which has been demonstrated to have the lesser adverse impact”).

At summary judgment, defendant argues that plaintiff's proof falls fatally short at three, independent stages of the disparate impact analysis. First, defendant argues that

10

plaintiff has not made out a prima facie case of disparate impact as to female applicants generally, or at minimum, as to plaintiff in particular given her disqualification was at the component stage of the PAT at issue. Second, defendant argues that even if there was a disparate impact, it successfully established the PAT was “job related” and “consistent with business necessity.” Finally, defendant argues that plaintiff failed to prove a viable alternative employment practice with a less adverse impact. The court addresses each argument in turn below.

I. The PAT Produced a Disparate Impact on Female Applicants

Based on the statistical analysis of the 2014 PAT results described at summary judgment, and summarized above, as supported by expert testimony proffered by plaintiff, the court found that the differences in the performance of male and female applicants during the PAT were highly significant statistically and that a reasonable factfinder could conclude that the PAT produced a disparate impact on female applicants, sufficient to satisfy her burden on the first element of a disparate impact claim. Moreover, at trial, the defendant again did not dispute that plaintiff had satisfied the first element of the claim if the court considered the performances of applicants attempting the PAT as a whole, rather than considering the discrete elements of the PAT that resulted in plaintiff's failure in particular -- the ladder and pike pole elements -- which she also argues were particularly disadvantageous for females. In its decision on summary and judgment, the court also discussed at length its reasons for analyzing the PAT as a whole as the challenged hiring practice (9/19/18 Op. & Order (dkt. #47) 12-16), and invited additional briefing on this issue from both parties.

11

Nevertheless, defendant simply regurgitates the same arguments it previously made in its post-trial briefing. (Def.'s Br. (dkt. #54); Pl.'s Br. (dkt. #74) 5-9.) The court sees no reason to revisit this issue as a matter of law, and again concludes that the appropriate unit of analysis for plaintiff's claim is the PAT as a whole. Moreover, the court concludes based on the undisputed record that plaintiff has established by a preponderance of the evidence that the 2014 PAT had an adverse impact on female applicants.

II. The PAT was Job-Related and Consistent with Business Necessity

Given that plaintiff has satisfied her burden of demonstrating that the PAT as a whole produced a disparate impact on women, the burden shifts to defendant to show that the requirements of the PAT are job-related for the position and consistent with business necessity. 42 U.S.C. § 2000e-2(k)(1)(A)(i); Griggs, 401 U.S. at 432. “A test is ‘job-related' if it measures traits that are significantly related to the applicant's ability to perform the job.” Gillespie v. State of Wis. , 771 F.2d 1035, 1039 (7th Cir. 1985). “Employers are not required to support their physical-skills tests with formal validations studies, which show that particular criteria predict actual on-the-job performance.” Ernst , 837 F.3d at 796 (internal citation and quotation marks omitted). As is in this case, however, “[w]hen an employer relies on a validity study, federal regulations establish technical standards for these studies.” Id. (citing 29 C.F.R. § 1607.14(B)(4)).

While the parties appear to dispute whether § 1607.14 requires the defendant to prove the validity of the PAT itself or the study used to do so, the industrial psychologist firms that the Department employed to validate the PAT used content studies, and there is no dispute that a content study is an adequate method of validation. Similar to a

12

criterion study, the regulations for a content study require a selection procedure that: (1) ensures a “representative sample of the content of the job”; and (2) the skill to be tested is a “necessary prerequisite to successful job performance.” 29 C.F.R. § 1607.14(C)(1). Thus, in determining whether the content of the test is valid, the Seventh Circuit directed district courts to consider:

(1) the degree to which the nature of the examination procedure approximates the job conditions; (2) whether the test measures abstract or concrete qualities; and (3) . . . combin[ing] of these factors, . . . whether the test attempts to measure an abstract trait with a test that fails to closely approximate the working situation.

Bryant v. City of Chi ., 200 F.3d 1092, 1099 (7th Cir. 2000) (quoting Gillespie v. State of Wis. , 771 F.2d 1035, 1043 (7th Cir. 1985)).

As detailed above, defendant's experts Rick Jacobs, the developer of the original PAT in 1997, and Carl Swander, who updated and validated the PAT in 2013, both provided extensive testimony about: (1) the job studies conducted to select physical tasks as a proxy for on-the-job requirements, which were conducted with the assistance of senior members of the Department; and (2) the process for setting cut and disqualification scores, utilizing a statistical method and checking it against the subjective assessment of senior members of the Department. (Facts ¶¶ 7-20.) Members of the Department also testified to their involvement in developing and updating the PAT leading up to the challenged 2014 PAT. Given this record, the court concludes that the 2014 PAT was valid under the Uniform Guidelines set forth in § 1607.14.

In fairness, while plaintiff's expert Arthur Weltman offered some challenges to the PAT: criticizing how the cut and disqualification scores were set; questioning whether the

13

PAT developers were sufficiently experienced in exercise physiology, which is Weltman's area of expertise; raising concerns about the qualifications of the Department's command officers to serve as subject matter experts; and faulting defendant for failing to engage in a test-retest review. However, none of these concerns directly challenge to the validity of the 2014 PAT under § 1607.14. Indeed, on this critical point, Weltman conceded that he had little familiarity with the Uniform Guidelines and offered no opinion as to whether the PAT was valid in light of them. (Trial Tr. (dkt. #67) 61.)

In addition to failing to show that any of these criticisms -- whether viewed alone or in combination -- undermine defendant's evidence of the validation process for its PAT § 1607.14, including the 2014 PAT at issue, defendant also countered with its own evidence that LJA's and EAPRI's reliance on statistical analysis to set cut and disqualification scores was a widely accepted practice, and witnesses from both consulting groups reliance on the Departments' senior officials in conducting a job study and utilizing other forms of reliability testing, including focusing on execution and administration of the test. EAPRI's President Swander also credibly testified to the difficulty in conducting a test-retest study with a sufficient number of participants given Madison's size (among other constraints), and critiqued the Austin study of the CPAT as having an insufficient survey base to serve its purpose.

The court further concludes that the Department met its obligation to investigate “alternative selection procedures with evidence of less adverse impact . . . to determine the appropriateness of using or validating it in accord with [the Uniform] guidelines.” 29 C.F.R. § 1607.3(B). Specifically, Swander testified that he reviewed the CPAT test

14

components with the Department, but it rejected the pike pole component of the CPAT. The court credits this testimony, and further finds that the Department offered significant evidence in support of the specific designs of its own ladder and pike pole test. In particular, the evidence demonstrated that the ladder test reflected the equipment used by the Department at the time of the test, and the pike pole configuration took into account safety considerations. The court sees no reason to find that these discrete decisions somehow invalidated the PAT.

Plaintiff also faulted the Department for its failure to consider the fairness of the test on female applicants. Specifically, plaintiff pointed out that other than Firefighter Frances Tatar's research into the CPAT and attempts to collect data about hiring of female applicants by other departments using the CPAT, the Department did not engage in a “fairness” assessment as required under the portion of § 1607.14 addressing criterion validity studies. However, defendant argued that a fairness assessment was not required for content validity studies, given the regulation's construction, a reading which the court indicated seemed correct during the closing discussion. (Def.'s Br. (dkt. #70) 10-13; Trial Tr. (dkt. #68) 178-79.) Moreover, in post-trial briefing, defendant again made this argument, which plaintiff did not dispute, effectively conceding that the fairness analysis under 29 C.F.R. § 1607.14(B)(8) was not applicable to content validity studies. (Def.'s Reply (dkt. #76) 2.)

15

Based on the record at trial as a whole, therefore, the court finds that defendant has met its burden of proving by a preponderance of the evidence that the 2014 PAT was job-related for the position and consistent with business necessity.

III. Alternative Test, the CPAT, Was Not Adequate.

As a result of this finding, the burden shifts back to plaintiff to prove that the employer refused to adopt an alternative hiring practice resulting in less disparate impact and serving the employer's legitimate needs. 42 U.S.C. §§ 2000e-2(k)(1)(A)(ii), (C); Ernst, 837 F.3d at 794; see also 29 C.F.R. § 1607.3B. (“[T]he user should use the procedure which has been demonstrated to have the lesser adverse impact.”). Here, Erdman contends that the CPAT is such an alternative test. In determining whether “the CPAT would be equally as effective as the challenged practice in serving the employer's legitimate business goals,” the court may consider “[f]actors such as the cost or other burdens of the proposed alternative selection devices.” Watson , 487 U.S. at 998.

As an initial matter, defendant challenges whether the evidence demonstrates that the CPAT has a lesser adverse impact. Specifically, defendant questions whether the

16

CPAT's passage rate cited by plaintiff of 48% is a fair comparison to the female passage rate under the 2014 PAT. Specifically, defendant points out that the CPAT study showed of the “naive” or non-firefighter participants (as compared to the incumbent participants) in the Austin study only two of nine passed, resulting in a passage rate of 22%, which is certainly much closer to the 14% passage rate experienced using the 2014 PAT. (Trial Tr. (dkt. #67) 85-86; Ex. 6 at p.5.) Moreover, the CPAT revalidation study noted that the ”comparison of male and female sample size is so disparate, it does not allow statistically meaningful comparison.” (Ex. 6 at p.6.) In considering the pike pole event in particular -- the element that eliminated plaintiff Erdman from the 2014 PAT-- 4 out of 20 women or 20% participating in the CPAT validation study did not complete that component, as compared to 1 out of 7 participants or 14% participating in the 2014 PAT.

Still, even conceding the likely lack of a statistically significant testing sample between a single department's use of CPAT and the Madison Fire Department's use of the 2014 PAT, plaintiff's expert Weltman also relied on a broader study of fire departments using the CPAT, in which the pass rates for women was 68.0% as compared to 49.0% in departments using other physical ability tests. (Weltman Rept. (dkt. #18-2) 24.) Based on the results of this broader survey, the court finds it more probably true than not true that the CPAT has less of an adverse impact on female applicants. The court further finds that the CPAT is valid generally, which the defendant does not really challenge. Indeed, defendant's expert Charles Swander testified that, in his role as a co-owner of a testing business, he has administered the CPAT over 10,000 times. (Trial Tr. (dkt. #68) 108.)

17

This leads to the final issue addressed at trial: whether plaintiff proved by a preponderance of the evidence that the CPAT meets Madison Fire Department's legitimate needs. In answering this question, both parties focus on certain, distinct elements of the PAT compared to the CPAT. At the outset, the court recognizes that this analysis proves an ill fit, since plaintiff is challenging the PAT as a whole, which is necessary to meet her prima facia burden of showing a disparate impact. Moreover, if plaintiff were pursuing a claim based on a discrete element of the PAT, the available statistics are not nearly so clear cut to support her claim of disparate impact, as described above. Nevertheless, plaintiff seeks to latch onto discrete elements of the PAT, as compared to the CPAT, to argue that the latter meets the Madison Fire Department's legitimate needs.

Even adopting plaintiff's approach, the court must consider the Department's specific arguments in support of the two elements that plaintiff asserts are unnecessary as compared to CPAT's alternative elements -- the ladder and pike pole. In particular, the Department points out that those elements of the test were specifically designed to replicate the tasks Madison firefighters would be expected to execute in light of the equipment available to the Department at that time and of concerns about safety. Evidence of the time spent by the Department's personnel and their expert consultants in developing these two elements of the 2014 PAT was overwhelming. In crediting this testimony, however, the court is skeptical of the Department's argument that its role as a forerunner in developing this type of physical abilities test -- one that is premised on job tasks, rather than just general fitness requirements -- as well as its relatively strong record of hiring women more generally when compared to other fire departments around the

18

country, should somehow excuse it from considering an alternative test. On the other hand, plaintiff simply points to the CPAT, assuming that it would fit Madison's needs without attempting to validate the test locally.

Regardless, defendant proffered credible evidence of numerous burdens associated with adopting the CPAT as an alternative test, including: (1) the need to perform a transferability study; (2) the PAT having been a good predictor of outcome historically, as defined by a high passage rate out of the academy; (3) the Department's comparatively high percentage of female firefighters, leading to a possible inference that the CPAT may have a favorable disparate impact on women but results in the washing out of ultimately unsuccessful applicants after the additional expenditure of time and money at the academy phase; and (4) certain elements of the PAT were designed specifically for Madison, in light of characteristics of the city, the Department's equipment or other considerations, including safety. Given plaintiff bears the burden to prove the CPAT would serve the Madison Fire Department's legitimate needs, when coupled with the Seventh Circuit's admonition that “courts are generally less competent than employers to restructure business practices, and unless mandated to do so by Congress they should not attempt it,” Ernst , 837 F.3d at 794 (quoting Furnco Constr. Corp. v. Waters , 438 U.S. 567, 578 (1978)), the court concludes that plaintiff has not demonstrated by a preponderance of the evidence that the CPAT meets the Department's legitimate needs as an alternative to the 2014 PAT.

19

ORDER

IT IS ORDERED that the court finds in defendant's favor on plaintiff's Title VII claim. The clerk's office is directed to enter judgment accordingly.