GSA Logo - U.S. General Services Administration

USA Hire℠ Agentic Platform

Closed-Loop Validity

Step 2 • Assessment Design Agent

Right method for every KSAO. No more one-size-fits-all.

The agent takes a KSAO portfolio, queries an evidence store of meta-analytic validity priors, and optimizes the assessment battery for predicted validity, time, cost, and adverse impact.

Conversation with Design Agent

Active
I need to assess for a GS-13 IT Specialist (INFOSEC). Pull in the KSAOs from our existing job analysis.
Loaded the KSAO portfolio from job analysis JA-2025-INFOSEC-013 — 11 KSAOs weighted by criticality. What constraints should I optimize for?
Total time under 120 min, adverse-impact d ≤ 0.25, predicted validity at least 0.50. We can afford simulations for this grade.
Running validity-generalization optimization across 7 method families and 32 evidence sources… proposing a 4-method battery below. Predicted r̂ = 0.57, total 105 min, weighted adverse-impact d = 0.18.

KSAO Portfolio

Loaded from JA-2025-INFOSEC-013

Information security principles

Knowledge

0.92

Federal cyber policy & FISMA

Knowledge

0.78

Risk assessment & analysis

Skill

0.95

Incident triage under pressure

Skill

0.88

Technical written communication

Skill

0.70

Critical reasoning

Ability

0.85

Information processing speed

Ability

0.60

Conscientiousness

Other

0.65

Integrity

Other

0.90

Recommended Battery

4-method battery for GS-13 IT Specialist (INFOSEC)

Predicted validity

r̂ 0.57

Hunter–Schmidt corrected

Total time

105 min

under 120-min cap

Adverse impact d

0.18

below 0.25 target

Cost per applicant

$$$

GS-13 grade approved

Method composition

Cognitive Ability (CAT)

25 minr̂ 0.51
A1A2

Highest mean validity for general cognitive ability per Schmidt–Hunter meta-analysis (k=85, r̄=.51).

INFOSEC Work Sample

35 minr̂ 0.48
K1K2S1S3

Job-specific work sample: applicant triages a synthetic incident and drafts a remediation memo. Content-valid for §14C(4).

High-Fidelity Simulation

30 minr̂ 0.45
S2S1

Branching scenario simulation under time pressure. Recommended for GS-13+ where stakes justify production cost.

Structured AI-Graded Interview

15 minr̂ 0.39
S3O1O2

Behavioral interview with consistency, integrity, and judgment probes. Transcript scoring with fairness audit.

Method validity contributions

Incremental validity computed via validity generalization; method covariance corrected to avoid double-counting.

Alternative configurations

Recommended

r̂ 0.57
Time105 min
Cost$$$
Adverse-impact d0.18

Lean (drop simulation)

r̂ 0.51
Time75 min
Cost$$
Adverse-impact d0.21

Fairness-first (drop cognitive)

r̂ 0.48
Time80 min
Cost$$
Adverse-impact d0.09

Evidence cited by the agent

  • Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology.Cognitive ability prior
  • Sackett, P. R., et al. (2022). Revisiting the design of selection systems in light of new findings.Method incrementality
  • Hunt, S. T. (2014). Common Sense Talent Management.Method-context matching
  • OPM Federal Competency Framework (2024).KSAO taxonomy

Evidence priors continuously updated by observed validity from the Post-Hire Validity feed — closing the loop.