TeamStation AI / Research / Team Topology / Human-Task-Agent Alignment Stress Test
A 24,000-profile synthetic test of human-task-agent alignment, team topology fit, score sensitivity, and queue pressure.
What a 24,000-profile synthetic stress test reveals about work-reasoning scores, software team topologies, measurement sensitivity, and queue pressure.
A work-reasoning score can appear precise while its answer still depends on the weights, the missing data, the task, the AI agent, the team shape, and the queue around the work.
That is the problem behind our new SSRN working paper, Human-Task-Agent Alignment Across Software Team Topologies. The paper does not claim that a score predicts human performance. It takes the earlier and narrower scientific step: disclose the calculation, build a fixed-seed synthetic experiment, and push the model until its assumptions become visible.
The experiment uses 24,000 synthetic profiles, four software team topologies, autonomy-dependent requirement shifts, input noise, missing domains, weight perturbations, queue saturation, topology-health scenarios, and deliberately injected coefficients. Synthetic people are not presented as real people; the experiment is designed to reveal where a proposed measurement bends before real people ever enter the study.
Short answer
The score passed basic mechanical checks, but the stress tests found meaningful sensitivity.
- Exact profile-to-requirement matches scored 100.
- Domain noise with standard deviations of 0.05 and 0.10 produced mean absolute score errors of 1.67 and 3.50 points.
- Rank correlation fell to 0.962 and 0.863 at those two noise levels.
- Omitting one domain produced mean absolute error from 1.54 to 2.12 points.
- Weight changes switched the highest-scoring team topology for 10.7% of profiles on average and as many as 48.4% in sampled draws.
- Under unit variability, estimated queue wait rose from 1 to 19 service-time units as utilization moved from 0.50 to 0.95.
- Known synthetic coefficients were recovered with small mean bias across 40 replications.
Those results do not validate the score on humans. They tell us which assumptions require stronger governance, human study design, fairness review, and operating telemetry.
What the paper actually tested
The paper starts with a provisional, nonclinical vector called mental shape. It represents six work-reasoning domains:
- Conceptual fidelity: how closely a person's explanation matches the real system and task.
- Architectural instinct: how they reason about boundaries, dependencies, failure paths, and tradeoffs.
- Problem-solving agility: how they update a plan when evidence or constraints change.
- Collaborative mindset: how they expose decisions, receive correction, and move work through a team.
- Learning orientation: how they absorb new evidence and repair an outdated model.
- Metacognitive conviction: how well confidence tracks the strength and limits of the available evidence.
Each synthetic profile is a point on a six-dimensional unit cube. Each work context has a requirement profile on the same cube. A weighted Euclidean distance measures how far the profile sits from the requirement, and that distance is converted into an alignment score where an exact match equals 100.
Think of it like measuring the distance between two pins on a map. The ruler may work perfectly, but the result can still change when we move a pin, change the map scale, remove one coordinate, or decide that one direction matters more than another.
That is why the study tests the ruler and the map before treating the number as a decision.
Four topologies change the requirement
The synthetic requirement profiles correspond to four established software team types:
- Stream-aligned teams own a flow of work tied to a product or customer outcome.
- Platform teams build internal services that reduce cognitive load for other teams.
- Enabling teams help other teams learn a missing capability or adopt a new practice.
- Complicated-subsystem teams own parts of the system that demand deep specialist knowledge.
These are not four labels for the same job. A platform team may need stronger boundary and service thinking. An enabling team may put more weight on teaching, learning, and collaborative change. A complicated subsystem may demand deeper specialist reasoning. A stream-aligned team may need broader product context and end-to-end ownership.
The paper also shifts requirements as agent autonomy changes. That matters because adding AI is not only adding a faster tool. Higher autonomy can move review work, exception handling, system supervision, and decision ownership. The human requirement can change even when the job title stays still.
Weight sensitivity is a governance result
The largest practical warning in the paper is not that the calculation failed. It is that small changes in importance can change the winning topology.
Across sampled weight perturbations, the highest-scoring topology changed for 10.7% of profiles on average. In the most sensitive sampled draw, it changed for 48.4%.
That means the weights are not harmless configuration. They carry an operating judgment about what the work demands. If a buyer, evaluator, or model owner changes those weights without evidence, the same profile can receive a different recommendation while every arithmetic step remains correct.
A production scoring system therefore needs a weight-governance record:
- who proposed the weights
- which role and topology they apply to
- which evidence supports them
- which version was used for the decision
- how sensitive the result is to reasonable alternatives
- who can approve or reject a change
The right question is not only, "Did the formula run?" The stronger question is, "Would a small defensible change in assumptions alter the decision?"
Noise and missing data create decision bands
Real work data is messy. A response can be incomplete. A work sample can be scored with uncertainty. One domain can be missing because the task never exposed it. Translation, disability accommodation, time pressure, and evaluator disagreement can also affect what gets observed.
The synthetic test added controlled noise. At a domain-noise standard deviation of 0.05, the average absolute score error reached 1.67 points and rank correlation fell to 0.962. At 0.10, error reached 3.50 points and rank correlation fell to 0.863.
Removing one domain caused average absolute error between 1.54 and 2.12 points, depending on which domain was omitted.
The operating lesson is that a single displayed score should not pretend to have unlimited precision. A governed system should preserve the observed inputs, missingness, evaluator confidence, sensitivity band, and decision threshold. Close scores may be operationally tied even when the dashboard prints different decimals.
That also supports human review. The reviewer can see whether a recommendation is stable, fragile, or based on incomplete evidence instead of receiving one unexplained number.
Queue pressure can erase local fit
The paper connects the measurement model to queueing because a capable person and a useful AI agent can still be trapped inside a bad delivery system.
Under unit variability, estimated queue wait rose from one service-time unit at 50% utilization to 19 at 95% utilization. The exact number comes from the paper's Kingman queue saturation scenario, but the plain-English meaning is easy to see: the last slice of utilization is expensive.
Imagine a one-lane bridge. At light traffic, a new car moves through. Near full capacity, a small delay creates a line, and each new car has to wait behind the delay already in the system. Software review, QA, security approval, data access, and architecture decisions behave the same way when one shared lane becomes saturated.
Human-task-agent alignment includes the queue because a score can describe local fit while the flow system controls the actual outcome. Faster code generation may increase arrival rate into review, and when review capacity does not move, the operating system can produce more waiting, more context switching, and more unfinished work.
Use engineering telemetry to keep both views together: profile-to-work alignment and the flow constraints around the work.
Why coefficient recovery matters
The experiment deliberately injected known logistic coefficients into synthetic outcomes, then checked whether the analysis could recover them. Across 40 replications, the mean bias was small.
The coefficient check is a mechanical validation step showing that the code can recover a relationship intentionally planted by the simulator; it does not show that the same relationship exists in a workforce, a client, a country, or a real delivery program.
That boundary is important. A simulation can test equations, implementation, sensitivity, and study design. It cannot manufacture external validity.
What the results mean for engineer evaluation
Inside Axiom Cortex engineer vetting, one score should never become the whole person. The stress test points toward a stronger evaluation protocol:
- Define the work first. Name the task, topology, autonomy level, system boundary, and human authority.
- Collect multiple evidence lanes. Use realistic work samples, architecture reasoning, review behavior, collaboration evidence, and uncertainty calibration.
- Expose the calculation. Preserve domain values, weights, missingness, version, and transformation logic.
- Run sensitivity checks. Test nearby weights, noisy observations, missing domains, and alternate topology assumptions.
- Keep a human decision owner. Let an accountable reviewer inspect the evidence, accommodation context, and stability before acting.
- Measure the operating result. Compare the recommendation with delivery telemetry after onboarding without turning private worker data into public content.
Neuro-psychometric vetting must therefore remain nonclinical and job-related because the goal is not diagnosis; the goal is better evidence about work reasoning, team context, and delivery fit, with clear limits and human control.
LATAM is the validation layer, not the claim
The paper proposes a future temporal and project-held-out LATAM outcome study. That is a next gate, not a completed result.
A serious study would need preregistered hypotheses, defined outcomes, time-separated evaluation, project-level holdouts, fairness and accommodation review, and enough data to test whether the score adds value beyond simpler baselines. It should also examine language, country, role, seniority, disability accommodation, and evaluator effects without treating any demographic trait as a proxy for capability.
Aggregate-only reporting matters. Candidate and worker evidence is protected data. Public research should report group-level methods and results while keeping personal records inside the approved system of record.
For a distributed LATAM engineering model, the useful question is not whether a synthetic result proves the workforce. It is whether the disclosed simulation gives us a safer, more falsifiable plan for the human study.
What the study does not prove
The paper is a working paper, not a peer-reviewed human validation study.
It does not prove that mental shape is a validated psychological construct. It does not prove that the score predicts software performance. It does not establish causal effects of AI agents, team topology, or any domain on real people. It does not prove fairness across countries, languages, disability states, roles, or protected groups. It does not authorize automated hiring, ranking, discipline, or termination.
Its contribution is narrower and useful: the outer calculation is now visible, reproducible, and stress-tested enough to show where measurement and operating assumptions matter.
The operating decision
Do not ask a score to carry more truth than the evidence can support.
Use the score as one inspectable evidence object. Test whether its recommendation survives reasonable changes. Keep queue pressure, team topology, task design, agent autonomy, and human authority in the same operating model. Then move to human validation with preregistration, fairness controls, protected-data governance, and held-out outcomes.
The math did not give us a crystal ball. It gave us a map of where the model can break, which is exactly what a responsible next experiment needs.
Related TeamStation research
Sources, authorship, and limitations
Lonnie McRorey directed the research question and simulation and remains responsible for the paper and the applied field guide. The paper discloses AI assistance with source organization, equation implementation, synthetic simulation, draft structure, citation checks, figure production, and mechanical validation under the TeamStation Science Paper OS.
The article explains a synthetic, nonclinical stress test and does not present clinical advice, a validated employment test, a guarantee of engineer performance, or evidence for an automated employment decision.
