Incarceration diversion programs aim to rehabilitate justice-impacted individuals and reduce recidivism, but limited capacity requires careful admission decisions that often rely on predictions of the risk of re-offending during the program. This talk, based on the paper [1], explores how prediction errors affect decision quality in high-stakes settings where online exploration is infeasible. To address this, we develop a framework that integrates queueing models with uncertainty quantification to evaluate the correctness of decisions and assess potential interventions.

Theoretically, we demonstrate that a priority score policy solves both the ground truth and estimated admission control problems, maintaining optimality under a likelihood ratio ordering condition even in the presence of prediction errors. By decomposing decision uncertainty into components related to priority scores and decision boundaries, we further identify when decisions can or cannot be considered reliable.

Practically, we apply this framework in a simulation-based case study using data from Adult Redeploy Illinois to evaluate two interventions: collecting more data and incorporating human-in-the-loop decision-making. While both strategies enhance average decision quality, challenges such as non-monotonic cost reductions highlight the importance of designing interventions that thoughtfully balance algorithmic automation with targeted human oversight in high-stakes decision-making.