Distributionally Robust Selection of the Best

Abstract

Specifying a proper input distribution is often a challenging task in simulation modeling. In practice, there may be multiple plausible distributions that can fit the input data reasonably well, especially when the data volume is not large. In this paper, we consider the problem of selecting the best from a finite set of simulated alternatives, in the presence of such input uncertainty. We model such uncertainty by an ambiguity set consisting of a finite number of plausible input distributions, and aim to select the alternative with the best worst-case mean performance over the ambiguity set. We refer to this problem as robust selection of the best (RSB). To solve the RSB problem, we develop a two-stage selection procedure and a sequential selection procedure; we then prove that both procedures can achieve at least a user-specified probability of correct selection under mild conditions. Extensive numerical experiments are conducted to investigate the computational efficiency of the two procedures. Finally, we apply the RSB approach to study a queueing system's staffing problem using synthetic data and an appointment-scheduling problem using real data from a large hospital in China. We find that the RSB approach can generate decisions significantly better than other widely used approaches.

Extracted equations

• μ_i^* = arg max_i min_f ∈ F E_f[Y_i(f)]
• P(μ_{i*}^1 - μ_1^1 ≤ δ) ≥ 1 - α

Paper claims vs. our run

• Two-stage RSB procedure achieves PCS ≥ 1-α in finite-sample regime
  not-testable
  no fidelity score recorded
• Sequential RSB procedure achieves PCS ≥ 1-α asymptotically
  not-testable
  no fidelity score recorded
• Sequential procedure sample size is insensitive to δ when δ is small
  not-testable
  no fidelity score recorded
• RSB approach generates staffing decisions with significantly lower and more stable cost than best-fit distribution approach
  not-testable
  no fidelity score recorded
• RSB approach outperforms increasing-order-of-variance scheduling rule in hospital appointment scheduling
  not-testable
  no fidelity score recorded

Parameters

Run notes

Stub simulator: Monte Carlo runner not yet wired

Cross-field hypotheses involving this paper

• building energy systems and control → simulation optimization under input uncertainty
  Physics-informed machine learning priors embedded in distributionally robust decision-making

  If we embed physics-based constraints and domain knowledge (from BESTOpt's PIML approach) into the ambiguity set construction and selection procedure of the RSB framework, we can reduce the conservatism of worst-case robust selection while maintaining guarantees under input distribution uncertainty. This would allow the robust selection procedure to exploit structural knowledge about the problem domain to tighten the ambiguity set and improve decision quality.

  Experiment: Implement RSB for a hospital appointment-scheduling problem (as in Paper B) in two variants: (1) baseline RSB with empirical ambiguity set from data alone, and (2) RSB with physics-informed constraints (e.g., service-time distributions bounded by clinical workflow physics, no-show rates structured by appointment type). Compare probability of correct selection, computational cost, and out-of-sample performance on held-out hospital data.

  novelty 0.55testability 0.62business 0.48sim: discrete-event
• healthcare operations / appointment scheduling → simulation optimization under input uncertainty
  Heavy-traffic fluid approximation for asymptotic optimization under distributional uncertainty

  If we apply Paper A's heavy-traffic fluid control problem (FCP) framework to Paper B's distributionally robust selection problem, we can derive asymptotically optimal selection policies that are robust to input distribution ambiguity. Specifically, we could formulate the worst-case selection criterion as a fluid-scale deterministic control problem, then solve it via discretized quadratic programming to obtain robust staffing or scheduling decisions.

  Experiment: Implement Paper B's RSB procedure for an appointment-scheduling problem (already mentioned in their abstract) using both the standard monte-carlo approach and a new variant that first solves a fluid control problem over the ambiguity set, then discretizes to obtain candidate schedules. Compare solution quality, computational time, and robustness across synthetic and real hospital data.

  novelty 0.62testability 0.68business 0.55sim: discrete-event