Real-time decision-making is a crucial aspect of digital twins, facilitating the effective operation of physical assets. Specifically, optimal control solutions are employed to guide these assets in achieving their operational objectives. To enable timely decision-making that aligns with the dynamics of physical assets, reduced-order models are essential. However, despite their real-time capabilities, it is imperative to address modeling errors. To tackle this challenge, we introduce Hyper-Differential Sensitivity Analysis (HDSA), a computationally efficient technique designed to assess how optimization solutions are affected by modeling inaccuracies. By integrating HDSA with data from a limited number of high-fidelity forward evaluations, we can effectively update the reduced-order optimal control problem. We demonstrate the efficacy of our algorithmic strategy through several non-trivial examples.