Decision-Calibrated Multimodal Sequence Models for Rare-Event Forecasting in Heterogeneous Streams

• Gerald Navarro

  Department of Information Technology, Mindoro State University, Victoria Campus, Alcate Road, Victoria 5205, Philippines

  Author

• Marvin Tolentino

  College of Computer Studies, Eastern Samar State University, Arteche Boulevard, Borongan City 6800, Philippines

  Author

• Ramon Villareal

  Department of Computer Engineering, Palawan State University, Rizal Avenue Extension, Puerto Princesa City 5300, Philippines

  Author

Data streams in safety-critical environments increasingly combine irregular multivariate telemetry with sparse, high-level textual context. These heterogeneous signals are temporally misaligned, partially observed, and generated under nonstationary measurement policies, making rare-event forecasting a difficult learning problem despite the apparent abundance of logged data. Modern deep sequence models can rank future adverse events well, yet ranking alone is insufficient when downstream systems must trigger interventions under resource constraints and asymmetric costs. This paper develops a computer-science–focused methodology for decision-calibrated multimodal forecasting that treats reliability as a first-class design target rather than a post-hoc diagnostic. We formalize horizon-based risk prediction under censoring and repeated sampling, define modality-aware representations that degrade gracefully under missing channels, and derive training and calibration procedures that preserve discrimination while stabilizing operating thresholds. We further connect calibration to decision-theoretic optimization, showing how miscalibration interacts with class imbalance, prevalence drift, and selection bias induced by observation masks. The approach integrates gated fusion, uncertainty-aware objectives, and selective prediction to reduce false-trigger rates without sacrificing early-warning lead time. We outline evaluation protocols that separate improvements due to information gain from those due to base-rate stratification and propose drift monitors that are compatible with delayed labels. The resulting framework provides a principled path from multimodal encoders to actionable risk scores suitable for streaming deployment, emphasizing reproducible abstractions, provable properties where possible, and auditability under distribution shift.

• PDF

Copyright (c) 2025 International Journal of Science, Engineering and Applied Technologies

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.