"Out-of-Distribution (OOD) generalization remains both a fundamental challenge and an often-overlooked aspect of modern machine learning--especially in the context of Deep Neural Networks (DNNs), which are highly expressive yet prone to overfitting under distributional stress. Classical learning theory highlights the role of regularization in managing the bias-variance trade-off--particularly important for compact models with lower VC dimension. In this work, we explore stochastic data regularization techniques--such as random augmentations and noise injection--applied not only as isolated strategies but also organized through a Curriculum Learning-based framework. By progressively increasing input difficulty during training, the curriculum aligns model capacity with task complexity, promoting more robust generalization. We also propose a novel statistical procedure to assess the consistency of performance estimates across cross-validation folds, mitigating miscoverage effects in confidence interval estimation. Altogether, our findings highlight the importance of a tailored data regularization, where the selection, combination, and scheduling of perturbations become key to achieving OOD robustness in DNNs"