Catalyst surfaces are dynamic objects that constantly rebuild in response to stimuli like temperature, electrochemical potentials, and adsorbates under reactive conditions. Conventional catalyst design paradigms, relying on static pre-catalyst structures, fail to account for this intrinsic dynamism, leading to imprecise predictions of catalytic activity and stability. This review critically analyzes the shortcomings of empirical and simulation-based design approaches while synthesizing basic surface restructuring phenomena (such as oxidation-state dynamics in PtCu single-atom alloys and adsorbate-induced phase transitions in AgPd nanoalloys). We investigate the possibilities of World Models guided Reinforcement Learning frameworks based on neural networks as a viable method for controlling and predicting surface reconstruction. These methods allow adaptive policy optimization in dynamic catalytic systems by combining experimental data with physics-informed atomistic simulations. The review describes important challenges in uncertainty quantification, reward balancing, and latent state interpretability for future catalyst-specific world models, while highlighting how AI frameworks trained on operando data, when paired with simulations informed by physics, opens new avenues for predictive catalyst design.