Industrial robot accuracy degrades under changing configurations, workspace regions, and loading conditions, which limits its use in high-precision manufacturing. To address this issue, this paper proposes a vision-informed unified adaptive incremental learning framework for industrial robot accuracy maintenance via positioning-error and deformation-error prediction. The framework combines metaheuristic hyperparameter optimization, eXtreme Gradient Boosting (XGBoost)-based nonlinear regression, and incremental model updating to improve initial-model quality and adaptability. The method is validated on two representative tasks: positioning-error prediction using laser-tracker-based measurements and deformation-error prediction using an optical C-Track system under external loading. For the positioning-error task, evaluated under a grouped configuration–workspace protocol, incremental updating reduced the mean absolute error (MAE) from 1.080 mm to 1.003 mm and increased R² from 0.883 to 0.903. For the deformation-error task, Random-Walk Grey Wolf Optimizer (RWGWO)-XGBoost achieved the best overall performance among the compared methods under the same evaluation protocol, and incremental updating further reduced the mean squared error (MSE) from 0.75807 mm² to 0.41582 mm². Additional analyses of the incremental adjustment coefficient, update strategy, and computational cost further clarified the update behavior and deployment characteristics of the framework. Overall, the proposed framework provides accurate nonlinear error prediction and sustained adaptive improvement, offering practical evidence for vision-informed industrial robot accuracy prediction and adaptive maintenance under the tested operating conditions.