To develop a comprehensive framework to simulate the response to immune checkpoint inhibitors (ICIs) and radiotherapy (RT). We further apply the framework, fitting to clinical data of advanced hepatocellular carcinoma (HCC) patients and investigate potential combination regimen. The mechanistic mathematical model is built upon the dynamic biological interactions between the immune system and tumor, based on blood samples data and outcomes of clinical trials. Our simulations indicate that a significant amount of the benefit stems from the reduction in irradiated tumor load and associated immune suppression, and this important aspect needs to be included in the interpretation of outcomes and design of novel combination trials.