Skip to contents

Evaluate Prognostic Model Performance

Source: R/prognosis.R
evaluate_model_pro.Rd

Evaluates the performance of a trained prognostic model using various metrics relevant to survival analysis, including C-index, time-dependent AUROC, and Kaplan-Meier (KM) group analysis (Hazard Ratio and p-value).

Usage

evaluate_model_pro(
  trained_model_obj = NULL,
  X_data = NULL,
  Y_surv_obj,
  sample_ids,
  years_to_evaluate = c(1, 3, 5),
  precomputed_score = NULL,
  meta_normalize_params = NULL
)

Arguments

trained_model_obj

A trained model object (of class "train" as returned by model training functions like lasso_pro, rsf_pro, etc.). Can be NULL if precomputed_score is provided.

X_data

A data frame of features corresponding to the data used for evaluation. Required if trained_model_obj is provided and precomputed_score is NULL.

Y_surv_obj

A survival::Surv object for the evaluation data.

sample_ids

A vector of sample IDs for the evaluation data.

years_to_evaluate

A numeric vector of specific years at which to calculate time-dependent AUROC.

precomputed_score

Optional. A numeric vector of precomputed prognostic scores for the samples. If provided, trained_model_obj and X_data are not strictly necessary for score derivation.

meta_normalize_params

Optional. A list of normalization parameters (min/max values) used for base model scores in a stacking ensemble. Used when trained_model_obj is a stacking model to ensure consistent normalization during prediction.

Value

A list containing:

  • sample_score: A data frame with ID, outcome, time, and score columns.

  • evaluation_metrics: A list of performance metrics:

    • C_index: Harrell's C-index.

    • AUROC_Years: A named list of time-dependent AUROC values for specified years.

    • AUROC_Average: The average of time-dependent AUROC values.

    • KM_HR: Hazard Ratio for High vs Low risk groups (split by median score).

    • KM_P_value: P-value for the KM group comparison.

    • KM_Cutoff: The score cutoff used to define High/Low risk groups (median score).

Examples

# \donttest{
# Generate dummy data
set.seed(42)
n <- 50
X <- as.data.frame(matrix(rnorm(n * 5), n, 5))
Y_surv <- survival::Surv(time = runif(n, 1, 5*365), event = sample(0:1, n, TRUE))
ids <- paste0("s", 1:n)

# Train a simple model
initialize_modeling_system_pro()
#> Prognosis modeling system already initialized.
model_obj <- lasso_pro(X, Y_surv)

# Evaluate the model on the same data
eval_results <- evaluate_model_pro(
  trained_model_obj = model_obj,
  X_data = X,
  Y_surv_obj = Y_surv,
  sample_ids = ids,
  years_to_evaluate = c(1, 2, 3)
)
#> Warning: Cannot perform KM analysis due to constant, all NA, or non-varying scores.
str(eval_results$evaluation_metrics)
#> List of 6
#>  $ C_index      : logi NA
#>  $ AUROC_Years  :List of 3
#>   ..$ 1: num 0.5
#>   ..$ 2: num 0.5
#>   ..$ 3: num 0.5
#>  $ AUROC_Average: num 0.5
#>  $ KM_HR        : logi NA
#>  $ KM_P_value   : logi NA
#>  $ KM_Cutoff    : logi NA
# }