Table 1 RiGoR: reporting guidelines for risk models

Title/Abstract/Keywords

1

Identify the article as reporting the development of a risk model combining multiple predictors (MeSH “Risk”, possibly “risk factor” and/or “biomarker”)

1

1

1

Introduction

2

Identify the overarching goal – why would an effective risk model be valuable to clinical care, public health, or research?

2

20

2

Methods

Participants

3

Describe the study subjects: The inclusion and exclusion criteria (and resulting sample sizes), setting and locations where the data were collected. Descriptive statistics should include variable ranges.

3

2

4,5,14

4a

Describe participant recruitment.

4

5

4b

Report when study was done, including beginning and ending dates of recruitment.

14

6

4

5

Describe the study design. Was this a cohort study? A case–control study? Note: matched case control studies are generally not suitable for risk model development unless special methods and external data are used.

5

6

4

Biomarker Data

6

Describe data collection, including timing of specimen collection for biomarker measurement. Document where there was blinding to clinical outcomes.

8

4,5

7

Document technical specifications of biomarker materials and methods, including marker units. Describe possibility of batch effects, storage effects, number of freeze/thaw cycles, assay upper and lower limits. Document how biomarker values at the limits of detection were handled.

4,5

7

8

For multi-center studies, document whether biomarker measurements can be considered comparable between study sites, or whether lab effect, platform differences, or variations in clinical practice may affect biomarker levels.

23

Outcome variable

9

Describe how the outcome is defined (e.g., precise definition for disease diagnosis, or death from any cause vs. specific cause)

7

6

Statistical Methods

10

Document measures of model performance, e.g. AUC for risk models; sensitivity and specificity for a pre-selected risk threshold; report methods to quantify uncertainty (e.g., 95% confidence intervals via bootstrapping)

12

12

11

Document how markers were used: transformations (e.g., log)? categorization of continuous variables? Other adjustments (e.g. kidney biomarkers adjusted for urine creatinine)?

9

11

8

12a

List all variables initially considered as candidates

8

9

12b

Describe variable selection: how were variables selected to include in the risk model or classifier? Pre-specified prior to any analysis of the data? Selected based on univariate analysis? An exhaustive search over a set of models? Stepwise procedure?

10

9

12c

Describe how model-selection bias was addressed in assessing the performance of final reported model(s). If model-selection bias was not addressed, state this explicitly.

10

13

Document methodology used to develop risk model or classifier: logistic regression? logic regression? relative risk regression?

10

14a

Document methodology to avoid or correct for resubsitution bias in measures of the performance of the final reported model(s).

10

14b

If an independent validation “test” dataset was used, document that the test data were not used for any part of model development, including variable selection. Document that these data were accessed only when models were finalized. Report the number of models evaluated on the “test” data and how these were selected.

10

14c

If cross-validation is used, state how final reported model is derived.

10

15

For multi-center studies with the possibility of confounding by center, describe methods for adjusting or accounting for center effects.

16

Describe how indeterminate results and missing data were handled, or report that there were no indeterminate results or missing data.

22

11

17

Describe methods for assessing model calibration.

Results

18

Report clinical and demographic characteristics of the study population (e.g. age, sex, presenting symptoms, co-morbidity, current treatments, recruitment centers).

15

13

15

19

Report final risk model or classifier

20

Report estimates of model performance with measures of uncertainty when possible (e.g., 95% confidence interval)

21

18,19

21

Assess and report evidence of risk model calibration.

Discussion

22

Discuss prospects of final risk model for satisfying the research goal

25

22, 23

23

Discuss known and possible limitations to generalizability or applicability of risk model

19

21