---
title: "Getting Started with `integrity`"
author:   Sol Libesman, David Nguyen, Dario Strbenac, Jie Kang, Anna Lene Seidler, Kylie Hunter <br>
        The University of Sydney, Australia.
output:
  html_document:
    toc: true
vignette: >
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteIndexEntry{An Introduction to the integrity Package}
---

<style>
body {
  color: #263238;
}

h1, h2, h3, h4 {
  color: #17324d;
}

h2 {
  border-bottom: 2px solid #d9e2ec;
  padding-bottom: 0.25em;
  margin-top: 1.5em;
}

h3 {
  margin-top: 1.4em;
}

h4 {
  border-left: 4px solid #2f80ed;
  padding: 0.35em 0 0.35em 0.75em;
  background: #f5f9ff;
}

table {
  width: 100%;
  margin: 0.75em 0 1.5em 0;
}

thead {
  background: #edf2f7;
}

th, td {
  padding: 0.45em 0.6em;
  vertical-align: top;
}

blockquote {
  border-left: 4px solid #7b8794;
  color: #52606d;
  background: #f7f9fb;
  padding: 0.5em 1em;
}
</style>


## Package overview

Increasing concerns about the trustworthiness of research have prompted calls to scrutinise studies' individual participant data (IPD), that is, de-identified raw line-by-line data for each participant in a study.

`integrity` was developed to support application of the IPD Integrity Tool [(Hunter et al. 2024A)](https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1738). It enables structured and transparent assessment of the integrity and trustworthiness of randomised controlled trials (RCTs) using IPD and informs decisions about  whether RCTs should be included in evidence synthesis or considered suitable for publication. Further information may be found about the development of the tool [here - (Hunter et al., 2024B)](https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1739). 

If you use our package please cite:

&ensp; &ensp; Hunter KE, Aberoumand M, Libesman S, Sotiropoulos JX, Williams JG, Aagerup J, Wang R, Mol BW, Li W, Barba A, Shrestha N. Webster AC, Seidler AL. The Individual Participant Data Integrity Tool for assessing the integrity of randomised trials. *Research Synthesis Methods*. 2024 Nov;15(6):917-39.


## How to use the package 

Each step of the workflow is illustrated using a case study on umbilical cord management at preterm birth, based on a de-identified and altered data set from the [iCOMP study](https://doi.org/10.1016/S0140-6736(23)02468-6). The main goal was to determine the optimal umbilical cord management strategy at preterm birth, such as milking or delayed cord clamping.

You can download the editable R Markdown (`.Rmd`) vignette to run the case study on your own computer or adapt it to generate a report using your own data.

1. `r xfun::embed_file(knitr::current_input(dir = TRUE), name = "my_integrity_report.Rmd", text = "Download the R Markdown report template")` and save it as `my_integrity_report.Rmd`. You can also find the vignette under **Vignettes** on the [integrity CRAN package page](https://CRAN.R-project.org/package=integrity), right-click its **source** link and choose **Save link as…**. If the source opens as plain text, choose **Save As** and check that the filename ends in `.Rmd`.
2. Open `my_integrity_report.Rmd` in RStudio using **File > Open File**. Follow the one-time installation instructions in **Step 1: Data loading**. To use your own data, replace the example import in that section. Use the full path to your data file, or place it beside the report and use its filename. On Windows, use forward slashes in R file paths, as in `C:/Users/YourName/Desktop/my_data.xlsx`.
3. If you are running the tool with the example data included in the package, skip to point 4. If you are using your own data, follow **Step 2: Data preparation** to update `dataset_info`. Set the column names, variable types, and expected values to match your dataset, and remove any optional entries that do not apply. Update the report title, author, and case-study text as needed.
4. Save the report and click **Knit** in RStudio to generate the HTML report. Review the status tables, caution messages, and plots. The output for each item is intended to be reviewed consecutively and rated using the [tool's decision guide and rating sheet](https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1738).


## Step 1: Data loading

Before knitting this report for the first time, run the following command in the RStudio Console to install the package and the packages used by this example. This installation command is not run automatically when you knit the report. Install any additional packages needed to import your own data separately.

```{r, eval=FALSE}
install.packages(c("integrity", "rmarkdown", "knitr", "readxl"))
```

Load the integrity package into R each time you run the report.

```{r}
library(integrity)
```

Next, import the data set you wish to examine into R. There are a variety of functions in R or CRAN packages to do this:

- `read.csv` and `read.table` functions to import comma-separated and tab-separated text files.
- `read_sas()` for SAS, `read_sav()` for SPSS and `read_dta()` for STATA in the CRAN package [haven](https://haven.tidyverse.org/).
- `read_excel` function for Microsoft Excel in the CRAN package [readxl](https://readxl.tidyverse.org/).


Case study: The altered iCOMP case study is loaded with the `integrity` package. The data are in a Microsoft Excel file.

```{r, warning =F}

library(readxl)
examplePath <- system.file("extdata", "dataset.xlsx", package = "integrity")
dataset <- read_excel(examplePath, sheet=1)
dataset[1:5, ]
```


In the tibble above, the sample identifiers can be seen (`infant_ID`), as well as the date of randomisation (`rand_date`) and the first few clinical covariates. 


## Step 2: Data preparation 


The `integrity` tool requires a list of trial metadata that maps the variables in your dataset to the relevant integrity checks. For each element you supply, specify the corresponding column name(s) in your dataset:

- `participantID`: The name of the column which corresponds to the unique participant identifier (this variable is mandatory).
- `enrolment`: lists the names of three columns corresponding to `start` (date of first participant enrolment), `randomisation` (date of participant randomisation) and `end` (date of the last participant enrolment).
- `baseline`: lists named `dichotomous`, `polytomous`, `numeric` are for specifying the column name(s) of
the column(s) which correspond(s) to baseline measurements.
- `intervention`: the name of the column specifying the intervention or group allocation for each individual (required for comparisons between study groups).
- `outcome`: lists named `common` and `rare`, with sublists named by `dichotomous`, `numeric` or `polytomous`, containing the names of columns of those data types for outcomes assessed.
- `correlated`: A named list of variable pairs expected to be correlated, with two column names per pair.
- `unexpected`: A named list of column names with values that are not expected to be seen. `days` is a special sublist and applies to date columns, which are converted into days of the week before comparison. It must have two elements: `names`, which are the unexpected day names, and `locale`, which is the locale of the unexpected day names specified.

Only `participantID` is strictly required. Supply `enrolment`, `baseline`, `intervention`, `outcome`, `correlated`, and `unexpected` when available. If your dataset does not contain a variable, remove its optional metadata entry or comment it out by placing `#` at the start of the relevant line(s). For example, no polytomous baseline variables exist in the case study and so that entry is commented out in the code below. Checks that require the omitted variable will be skipped.

If date columns are imported as text, convert them to R dates or date-times before running the checks. For example, `as.Date(x, format = "%Y-%m-%d")` converts text dates such as `2026-10-12`; adjust the format to match your data. Randomisation-date plots require a recognised date or date-time column.

### Option 1: defining metadata directly in R

The variable types and expectations need to be defined before running the checks. The example below defines this metadata as a named list written directly in R.

Coding the list directly in R as below, is the simplest option for users already working inside an R script or an R Markdown document, because the metadata can be written directly next to the analysis code. The R code below may be used as a template and altered based on relevant variables in a new dataset.

```{r}
dataset_info <- list(
  participantID = "infant_ID",
  enrolment = list(
    start = "enrol_start",
    randomisation = "rand_date",
    end = "enrol_end"
  ),
  baseline = list(
    dichotomous = c("sex"), # add more variables if needed e.g., c("sex", "respiratory_support")
    #polytomous = c("ordinal_or_nominal_variable"), # no polytomous baseline variables in this data set so it's commented out 
    numeric = c("mat_age", "GA_weeks", "birthweight")
    # can add polytomous variables if needed
  ),
  intervention = "treatment_cat",
  outcome = list(
    common = list(
      dichotomous = c("IVH", "NEC"),
      polytomous = c("CLD"), # if certain variable types don't exist, just delete the relevant line.
      numeric = c("hospital_days")
    ),
    rare = list(
      dichotomous = c("inf_death") # add more variables if needed e.g., c("inf_death", "severe_IVH")
    )
  ),
  correlated = list(
    timeAndSize = c("GA_weeks", "birthweight")
  ),
    # Example with two variable pairs (replace names with your own numeric columns):
  # correlated = list(
  #   timeAndSize = c("GA_weeks", "birthweight"),
  #   heightAndWeight = c("height", "weight")
  # ),
  unexpected = list(
    days = list(
      names = c("Saturday", "Sunday"),
      locale = "C"
    ),
    mat_age = c("less than 10", "greater than 50"),
    GA_weeks = c("less than 22", "greater than 37")
  )
)
```

In the `unexpected$days` section, `locale` controls the language used when R converts dates into weekday names. `locale = "C"` is the most likely option to use because it returns standard English weekday names such as `Saturday` and `Sunday`, which usually match the values entered in `names`. Other locale values are possible if your system or dataset uses a different language or naming convention, but `"C"` will usually be the safest default.


### Option 2: defining the metadata indirectly using the Excel template

Use this option to describe your dataset in Excel, then import those descriptions into R as `dataset_info`.

```{r metadata-template-download, echo=FALSE, results='asis'}
template_path <- system.file("extdata", "variables_template.xlsx", package = "integrity")
if (file.exists("../inst/extdata/variables_template.xlsx")) {
  template_path <- "../inst/extdata/variables_template.xlsx"
}
xfun::embed_file(template_path, text = "Download the Excel metadata template")
```

1. Download the template and save it as `my_variables.xlsx` beside your report's `.Rmd` file. Open the `metadata` sheet in Excel.
2. In columns A:D, replace the example entries with your own. The `level_1`, `level_2` and `level_3` columns identify the category; `value` contains the exact dataset column name (or expected-value rule). Keep the category labels as shown and delete unused example rows.

For several variables in one category, copy a matching row for each variable and change only `value`. For example, three numeric baseline variables need three rows:

| level_1 | level_2 | level_3 | value |
| --- | --- | --- | --- |
| baseline | numeric | | mat_age |
| baseline | numeric | | GA_weeks |
| baseline | numeric | | birthweight |

Leave `level_3` blank for baseline variables. For outcomes, repeat the frequency (`common` or `rare`) and type on each row. Enter one variable per cell, without commas or R code. Use only one variable for `participantID`, `intervention`, and each enrolment date role.

3. Save the workbook. Replace the entire `dataset_info <- list(...)` block in Option 1 with this line (adjust the file path if saved elsewhere):

```{r, eval=FALSE}
dataset_info <- read_metadata_excel("my_variables.xlsx", sheet = "metadata")
```

Then continue to Step 3 using the imported `dataset_info`.

## Step 3: Running integrity checks


Simply provide the data frame from the trial you are checking (in this case `dataset`) and data information list (`dataset_info`) to `run_checks`. The function first performs automated data checking and cleaning to check whether the nominated variables are present and report missing columns. The function will also convert columns nominated as factors into factors where required, and remove any columns containing only missing values.

```{r}
result <- run_checks(dataset, dataset_info)
names(result)
```

This creates a list of result objects, including overall check tables, detailed per-variable tables for selected checks, plots, and summary tables. The output for each item below should be reviewed consecutively and rated using the decision guide and rating sheet [(found here - Hunter et al. 2024A)](https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1738)

## Step 4: Reviewing results by integrity domain

Use the R Markdown template from [How to use the package](#how-to-use-the-package) to run the checks and generate these plots with your own data.

The sections below present the output from the integrity  `run_checks` function, split under each domain and item.

### Domain 1: Unusual or repeated data patterns

**Item 1.1**: Repeating patterns within baseline variables

This item involves sorting and visually inspecting baseline variables for repeating patterns that may indicate rows have been copied and pasted. Look for values that recur at regular intervals. Rare or unusual entries can be particularly informative, for example, an unusual value that appears every 11 rows.

The plots below support visual inspection of continuous and categorical baseline variables. Participants are ordered by randomisation date when available, and points are coloured by study group. If no usable randomisation date is available, participants are plotted in dataset row order. In this case, interpret patterns cautiously, as they may reflect how the dataset was sorted. For continuous variables, values should generally display a random scatter across participants, with no obvious repeating sequences or regular spacing. For categorical variables, categories should also appear randomly distributed. Large, recurring gaps between participants with the same category may indicate a non-random pattern, although such gaps can occur naturally when a category is rare. Points are coloured by study group to help visualise patterns both within and between groups. Consider whether the overall distribution appears genuinely random or whether repeating sequences, regular intervals, unusual clustering, or other systematic patterns suggest potential data irregularities.

```{r}
item_1_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "1.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_1_1, row.names = FALSE)
baseline_pattern_plots <- grep("^Repeated Baselines Within Variables - ", names(result[["images"]]), value = TRUE)
for(plot_name in baseline_pattern_plots) {
  print(result[["images"]][[plot_name]])
}
```

**Item 1.2**: Repeating data patterns across baseline variables

This item looks for duplication across participants, e.g. do all participants with a height of 180cm have the same weight? Duplicate entries for baseline variables are listed below (if there are none, no input will be printed).  

```{r}
item_1_2 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "1.2", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_1_2, row.names = FALSE)
```

**Item 1.3**:  Repeating data patterns across baseline variables and rare variables. 

```{r}
item_1_3 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "1.3", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_1_3, row.names = FALSE)
```

**Item 1.4**: Bias in the terminal (rightmost) digits.

This item plots the terminal digit for the selected continuous variables (avoid variables that tend to be rounded or that lack precision). Each variable has its own y-axis scale. Inspect the bar charts for biased or unexpected distribution.


```{r}
item_1_4 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "1.4", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_1_4, row.names = FALSE)
if("Terminal Digits" %in% names(result[["images"]])) {
  result[["images"]][["Terminal Digits"]]
}
```

### Domain 2: Baseline characteristics

**Item 2.1**:  Excessively homogeneous distribution of binary baseline variables, i.e. loss of independence between consecutive observations

The two-sided Wald-Wolfowitz runs test checks whether binary baseline values have unusually few or unusually many runs (consecutive blocks of the same value). The detailed table reports observed and expected runs, p values, and the test method. When either category has 10 or fewer observations, an exact conditional test is used: the two-sided p value is twice the smaller inclusive tail probability, capped at one. Otherwise, the test uses a normal approximation without continuity correction.

The test automatically orders participants by randomisation date. If no usable randomisation date variable is available, the test uses dataset row order and results may be invalid. A significant result requires contextual review, particularly for rare categories, time trends, and multiple tests; it does not establish an integrity problem by itself.

```{r}
item_2_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "2.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_2_1, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["2.1"]])) knitr::kable(result[["detail_tables"]][["2.1"]], row.names = FALSE)
```

**Item 2.2**: Excessive imbalances between groups in continuous baseline variables.

Evaluates mean and standard deviation for key prognostic factors that are continuous, split by treatment group. 

```{r}
item_2_2 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "2.2", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_2_2, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["2.2"]])) knitr::kable(result[["detail_tables"]][["2.2"]], row.names = FALSE)
```

**Item 2.3**: Excessive imbalances in baseline categorical variables between groups.

This item assesses whether counts of baseline categorical variables are significantly different (p<0.05) between groups. 

```{r}
item_2_3 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "2.3", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_2_3, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["2.3"]])) knitr::kable(result[["detail_tables"]][["2.3"]], row.names = FALSE)
```

**Item 2.4**: Significant difference in variance of continuous baseline variables between groups.

This item uses Levene's test, which checks whether there is a significant difference in variability between groups.

```{r}
item_2_4 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "2.4", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_2_4, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["2.4"]])) knitr::kable(result[["detail_tables"]][["2.4"]], row.names = FALSE)
```

### Domain 3: Correlations

**Item 3.1**: No association between variables known to be highly correlated.

This item plots the correlation between selected continuous variables and calculates the Pearson correlation coefficient (R) and associated p value. Check whether expected correlations are present. 

```{r}
item_3_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "3.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_3_1, row.names = FALSE)
correlation_plots <- intersect(names(dataset_info[["correlated"]]), names(result[["images"]]))
for(plot_name in correlation_plots) {
  print(result[["images"]][[plot_name]])
}
```

### Domain 4: Date violations

**Item 4.1**: Individual enrolment dates do not fit within study start and end dates.

This item examines whether randomisation dates for each individual fall within the enrolment period. 

```{r}
item_4_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "4.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_4_1, row.names = FALSE)

item_4_1_dates <- result[["detail_tables"]][["4.1"]]
if(!is.null(item_4_1_dates)) {
  knitr::kable(item_4_1_dates, row.names = FALSE)
}
```

**Item 4.2**:  Dates (or visits) are not in logical order.

Requires study-specific repeated visits or event-date variables; for example, a follow-up date occurring before enrolment.

```{r}
item_4_2 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "4.2", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_4_2, row.names = FALSE)
```

### Domain 5: Patterns of allocation

**Item 5.1**:  Non-random allocation patterns: plot.

The plot below shows the cumulative number of allocated participants to each treatment arm by date of randomisation. We expect the cumulative number of randomised participants in each group to be similar if 1:1 allocation is used. Assess whether cumulative lines for treatment groups deviate from each other drastically.
Note: the graphs will only appear when the date of randomisation is provided.


```{r}
item_5_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "5.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_5_1, row.names = FALSE)
if("Cumulative Allocation" %in% names(result[["images"]])) {
  result[["images"]][["Cumulative Allocation"]]
}
```

**Item 5.2**:  Non-random allocation patterns: statistical test

This item evaluates randomness of allocation using two approaches: a runs test and a chi-squared test comparing observed adjacent intervention runs with the expected number under random allocation. A statistically significant result (p<0.05) from either test may be indicative of an issue with randomisation.

If no usable randomisation date is available, the tests use dataset row order and the results table flags that this order may differ from randomisation order. If only some dates are missing, both tests exclude participants with missing dates and the results table reports how many were excluded.

```{r}
item_5_2 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "5.2", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_5_2, row.names = FALSE)
item_5_2_test <- result[["detail_tables"]][["5.2"]]
if(!is.null(item_5_2_test)) {
  knitr::kable(item_5_2_test, row.names = FALSE)
}
```

**Item 5.3**: Unexpected imbalance in randomisation day of week.

The table below reports two chi-squared tests: one assessing whether randomisation is distributed evenly across weekdays overall, and one assessing whether randomisation day differs by intervention group. The graph below shows the number of participants randomised on each day of the week by group. We expect numbers to be balanced between groups for each weekday, and fewer enrolments on the weekend for non-urgent interventions. 
Note: the graph will only appear when the date of randomisation is provided.


```{r}
item_5_3 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "5.3", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_5_3, row.names = FALSE)
item_5_3_test <- result[["detail_tables"]][["5.3"]]
if(!is.null(item_5_3_test)) {
  knitr::kable(item_5_3_test, row.names = FALSE)
}
if("Days" %in% names(result[["images"]])) {
  result[["images"]][["Days"]]
}
```

### Domain 6: Internal inconsistencies

**Item 6.1**: Inconsistent or illogical values across variables within individual participants.

Derive logic rules for each variable to be collected, e.g. date of hospital discharge = date of admission + days in hospital; if number of transfusions ≥1, then any transfusion = yes. Incorporate these rules into the package so that any breaches are displayed in the output

```{r}
item_6_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "6.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_6_1, row.names = FALSE)
```

### Domain 7: External inconsistencies

**Item 7.1**: IPD do not correspond to publications or reports.

The table below shows summary statistics for each variable provided in the IPD dataset, e.g. mean, median, range, etc. Manually cross‐check these against any published trial reports, including appendices and supplements. Record any inconsistencies identified, for example, discrepancies in summary variable values between IPD and publication, inclusion of participants in IPD that do not meet eligibility criteria in publication, published variables that are missing from IPD dataset. If data are provided for excluded participants, check whether reasons for exclusion are consistent with publication.

```{r}
item_7_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "7.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_7_1, row.names = FALSE)
if(!is.null(result[["summary_table"]])) {
  result[["summary_table"]]
}
```

### Domain 8: Plausibility of data

**Item 8.1**: Too few missing data or missing data are overly similar between groups.

The table below shows missingness by intervention group for outcome variables, including the percentage missing in each group.

```{r}
item_8_1 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "8.1", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_8_1, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["8.1"]])) knitr::kable(result[["detail_tables"]][["8.1"]], row.names = FALSE)
```

**Item 8.2**: Implausible event rates: outcomes and demographics.

The table below shows events and totals for dichotomous baseline variables and dichotomous common and rare outcomes, split by intervention group.

```{r}
item_8_2 <- result[["check_table"]][result[["check_table"]][["ItemNumber"]] == "8.2", c("Item description", "Status", "Details"), drop = FALSE]
knitr::kable(item_8_2, row.names = FALSE)
if(!is.null(result[["detail_tables"]][["8.2"]])) knitr::kable(result[["detail_tables"]][["8.2"]], row.names = FALSE)
```

## Computing Environment

This vignette was executed on the following computing system:

```{r}
sessionInfo()
```