Using fingerprinting as a testbed for strategies to improve reproducibility of functional connectivity

Introduction

Despite the promise of increasingly representative and deeply phenotyped consortia, reproducible functional connectivity-based biomarkers have remained elusive. A biomarker is “a characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention”.^1–4 The identification of clinically useful brain-based biomarkers requires that our measures of the brain (e.g., functional connectivity, which captures the statistical dependence of BOLD signals between brain regions) and of behavior (e.g., clinically important symptoms or behaviors) are reproducible. One plausible way of achieving brain-based biomarkers lies in employing precision fMRI strategies by “deep” sampling functional data across repeated measurements (e.g., Midnight Scan Club [MSC]).⁵ This is because a key component of reproducibility is reliability³ — the stability of a measure over time and under similar plausible scenarios.^6,7 The reliability of a measure establishes an upper limit on its validity, where validity is measured in terms of its correlation with a target measure (e.g., behavior).⁸ Assessing the reliability of functional connectivity metrics is therefore crucial to advancing reproducible and clinically meaningful connectivity-based biomarkers.

One challenge to assessing functional connectivity metrics is the lack of a “ground truth” against which to evaluate their validity and which could therefore serve as a basis for comparing methods and pipelines for computing functional connectivity.⁹ For example, while functional connectivity shows good correspondence with measures of anatomical or structural connectivity (e.g., as revealed by retrograde tracer studies in non-human animals, or diffusion-based connectivity studies in humans),^10,11 it is clear that functional connectivity metrics reflect more than is captured by these anatomical measures, so correspondence with such measures alone cannot be used to evaluate functional connectivity metrics or methods for their computation. In the absence of a ground truth for evaluation of validity, many studies have focused on test-retest (TRT) reliability as an outcome measure; if a particular functional connectivity metric or analysis step or pipeline is advantageous, it should exhibit improved TRT reliability. The most consistently used index to evaluate TRT reliability of functional connectivity-based measures is the intraclass correlation coefficient (ICC).^8,12,13 While the ICC has been informative, it has a number of limitations. First, it has high dimensionality, since it is typically computed either as a voxelwise or an edge-level measure (e.g., Shehzad and colleagues¹²). This means that ICC for a given ROI or network is often computed by averaging across a large number of functional connections, or even the whole brain, leading to a loss of precision and potentially deflating the metric. In addition, ICC is computed as the ratio of between-subject variance to within-subject variance,¹² with between-subject variance accounting for majority of the variability in the ICC.¹² Low between-subject variance for some functional connectivity measures or networks may also depress ICC.^14–16 These observations have previously been put forward to explain why the ICC for functional connectivity-based measures is poor-to-fair, at best (see Noble and colleagues for a review⁸).

Here, we test a potential complementary approach to the ICC for comparing analyses and pipelines — connectome fingerprint identification. This process, first described by Finn and colleagues,¹⁷ involves treating the functional connectivity profile (connectome) of an individual as a “fingerprint”. Fingerprint identification is performed by matching (using correlation) an individual’s connectome fingerprint derived from a given fMRI scan (taken from a “database” set of scans) with their connectome fingerprint derived from another, separate scan (taken from a “target” set of scans, typically acquired during a different scan session). If the correlation between a given individual’s two functional connectivity matrices is greater than the correlation between the individual’s database connectome and the connectome of other individuals in the target dataset, then it is considered a “match” — a correct identification (ID). If the self-self correlation is not greater than the self-other correlations, the individual has not been successfully identified by the matching algorithm. Studies applying the connectome fingerprinting approach across populations and scan conditions show that mean ID accuracy across a given sample can be >95%,^17–19 providing a different perspective on the reproducibility of functional connectivity metrics, relative to the ICC. Connectome fingerprint ID could be a complementary metric to the ICC for assessing analysis pipelines and novel methodologies due to its applicability, lower computational demands, and straightforward interpretability. We extend this logic to hypothesize that if a particular analysis step or pipeline is advantageous in ways that relate to reliability and/or capturing intersubject differences in functional connectivity, then it should improve fingerprint ID. However, it is important to note that a particular analysis step or pipeline can produce higher fingerprint ID accuracy but not necessarily enhance validity. While a reliability-based evaluation metric is helpful, it could be an incomplete means to evaluate an analysis step or pipeline.⁸ Therefore, some measures of validation would be preferred where available.

Several papers have already shown that ID accuracy is influenced by a number of processing choices. For example, Finn and colleagues¹⁷ reported lower ID accuracy when a cortical parcellation scheme was used to derive the functional connectivity profiles, relative to a whole-brain atlas. This may reflect the impact of differential parcel resolutions or spatial coverage (cortical vs. subcortical regions). Indeed, fingerprint ID accuracy exhibits variation across functional networks, with the highest ID accuracies being found in the medial frontal and frontoparietal networks.^17,19–24 Somewhat predictably, restricting analyses to a small subset of the most distinct connectome edges (e.g., < 1%) also improves ID accuracy, producing near-ceiling accuracy.²⁴ Other factors known to affect ID accuracy include scan duration,^17,19,20 parcellation resolution,¹⁹ and participant age,^23,25 with the youngest participants showing lowest ID, reflecting poor connectome distinctiveness, particularly between sessions.²⁶ Recent work also indicates that denoising strategies (e.g., global signal regression (GSR), temporal censoring, ICA-AROMA) improve ID accuracy in early childhood.²⁷ Importantly, and contrary to the benefits commonly inherent to increasing sample size, increasing sample size has been shown to decrease ID accuracy, making it more difficult to identify an individual from a group.^28,29

The present study has twin aims. First, we aim to show how fingerprint accuracy can serve as a robust, easily computed, intuitive, and sensitive index for comparing or benchmarking pipeline parameters. To achieve this aim, we expand on recent efforts to maximize functional connectome identification accuracy by examining the contributions of data pre- and post-processing factors, as well as acquisition parameters, using five independent datasets spanning childhood to adulthood, obtained from the Consortium for Replicability and Reproducibility.³⁰ By identifying several factors that maximize ID accuracy across the independent datasets, we delineate components of a pipeline that is optimized for the reliable measurement of functional connectivity at the individual level. The use of fingerprint accuracy as part of a workflow that seeks to maximize the identifiability of individual connectomes may also help improve the identification of robust and reproducible brain-behavior relationships.

Methods

Datasets

Five datasets from the Consortium for Replicability and Reproducibility (CoRR; http://fcon_1000.projects.nitrc.org/indi/CoRR/html/samples.html) were used: Beijing Normal University (BNU), New York University (NYUado, NYUadu), Southwest University (SWU), and University of Pittsburgh School of Medicine (UPSM). We chose to examine a diversity of datasets from CoRR in terms of developmental samples, recruitment location, sample size, inter-scan interval, and scan duration. By selecting these five datasets, we ensured our samples captured participant inclusivity based on the recruitment protocols described by CoRR. This is particularly important because many fingerprinting studies tend to use the Human Connectome Project (age range = 22-35 years) due to their high-resolution structural and functional MRI scans^{17,21,28,29,31–34} — which can overrepresent the range of fingerprint ID accuracies reported in the field. In general, fingerprint identification appears to be relatively stable across development and even spanning several years.^22,23 Ethical approval was obtained from the relevant committee at each imaging site. Written informed consent was obtained from participants ages 18 or older. For participants younger than 18 years, written consent was obtained from their legal guardians, and written assent was obtained from the participant. Each dataset included resting-state fMRI scans obtained from two imaging sessions either on the same day or several months or years apart. Demographics and fMRI acquisition parameters are provided in Table 1.

Preprocessing

All data were preprocessed with the same basic pipeline as described by C-PAC (https://fcp-indi.github.io/docs/latest/user/quick.html).^35–37 Structural and functional data were preprocessed using AFNI (http://afni.nimh.nih.gov/afni/), FSL (http://fsl.fmrib.ox.ac.uk/), and ANTs (http://stnava.github.io/ANTs). The minimal preprocessing pipeline follows common steps that are typical for structural and functional magnetic resonance imaging (MRI) data.^35–37 Our preprocessing pipeline, which follows the 36-parameter model, has been evaluated using different functional connectivity metrics and datasets.^38,39 The preprocessing pipeline included: (1) volume-based motion correction; (2) grand mean scaling; (3) linear and quadratic detrending; (4) nuisance signal regression on 24 motion parameters (i.e., 3 translational, 3 rotational, their first derivatives, and the squares of each of these terms) and 12 nuisance regressors (i.e., white matter, CSF, global signal, [though global signal was also a factor of interest so for some analyses, GSR was excluded from the nuisance regressors], their squares, and the squares of their first derivatives); (5) spatial smoothing using a Gaussian kernel at 6mm FWHM; and (6) temporal filtering (0.009 \(\leq f \leq\) 0.1 Hz). Functional to session-averaged anatomical co-registration was performed using boundary-based registration in FSL.⁴⁰ Diffeometric normalization of individual anatomical to the MNI152 standard space with 2 mm resolution was carried out non-linearly using ANTs; the same non-linear transformation was applied to the fMRI data. Of note, we did not vary the smoothing kernel size as prior evidence shows that it generates no-to-minimal difference in fingerprint ID accuracies.¹⁷ Participants were excluded if either scan exhibited root-mean-square framewise displacement (rmsFD) > 0.20 mm.⁴¹ Applying a rmsFD < 0.20 mm retained BNU N: 60/60, NYUado N: 24/24, NYUadu N: 31/31, SWU N: 82/82, and UPSM N: 67/92. This threshold provided a reasonable balance between sample size requirement to achieve adequate statistical power for functional connectivity-based studies and good data quality standards to mitigate motion artifacts in these studies. For each dataset, the rmsFD averaged across the two functional scans is as follows: BNU = 0.06 mm [range = 0.02-0.09 mm], NYUado = 0.07 mm [range = 0.03-0.14 mm], NYUadu = 0.07 mm [range = 0.03-0.19 mm], SWU = 0.06 mm [range = 0.02-0.13 mm], and UPSM = 0.09 mm [range = 0.04-0.19 mm]. Traditionally, a threshold < 0.10 mm is regarded as strict whereas a threshold > 0.50 mm is considered liberal.^42–47 There is no one-threshold-fits-all populations as head motion varies by several factors including age (e.g., infants, adolescents, adults) and mental health diagnosis.^48–51

Data pre- and post-processing parameters

To demonstrate how fingerprint ID accuracy can be used to test pipeline parameters, and to compare the utility of ID accuracy vs. ICC for this purpose, we examined how several data acquisition, pre-, and post-processing parameters affected ID accuracy and ICC (Figure 1), specifically: (1) GSR; (2) parcellation schemes (i.e., scheme, resolution, spatial coverage); (3) network organization; (4) use of discriminative edges; (5) scan duration; and (6) sample size.

Global signal regression

Although debated, GSR is a highly effective denoising strategy.^38,52,53 It remains the most effective strategy for reducing the associations between motion and functional connectivity-based measures^35,36,54,55 in addition to increasing the biological plausibility of patterns of functional connectivity.^56,57 We examined the impact of including vs. excluding the global signal from the 36-parameter nuisance signal regression step during preprocessing.

Parcellation schemes (selection, spatial coverage & resolution)

Parcellation schemes differ in aspects such as spatial coverage (e.g., cortex, subcortex), space (e.g., volume, surface), and resolution (e.g., 10-1000). We parcellated the fMRI data using four publicly available brain atlases: Shen 268⁵⁸, Glasser 360⁵⁹, MIST 444⁶⁰, and Schaefer 1000-17Networks,⁵⁸ to examine whether these different commonly applied schemes are associated with differences in ID accuracy. The Shen parcellation was derived from activity patterns using resting-state fMRI data.⁵⁹ The Glasser parcellation was derived from areal properties of architecture, function, connectivity, and topography using high-resolution structural MRI as well as resting-state and task-based fMRI data.⁶⁰ The MIST parcellation was derived from patterns of functional connectivity using resting-state fMRI data.⁶¹ The Schaefer parcellation was derived from resting-state and task-based fMRI data with properties related to function, architectonics, connectivity, and visuotopy.⁵⁸ We also compared ID accuracies with cortical-only parcellations to parcellations that included subcortical and cerebellar regions. Finally, we assessed the relationship between resolution and ID accuracy by examining two parcellation schemes with 8 (Schaefer 17Networks, 100-1000) and 10 (MIST, 7-1095) levels of dimensionality.

Network organization

Several studies have demonstrated that some functional networks have stronger predictive power in facilitating the identification of an individual.^17,22,23 To assess differences between functional networks in terms of ID accuracy, we examined 8 functional Finn et al.¹⁷ Networks: (1) medial frontal; (2) frontoparietal; (3) default mode; (4) subcortical and cerebellum; (5) motor; (6) visual I; (7) visual II; and (8) visual association¹⁷ and 7 Yeo et al.⁶² Networks: (1) ventral attention; (2) dorsal attention; (3) frontoparietal; (4) default mode; (5) somatomotor; (6) limbic; and (7) visual.⁶²

Discriminative edges

Fingerprint identification is influenced by the degree to which different edges contribute to the identification of an individual.^17,19,23,24 We employed the discriminative power (DP) measure to quantify the usefulness of an edge in successfully identifying an individual from a group. The DP of a given edge reflects the empirical probability of an individual being more correlated to themself relative to another individual based on the same edge. It is computed as the product of the edge values between sessions from an individual and compared with the product of the edge values between sessions from other individuals in a dataset.^17,23 An edge with a high DP means that the within-subject product is greater than the between-subject product across all the participants and that it is useful for the identification of an individual. To compute ID accuracy with the most discriminative edges across all individuals in the five datasets, we selected the edges whose DP values fell between the 25^th and 99^th percentiles of the functional connectome, i.e., we retained the top 1-75% of edges in the whole-brain connectome to compute ID accuracy for each dataset.

Scan duration

Converging evidence has revealed that the reliability of functional connectivity-based measures depends on scan duration^63–67 and that longer scan duration improves ID accuracy.^17,19,20,68 We assessed the relationship between scan duration and ID accuracy by increasing the scan lengths from 0.5 min to the full acquisition time of each dataset (scan parameters for each site in Table 1). Of note, the range of the scan lengths is limited based on the relatively short scan durations of the five datasets (e.g., the maximum scan length is approximately 8 min derived from the BNU and SWU datasets).

Sample size

While sample size typically improves the reliability of functional connectivity-based measures,^69,70 identification accuracy tends to decrease with increasing sample sizes.^28,29 To assess the impact of sample size on ID accuracy, and to avoid site/sample effects, we pooled the five datasets (N = 264), then randomly selected subsets of individuals such that N \(\in\) {20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 264} without replacement and computed ID accuracy over 100 of each N iterations.

Functional connectivity matrices — fingerprints

For every individual and each scan, the mean timeseries of each region of interest (ROI) within a given parcellation scheme was extracted and the Pearson’s correlation coefficient (\(r\)) between all possible ROI pairs was computed, yielding two functional connectivity matrices (fingerprints) per individual that were subsequently used in the ID procedure.

Identification procedure

To identify an individual based on their functional connectivity fingerprints, we followed the identification procedure as previously described.¹⁷ Identification was performed for each dataset separately by creating a “database” D = {\(X_{i}\), i = 1, 2, 3….N} where \(X_{i}\) refers to the vectorised functional connectivity matrix for Session 1 and the subscript \(i\) represents the \(\ i\){}^th participant in D. Iteratively, \(X_{i}\) is compared with functional connectivity matrix \(X_{j}\) { j = 1, 2, 3….N} from the Session 2 target set, T, using correlation. Correlations were rank ordered, and an individual was assigned a binary identification (BID) score of 1 if the predicted identity \(X_{j}\) (rank = 1) in T matched the true identity \(X_{i}\) of the individual in D, otherwise the BID score was 0. Mean BID accuracy for each dataset and each combination of analysis factors was computed as the number of correctly identified participants divided by N (sample size). To reach a final ID accuracy per dataset, we recomputed mean BID by exchanging the roles of the database-target matrix (D \(\leftrightarrow\) T), and averaging across the two mean BIDs.

In addition to BID, we computed the relative rank (RR), which is a continuous measure ranging from 0 to 1 that quantifies the degree of “confusion” for inaccurately identified individuals. RR was calculated as the j^th position of the rank ordered correlation that matches the predicted identity \(X_{j}\) in T with \(X_{i}\) in D divided by N. For example, if the ranked ordered correlation of participant 1 in the SWU dataset is at the first position, then RR = 1\(/\)82 = 0.012 whereas if the ranked ordered correlation for the same participant is at the last position, then RR = 82\(/\)82 = 1. The fewer individuals inaccurately ranked above their true identity, the lower the degree of confusion, and lower the RR (RR → 0).

TRT reliability

We computed TRT reliability, as indexed by ICC, for each edge and each dataset. We applied the ICC (2,1) model to refer to “absolute agreement” for which the sources of error are known (e.g., multiple sessions) and modeled as random effects.⁸ For each dataset, the mean ICC was obtained by averaging the edgewise ICC values in the functional connectome in addition to setting the negative values to 0 to indicate complete unreliability.^8,63,71

\[ \operatorname{ICC}(2,1)=\frac{\mathrm{MSBS}-\mathrm{MSE}}{\mathrm{MSBS}+(\mathrm{K}-1) \mathrm{MSE}+\frac{\mathrm{K}}{\mathrm{~N}}(\mathrm{MSBM}-\mathrm{MSE})} \]

where MSBS = mean square between subjects, MSBM = mean square between measurements, and MSE = mean square error.^72,73

Comparing analysis pipelines using fingerprint ID vs. ICC

For each dataset, the identification accuracy was computed with each of the data acquisition, pre-processing, and post-processing parameters, and then compared with a “standard” functional connectivity pipeline as described in the preprocessing stages. The standard pipeline comprised a 36-parameter confound regression model including GSR and its derivatives, whole-brain Shen 268 parcellation, full scan length, and full sample size of each dataset. On the basis of the highest contributions of the parameters on ID accuracy (i.e., BID → 100% and RR → 0%), we identified two optimal fingerprint-informed pipelines which are as follows:

• Pipeline 1: minimal pre-processing, GSR, Schaefer 1000-17Network parcellation, full scan length, full sample size;

• Pipeline 2: minimal pre-processing, GSR, frontoparietal networks (medial frontal + frontoparietal) derived from “Finn” Networks, full scan length, full sample size.

It is important to note that selecting an optimal fingerprint-informed pipeline is context/study-dependent and non-trivial and can increase the degrees of freedom as the analytic decisions depend on a broad array of parameters. We optimized the pipeline with regards to specific parameters manipulated in the present study though further optimization through consideration of other potential factors such as motion correction strategies or the influence of signal-to-noise ratio (SNR) is always possible. Here, our main goal was to demonstrate that functional connectome fingerprinting provides a simple, straightforward, and intuitive way to assess such parameters/factors. We selected common data acquisition and pre- and post-processing parameters that have shown a direct impact on fingerprint ID accuracies based on prior fingerprinting studies. For each dataset, the mean ICC was further computed with Pipeline 1 and Pipeline 2. To demonstrate the utility of the two fingerprint-informed pipelines, they were employed to derive the functional connectivity matrices of the participants for their two fMRI scans for each dataset. Given that we assessed how fingerprint ID accuracy can be applied to test analysis pipeline parameters, we did not perform permutation testing to assess the statistical significance of the ID accuracies themselves across the five datasets.

Data sharing and code availability

The analyses in this study were conducted in Python and the codes are available via our GitHub repository (https://github.com/JRam02/conn_fingerprint_neurodev). We used publicly available packages such as pingouin (https://pingouin-stats.org/api.html), scikit-learn (https://scikit-learn.org/stable/), and SciPy (https://www.scipy.org). We also used Python libraries for data manipulation and visualization such as Numpy (https://numpy.org), Pandas (https://pandas.pydata.org), Matplotlib (https://matplotlib.org), and Seaborn (https://seaborn.pydata.org). The ICC analysis was conducted using the intraclass_corr package in pingouin. The fMRI data of the five datasets that were used in this study are available on the CoRR platform. We also provide the mean timeseries of the participants for each dataset via our GitHub repository.

Figure 1.Analysis framework to delineate an optimal pipeline for maximizing connectome-based identification. A. Applying global signal regression (GSR) as a denoising strategy. B. Delineating the functional connectome with four publicly available parcellation schemes such as Shen 268, Glasser 360, MIST 444 and Schaefer 1000-17Networks. C. Delineating cortical and subcortical regions (Harvard-Oxford atlas and Buckner atlases). D. Computing the functional connectome using multi-resolution parcellation schemes with 8-10 levels of dimensionality. E. Examining ID within Finn et al.¹⁷ Networks and Yeo et al.⁶² Networks. F. Using the top 1-75% of most discriminative edges. G. Examining scan duration — from 0.5 min to the full acquisition time of each dataset. H. Examining the impact of sample size (N) by pooling data across the five datasets and resampling such that N \(\in\) {20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 264}.

Results

Identifiability of the individual functional connectome

Using whole-brain functional connectomes collected from two scans per participant derived from CoRR, we initially applied a standard functional connectivity pipeline which involves minimal data pre-processing (including GSR), post-processing (i.e., whole-brain Shen 268 parcellation), and acquisition (i.e., full scan duration and sample size) parameters to compute the identification accuracy across the five datasets. We first computed binary identification (BID) such that the BID accuracy is 1 if an individual is successfully identified and 0 if they are not, using all five datasets. BID accuracies ranged from 42% to 77% which were averaged across the two database-target matrix configurations (Figure 2A and Figure 2B). On average for both directions (D → T and T → D), the NYU developmental dataset (NYUado; N = 24) achieved the strongest identification accuracy (BID = 77%) whereas the UPSM adolescent group (N = 67) had the lowest (BID = 42%). We then employed relative rank (RR) to measure the degree of confusion for misidentifying an individual from a group such that the lower the RR, the lower the confusion for incorrectly identifying that individual. RR showed that the degree of confusion varied from 6% to 13% for the first direction (D → T) (Figure 2A), and 4% to 16% for the second direction (T → D) (Figure 2B). On average, the adult dataset (SWU) achieved the lowest degree of confusion for incorrectly identifying an individual (RR = 5%) between sessions whereas the UPSM adolescent dataset exhibited the highest degree of confusion (RR = 15%).

TRT reliability of the functional connectome is consistently poor

We then evaluated the TRT reliability of the functional connectome by applying the ICC(2,1) model using the standard functional connectivity pipeline across the five datasets. We computed the average ICC across 268 x 268 edges for the Shen ROIs. We found that the mean TRT reliability was consistently “poor” (i.e., ICC < 0.4) across the independent datasets (Figure 2C). The highest mean TRT reliability was observed for the adult dataset (SWU; ICC = 0.29) whereas the lowest mean TRT reliability was found for the adolescent group (UPSM; ICC = 0.15). Although the ICC and BID accuracy can only be compared qualitatively, the two metrics track each other well (\(r\) = 0.82; Figure 2C).

Applying GSR as a denoising strategy improves identification accuracy

While investigating the impact of data pre-processing factors on identification accuracy across the five datasets, we assessed the contribution of GSR by computing identification accuracy following pre-processing with and without GSR (Figure 2D). For the purpose of this comparison, all other noise regressors were held constant (i.e., the global signal was the only regressor removed from the 36-parameter nuisance signal regression). The range of the BID accuracies was higher with GSR (42-77%) than without GSR (35-69%). GSR significantly improved the BID accuracies across all five datasets (paired sample t-test; t (4) = 6.14, P = 0.004, Cohen’s d = 0.52).

Figure 2.(A) ID accuracy derived from BID in addition to the degree of confusion as indexed by RR for the database-target matrix configuration. (B) ID accuracy derived from BID in addition to the degree of confusion as indexed by RR for target matrix-database configuration. (C) Reliability estimates derived from the fingerprint approach using BID and the ICC(2,1) model. The Pearson’s correlation coefficient (\(r\)) between ID accuracy and ICC is 0.82. (D). ID accuracy derived from BID with and without applying GSR as a denoising strategy. The five independent datasets are as follows: BNU (N = 60), NYUado (N = 24), NYUadu (N = 31), SWU (N = 82), and UPSM (N = 67).

Finer cortical parcellations yield stronger identification accuracy

To understand how different parcellation features influence the identification accuracy, we computed functional connectivity using four parcellation schemes with different spatial resolutions and spatial coverage. We found that the BID accuracies differed significantly across parcellation schemes (repeated measure one-way ANOVA test; F (3,12) = 31.1, P < 0.001). There was a tendency to achieve greater BID accuracies across the five datasets when the functional connectome was derived from finer parcellation schemes, i.e., parcellations with higher resolutions. While the lowest BID accuracies were obtained with the Shen 268 parcellation (42-77%), the highest BID accuracies were found with the Schaefer 1000-17Network parcellation (68-94%) (Figure 3A). Both Glasser 360 and MIST 444 parcellation schemes produced intermediate BID accuracies ranging from 55% to 86% across the five datasets (Figure 3A).

To assess the effect of resolution more cleanly and within the same parcellation scheme, we applied the multi-resolution Schaefer 17Networks (i.e., 100-1000) and MIST (i.e., 7-1095) parcellations to examine the relationship between parcel resolution and identification accuracy. There was a non-linear relationship between parcel resolution and BID accuracy that was evident across the five datasets (Figure 3B). BID showed that there were large gains in ID accuracy as parcel resolution increased from 7 to 300, with smaller but still significant gains thereafter (Figure 3B). Similarly, there was a non-linear relationship between parcel resolution and RR across the five datasets such that there was a rapid decline in RR from 7 to 300 parcels, and then plateaued until reaching the finest resolutions (Figure 3B).

We also investigated how using regions from the cortex, subcortex, and cerebellum influences identification accuracy. Fingerprint BID accuracies differed significantly across different combinations of the spatial coverage (repeated measure one-way ANOVA test; F (5,20) = 54.4, P < 0.001). The highest BID accuracies were derived when the functional connectome included cortical regions (41-77%), with the addition of subcortical and cerebellar regions making little-to-no difference to BID accuracy (Figure 3C). Fingerprints computed on the basis of subcortical (17-49%) and/or cerebellar (6-36%) structures alone were associated with much lower BID accuracies across the five datasets (Figure 3C). The higher fingerprint BID accuracies associated with the cortical regions could be attributed to their macroscale divisions having larger number of edges, as we found significant associations between number of edges and BID accuracies for 4 out of 5 datasets (except for the BNU dataset).

Identification accuracy across functional brain networks

To examine variation in fingerprint identifiability across functional networks, we compared the BID accuracies across eight Finn et al.¹⁷ networks and seven Yeo et al.⁶² networks. For the Finn et al.¹⁷ networks, BID accuracies were highest when the fingerprints were restricted to the medial frontal (except for the UPSM dataset) and frontoparietal networks, relative to the other functional networks (Figure 4A). In contrast, fingerprinting using the visual networks (i.e., visual I, visual II, visual association) produced the lowest BID accuracies (Figure 4A). Given previous research showing that combining medial frontal and frontoparietal networks improved BID accuracies,^17,23 we examined the impact of combining these functional networks into the “frontoparietal networks”. Restricting the fingerprint to these frontoparietal networks was associated with significantly greater identification accuracies compared to the whole-brain functional connectome (paired sample t-test; t (4) = 2.83, P = 0.047, Cohen’s d = 0.77). When we applied the Yeo et al.⁶² networks, BID accuracies revealed that the fingerprints restricted to the frontoparietal (except for the NYUadu dataset; 57-87%) and default mode networks outperformed the other functional networks across the five datasets (Figure 4B). Across the five datasets, fingerprinting with the limbic network exhibited the lowest BID accuracies (21%-67%) followed by fingerprinting with the visual network (25%-69%) (Figure 4B). When testing if the advantages in fingerprint BID accuracies for the frontoparietal networks might be due to a network size (i.e., networks having larger number of edges), we found that for the Finn et al.¹⁷ networks, only the SWU dataset produced a significant relationship between network size and BID accuracies (\(r\)_s = 0.72, P = 0.028). For the Yeo et al.⁶² networks, only the NYUado (\(r\) = 0.79, P = 0.035) and UPSM (\(r\) = 0.77, P = 0.041) datasets yielded a significant relationship between network size and BID accuracies.

Greater number of discriminative edges yields stronger identification accuracy

To examine whether restricting the fingerprint to discriminative edges improves identification accuracy, we restricted the fingerprint to those edges whose DP values fell between the 25^th and 99^th percentiles of the functional connectome for each dataset (i.e., selecting the top 1-75% of all edges, ordered by discriminality). We observed a non-linear relationship between the number of discriminative edges and BID accuracies across the five datasets (Figure 5). When the fingerprint was restricted to the top 1% of DP edges, BID accuracies (21-64%) were significantly lower compared to using all the edges from the whole-brain connectome (paired sample t-test; t (4) = 6.54, P = 0.003, Cohen’s d = 1.1). When the fingerprint was computed with the top 10% of DP edges, BID accuracies (37-77%) were significantly higher compared to the top 1% of DP edges (paired sample t-test; t (4) = 8.22, P = 0.001, Cohen’s d = 1.1). On the other hand, when the fingerprint was computed with the top 75% of DP edges, BID accuracies slightly improved (43-77%) compared to the whole-brain connectome. However, this improvement was not significantly different from the whole-brain connectome (paired sample t-test; t (4) = 1.84 P = 0.14, Cohen’s d = 0.1). Similar observation was noted when comparing the BID accuracies from the top 10% of DP edges to the whole-brain connectome (paired sample t-test; t (4) = 0.14, P = 0.90, Cohen’s d = 0.01). The non-linear relationship between number of discriminative edges and BID accuracy is striking — there is a large jump in BID accuracies between 1% and 10%, but little-to-no improvement after that, suggesting that good BID accuracies can be achieved with as little as the top 10% of the most discriminative edges. Of note, the potential benefits of using the most discriminative edges to perform fingerprinting can be outweighed by the computational challenge in estimating the DP for high-resolution parcellation schemes. For example, a coarser parcellation scheme such as the Shen 268 has 35,778 edges whereas a finer parcellation scheme like the Schaefer 1000-17Networks has 499,500 edges. Given that DP is computed as the product of an edge value from Scan 1 and Scan 2 from the same individual and compared with the product of the same edge value from Scan 1 and Scan 2 from other individuals, identifying the most discriminative edges based on their DP values will be feasible for smaller parcellation sets (e.g., 35,778 edges) but will create computational difficulties at larger (e.g., 499,500 edges).

Longer scan duration facilitates higher identification accuracy

We evaluated the effort of acquiring increasing scan length to boost identification accuracy by varying the scan duration from 0.5 min to the full acquisition length of each dataset. There was a non-linear relationship between scan duration and identification accuracy across the five datasets (Figure 6A), such that BID accuracies were very negatively impacted by very short fMRI scans (e.g., 0.5 min — ranging from 5% to 21% across the five datasets). BID accuracies were optimal (42-77%) when the full acquisition length of each dataset was used to compute the functional connectivity matrices of each individual. RR revealed a similar pattern such that the degree of confusion for unsuccessfully identifying an individual was highest (23-41%) with short acquisition time (e.g., 0.5 min) but minimal (5-15%) when the entire scan length was used to derive the functional connectivity profiles of the individuals across the five datasets.

Increasing sample size does not necessarily improve identification accuracy

We also examined the effect of sample size (N) on identification accuracy by creating random subgroups of individuals N ∈ {20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 264} across the five datasets. For each N, 100 samples were selected, and the average BID accuracy was computed across samples. There was a non-linear relationship between sample size and BID accuracy such that the ability to identify an individual based on their functional connectivity profiles decreased non-linearly as the sample size increased (Figure 6B), with the steepest decrease observed as sample size increased from 20 to 100, and a more gradual decrease in BID accuracy as sample size increased further to 264. The overall change across sample sizes was relatively small, however — BID accuracies ranged from 70% when N = 20 to 53% when N = 264. As a mathematical reality, increasing sample sizes also increases the potential for “confusion” (high correlation) amongst individuals — it increases the pool of potential matches, thus decreasing identification accuracy. The aggregated/subsampling approach produced higher BID accuracies for sample sizes equivalent to those of the UPSM, BNU, and NYUadu sites, relative to the BID accuracies for those sites alone, suggesting some site or sample effects that were overcome by the aggregating and subsampling approach. Further, and contrary to observations for the preceding manipulations, for which the trend for RR was inverted relative to the trend for the BID accuracy, RR decreased slightly but non-linearly from 12% to 7% as sample size increased (Figure 6B).

Selecting a fingerprint-informed pipeline

On the basis of the results presented above, we identified two optimal fingerprint-informed pipelines featuring those parameters associated with the highest BID accuracy (Table 2; Figure 7). The two pipelines have been described given that predictive networks such as the frontoparietal networks and finer cortical parcellation schemes such as the Schaefer 1000-17Network scheme, separately, encapsulate the most distinct features of an individual’s functional connectome. As expected, relative to the standard pipeline (minimal pre-processing, GSR, whole-brain Shen 268 parcellation, full scan length, and full sample size), BID accuracy was significantly higher for both Pipeline 1 (paired sample t-test; t (4) = 6.11, P = 0.008 FDR, Cohen’s d = 1.5) and Pipeline 2 (paired sample t-test; t (4) = 2.83, P = 0.047 FDR, Cohen’s d = 0.8). Notably, some datasets (e.g., NYUado; SWU) exhibited BID accuracies of 90% or above following processing with Pipeline 1.

Applying functional connectome fingerprinting informed pipelines to improve ICC

To further assess the utility of these two fingerprint-informed pipelines, we computed TRT reliability as indexed by the ICC for each dataset following the application of these pipelines to compute functional connectivity (Table 2; Figure 7). The mean ICCs increased significantly with Pipeline 1 (paired sample t-test; t (4) = 7.22, P = 0.002 FDR, Cohen’s d = 1.0) and Pipeline 2 (paired sample t-test; t (4) = 7.22, P = 0.002 FDR, Cohen’s d = 1.6), although it is worth nothing that the mean ICCs across the five datasets would still be considered “poor” according to the Cicchetti and Sparrow (1981) definition.

Figure 3.(A) ID accuracy obtained from BID across the five independent datasets using four publicly available parcellation schemes including Shen 268, Glasser 360, MIST 444 and Schaefer 1000-17Networks. (B) Relationship between parcel resolution and ID accuracy measured by BID in addition to the degree of confusion as indexed by RR using 8 and 10 levels of parcel granularity derived from the Schaefer 17Networks and MIST parcellation schemes across the five independent datasets. (C) ID accuracy derived from BID based on the spatial coverage of the functional connectome covering cortical, subcortical and cerebellar structures across the five independent datasets. Cor: cortical; Cor+Sub: cortical and subcortical; Cor+Cer: cortical and cerebellum; Sub: subcortical; Sub+Cer: subcortical and cerebellum; and Cer: cerebellum.

Figure 4.(A) ID accuracy derived from BID for the Finn et al.¹⁷ Networks in addition to the combined medial frontal and frontoparietal networks [FPNs] and whole-brain connectome [WBN] across the five datasets. The Finn et al.¹⁷ Networks are as follows: (1) medial frontal [MFN]; (2) frontoparietal [FPN]; (3) default mode [DMN]; (4) subcortical [SUB]; (5) motor [MOT]; (6) visual I [VISI]; (7) visual II [VISII]; and (8) visual association [VISA]. (B) ID accuracy derived from BID for the Yeo et al.⁶² Networks across the five datasets. The Yeo et al.⁶² Networks are as follows: (1) ventral attention [VAN]; (2) dorsal attention [DAN]; (3) frontoparietal [FPN]; (4) default mode [DMN]; (5) somatomotor [SOM]; (6) limbic [LIM]; and (7) visual [VIS]

Figure 5.Relationship between number of discriminative edges and ID accuracy derived from BID in addition to the degree of confusion as indexed by RR across the five independent datasets. The discriminative edges are selected based on their DP values falling between the 25^th and 99^th percentiles of the whole-brain connectome, i.e., the top 1-75% of edges in the whole-brain connectome were retained to compute the ID accuracy of each dataset. The black box represents the ID accuracy derived from BID when all the edges in the whole-brain connectome were used for each dataset.

Figure 6.(A) Relationship between scan duration and ID accuracy measured by BID in addition to the degree of confusion as indexed by RR across the five independent datasets. The scan duration ranges from 0.5 min to the full acquisition time of each dataset. (B) Relationship between amount of resting-state fMRI data and ID accuracy measured by BID in addition to the degree of confusion as indexed by RR. The resting-state fMRI scans were concatenated across the five datasets (N = 264) for each imaging session, and then subsets of individuals were randomly selected across 100 iterations such that N ∈ {20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 264}.

Figure 7.ID accuracy and TRT reliability of the functional connectome measured by BID and ICC, respectively, using the standard and optimal fingerprint-informed pipelines. Dots and error bars represent the mean and SD of the BID accuracies and ICC values. Standard: GSR + Shen 268 parcellation + full scan length + full sample size. Pipeline 1: GSR + Schaefer 1000-17Network parcellation + full scan length + full sample size. Pipeline 2: GSR + frontoparietal networks derived from Finn et al.¹⁷ Networks + full scan length + full sample size. *P < 0.05, two tailed paired sample t-test. For ID accuracy, the effect sizes measured by Cohen’s d are: 1.5 (Pipeline 1) and 0.8 (Pipeline 2). For TRT reliability, the effect sizes measured by Cohen’s d are: 1.0 (Pipeline 1) and 1.6 (Pipeline 2).

Discussion

The present study aimed to demonstrate how connectome-based fingerprint identification accuracy could be used as a simple, intuitive, and sensitive index to adjudicate between functional connectivity-based analysis parameters. Across five small-moderately sized, independent Open Science datasets spanning childhood to adulthood, we found that functional connectome fingerprint identification accuracy ranged from 42% to 77%. We then tested the effect of a range of analysis parameters on ID accuracy and found that GSR, fine-grained (i.e., ~1,000 ROIs) brain parcellations encompassing cortical regions, number of discriminative edges, and longer scan duration improved identification accuracy — for some datasets to ≥ 90%. Interestingly, pooling fMRI data across datasets to increase sample size decreased, rather than increased, ID accuracy. Importantly, the effects of many of these manipulations on ID accuracy were quantitatively and qualitatively unambiguous, allowing for easy selection of the “best performing” parameters. These “best performing” parameters were then combined into two analysis pipelines that optimize the detection of individual differences in the functional connectome. It is important to highlight that the parametric statistical tests performed may not always meet their assumptions, which in turn, may undermine their validity in testing significant differences between pipelines (e.g., mean ID accuracies bounded between 0 and 1 may not be normally distributed). Others may argue that selecting the optimal pipelines involves a level of circularity that may challenge the way we compared the pipelines that maximize fingerprint accuracy. However, the discussions and conclusions made in the present study did not reply on the statistical tests per se; instead, we sought to demonstrate the use of connectome fingerprinting as a testbed for different strategies rather than establishing gold standards across all contexts and studies. If the different strategies/parameters improve fingerprint ID accuracy, then they also improve the reproducibility of the functional connectome.

Following the investigation of the impact of a set of data acquisition, pre-, and post-processing parameters on fingerprint accuracy, we described two analysis pipelines as follows: (1) Pipeline 1 — GSR, cortical-based parcellation scheme with 1,000 parcels, top 10% of discriminative edges, full scan length, and sample size and; (2) Pipeline 2 — GSR, frontoparietal networks (medial frontal and frontoparietal networks derived from the Finn et al.¹⁷ Networks), full scan length, and sample size. With Pipeline 1, accuracies that were 42-77% increased to 68-94%, and with Pipeline 2, they increased to 44-86%. The utility of these pipelines was also supported by the fact that they significantly increased TRT reliability (indexed by ICC) relative to the “standard” pipeline — although it is notable that the ICC values remained poor, according to the traditional categorization (i.e., ICC < 0.4). This echoes the TRT reliability literature which suggests that univariate measures of TRT like ICC for functional connectivity metrics are generally poor (mean: 0.29; 95% CI: 0.23-0.36).⁸ While a direct quantitative comparison of ICC and fingerprint BID is not possible, we can compare them qualitatively given that fingerprint accuracy tracks the ICC values. For example, an ICC = 0.40 implies that the TRT reliability of a functional connectivity measure is fair/moderate, but is above the upper meta-analytic estimate for the ICC across studies.⁸ This means that 40% of the total variance in the measurement of interest is attributed to agreement between the “raters” while the remaining 60% of the variance corresponds to individual variation or measurement errors. In contrast, an ID = 40% would be considered a low level of self-self correlation within a sample. What these comparisons suggest is that, while the ICC values that are typical for the field are considered to reflect “poor” reliability, fingerprint identification accuracy suggests a more optimistic perspective on functional connectome stability. To take one example, BID accuracy for the NYUado dataset processed with Pipeline 1 is 94% (ICC = 0.28), indicating that on average, 23 of 24 participants are correctly identified based on their functional connectomes. Similarly, the BID accuracy derived from Pipeline 1 for the SWU dataset is 90% (ICC = 0.35), which successfully identified, on average, 74 of 82 participants. Hence, connectome-based identification provides a metric of reproducibility of an individual’s functional connectome which is stable on both short-term (days) and long-term (years) from childhood to adulthood. Taken together, these findings mirror the emerging consensus that univariate measures of functional connectivity reliability are generally low whereas multivariate measures (e.g., fingerprinting) tend to be better.^63,74

Although we found that GSR was beneficial in terms of its impact on fingerprint ID accuracy, it remains the subject of dispute and there is no clear consensus on its application in publicly available pre-processing pipelines.⁷⁵ This disagreement has been traditionally rooted in evidence that showed that GSR introduces negative correlations between brain regions,^56,76,77 but often countered by its efficacy at mitigating the effects of motion and other sources of noise.^27,35,36 The debate is ongoing with recent work showing that the global signal could reflect neural information of behavioral relevance: for example, in a sample of 1,094 adults, the topography of the global signal has been shown to capture a positive-negative axis of life outcomes and psychological functions linking to an increased component loading of the frontoparietal network.⁷⁸ Across the lifespan, the topographical patterns of the global signal have revealed spatially specific linear and quadratic associations with age ranging from 6 to 86 years.⁷⁹ Given the significant and positive effect of GSR on fingerprint accuracy, in combination with strong evidence that GSR reduces the relationships between functional connectivity and head motion, particularly in age groups and vulnerable populations that exhibit heightened in-scanner movement, we recommend its inclusion in processing pipelines.

Describing the functional brain with high-resolution parcellation schemes produced the highest identification accuracies. This likely reflects the fact that increasing the parcel resolution preserves finer properties of the functional connectome from large-scale functional networks to localized brain areas, thus improving the sensitivity and specificity of functional connectivity analyses.⁸⁰ The choice of selecting a parcel resolution to investigate individual differences in the functional connectome typically depends on the desired spatial resolution of the effects being investigated. We found that cortical regions produced the highest fingerprint accuracy, which is consistent with observations that their functional connectivities are highly reliable compared to that of subcortical and cerebellar structures,^8,63,81 which may be driven by factors such as poor signal-to-noise ratio (SNR) due to signal dropout and susceptibility to physiological noise.⁶³ It is important to note that there may not be a unique parcellation scheme that captures individual differences in the functional connectome across all brain states.⁸² Instead, there exists a set of brain parcellations that is condition dependent which provides better parcel definition to detect such individual properties across different states.⁸² Investigating the impact of such state-specific parcellations on fingerprint ID would be an important future direction, especially in the context of clinical populations. These populations could have increased variations in their functional connectome that may not be captured with standard space parcellations as opposed to individualized schemes — thus making standard space parcellations less accurate when identifying the unique properties of the individual functional connectome.

Fingerprints restricted to the medial frontal and frontoparietal networks also yielded the highest fingerprint accuracies compared to the whole-brain connectome. These networks have been shown to exhibit high predictive power and are reproducible in both neurotypical and developing populations.^17,22,23,83 The frontoparietal network exhibits higher inter-subject variability in its functional connectivity relative to other functional networks including subcortical, limbic, and visual networks,^19,84 suggesting that this functional network can better discriminate an individual from a group. Restricting the functional connectome fingerprint with the frontoparietal networks (Pipeline 2) would be suitable for some functional connectivity applications but not necessarily for all measures, pipelines, and procedures. Therefore, describing the individual functional connectome with a second fingerprint-informed pipeline that includes a fine-grained parcellation scheme (Schaefer 1000-17Networks; Pipeline 1) offers another optimal strategy to test the reproducibility of functional connectivity applications.

As others have shown,^17,19,20,68 identification accuracies further increased non-linearly with longer scan duration. Scan duration has long been identified as a factor that affects the reliability of functional connectivity measures,^{8,12,63–65,67,70,85} along with resting-state conditions such as fixation, eyes-opened, and eyes-closed.^86–89 While initial reports suggested that acquisition times of ~5 min are sufficient to yield fair-to-excellent reliability,^12,67 more recent studies have suggested that much longer duration scans are required to yield the most reliable (e.g., > 36 min)⁶³ and reproducible (e.g., 15-25 min)⁶⁶ metrics. Researchers have employed several strategies to increase the amount of fMRI data available per participant, including longer scan duration,^8,63,81,90 repeated scans within-/between- sessions,^63,69,70,85 and higher sampling rates.⁶⁵ Acquiring and combining more scans for each individual across different brain states could further improve the fingerprint accuracy as within-subject variability may be better captured than at rest.⁹¹ This echoes recent evidence which suggests that having more fMRI data per participant increases the ability to capture unique properties of the functional connectome using resting-state and naturalistic conditions.¹⁹ It is, however, challenging to identify an ideal amount of fMRI data that is needed per participant to optimize the reliability of functional connectivity — longer scan lengths undermine the practical utility of fMRI, especially in developing and clinical populations with lower in-scanner tolerance or wakefulness.

In agreement with previous work,^19,20 identification patterns were improved when a greater number of discriminative edges was used to compute a given individual’s functional connectivity profiles. This finding showed that as few as 10% of the most discriminative edges were needed to yield good fingerprint accuracies as there was no significant difference in identification accuracies for the top 10% or 75% of the most discriminative edges relative to the full whole-brain connectome. This suggests that computational complexity and resource demands could be used by downsizing the functional connectome to the top 10% of discriminative edges. This also suggests that there are some edges that do not contribute to the identifiability of an individual’s functional connectome, thus positing that the entire functional connectome may not be required to compute functional connectome fingerprints. Moreover, it is difficult to suggest the ideal number of discriminative edges that should be obtained across populations because it may be challenging to obtain for clinical group comparisons. A recent study showed that there was a weak relationship between edge-level effect size obtained from between-group differences (depressed youth vs. healthy volunteers) and differential power.⁷⁴

We also pooled fMRI data across the five datasets to investigate the impact of sample size, but this did not improve the fingerprint accuracies. That is, larger sample sizes (within the ranges tested here) seem to increase the difficulty of differentiating an individual from a group. Similar observations have been reported which showed that fingerprint accuracy decreases non-linearly when the sample size derived from the Human Connectome Project increases from typical (e.g., N = 100) to large (e.g., N = 1,000).^28,29 A potential limitation of this sampling strategy is that we aggregated the fMRI data across a wide range of age groups (i.e., 7-43 years) which may not be ideal to detect individual differences in the functional connectome. There could also be a site effect that requires further investigation, particularly in a larger dataset acquired from several more sites. This may be a challenge as a downside of the connectome-based identification procedure is that a minimum of two imaging scans is required which is not always feasible during cross-sectional study designs. Future work may investigate the utility of split-half fingerprinting as a proxy method.^92,93 Along the same lines, employing a larger multisite dataset could allow the investigation of inter-scan intervals on fingerprint accuracy. The CoRR datasets used in this study make the comparison very difficult due to the wide range of inter-scan intervals from days to years.

Although we aimed to identify a pipeline that would optimize functional connectome fingerprint ID, and while high identification accuracies were attained for some samples, the accuracy did not reach 100% for any sample. This demonstrates that there are some individuals who remain incorrectly identified even with the fingerprint-informed pipelines. This suggests that there is scope to identify further factors that might influence fingerprint identification accuracy. On the other hand, it may also suggest that there are individuals who remain misidentified,⁹⁴ even in the context of overall high ID accuracy. We attempted to quantify the degree of misidentification for a given individual from a group using the RR metric which was as low as 2% when the optimal pipelines were employed. The RR metric did not provide additional discriminative information beyond BID accuracy, however. It is possible for fingerprint ID accuracy (as well as TRT reliability) to produce misleading conclusions.^95,96 As such, another multivariate metric referred to as discriminality has been proposed which calculates the probability of the distance between-sessions within each individual being lower than the distances to other individuals.^96,97 An interesting property of discriminality is that it can be equivalent to the ICC under the Gaussian assumption for univariate data although it can be broadly applied to assess the reliability of high-dimensional data given that it is a non-parametric multivariate generalization to the ICC. Discriminality has been used to examine the impact of analysis pipelines on the functional connectomes.⁹⁶ One main advantage of discriminality is that it provides stronger statistical power with increasing number of participants and repeated measurements compared to the fingerprint identification procedure.⁹⁷ It also generated estimates that tend to be constant as the sample size grows whereas fingerprint ID accuracy decreases non-linearly with more participants.⁹⁷ Therefore, discriminality could provide complementary information about the identifiability of an individual’s functional connectome together with fingerprint ID accuracy — indexing the reproducibility of functional connectivity estimates.

Can we expect resting-state functional connectome fingerprint ID accuracy to be perfect? There is evidence for 100% identification accuracy, achieved using naturalistic paradigms, albeit in a small sample.¹⁹ The answer may lie in the cognitive states of participants being scanned,^91,95 scanning conditions that evoke more specific cognitive states tend to be associated with higher fingerprint accuracy relative to rest.^19,91 It is possible that connectome-based identification accuracy at rest may reflect the influence of arousal, thought, and even physiological processes.⁹¹ Another important consideration is the influence of anatomical variability on individual differences in functional connectivity estimates that have been characterized as a mixture of morphological and functional variability.⁹⁶ Converging evidence from human and primate studies have shown that there is a significant relationship between structural and functional connectivities such that structural connectivity may predict inter-individual variability in cognition.^97,98 This relationship may, however, be heterogeneous as the structure-function coupling could be stronger for some networks but weaker for others.^99–101 Future work could investigate the contributions of anatomical variability (e.g., morphology) in measuring individual differences in functional connectome identifiability. Yet another avenue could focus on investigating whether unique or similar structure-function coupling across individuals relates to better or worse connectome-based identification.

In conclusion, we showed that functional connectome fingerprint accuracy can be used as a simple, intuitive, and sensitive metric for examining the effects of different functional connectivity analysis parameters in smaller and moderately sized samples, and in both developmental and adult datasets. By examining a set of data acquisition, pre-, and post-processing parameters, we identified a number of recommended steps that form two pipelines optimized for reproducible and stable functional connectivity at the individual level (Table 3). The pipelines comprise the following strategies: (I) applying GSR as a denoising strategy; (II) fine-grained brain parcellations; (III) cortical regions relative to subcortical and cerebellar structures; (IV) predictive functional networks such as medial frontal and frontoparietal networks; (V) discriminative edges; and (VI) increasing scan duration. In contrast, increasing the sample size by pooling resting-state fMRI data across different populations do not necessarily yield higher fingerprint accuracies. We suggest that the inclusion of fingerprint accuracy in analysis workflows can help researchers identify the most reproducible pipelines to ultimately achieve robust and reproducible brain-behavior relationships for biomarker discovery.

Authorship contributions

Jivesh Ramduny: Conceptualization, Methodology, Formal analysis, Investigation, Writing — original draft, review & editing. Tamara Vanderwal: Conceptualization, Investigation, Writing — original draft, review & editing. Clare Kelly: Conceptualization, Funding, Methodology, Formal analysis, Investigation, Writing — original draft, review & editing, Supervision, Project administration.

Funding sources

This work has been supported by the Trinity College Dublin Provost Award (2018-2019) awarded to Clare Kelly. Jivesh Ramduny is currently funded by the Kavli Postdoctoral Award for Academic Diversity at Yale University.

Ethical approval

Ethical approval was obtained from the relevant committee at each imaging site. Written informed consent was obtained from participants ages 18 or older. For participants younger than 18 years, written consent was obtained from their legal guardians, and written assent was obtained from the participant.

Conflicts of interest

The authors declare no conflict of interest.

Acknowledgements

We kindly thank the Consortium for Replicability and Reproducibility for allowing the neuroimaging data to be publicly accessible.

Corrresponding Author

Jivesh Ramduny
Department of Psychology
Yale University
100 College Street
New Haven
CT 06510
USA
jivesh.ramduny@yale.edu