Ruff Figural Fluency Test

The Ruff Figural Fluency Test (RFFT) (section: mental health) was measured in all Lifelines participants aged 18 years and older during:

• 1A Visit 1 (n = ~91.000)
• 3A Visit 1

The RFFT is a measure of nonverbal fluency, a parameter of executive cognitive functioning (section: mental health). The RFFT consists of five parts, each presenting a different stimulus pattern of dots, in which the task is to draw as many unique designs as possible within a set time period.

The RFFT was measured on paper in adult and elderly Lifelines participants during 1A Visit 1. Participants underwent either the RFFT or Spirometry, as these tests were performed in alternate weeks.
In assessment 2A, executive cognitive functioning was measured with CogState instead of the RFFT.
In assessment 3A, both the RFFT and CogState are measured with tablet and PC.

During 1A visit 1, the RFFT was performed on paper using a red marker.
The RFFT contains five parts, each presented on a separate stimulus sheet containing 35 identical replications of a unique stimulus item (see figure).

• Part 1 contains a standard dot pattern.
• Parts 2 and 3 contain the same dot pattern but with different distractors;
• Parts 4 and 5 contain a variation of the original dot pattern.

The task is to draw as many unique designs as possible within a set time period (60 seconds for each part). The entire RFFT takes approximately 10 minutes to be completed.
Each stimulus sheet is preceded by a sample page containing three examples of the specific stimulus item to afford the respondent an opportunity to practice.

Participants received the following practice instructions from a research nurse:

“In front of you are three squares, each containing five dots. Note that the arrangement of the five dots is always the same. I want you to connect two or more dots by always using straight lines. The purpose of the test is for you to make as many patterns (or figures) as possible, but each pattern has to be different in some way from all the others.”

Research nurses were instructed to give feedback on errors after the practice sample was completed (e.g., to point out two identical patterns), and to reemphasize that a figure can be drawn by connecting two or more dots if the respondent's patterns were very elaborate (e.g., all five dots were connected in each design).

Research nurses then gave the following instructions:
“Turn to the first test page. On this page, please draw as many different patterns (or figures) as possible. Start in the upper left square and work from left to right (examiner points out correct order). Just connect at least two dots with a straight line. Remember, work as quickly as possible and make every pattern different. Get ready-go!.”
Similar instructions were given for all five parts of the RFFT.

For each part of the test, the total number of unique designs (patterns), and the number of perseverative errors (i.e., repetitions of the same pattern within a single stimulus sheet) are counted.
Lines drawn between the dots do not have to be connected to each other. However, lines drawn to a neighboring dot configuration are considered invalid and are not counted as unique designs or perseverative errors, but as violations.
Also lines that are not connecting two dots are considered invalid, and are counted as violations.
Further, the number of empty dot configurations (no lines drawn) are counted, which results in four different outcomes for each part:

1. the number of unique designs;
2. the number of perseverative errors;
3. the number of violations;
4. the number of empty dot configurations.

As an index of planning efficiency, the error ratio can be calculated. The error ratio can be calculated by summing the number of unique designs and the number of perseverative errors across the five different parts, and then dividing the total number of perseverative errors by the total number of unique designs. The error ratio assesses the degree to which the respondent is able to minimize repetition while maximizing unique productions ¹⁾.

Exclusion criteria for performing the RFFT in Lifelines are:

• The participant is not sufficiently able to hold a pencil.
• The participant has impaired sight.
• The participant scores <26 on the Mini Mental State Examination.
• The participant has participated in the PREVEND study less than a month prior to his/her visit to the LifeLines facility.

The RFFT is a measure of nonverbal fluency, an example of executive functioning, which is the ability to utilize one or more strategies that maximize production of responses and avoid response repetition. The RFFT was developed with the aim of providing clinical information regarding nonverbal capacity for fluid and divergent thinking, ability to flexibly shift cognitive set, planning strategies, and executive ability to coordinate this process.

Criterion validity of the RFFT was assessed in a study which focused on the dependence of RFFT performance on right frontal lobe (which is generally believed to play an important role in figural fluency) activity ²⁾. A total of 45 right-handed men without history of significant head injury and cerebral dysfunction participated in the study. Based on the total number of unique designs produced on the RFFT, two groups were formed: one group consisted of 15 participants (mean age = 19.8 yrs, +/- 2.8 yrs) with the fewest number of unique designs produced, the other group consisted of 15 participants (mean age = 20.3 yrs, +/- 1.8 yrs) with the greatest number of unique designs produced. Via quantitative electroencephalography (EEG) delta magnitude at three different sites in the right frontal brain was measured. The study found that right frontal brain delta magnitude was significantly higher in the group which produced the lowest number of unique designs (F (1,28) = 5.44, p = .027), which supports the involvement of the right frontal brain in performing the RFFT.

Further support for the criterion validity of the RFFT is provided by another study ³⁾. In this study four groups of patients were included: patients with 1) right frontal (N=8, mean age = 30.8 yrs, +/- 6.2 yrs); 2) left frontal (N=8, mean age = 24.6 yrs, +/- 5.1 yrs); 3) right posterior (N=5, mean age = 27.2 yrs, +/- 11.2 yrs); and 4) left posterior (N=8, mean age = 37.0 yrs, +/- 14.0 yrs) brain lesions. All patients performed the RFFT at least 6 months post trauma, while being oriented and not in a state of amnesia. Results of the study show that a significant main effect for lesion group was found (F (1,28) = 5.44, p = .027). Patients with right frontal lobe lesions produced significantly fewer designs than patients with left frontal lobe lesions (t(14) = 3.04, p = .004), patients with right posterior lesions (t(12) = 2.63, p = .011), and patients with left posterior lesions (t(14) = 3.21, p = .003). This study also supports that the RFFT is a measure of right frontal lobe functioning.

A study examining the relationship between performance on the RFFT and other measures of executive functioning was conducted, to investigate convergent and divergent validity. A total of 102 healthy participants (84% women; mean age = 21.7 yrs, +/- 3.7 yrs) were included in the study. All participants performed the RFFT, alongside several other measures of cognitive functioning. These measures included tests for general intelligence, working memory, and verbal learning. Further, several tests measuring executive functioning were included. Results of the study show that significant modest correlations (r = 0.20 - 0.41) are seen between the number of unique designs on the RFFT and several measures of executive functioning, which supports convergent validity. The number of unique designs on the RFFT also correlated (r = 0.25) with a measure of general intelligence, and the RFFT error ratio correlated (r = 0.46) with a measure of nonverbal working memory. These results suggest that general intelligence and working memory may influence the performance on the RFFT. However, these RFFT indices did not correlate with any other (mostly verbal) measure of intelligence, working memory, or verbal learning, which supports convergent validity.

The study described in the previous section also examined inter-rater reliability of the RFFT ⁴⁾. The RFFT protocols of all participants (N = 102, as described earlier) were scored by six raters. The raters were undergraduate psychology majors, who had all received prior training in the administration and scoring of several neuropsychological tests. Intra-class correlation coefficients (ICC’s) were .98 for the number of unique designs, .94 for the number of perseverations, and .84 for the error ratio. These ICC values are generally considered excellent ⁵⁾.

A second study that examined the inter-rater reliability of the RFFT used a quite similar sample ⁶⁾. In this study five RFFT protocols obtained from a previous study were rated by 124 undergraduate psychology students independently. Additionally, 60 of the 124 raters rated five additional RFFT protocols, which led to an additional 150 RFFT protocols that were double rated. ICC’s were .93 for the number of unique designs, .74 for the number of perseverations, and .66 for the error ratio. However, a practice effect was found in rating. ICC’s improved from .87 to .97 for the number of unique designs, from .52 to .97 for the number of perseverations, and from .34 to .54 for the error ratio.

The longitudinal performance on the RFFT was tested in a study in which 2515 participants (47% women; mean age = 53 yrs, +/- 10 yrs) from a longitudinal study in the Netherlands participated ⁷⁾. Participants were recruited from the general population, and selection was based on their urinary albumin excretion. All participants completed the RFFT at three moments: the mean follow-up time between the first and second measurement was 2.8 (+/- 0.5) years; the mean follow-up time between the second and third measurement was 2.7 (+/- 0.5) years. Results of the study show that the mean number of unique designs increased significantly from 73 (+/-26) at baseline to 79 (+/- 27) at first follow-up, and 83 (+/- 26) at second follow-up overall (p < 0.001).

The RFFT may be analyzed by humans or by computer software, yielding similar results ⁸⁾.

Lifelines uses both RFFT and CogState to assess executive cognitive functions. In how far do these two instruments measure the same thing? In a subset of 509 participants (mean age (SD): 53 years (14.6); range 18–87 years), correlations between the scores of the RFFT and the CogState were statistically significant (except for the correlation between the RFFT error ratio and the CogState One Back Task), hoewever they were weak to moderate (ranging from -0.39 to 0.28). Stratifying the analyses for age, education, and gender did not substantially affect our conclusions. Sensitivity analyses showed no substantial influence of level of computer experience or (physical) impairments ⁹⁾.