An Icelandic version of the Kiddie-SADS-PL: Translation, cross-cultural adaptation and inter-rater reliability

´ BERTRAND LAUTH, PA ´ LL MAGNU ´ SSON, PIERRE FERRARI, HANNES PETURSSON

Lauth B, Magnússon P, Ferrari P, Pétursson H. An Icelandic version of the Kiddie-SADS-PL: Translation, cross-cultural adaptation and inter-rater reliability. Nord J Psychiatry 2008;62: 379�385. Oslo. ISSN 0803-9488.

The development of structured diagnostic instruments has been an important step for research in child and adolescent psychiatry, but the adequacy of a diagnostic instrument in a given culture does not guarantee its reliability or validity in another population. The objective of the study was to describe the process of cross-cultural adaptation into Icelandic of the Schedule for Affective Disorders and Schizophrenia for School-Age Children*Present and Lifetime Version (Kiddie-SADS-PL) and to test the inter-rater reliability of the adapted version. To attain cross- cultural equivalency, five important dimensions were addressed: semantic, technical, content, criterion and conceptual. The adapted Icelandic version was introduced into an inpatient clinical setting, and inter-rater reliability was estimated both at the symptom and diagnoses level, for the most frequent diagnostic categories in both international diagnostic classification systems (DSM-IV and ICD-10). The cross-cultural adaptation has provided an Icelandic version allowing similar understanding among different raters and has achieved acceptable cross- cultural equivalence. This initial study confirmed the quality of the translation and adaptation of Kiddie-SADS-PL and constitutes the first step of a larger validation study of the Icelandic version of the instrument. � Child psychiatric interview, Diagnostic instruments, Inter-rater reliability, K-SADS-PL, Translation and cross-cultural adaptation.

Bertrand Lauth, University of Iceland, Landspitali University Hospital, Department of Child and Adolescent Psychiatry, Dalbraut 12, 105 Reykjavı́k, Iceland, E-mail: [email protected] is; Accepted 20 September 2007.

T he development of structured diagnostic instruments

has been an important step for research in child and

adolescent psychiatry. These standardized instruments

are based on the more descriptive taxonomy of recent

editions of international classification systems, DSM-IV

(1) and ICD-10 (2). They are essential for research and

are good aids for clinicians, since they provide a

structured and systematic assessment procedure that

has increased diagnostic reliability.

However, for researchers belonging to cultural back-

grounds and speaking languages other than English, the

adoption of such an instrument involves a process of

translation and adaptation of the instrument. The

adequacy of the diagnostic instrument in a given culture

does not guarantee its reliability or validity in another

population (3). The challenge for these researchers is then

to develop a translated and adapted version of the

assessment tool, which is equivalent to the original

version of the instrument. Only by achieving this equiva-

lence will it be possible to assure the degree of compar-

ability in studies carried out in different cultures (3).

The need for diagnostic instruments has led the

authors to start the development of a local version of

the Kiddie-SADS-PL for research in clinical populations.

Because of the lack of European consensus statement

about standard and validated practices on that topic, we

started with a classic translation/back-translation process

to try to achieve semantic equivalence and then started to

introduce the structured interview into an adolescent

inpatient clinical setting.

The impact of the introduction of the diagnostic

instrument is described in another publication, as well

as additional studies on the psychometric properties of

the Icelandic version in an adolescent clinical population.

# 2008 Informa UK Ltd. (Informa Healthcare, Taylor & Francis As) DOI: 10.1080/08039480801984214

B LAUTH ET AL.

This article describes the process of translation and

adaptation of the instrument as well as inter-rater reliability data.

The importance of structured psychiatric interviews in child and adolescent psychiatry Research does not support unstructured interviews as

reliable means to standardized diagnoses (4). Even experienced clinical interviewers are not reliable diag-

nosticians when compared which each other or when

compared with structured interviews (5�8). Agreement is

usually low in studies involving outpatients or hospita-

lized children and adolescents (5, 6, 9�12). Agreement is

usually higher for externalizing disorders than for inter-

nalizing and many authors suggest that diagnoses of

anxiety and depression are often missed using an unstructured interview. Miller (7, 8) has suggested that

because the structured interview yields precise diagnostic

data, appropriate treatments may be delivered earlier,

leading to more rapid recovery and shorter hospital stays.

Research is still needed to establish the utility of

diagnostic interviews in clinical settings, but by encoura-

ging clinicians to follow standard diagnostic and inter-

viewing methods, they promote more consistent diagnostic practices and help justify therapeutic inter-

ventions and outcomes. However, the instruments are

not a substitute for clinical judgement. As McClellan &

Werry (13) pointed out, psychiatric decision making

depends on the integration of information from diverse

sources and perspectives, including the patient and

family interviews, the mental status examination, collat-

eral informants (teachers) and other treatment providers. The pre-eminent role of the clinician must be recognized

and preserved.

Choice of a ‘‘state of the art’’ psychiatric diagnostic interview In this project, an interviewer-based diagnostic interview that requires some clinical decision making on the part

of the interviewer was chosen. This kind of instrument is

often used in studies of clinical populations and is

preferred by clinicians to respondent-based interviews

(13), even though it usually shows poorer reliability.

The ‘‘Kiddie-Schedule for Affective Disorders and

Schizophrenia for School-Age Children’’ (‘‘K-SADS’’)

has changed since its original publication (14) and is currently available in different DSM-IV format versions:

the Kiddie-SADS-P IVR (Present State), the Kiddie-

SADS-L (Lifetime), the Kiddie-SADS-PL (Present and

Lifetime Version) and the Kiddie-SADS-E (Epidemio-

logical) (15, 16).

The ‘‘Present and Lifetime’’ version (K-SADS-PL)

has several strengths (16). It has strong content validity

because it was designed to tap pre-specified diagnostic criteria. It has been designed to lead the clinician or the

therapist to make DSM-IV diagnoses during interviews

of the young patient and his parent, including detailed probes useful in eliciting clinically meaningful informa-

tion. One strength of the Kiddie-SADS-PL is its high

degree of precision and detail in assessing child symp-

toms and their onset, severity, duration and associated

impairments. It is the only instrument that provides

global and diagnosis-specific impairment ratings to

facilitate the determination of ‘‘caseness’’ (17). The

Kiddie-SADS-PL provides also a clinician-friendly front and screening examination, which may result in a more

efficient shorter interview. Skip-decisions seem reliable

and have been validated by comparison with other

symptom measures (18).

In comparison to other child diagnostic instruments,

the K-SADS-PL compares favourably with test�retest

reliability estimates (17) and adequate concurrent valid-

ity of the K-SADS-PL diagnoses was established against several standard self-report scales (18). Its strength

remains in diagnoses of affective and anxiety disorders;

it has also been demonstrated to have good reliability for

attention-deficit/hyperactivity disorder.

Use of the Kiddie-SADS-PL requires extensive clin-

ical experience and instrument specific training. Diag-

nostic algorithms have not been computerized, so

diagnoses are formulated by expert judgement.

Translation and adaptation of Kiddie-SADS-PL We followed the cross-cultural adaptation model used to

translate into Spanish and culturally adapt the Diag- nostic Interview Schedule for Children (DISC) (19).

Even if cultural differences between American and

Icelandic children and adolescents may be to some

extent less important than between American and

Spanish-speaking child and adolescent populations, we

aimed at attaining cross-cultural equivalency by addres-

sing five important dimensions: semantic, technical,

content, criterion and conceptual. This model provided the frame of reference employed

in the development of an Icelandic version of the K-

SADS-PL.

SEMANTIC EQUIVALENCE

It involves the choice of terms and sentence structures

that ensure that the meaning of the source language

items is preserved in the translation. Semantic equiva- lence can be achieved with a combination of translation

and back-translation techniques (20).

In our case, two different translaters have indepen-

dently translated and back-translated the instrument.

Following Geisinger’s recommendations (21), they are

fluent in both languages, knowledgeable about both

cultures and have a good understanding of clinical

characteristics and content that the instrument is supposed to evaluate. Translation and back-translation

NORD J PSYCHIATRY �VOL 62 �NO 5 �2008 380

AN ICELANDIC VERSION OF THE KIDDIE-SADS-PL

drafts have been examined by a bilingual committee

composed of the two translaters and four bilingual and

experienced clinicians in child and adolescent psychiatry.

Feedback from interviewers after initial field testing was

essential in further improving the translation.

A few examples of differences between original and

back-translated instruments are following, illustrating

some of the semantic problems that had to be solved:

. Sadness/depressed mood;

. Separation from/absence of the mother;

. Racing/quick thoughts;

. Fits/episodes;

. Diurnal mood variations/daily mood fluctuations;

. Impulsivity/impetuousness.

One difficulty consisted in that the English vocabulary is

broader and more detailed than the Icelandic one, which

in particular includes fewer words to describe emotions

and affects.

CONTENT EQUIVALENCE

The purpose here is to examine whether the content of

each item is meaningful to the population under study

and, if necessary, to remove some of them. In the

Icelandic version of Kiddie-SADS, we had for example

to remove ‘‘Walking on the train tracks’’, since transport

by train does not exist in Iceland. Additionally we had to

remove some references to famous US teenagers’ gangs,

as well as several examples of social games common with

American children and replace them by equivalent

Icelandic ones.

Another issue in the field of content equivalence is to

determine whether the operationalization of what con-

stitutes normal behaviour is relevant to the culture under

study, with special consideration to evaluation of im-

pairment and social adaptation (normative equivalence).

Since Kiddie-SADS items include systematic estimation

of impairment or social adaptation, those issues have

been carefully examined by our bilingual committee.

TECHNICAL EQUIVALENCE

The issues addressed in this field are related to under- or

over-reporting of certain problems by some cultural

groups compared to others (substance abuse, sexual

behaviour and thoughts, certain feelings). We considered

here the openness with which particular topics are

discussed by Icelandic teenagers, the manner in which

ideas are expressed and the way in which strangers

asking questions are treated. The most significant

particularity we described (but not systematically as-

sessed) was related to difficulties, showed in our

experience by many Icelandic boys, to express both

positive and negative emotions, especially anxiety.

CONCEPTUAL EQUIVALENCE

This refers to the fact that the same theoretical construct

should be evaluated by the two versions of the assessment

instrument in the different cultures involved. Does the

concept operationalized in the source instrument (in-

dependently of the words and phrases used to represent

it) exist in the same form in the thinking of members of

the target culture?

In our case, two important discrepancies revealed by

differences between original and back-translated ver-

sions illustrated conceptual problems:

. Conduct/behavioural disorder;

. Attachment figure/someone close

The concept of attachment as it exists in English

semiology is almost impossible to translate into Icelandic.

Behling & Law (22) pointed out that translation/back-

translation was not an adequate test of the equivalence

of the target and source-language documents, but deals

only with semantic equivalence. The two versions may

correspond with one another for the wrong reasons and

target-language version may still not convey the in-

tended meaning to potential respondents.

Solving conceptual problems needs careful considera-

tion of: the constitutive definition of the concept of

interest, the theory that explains it and the nature of any

difference between the source and the target culture.

Usually this kind of equivalence cannot be directly

assessed except for rating scales with a known factorial

structure, which could be empirically tested with a

confirmatory factor analysis, trying to find the same

factor structure in the translated version.

CRITERION EQUIVALENCE

Solving problems in that field involves identifying

pertinent norms in each culture and assessing when the

trait or disorder evaluated is said to exist according to

these norms. In our case, the culture for which Kiddie-

SADS is to be adapted is not very different from the

culture for which the instrument was developed and we

were not able to identify significant symptoms of mental

distress that would appear different culturally from those

included in the original version of the instrument.

However, we were aware of cultural differences in how

parents and professionals view behavioural problems.

In this study, we were not able to validate the

instrument against best estimate clinical judgement

from various Icelandic child and adolescent psychia-

trists, so we designed a study of the concurrent validity

using 11 well-known checklists and rating scales,

which had already been translated and validated in

Iceland and that assess several important dimensions of

psychopathology. This validation study will be the

subject of a separate paper.

NORD J PSYCHIATRY �VOL 62 �NO 5 �2008 381

� �

� � �

�

� �

B LAUTH ET AL.

Methods Clinical context The adolescent unit of the Department of Child and Adolescent Psychiatry of the Landspı́tali University

Hospital in Reykjavı́k, is the only psychiatric ward for

adolescents in Iceland, admitting each year between 70

and 80 patients from 11 to 18 years of age, from all parts

of the country. The main reasons for admission are

severe behavioural or emotional disturbances with severe

functional impairment and often suicidality (61% of

cases in the period 2001�2004), 53% being acute admis- sions. Mean length of stay was 43 days (2001�2004).

Adolescents presenting with alcohol and drug abuse as a

predominant problem are referred to other service

providers, such as social services. The population

admitted is culturally homogeneous and its geographic

distribution throughout the country is representative of

the general population for the same age categories.

Since the unit is the only facility in the country providing psychiatric inpatient treatment for adoles-

cents, we assume that our population is representative

of the most severe range of psychiatric morbidity of the

adolescent clinical population in Iceland.

Participants Fifteen subjects were included in the inter-rater relia-

bility data pool; the mean age was 15.2 (standard

deviation, s 1.0, range 14�17), 55% being females and all were Icelandic.

These subjects were chosen at random among parti-

cipants in the validation study (n 86, mean age

15.0 years; s 1.34; range 11�18), females accounted

for 55.7% and again, all were Icelandic. They were

severely affected in most of the cases and some of them

could not complete the diagnostic interview.

Measures As the main official diagnostic classification system in

European countries is ICD-10 for both clinical and

research purposes, results of Kiddie-SADS interviews

algorithms have been translated into the ICD-10. A few

additional questions were included in the interviews for

ICD-10 criteria not covered by the K-SADS-PL.

Both DSM-IV and ICD-10 diagnoses were generated,

and numbers of diagnostic criteria met were calculated for the most frequent diagnostic categories in both

classification systems.

Two coders checked to verify accurate utilization of

DSM-IV and ICD-10 algorithms in the calculation of

numbers of diagnostic criteria met and assignment of

final diagnosis.

Procedure Inter-rater reliability was estimated with 15 interviews being re-rated independently by other experienced and

trained clinicians. Three clinical psychologists and one

child and adolescent psychiatrist took part in the project. Interviews were either videotaped or attended

at the same time by another rater.

Statistical analysis The Statistical Package for Social Sciences was used for

data analysis (23). For comparisons of categorical

variables, chi-square tests were applied. Cohen’s Kappa

(24) was used for reliability measures. Criteria proposed

by Landis & Koch (24) were used to interpret the Kappa

coefficients: excellent reliability (Kappa 0.75), good

reliability (Kappa 0.59�0.75), fair reliability (Kappa 0.40�0.58) and poor reliability (KappaB0.40).

Kappa values were calculated in both classification

systems and for most frequent diagnostic categories. In

each diagnostic area, inter-rater reliability was also

examined at the symptom level, with Kappa values

calculated for each item. Correlations between numbers

of diagnostic criteria met generated by different K-

SADS raters were calculated. Inter-rater reliability was also examined for main

diagnostic areas surveyed in the screen interview, in order

to estimate agreement in utilization of skip-out criteria.

Since several severely affected patients could not

complete the diagnostic interview, the number of sub-

jects allowing calculation of inter-rater reliability varied

according to diagnostic categories.

Results Inter-rater reliability of skip-out criteria The average agreement evaluated by calculation of

Kappa statistics across the nine diagnostic categories

studied here is fair to excellent (Table 1).

Inter-rater reliability of diagnosis assignment Kappas for diagnoses ranged from 0.31 to 1.0 (Table 2).

In each diagnostic category, inter-rater reliability was

also examined at the symptom level with Kappas

calculated for each item. Table 3 summarizes the results

Table 1. Inter-rater reliability: Agreement in utilization of skip-out criteria.

Diagnostic area in screen interview Kappa value n

Depressive disorders 1.00 15

Bipolar disorders 0.86 14

Attention-deficit/hyperactivity disorder 1.00 11

Oppositional defiant disorder 1.00 12

Conduct disorder 0.67 12

Post-traumatic stress disorder 1.00 8

Social phobia 1.00 12

Separation anxiety disorder 0.81 12

Generalized anxiety disorder 0.57 12

NORD J PSYCHIATRY �VOL 62 �NO 5 �2008 382

Table 2. Inter-rater reliability: measure of agreement on diagnoses.

Diagnostic category Kappa value n

Major depressive disorder (DSM-IV) 1.00 15

Melancholic depression (DSM-IV) 0.55 15

Dysthymia (ICD-10) 0.63 15

Moderate depressive episode (ICD-10) 0.32 15

Severe depressive episode (ICD-10) 0.44 15

Somatic syndrome (ICD-10) 0.47 15

Mania (DSM-IV) 0.31 14

Hypomania (DSM-IV) 1.00 14

Bipolar disorder not otherwise 0.44 14

specified (DSM-IV)

Mania (ICD-10) 0.44 14

Hypomania (ICD-10) 1.00 14

ADHD*predominantly inattentive 0.81 11

type (DSM-IV)

ADHD*predominantly 1.00 11

hyperactive-impulsive type (DSM-IV)

ADHD*combined type 1.00 11

(DSM-IV)/hyperkinetic disorder (ICD-10)

Oppositional defiant disorder (DSM-IV) 1.00 12

Oppositional defiant disorder (ICD-10) 1.00 12

Conduct disorder (DSM-IV and ICD-10) 1.00 12

Post-traumatic stress disorder (DSM-IV) 0.67 6

Post-traumatic stress 0.57 6

disorder*chronic type (DSM-IV)

Post-traumatic stress disorder (ICD-10) 0.67 6

Social phobia (DSM-IV) 0.82 12

Social anxiety disorder of 1.00 12

childhood (ICD-10)

Separation anxiety disorder (DSM-IV) 1.00 12

Separation anxiety disorder of 1.00 12

childhood (ICD-10)

Overanxious/generalized anxiety 1.00 12

disorder (DSM-IV)

Overanxious/generalized anxiety 0.82 12

disorder (ICD-10)

ADHD, Attention-Deficit/Hyperactivity Disorder.

with average values calculated within each diagnostic

category.

The inter-rater reliability data pool allowed calcula-

tion of correlations between numbers of diagnostic criteria met generated by different K-SADS raters and

calculated in main areas of psychopathology, related to

both classification diagnostic systems. Statistically sig-

nificant correlations between different raters’ severity

scores were found at the 0.01 level in all diagnostic

categories (Table 4).

Discussion The translation and adaptation process of the Kiddie-

SADS-PL into the Icelandic language and culture aimed

at attaining cross-cultural equivalency by addressing five

important dimensions: semantic, technical, content,

criterion and conceptual.

The results of the study on inter-rater reliability must be interpreted cautiously given the small size of the

AN ICELANDIC VERSION OF THE KIDDIE-SADS-PL

Table 3. Inter-rater reliability: average values at the symptom level within each diagnostic category.

Diagnostic category Kappa value n

Depressive disorders 0.83 15

Bipolar disorders 0.85 14

ADHD*inattention 0.90 11

ADHD*hyperactivity-impulsivity 0.94 11

Oppositional defiant disorder 0.98 12

Conduct disorder 0.96 12

Post-traumatic stress disorder*current 0.93 9

Post-traumatic stress disorder*msp 0.86 3

Social phobia 0.48 12

Separation anxiety disorder 0.78 12

Generalized anxiety disorder 0.82 12

ADHD, Attention-Deficit/Hyperactivity Disorder msp, most severe

past episode.

research sample. Additionally, two different measuring

procedures were used: in-session observation and video-

taped observation.

The data collected in this study were, however, in line

with those reported by other investigators (4, 12, 18, 25� 27). Agreement in utilization of skip-out criteria was

excellent for most diagnostic categories but moderate for

generalized anxiety disorder. Agreement in the assign-

ment of most frequent diagnostic categories was ex-