The effect of a shared visual context during the presentation of elicitation stimuli on the narratives of young children with and without language impairment

This study aims to ascertain whether a shared visual context between examiners and children during narrative assessment influences the narratives produced by the children. Participants were 20 typically developing (TD) children and 10 children with language impairment (LI), aged 6 to 8 years. They were randomly assigned to two groups and assessed with two different presentation methods. Narrative performance was measured in terms of microand macrostructure. Microstructural variables included productivity (total number of words, total number of T-units), syntactic complexity (mean length of T-unit) and lexical diversity measures (total number of different words, number of internal state terms). Macrostructural variables included cohesion measures (number of complete, incomplete and error ties) and story structure scores measured by the Afrikaans translation (Klop, Visser and Oosthuizen 2012a) of the Language Impairment Testing in Multilingual Settings-Multilingual Assessment Instrument for Narratives (LITMUS-MAIN) (Gagarina et al. 2012). Both presentation methods elicited narratives of similar quality in terms of the microand macrostructural variables in all the groups. A shared visual context between examiners and children during narrative assessment therefore did not influence the narratives produced by children with LI and TD children.


Introduction
The goal of assessment in speech-language therapy is to obtain data on a person's communication performance so that a diagnosis and appropriate goals for clinical management can be formulated (Owens 2004:58).It is therefore important to select assessment procedures that provide valid representations of a child's speech and language abilities (Boudreau 2008:105) and are challenging enough to reveal linguistic vulnerability (Hadley 1998:138).
The assessment of narratives is considered by researchers and clinicians to be an ecologically valid way to investigate communicative competence (Botting 2002:3) and is often more valid, sensitive and less biased than norm-referenced assessment tools in some populations (Manolitsi and Botting 2011:42;Norbury and Bishop 2003:310).Extensive research over the past 25 years has demonstrated the relationships between children's narrative skills and academic success (e.g., Botting 2002;Cain and Oakhill 2007;Feagans and Short 1984;Hester 2010:8;Milosky 1987), thereby highlighting the value of including narrative assessment in the assessment battery.Furthermore, oral narratives allow clinicians to analyse multiple linguistic features, such as microstructural aspects (e.g., lexical diversity), and macrostructural elements (e.g., story grammar), using a relatively short language sample (Heilmann, Miller, Nockerts and Dunaway 2010:154).
Narrative performance represents a set of dynamic skills that are influenced by contextual factors such as elicitation frameworks and task demands (Boudreau 2008:105).Research has shown that both the comprehension and production of narratives can be influenced by the elicitation context (e.g., structured or unstructured), story genre (e.g., story retellings, story generations or personal stories), narrative themes, the child's experiences and world knowledge, the presentation modality (e.g., audio, visual or combined audio-visual input), and the familiarity of the listener (Gazella and Stockman 2003:65;Liles 1993:873).
The use of a so-called "naive" or "unfamiliar" listener has become part of standard operating procedures in research on children's narrative abilities.A naive listener is someone who is presumed not to share a mutual knowledge base with the narrator about the topic.The underlying assumption is that shared knowledge between communicative partners influences the interaction between them.For example, if a clinician and a child are both looking at the same set of pictures, the child may only use pronouns to refer to the characters while telling the story, because the shared context and mutual knowledge base make it unnecessary to refer to the referents in a more specific way (Hughes, McGillivray and Schmidek 1997:24).
Researchers employ various methods to create naive listener conditions during narrative assessment.The procedure used most often is one in which the naive listener is not present when the child views a film or listens to a story.Thereafter, the naive listener enters the room, and the child is then usually instructed to tell the story to this listener who is supposedly unfamiliar with the content (e.g., Gazella and Stockman 2003:65;Hadley 1998:134;Liles 1985:124;1993:873;Schneider and Dubé 2005:55).
Other methods include telling the story to a listener who is blindfolded (Kail and Hickmann 1992:79) or positioning the material so that the examiner cannot see the pictures during the narration (e.g., Schneider and Hayward 2010:465).The instructions of formal tests, for example, the Edmonton Narrative Norming Instrument (ENNI) (Schneider, Dubé and Hayward 2005) and the Diagnostic Evaluation of Language Variation (DELV) (Seymour, Roeper and De Villiers 2005), often stipulate that the examiner must be unable to see the pictorial stimuli during the elicitation of narrative items.
In contrast with research studies, clinicians are seldom able to use a naive listener during narrative assessments.It is often impracticable to include another person in the assessment environment, and doing so may also infringe upon client confidentiality.
The question arises as to whether this means that narrative assessment in clinical practice is compromised because of the lack of control over listener conditions.Are children's narrative abilities routinely underestimated because of flawed sampling conditions?Furthermore, do children really believe us when we tell them that the naive listener has never seen or heard the story before?Is it sufficient to only control for joint attention, in other words, not looking at the picture together during narrative assessment?
This article reports the findings of a study that investigated the impact of shared knowledge and joint attention on the micro-and macrostructure of narratives of 6-to 8-year-old children with and without language disorders.The next section will provide a brief overview of the existing literature about the impact of task and listener conditions on children's narratives.

Shared versus unshared information: existing evidence
For successful communication to take place, the speaker must take the knowledge state of the listener into consideration when conveying information.This implies that the speaker must evaluate what the listener already knows about a topic and what new information s/he requires.According to Sonnenschein (1986:549), even young children understand that certain procedural rules, such as considering the listener's status, knowledge and abilities, must be followed to facilitate good communication.For example, research has shown that children as young as 3 years associate seeing an object with knowledge of that object, and that they will adjust their communicative behaviour according to their perception of a listener's knowledge state (see Nayer and Graham 2006 for a review).Blank (1975:255) found that young children use fewer gestures and more verbal explanations if the referents are absent, and cautioned that the physical presence of elicitation stimuli during testing could lead to underestimation of children's verbal abilities.
Previous studies have shown that the use of a naive listener during narrative assessment elicits more complex language from children.Short- Masterson (2010:220) found that preschool children used more complex language when they were aware of the need to establish shared knowledge with their listeners.Even after mutual knowledge about a topic was established, the children continued to use more abstract and less descriptive language.Sonnenschein (1986:552) found that school-age children will provide more redundant information, in other words, additional explanations, to unfamiliar listeners to support listener understanding.Several studies indicated that referencing, in particular, will be influenced by shared knowledge with the listener (e.g., Kail and Hickmann 1992;Kail and Lopez 1997).It was found that when children and listeners could both see the picture, children were less likely to introduce referents.In contrast, in the absence of shared visual knowledge, children were more likely to use indefinite determiners to introduce referents (Kail and Hickmann 1992:88).
Only two studies, those of Liles (1985) and Masterson and Kamhi (1991), were found in the speech-language therapy literature that investigated the impact of listener conditions on the narratives of children with and without language impairment.Liles investigated cohesive adequacy and the use of cohesive markers in the narratives of 20 children with language impairments and 20 typically developing children aged 7 years 6 months to 10 years 6 months.Cohesive adequacy was defined in terms of the number of complete ties 1 included in the narratives.The children were shown a film and then asked to retell the story, first to the examiner, and then to another person who was not present when the film was viewed.Liles found that both groups adjusted their narratives according to the listener condition.Their narratives to the listener who was not in the room when the film was shown contained more complete ties, whereas their narratives to the listener who shared the viewing contained more incomplete ties.Masterson and Kamhi (1991) investigated the impact of listener knowledge on the sentence production of 10 language-learning-disabled, 10 reading-disabled and 10 typically developing children between the ages of 6 and 9 years.Participants provided descriptions of an experiment and retellings of stories to the examiner who presented the experiment and stories to them.They then described the experiment and told the stories to another child who was not present during the procedures and therefore did not share any knowledge about them.The results were similar across the three groups: simple sentences were used more often when providing information already known to the listener, and compound sentences were used more often in the information provided to the naive listener.After examining the simple sentences that were used in both conditions, more lexical errors were found in the shared information condition (Masterson and Kamhi 1991:554).
The results from the abovementioned studies indicate that children are indeed sensitive to the knowledge states of their listeners and that they adjust their communicative behaviour accordingly.According to O'Neill (1996:673), children as young as 2 years are aware that a person's knowledge depends on his or her sensory access to information.They therefore understand that a person who was not physically present in a situation will have no knowledge of what took place and needs to be informed or updated by the persons who witnessed the situation.However, not all researchers interpret these findings as evidence that children consciously consider and assess their listeners' knowledge states during narrative tasks (Nayer and Graham 2006:405) or that the adjustments in their language reflect a sophisticated, causal understanding of listeners' needs.

The present study
We aimed to expand on the studies by Liles (1985) and Masterson and Kamhi (1991) by investigating the effect of different listener conditions on the narratives of children with and without language impairments.As stated before, the use of a naive listener is often impractical in clinical situations.We therefore wanted to investigate an elicitation method that did not include a naive listener, but still controlled for mutual knowledge and joint attention between the child and examiner.
We were particularly interested in the effect of different listener conditions on the cohesive adequacy in terms of number of complete, incomplete and erroneous ties in our participants' narratives.A number of studies investigating cohesive adequacy in narratives have found that, when compared with children with normal language abilities, children with language impairment use fewer cohesive ties in general, fewer complete cohesive ties, and more dam because he was thirsty; ii) incomplete, when the information referred to by the cohesive marker is not provided in the narrative, but has to be recovered outside the narrative by inferring the meaning from the story context, e.g., the boy walked home / he sat in the tree; and, iii) erroneous, when the listener is guided to ambiguous or incorrect information, or omission of an article, e.g., [ ] man wanted to shoot the bird.incomplete and/or erroneous ties (e.g., Finestack, Fey and Catts 2006:245;Liles 1985:130;1993:880;Strong and Shaver 1991:107) and are less successful in repairing cohesive breakdowns (Purcell and Liles 1992:359).One possible explanation for these findings is that the errors in cohesion displayed by children with language problems in narrative production may reflect their problems with the management of information.They tend to include irrelevant and redundant information, exclude critical and necessary elements, use unclear and ambiguous references, and confuse the sequencing and marking of new and old information with articles and pronouns (Merritt and Liles 1987:540;Owens 2004:147).
Depending on the goals of the examiner, narratives are usually analysed at two levels, namely microstructure and macrostructure.Microstructural analyses focus on the internal linguistic structures used in the construction of the narrative, such as noun phrases and conjunctions.Macrostructural analyses, by contrast, focus on higher-order hierarchical organization such as episodic structure and story grammar components (Heilmann et al. 2010:155).Micro-structure and macrostructure variables therefore represent two distinct areas underlying narrative competence.The variables investigated in this study were grouped into these two broad categories, namely microstructure and macrostructure.Following Heilmann et al. (2010:156), we regarded cohesive adequacy as a macrostructural measure because it goes beyond the internal linguistic structure of individual utterances to establish coherence at the organizational level.
Our study fell within a quantitative, comparative, between-subjects paradigm and was guided by the following general research question: What is the effect of joint attention between children and examiners on the narratives produced by 6-to 8-year-old children with typical language development (TD) and children with language impairments (LI)?
Our specific research questions were: 2. Are there group differences between children with and without language impairment?
Our expectations, based on previous research, were that: i) the presentation modality without shared knowledge and joint attention would result in more complex narratives; ii) the expectation in (i) would hold for children with and without language impairment, and iii) the narratives of the children with language impairments would differ from those of the typically developing children.

Ethical approval
The original study from which the present study's data was extracted was approved by the institutional ethical review committee of Stellenbosch University (N12/01/026).Formal written consent was obtained from the Western Cape Education Department, school principal, parents and participants prior to the study.

Typically developing (TD) children
The study's TD participants were recruited from a mainstream primary school in the Western Cape, South Africa.Selection criteria specified that they were in Grade 1, Afrikaans homelanguage speakers receiving their formal education in their home language, and that they have normal hearing and no speech or language problems.The children came from the same suburban community, cultural group and a middle-class socio-economic background.A parent questionnaire was used to exclude participants with a history of speech or language problems, chronic otitis media or hearing problems.For inclusion in the study, children had to pass otoscopic examinations and pure tone hearing screening at 20 dB levels at 500, 1000, 2000 and 4000 Hz (Stach 1998).Children who failed the hearing screening test were excluded from the study and referred for hearing tests.The children's receptive language skills were assessed by means of the Afrikaanse Reseptiewe Woordeskattoets (ARW) (Buitendag 1994), a formal standardised test developed for use with the South African population, in order to exclude children with language impairments.Twenty children (8 girls and 12 boys) fulfilled the abovementioned selection criteria and participated in the study.The mean age of this group was 73 months (SD 4) and their mean ARW standard score was 105 (SD 2.8).

Children with language impairment (LI)
Participants in the LI group were six children (3 girls and 3 boys) recruited from a school for children with special needs, who had clinical diagnoses of language impairment.Their language, cultural and socio-economic backgrounds were similar to those of the TD group.An additional four children (all boys) were recruited from the same population as the TD children on the basis of their performance on the ARW.These four children obtained standard scores ranging from 55 to 75, in other words, more than two standard deviations below the mean standard score (-1Z), indicating language impairment (Buitendag 1994:27).All the children with LI passed the same hearing tests described in section 4.2.1.The mean age of this group was 83 months (SD 8) and their mean ARW standard score was 71 (SD 3.9).The LI group's ARW standard scores were significantly lower than the TD group (F(1,26)=47.09;p<.001).

Group assignment
Children from the LI and TD groups were randomly assigned to two elicitation procedures: Procedure 1 where participants and examiners did not share the visual context during elicitation, and Procedure 2 with joint attention between examiners and children during elicitation.

Narrative assessment material
Participants' narratives were assessed with the LITMUS-MAIN (Gagarina et al. 2012).The LITMUS-MAIN was developed as a tool for the assessment of narrative abilities of children between the ages of 4 to 9 years by a workgroup of COST Action ISO8042 .The LITMUS-MAIN was developed after extensive pilot studies with more than 500 children in 15 different languages and language combinations.It was translated into Afrikaans (Klop et al. 2012a) and piloted in Afrikaans-and English-speaking South African populations (Klop, Visser andOosthuizen 2011, 2012b).The LITMUS-MAIN comprises four parallel stories, each with a carefully designed six-picture sequence, that are controlled for cognitive and linguistic complexity, parallelism in macro-and microstructure, as well as for cultural appropriateness.
The LITMUS-MAIN provides examiners with a choice of elicitation procedures, namely telling with or without a model story, or retelling.In the present study, the telling procedure was used.
A novel approach was developed by the workgroup for the presentation of the narrative elicitation stimuli in an attempt to control for the effect of shared knowledge and joint attention: the picture sequences are presented to the child in three closed envelopes with the instruction to choose one without showing the examiner the story that was selected.The child is then guided by the examiner to hold and unfold the pictures during the initial viewing and the telling of the story in such a way that the examiner cannot see the pictures (see Gagarina et al. 2012 for instructions).There is therefore no mutual sharing of the visual context between the child and examiner during this procedure.

Assessment procedures
Participants were assessed with the LITMUS-MAIN procedures for telling, in other words, story generation after viewing the pictures.The LITMUS-MAIN Cat Story was selected for the study.Each participant was exposed to the same standardised test protocols and procedures to increase the validity and reliability of the assessments.Narratives were collected by four speech-language therapy students who were familiar with the use of narratives in assessment and intervention procedures.Assessment took place over a period of one week.Each child was assessed individually in a quiet room at the school.This procedure followed the LITMUS-MAIN procedures where children select a story from one of three envelopes, unfold the pictures contained therein, and tell the story without the examiner being able to see the pictures.The examiner reminded the child that she did not know the story and could not see the pictures.The child was asked to preview the pictures and then tell the story to the researcher while looking at the pictures but not permitting the researcher to see them.During the story telling, the examiner responded with neutral responses such as "hmm", "yes" and "what else?" to prompt the child to continue the story or to provide more information.If the child stopped the story without providing a formal ending, the researcher told the child, "Tell me when you have finished".

Procedure 2: Shared visual context
In this procedure, the examiner unfolded the same pictures as in Procedure 1, but on the table in front of the child where both the child and the examiner could see the pictures during the viewing and telling of the story.The child was then asked to tell the story to the examiner, while looking at the pictures in front of them.During the story telling, the examiner used the same neutral responses described in Procedure 1.

Transcription and pruning
All data coding was done manually by the researchers.After the narratives were collected, each participant's narrative was transcribed and pruned of revisions, repetitions, interjections and incomplete utterances and then coded into T-units3 .The pruned narratives were then coded for the microstructural variables, namely total number of words, total number of T-units, number of different words and mean length of T-unit.ISTs were coded according to the format of Section B of the LITMUS-MAIN.The ISTs that featured in the data were further categorized according to the following IST groups: perceptual state terms (e.g., see, hear), physiological state terms (e.g., thirsty, hungry), consciousness terms (e.g., alive, awake, asleep), emotion terms (e.g., sad, happy), mental verbs (e.g., want, think) and linguistic verbs or verbs of saying/telling (e.g., say, call, shout).For macrostructure, the narrative was scored according to the scoring protocol of Section A of the LITMUS-MAIN (entitled "Story Structure"), and also coded for cohesive adequacy in terms of complete, incomplete and erroneous ties.

Data analyses
To establish inter-rater reliability, two speech-language therapists who were blind to the participants' group assignment re-coded all the data sets.Results were compared and differences were resolved through discussion so that consensus was reached for coding decisions.
A commercial software package, Statistica 9, was used by a biostatistician for the statistical analyses of the data.Two-way ANOVAs were used in order to examine the differences between the participants in the groups exposed to either the shared visual context or the non-shared visual context.A 5% significance level (p<0.05) was used as a guideline for determining significant effects of variables.

Results and discussion
The main aim of this study was to determine whether the narratives of children with and without language impairment in the absence of joint attention differ from those elicited by a listener with whom they shared the same visual context.Table 1 provides a summary of the descriptive statistics of the participants in each group with regard to receptive vocabulary scores and microstructural variables.Based on previous research, we expected that the group which was exposed to the elicitation procedure without a shared visual context would produce longer and linguistically more complex narratives and that the children with LI would produce shorter and less complex narratives than the TD children.In contrast with the findings of previous research (e.g., Blank 1975;Sonnenschein 1986), our study found no group differences for microstructural variables as a result of shared visual contexts between examiners and children.Possible explanations for these results will be offered later in this section.Table 2 provides a summary of the descriptive statistics of the participants in each group with regard to macrostructural variables.Kail and Hickmann 1992;Liles 1985) has shown that children usually produce more cohesive markers in their narratives in the absence of mutual knowledge and shared visual context with listeners.
Post-hoc comparisons with the Fisher LSD test also showed that TD children included more complete ties in their narratives than LI children, but only in Procedure 1 (p=.04).The fact that TD children's narratives contained more complete ties concurs with previous research (e.g., Finestack, Fey and Catts 2006:245;Liles 1985:130;1993:880;Strong and Shaver 1991:107).However, it was again unexpected that this difference was only observed for Procedure 1 and only for one of the variables for cohesion.
No significant differences between the two elicitation procedures were found in the narratives of both TD and LI children with regard to the other cohesive markers, total number of ties

Conclusions
In summary, our study did not provide evidence that children produce more complex and cohesive narratives when they do not share a visual context with their examiners.Our results indicated that the elicitation procedures had little impact on the micro-and macrostructural variables that were investigated in the narratives of TD and LI children.We offer three possible explanations for these findings: First, a telling or story generation task was used to elicit narratives from the participants.In contrast with telling, story retell procedures usually result in longer, more detailed and grammatically more complete language samples (Gazella and Stockman 2003:62;Schneider and Dubé 2005:57).Retelling is therefore considered to be more suitable to obtain representative language samples from younger children or children with language disorders (Hughes et al. 1997:17;Merritt and Liles 1987:540).All of the participants in this study obtained very low LITMUS-MAIN Story Structure scores, indicating low levels of macrostructural complexity.It could be that the demands of the telling task resulted in shorter and less detailed narratives from all the groups and therefore obscured the differences between them.The fact that the children with LI produced, in general, narratives similar to those of the TD children can be interpreted as further evidence that our measures were not sensitive enough to capture the differences between the groups.It is possible that retell narratives or a combination of telling and retelling would have revealed more information about the participants' narrative abilities and the differences between groups.
A second possible reason for the lack of differences between the groups is that our measures to create a non-shared visual context between the children and examiners may not have been explicit enough.Recall that even very young children understand that a person who was not physically present in a situation will have no knowledge of what took place and that they will adjust their narration to inform the person accordingly.It is generally accepted that the use of a so-called "naive listener" results in more complex and cohesive narratives.It would therefore seem that the listener's physical presence or absence during elicitation procedures is the decisive variable and not whether the visual context is shared or not.This hypothesis can be tested by replicating our study, but with the inclusion of a third elicitation procedure, namely the naive listener condition.
Finally, the small sample size of our study population was a limitation that could have influenced our results.A larger sample may have revealed more significant differences between the elicitation procedures and groups.

Implications of findings
The results of this study have several implications for narrative assessment practices.Recall that we contended that it is often impractical to use a naive listener during narrative assessment procedures.Our aim was to investigate if the absence of a shared visual context between the child and examiner would be sufficient to elicit more complex narratives from TD and LI children.Our results indicate that it made no difference to the participants in our groups if their listeners could see the pictures or not.
The lack of observed differences in the narratives between children with LI and TD children underlines the need for comprehensive assessment practices for diagnostic purposes.Although narratives provide valuable information about a child's linguistic performance at discourse level, it should be combined with assessments that provide more in-depth information about, for example, syntactic abilities.

1.
Do the narratives of both groups of children in the absence of a shared visual context differ from those elicited by a listener with whom they shared the same visual context, in terms of: a. Microstructure i. Quantity (number of words and number of T-units)?ii.Lexical diversity (number of different words and internal state terms)?iii.Syntactic complexity (mean length of T-units)?b.Macrostructure i. Cohesive adequacy (number of complete, incomplete and erroneous ties)?ii.Story structure scores in the Afrikaans version (Klop, Visser and Oosthuizen 2012a) of the Language Impairment Testing in Multilingual Settings -Multilingual Assessment Instrument for Narratives (LITMUS-MAIN) (Gagarina et al. 2012)?

Table 1 .
Means and (standard deviations) for the ages and scores of participants with language impairment (LI) and typical language development (TD) for the Afrikaans Reseptiewe Woordeskat (ARW) and microstructural variables

Table 2 .
Means and (standard deviations)for the scores of participants with language impairment (LI) and typical language development (TD) for macrostructural variables