As CLIL encompasses the teaching of both content and language, it follows that valid CLIL assessment will also be dual-focused (Llinares et al., 2012; Mehisto & Ting, 2017; Otto, 2017; Lo & Fung, 2020). What is less clear, however, is whether this means that the content and language taught and learned through CLIL should – and can – be assessed together or separately (Otto, 2019).

Some practical publications (e.g. Bentley, 2010; Coyle et al., 2010; Dale & Tanner, 2012) aim to help CLIL teachers identify the focus of assessment and to decide whether the emphasis should be on content, language or other related skills. They suggest that distinctions might also be made between formative and summative assessment, whereby content assessment alone is linked to grading, but feedback is provided on language. Other authors have questioned whether the separation of language and content in assessment is desirable considering the integrated goals of CLIL (e.g., Llinares et al., 2012; Mehisto & Ting, 2017; Morton, 2020). They suggest that language and content should be assessed together, as language is an integral aspect of subject content.

Calls for integrated assessment reflect recent thinking on CLIL from a disciplinary literacies perspective, which highlights that the content, language and cognitive processes involved in a subject are inextricably linked (e.g., Dalton-Puffer et al., 2018; Coyle & Meyer, 2021), or, as Morton (2020) formulates it: “Learning a school subject […] means being able to comprehend and produce the types of texts or genres (both oral and written) through which knowledge in the subject is communicated” (p. 9). In an integrated model of CLIL assessment (Llinares et al., 2012), this inextricable relationship between content and language is acknowledged at all stages of the learning process. The tasks and subject-specific language used in assessment will reflect those encountered in the classroom, so that learners know what is expected of them and how they will be evaluated.

Sato (2023) addresses the variety of perspectives on assessment in CLIL by focusing on the purposes of assessment. He discerns three types of CLIL assessment: ‘separate’, ‘weakly integrated’ and ‘strongly integrated’. Sato emphasizes that, while separation of language and content (e.g. through standardized language tests and content-focused testing in the first language) can be useful for research purposes, it does not align with the integrated goals of CLIL and is therefore a less logical choice for assessment in teaching practice. Weakly integrated assessments mitigate potential interference from lower L2 proficiency by managing linguistic demands, for example through contingent scaffolding during oral interaction or reducing the amount of active language production required in written tasks. This, Sato suggests, can be useful for diagnostic or formative assessment of content learning. For summative assessment that is aligned with CLIL’s integrated goals, however, Sato recommends a strongly integrated approach in which language is regarded as an aspect of the subject content. He argues that, if teaching introduces and supports the learning of subject-specific language as integral to subject learning, it follows that assessment should also be integrated.

While Lo and Fung (2020) agree that integrated assessment approaches are the best fit with the dual goal of CLIL, they also point to the fact that “CLIL assessments should avoid underestimating students’ content knowledge due to their inability to express their understanding with appropriate language” (p. 1194). Whereas Sato (2023) proposed to mitigate this interference between language proficiency and content-related performance in diagnostic and formative assessment only (i.e. ‘weak integration’), others suggest that support or ‘accommodations’ can also help increase the validity of summative assessment (Coyle et al., 2010; Lo & Lin, 2014; Otto, 2017) by “[helping] students access the content in English and better demonstrate what they know” (Butler & Stevens, 1997, p. 5). As defined by researchers in non-CLIL multilingual education settings, accommodations can be embedded in the assessment format (e.g. visual aids, text modifications, glossaries, translations) or made available as part of the assessment procedure (e.g. providing explanations on request; access to support materials such as dictionaries or online materials) (De Backer et al., 2019a; Yang, 2020). Accommodations have been identified as both helping learners and strengthening the validity of assessments (De Backer, 2020; Lo & Lin, 2014), although which types of accommodations are most effective appears to vary between learners and situations. De Backer et al. (2019a) therefore recommend the use of a range of accommodation strategies. Examples of accommodations that have been identified as both helpful to learners and beneficial in strengthening the validity of assessments are adaptations of texts to make them more accessible; providing aural input alongside written texts; and providing access to support materials such as glossaries or dictionaries (De Backer et al., 2019a; Lo & Lin, 2014).

A specific accommodation that can be used to balance the linguistic and cognitive assessment demands, is ‘translanguaging’: providing room for learners to use multiple languages in assessment and/or response formats (De Backer et al., 2019a; De Backer et al., 2019b). Translanguaging can support the learning of both content and language (Cenoz, 2017) and has been found to be an effective tool in minimizing interference of language in the assessment of subject content (Serra, 2007). In this sense, L1 support could be a helpful tool in what Sato (2023) terms ‘weakly integrated’ CLIL assessment where the main focus is on content outcomes. It is important to note, however, that learners do not always perceive choice of language in assessments as helpful, as identified in both CLIL and other L2 content classrooms (De Backer, 2020; Hönig, 2010). As De Backer et al. (2019a) observed, individual factors such as L2 proficiency and personal preference can influence the extent to which learners perceive translanguaging in assessments as useful.

Research on assessment practices in CLIL contexts is scarce. From the few studies that are available, it seems that CLIL teachers have a preference for separating language and content in assessments, but that they do not always achieve this in practice. Hönig (2010), for example, observed a contrast between teachers’ conviction that they focused only on content in CLIL assessment, and their actual assessments, which appeared to be influenced by linguistic aspects such as fluency and length of responses. Similarly, Otto and Estrada (2019) observed that content teachers found it difficult to ignore language in assessment, but at the same time did not feel equipped to assess it. These studies highlight how language can be an “invisible” criterion in CLIL assessment (Llinares et al., 2012, p. 284). Furthermore, the preference among CLIL teachers to take the same approaches to assessment with CLIL groups as they do with non-CLIL groups (Otto, 2017; Reierstam & Sylvén, 2019), namely with little attention for language, also seems to suggest that CLIL teachers may be insufficiently aware of the role language plays in assessment. According to Lo et al. (2019), an invisible role for language in CLIL assessment is not only a threat to fairness and validity, but can also affect the extent to which these aspects are integrated in CLIL teaching. They noted that a lack of explicit attention for language in assessment can have a washback effect, leading teachers to neglect the linguistic aspects of their subject in classroom teaching.

Multiple studies from CLIL contexts have shown that lack of L2 proficiency can hinder CLIL learners in the expression and demonstration of their content knowledge and skills, especially in the earlier stages of L2 development. Oattes et al. (2020) observed that learners in their first year of CLIL performed better in a multiple-choice history assessment conducted in Dutch (the main school language) than in a comparable assessment conducted in English (the CLIL language). This effect was not observed among CLIL learners in their third year, suggesting that learners with higher L2 proficiency coped better with the assessment’s demands. Likewise, Lo and Fung (2020) found that students’ performance in CLIL assessments declined with increasing linguistic demands. The latter study furthermore showed that, in CLIL assessments for science and biology in upper secondary education, both the cognitive and linguistic demands were generally high. In light of their findings pertaining to students’ performance, the authors warn that this situation is problematic as the high linguistic demands may prevent students with lower L2 proficiency from demonstrating their understanding of subject content. This also reflects evidence from non-CLIL contexts where limited mastery of the instructional language was found to negatively influence students’ performance on content exams (Trenkic & Warmington, 2019).

Little CLIL research has explored the potential of accommodations to mitigate interference between language and content in assessment. Regarding the presence of accommodations in CLIL assessment, while around two-thirds of the assessment materials analysed by Otto (2017) contained visual or pictorial support to limit language demands, only a quarter used graphic organizers to support language production, and none made use of glossaries or other types of linguistic support. To the best of our knowledge, there are no studies that explore if and how translanguaging can support CLIL assessment. This might be related to the fact that in some CLIL contexts, including the Netherlands, there is a strong conviction among schools and teachers that communication in CLIL should take place exclusively in the CLIL target language (Oattes et al., 2018), based on widespread misconceptions about multilingual language development (van Beuningen & Polišenská, 2019). That said, Hönig (2010) observed that, when students were offered a choice of language in which to complete an assessment, they all opted for the language in which they had been taught (English) rather than the main school language (German). This could corroborate Coyle et al.’s (2010) argument that it is inconsistent to assess learners in a language other than the one in which they have been taught. Considering the arguments presented above from other, non-CLIL, multilingual contexts, however (e.g. De Backer et al., 2019a), it seems that offering L1 translations or other materials as optional support during content-focused assessments is potentially helpful for some learners.

As explored above, based on the principle of constructive alignment (Biggs, 1996) and on the theoretical recommendations for CLIL practice, it could be assumed that (a) the integrated goals of language and content learning would translate into integrated approaches to assessment in CLIL contexts; and (b) scaffolded approaches as used in CLIL teaching would feed into the use of accommodations and translanguaging in assessment, to balance the cognitive and linguistic demands of assessment tasks. From the scarce research carried out in CLIL settings so far, however, it appears that practices in both of these regards are varied and that teachers are not always aware of the interaction between content and language in the way they assess learners’ performance. With this in mind, the current small-scale study explored ways in which summative assessment is approached in the first three years of Dutch-English bilingual secondary education (BSE) in the Netherlands, with the aim of increasing our understanding of CLIL assessment in this specific context.

Whereas CLIL is often associated with teaching and learning in non-language subjects, such as biology, history or art, many teachers of English in bilingual education (TEB s) in the Netherlands identify both as language acquisition specialists, and as content specialists in literature and language arts (Dale et al., 2018). This could lead to TEB s and teachers of other subjects (ST s) having different and possibly complementary perspectives on assessment, although we consider them all CLIL teachers. Therefore, although it was not the main goal of the current study to compare the assessment practices of ST s and TEB s, we analysed these groups’ practices separately.

The research question guiding this study is: How do ST s and TEB s in bilingual secondary education in the Netherlands approach summative assessment of language and content?

The answer to this question was sought in relation to the themes discussed in the preceding section, namely the extent of content and language integration; the interplay between cognitive and linguistic demands; and the use of strategies to balance these demands (i.e. accommodations and translanguaging).

The research took place in the context of lower secondary (learners aged 12-15) BSE in the Netherlands. BSE (known locally as tweetalig onderwijs or TTO) has existed since 1989 and has followed a national quality standard and accreditation scheme since 2003. At the time of writing, 134 schools offer BSE, amounting to about 21 % of the country’s 648 secondary schools (Mearns et al., 2023). BSE schools generally follow the same curriculum as mainstream schools, but offer a significant proportion of that curriculum in English. For pre-vocational (or vmbo) learners, the minimum proportion of English-medium teaching is 30 % in years 1-2, and for learners in the higher general (or havo) and pre-university (or vwo) tracks it is 50 % in years 1-3. By the end of their third year in BSE, learners are expected to reach A2 (pre-vocational/vmbo), B1 (higher general/havo) or B2 (pre-university/vwo) level English (according to the Common European Framework of Reference for Languages, CEFR). English-medium provision is spread across different subjects, so that learners develop L2 proficiency in a range of disciplines. In the senior years, most subjects revert to teaching in Dutch, in preparation for the centralized final examinations. The English-as-discipline curriculum in the senior years of BSE is enhanced with a content-oriented international examination program, in most cases the International Baccalaureate (IB).

The study combined two stages of data collection: a survey aimed at painting a broad picture of teachers’ (N = 42) self-reported summative assessment practices, and an in-depth analysis of example assessments (N = 17) submitted by thirteen teachers. The participants, instruments and analyses pertaining to each stage are described below.

We now move on to the results of the qualitative analysis of the assessment bundles submitted by teachers. For an overview of the contents of the bundles, see Table 1 (Method section). As with the questionnaire, the results will be presented under each of the three main analytical themes: roles of content and language, accommodations and translanguaging, and cognitive and linguistic demands.

It is perhaps not surprising that TEB s placed more emphasis on language in assessment, and ST s on content. This trend was apparent in both the self-report data and in the grading and feedback practices represented in the assessment bundles. This reflects earlier findings suggesting that teachers’ disciplinary identity influences the focus of their assessments (Otto & Estrada, 2019).

The content observed in TEB s’ assessment materials was not always linked to subject-relevant learning objectives (e.g. belonging to the fields of literature, cultural studies or linguistics), but in some cases was used as a vehicle to assess a linguistic goal (e.g. ‘appearing knowledgeable’ when giving a speech). A complexity that we observed when analysing the materials provided by teachers was the position of creative writing. Teachers identified content criteria in creative writing exercises, although in two of the three examples of this, it was not clear whether and how those criteria aligned with prior learning related to literary competence, or if they were extensions of learners’ writing skills. Literature on CLIL rarely refers to creative writing, and when it does, it tends to be positioned as a means of processing input (e.g. exploring a scene from a play, as in Dale (2020)) or of practising writing skills (e.g. Weiss et al., 2023), rather than of demonstrating literary competence as a form of content. That teachers in this study appeared to consider creative writing to have a content goal, although that goal was not always aligned with our understanding of disciplinary content, highlights the lack of clarity regarding what constitutes disciplinary content in the language classroom.

When ST s assessed language, they appeared to pay most attention to lower-order aspects of language (e.g. spelling, grammar, punctuation, pronunciation) or to unspecified language “use”, in spite of assessments being linguistically and cognitively demanding. While self-report data suggested that subject-specific language use was sometimes a focus of grading and feedback, this was rarely reflected in the assessment bundles. This echoes earlier findings from the Dutch CLIL context (Busz et al., 2014), where teachers provided micro-level feedback similar to that found in the current study, as opposed to feedback on subject-specific communication. Our findings might suggest that the increasing emphasis on disciplinary literacies in CLIL (Coyle & Meyer, 2021) as key to subject learning is not yet reflected in teaching practice, at least in terms of assessment. As Lo et al. (2019) emphasise, this could limit CLIL’s effectiveness in terms of the development of subject-specific language, as washback from assessment could lead ST s to neglect the linguistic aspects of their subject in their teaching. Furthermore, in line with earlier findings by Hönig (2010), analysis of the assessment materials and self-report data suggest that language does play a role in ST s’ assessment of content, even when it is not identified as a goal of assessment or mentioned explicitly in instructions. As identified by Lo and Fung (2020), this could lead to underestimation of learners’ content knowledge or to learners being penalized on the basis of “invisible” language criteria (Llinares et al., 2012, p. 274).

CLIL teachers in this study reported that they allow room for a range of accommodations to support completion of summative assignments (e.g. projects, presentations), and to a lesser extent for timed tests taken in classroom settings. In the assessment bundles, we saw that learners were allowed to draw on diverse support materials when working on summative assignments, although we found few instances of embedded accommodations such as visual support or text modification, to help learners understand input or instructions, or graphic organisers to support output. This contrasts to findings from a similar exploration by Otto (2017), who found visual support and guided questioning to be the most common forms of accommodation in the CLIL assessments she analysed, although she too observed an absence of support for language production. As earlier studies have suggested that a variety of accommodation strategies is needed to respond to different learner needs (De Backer et al., 2019a), this appears to be an area in which CLIL teachers may be able to support their learners better. As the multiple-choice options in the questionnaire were limited, however, we do not know whether this trend extends beyond the seventeen assessments analysed. Further research could explore this aspect of CLIL assessment in more detail, in order to make more concrete recommendations.

Translanguaging in assessment appeared not to be common practice. This confirms our expectation based on the known tendency for languages other than English to be excluded from CLIL classrooms in the Netherlands (van Kampen et al., 2018). On the few occasions where translanguaging was used to support learners or permitted in learners’ responses, teachers qualified their responses by stating that this was permitted only in the first year. This may reflect Oattes et al.’s (2018) observation that ST s are implicitly aware that translanguaging can be a useful pedagogical tool, and tend to use it mostly with younger learners. In light of Oattes et al.’s (2020) finding that assessing first year CLIL learners’ content knowledge in English can be detrimental to their performance, this might be considered a well-reasoned approach.

As exemplified in the analysed assessment bundles, both cognitive and linguistic demands of content-focused assessment were generally high. This was the case irrespective of year (1, 2 or 3) and educational track (pre-vocational/vmbo, general/havo or pre-university/vwo). Lo and Fung (2020) warn about the risks this combination of high demands can pose to assessment validity due to learners’ developing L2 proficiency, as they found that increased linguistic demands adversely affected learners’ performance in terms of content. In the context of our study where, in contrast to Lo and Fung’s findings, high demands on both axes were found in the earliest years of CLIL, and in light of Oattes et al.’s (2020) conclusion that language can be a barrier to younger learners in CLIL content assessments, the question can be posed as to whether it is fair and valid to assess them in this way. That said, emphasis on disciplinary literacies as integral to content learning emphasizes the inextricable connection between language and cognition in any learning situation (Dalton-Puffer, 2013). From this perspective, higher cognitive demands and higher linguistic demands will likely go hand in hand, but need to be built-up in structured and scaffolded ways (Coyle & Meyer, 2021). Rather than suggesting that teachers lower demands in assessments, it might be better to address the use of scaffolded approaches to teaching and assessment (e.g. using accommodations and translanguaging), in CLIL teacher education and professional development.