Validation of the DICAT Instrument for Assessing Motor Creativity in Dance Improvisation Tasks

Elena Pérez-Calzado

Cristina Calvo-Estelrich

Carlota Torrents Martín

Javier Coterón

Abstract

This study aimed to design and validate the Dance and Improvisation Creativity Assessment Tool (DICAT), an instrument to subjectively assess movement creativity in dance improvisation tasks performed by individuals without any specific dance training. The research was conducted to evaluate components of creativity in dance for which, to date, there is no specific instrument. We created an instrument to allow users to analyze both quantitative and qualitative aspects, addressing the dimensions of fluency, flexibility, originality, elaboration, and aesthetics. The instrument development and validation occurred across four phases: (a) initial design based on a literature review, (b) validation of the content by a panel of experts, (c) analysis of inter-observer reliability using the weighted kappa coefficient, and (d) criterion validation calculating the correlation with external measurements of motor skills diversity. The results show substantial validity, which indicated high reliability and consistency with external criteria. The DICAT is an innovative, valid, and reliable instrument for evaluating motor creativity in improvisation, offering a comprehensive approach suited to the specific characteristics of this field. This instrument has important applications in both research and education, offering a methodology that can be replicated and adapted to diverse contexts that promote creativity through movement.

Introduction

Guilford (1950) defined creativity as an ability to practice divergent thinking, an essential pillar in human development that is characterized by the production of original, innovative, and useful ideas. He differentiated between creativity and conventional intelligence, opening a line of research focused on divergent thinking as a process aimed at searching for innovative solutions (Runco, 2004).

Motor creativity, understood as an individual’s ability to respond in an appropriate, diverse, original, and unique way to a motor skills-based situation or problem (Murcia, 2001), constitutes a field of study of growing relevance within the Physical Activity and Sports Sciences. This ability to generate novel and adaptive motor responses is also a key performance factor in sports. Its study is particularly relevant in sports context where interaction with the environment and decision-making time play a decisive role in skill development and in problem solving, as well as in creative-expressive disciplines (Araújo et al., 2006).

There are multiple creativity evaluation tests that analyze the ability to generate ideas, such as the Remote Associates Test ( RAT; Mednick, 1962), the Consensual Assessment Technique (CAT; Amabile, 1982), or the Runco Ideational Behavior Scale (RIBS; Runco et al., 2001). One of the most common is the Tests of Creative Thinking (TTCT) created by Torrance (1966). This assess four dimensions of creativity: fluidity, understood as the ability to generate multiple responses to the same problem; flexibility, associated with the diversity of the solutions or ideas produced; originality, which evaluates the ability to produce innovative and unconventional ideas; and elaboration, which involves the amount of detail, complexity, and development of the ideas. These dimensions have been used as the theoretical research foundation in the Physical Activity and Sports Sciences field, with the main objective of assessing motor creativity in different disciplines (Canton et al., 2020). Specifically, this research has focused on studying the exploratory behavior of the individual (Hristovski et al., 2011), defined as the diversity and variability of their motor actions—understood as movement with a specific aim (for examples of the concept of motor actions, see the Annex)—produced by a complex system in a dynamic setting. On the other hand, Richard et al. (2020) developed the PLAY Creativity instrument for assessing motor creativity in children from grades 4 to 6 by measuring the following variables: fluency, originality/imagination, appropriateness, and flow.

In addition to specific tests, some sports research assesses the components of creativity in real-life settings. The following are some of the methods used to analyze variables of motor creativity taking into account the complexity of the system behavior: (a) Multiscale Entropy Measures (MSE), which assess the variability of the behavior; (b) Dynamic Overlap, which assesses the fluency and flexibility of motor behavior; and (c) Tucker’s congruence coefficient, which evaluates the level of similarity between patterns and provides information about their originality (see Canton et al., 2022). 

These methods have also been used to analyze motor creativity and exploratory behavior during dance improvisation tasks (Aragonés et al., 2021; Pérez-Calzado et al., 2024; Torrents et al., 2010). In the creative-expressive dance context, improvisation tasks represent the ideal setting to stimulate creativity and divergent thinking (Blom and Chaplin, 1988) and allow individuals to explore novel and spontaneous solutions during unpredictable situations, thus facilitating a continuous and inseparable interaction between the environment, the body, and the mind (Richard et al., 2021). This process stimulates not only the production of original movements, but also motor adaptability, which has emerged as a key element for enhancing creativity (Lewis and Lovatt, 2013).

All of the aforementioned research used observational instruments with categories (e.g., moving through space, levels, actions performed with the body, etc.) and subcategories (e.g. the “moving through space” category is subdivided into walking, running, crawling, rolling, etc.) that are specific to the field of dance. This makes it possible to quantify the number and types of motor patterns that a person performs during an improvised dance.

While these instruments have been useful for analyzing quantitative aspects, such as motor fluency or diversity, they do not fully capture the richness and depth of the creative process, which involves subjective variables. For example, originality, elaboration (difficulty level of the proposed movement), or aesthetics (ability of the performer to create visual and emotional impact on the spectator through expressiveness, coherence, and harmony among the sequences), (Coterón et al., 2008) are key aspects of creativity that cannot be assessed solely using quantitative criteria. Though prior research has shown that objective kinematic parameters significantly influence aesthetic perception (Coterón et al., 2008), and consequently the assessment of motor creativity, other aspects such as the quality of the movement cannot be systemically quantified. 

In this regard, specific tests have been created to evaluate motor creativity in dance using subjective variables. Brennan (1982) designed the Creative Movement Composition Test based on the Structure of Intellect by Guilford (1957), which consists of three components: operations, contents, and products. Brennan developed three tests based on those components: a) Position Tests, which evaluates originality, with an 8-item checklist for body positions; b) Composition Test, which measures the number of original movements performed by a person during the construction of a sequence based on four pre-established body positions; and c) Improvisation Test, which evaluates the number of new movements the person performs during an improvised dance with the constraint of keeping one foot on the ground. The research was conducted with 60 college dance students who had received less than one semester of training. Both this test and the TTCT (Torrance, 1966), considered a benchmark in overall creativity evaluation, have contributed significantly to the development of instruments for analyzing both creative thinking and creative expression, including their application in the field of movement. On the other hand, Pürgstaller (2020) validated the Creativity in Dance Test (CDT), analyzing the fluency, flexibility, and originality of movements in children from grades 3 to 6. 

Some research has also used previously validated tests to evaluate creativity in the field of dance. For example, Clements et al. (2018) used the CAT instrument (Amabile, 1982) to analyze overall motor creativity (without distinguishing specific variables) in choreography interpreted by contemporary dance students. 

Other instruments evaluate creativity in other creative-expressive arts, such as Body Language. In this case, Aranguren and Irrazabal (2012) designed the Evaluation of Creativity Behavior (ECC by its Spanish acronym) across different areas, including the Body Language area which includes the dance and theater disciplines. The ECC assesses creativity through items (e.g., “has performed a dance choreography” or “has attended dance classes”) based on the number of times (Never, Once or Twice, Sometimes, Often, Very Often) a person performed said actions throughout their life. On the other hand, Méndez-Martínez and Fernández-Río (2019) validated the Instrument to Measure Motor creativity (ICM). In this case, the ICM assessed the motor creativity in adolescents during a Body Language task, bearing in mind the variables of fluency, flexibility, imagination, and originality.

Considering the above, an instrument needs to be developed for the rigorous and comprehensive assessment of motor creativity in dance, which takes into account subjective variables such as elaboration and aesthetics that have not been addressed in the previously discussed instruments. These qualitative variables require subjective evaluation based on expert opinion and perception, as experts notice expressive and aesthetic nuances that elude quantitative measurements (tallies, measurements, etc.) Unlike objective evaluations, subjective evaluation allows for a more holistic and contextual interpretation of motor creativity. Additionally, we found no other validated instruments for evaluating motor creativity in adults without specific dance training, which highlights the need to develop an adequate instrument for this. Therefore, the aim of this study was to design and validate an instrument for the subjective assessment of creativity to facilitate the observation of improvised motor actions performed by individuals without specific dance training and to assess their level of creativity according to the variables of fluency, flexibility, originality, elaboration, and aesthetics. This instrument seeks to overcome current challenges and provides a precise and replicable methodology, contributing to enriching both arts research and educational programs that promote creativity through movement.

Methodology

Participants

We contacted professionals to join the panel of experts for the design and validation of the Dance and Improvisation Creativity Assessment Tool (DICAT). The established inclusion criteria were as follows: a) university professors; b) who conduct research in the field of creativity, dance, and Body Language; c) with at least 10 years of experience. Finally, seven experts collaborated (five women and two men) who were university professors with over 20 years of experience in the field of creativity, dance, and Body Language. Together with the principal investigators, they offered their knowledge during the first phase of the instrument’s evaluation. All the experts signed an informed consent form to participate in the study. The Catalonian Sports Administration Clinical Research Ethics Committee (09-2018-CEICGC) approved the study.

Instrument Design

The DICAT was designed using IBM Excel software (version 2411). Considering the five creativity variables we aimed to measure (fluency, flexibility, originality, elaboration, and aesthetics), we created five items to be evaluated on a 5-point Likert scale, where 1 = “very little” and 5 = “a lot.” Similarly, to explore in-depth the subjective assessment of the observer and supplement the quantitative assessment, we decided to include two open-ended questions. Both the items and the open-ended questions can be seen in Table 1. 

Table 1

Items and open-ended questions comprising the DICAT

See Table

Instrument Validation

Criterion and content validity (Cronbach and Meehl, 1955; Messick, 1989) were considered when validating the instrument. On the one hand, the content validity is aimed at qualitatively determining whether the instrument truly measures what it was designed to measure. In this study, we sought out the opinion of experts in the field when selecting and evaluating the creativity variables. On the other hand, the criterion validity was addressed, which involved comparing the results obtained from using the assessment instrument with an external criterion that aimed to measure the same construct (Thomas & Nelson, 2007), with the goal of ensuring there were no significant differences between both measures. In this case, we checked the correlation of the observation with quantitative data obtained from the study conducted by Pérez-Calzado et al. (2024), analyzing exploratory behavior through systematized observation of the same observation material (recordings of improvised dances by people without specific dance training).

Procedure

The instrument design and validation process took place across four phases following the procedure developed in similar research (Conejero et al., 2016; Sánchez-López et al., 2023): (a) literature review and provisional instrument design, defining five items to measure the five dimensions of creativity and two additional open-ended questions; (b) content validation based on the opinion of experts and modification of the initial version of the instrument; (c) interobserver validation of the instrument through two researchers’ observational analysis of improvised dances; and (d) criterion validation through comparison with the systematic observation.

In the first phase, we conducted a literature review of the study of creativity and research in the field of dance so as to theoretically justify the design of the instrument. We then started on the initial design of the provisional instrument.

In the second phase, we contacted a panel of experts and calculated the expert competence coefficient (k) to ensure their adequate selection. To this end, each professional was given a questionnaire containing a self-assessment of their expert competence (Cabero & Barroso, 2013)  in relation to the research subject. The k coefficient was calculated through the semi-sum of two coefficients taken from the questionnaire: the coefficient of knowledge (kc), which reflects the expert’s level of understanding of the research problem, and the coefficient of argumentation (ka) which indicates the evidence or sources the expert used to back up their opinion. The k coefficient is expressed as a score between 0 and 1 and divided into three levels: high (k > .8), medium (k = .7-.8), and low (k < .7). The final panel of experts comprised seven members who obtained a mean optimal k coefficient (M = .95; SD = .05; see Table 2).

Table 2

Coefficient of knowledge, coefficient of argumentation, and coefficient of expert competence obtained from the expert opinion

See Table

After creating the panel, the experts assessed the items and open-ended questions for: (a) clarity, clear writing, and unambiguous understanding of the meaning  (yes/no responses); (b) importance, level of importance, and meaningfulness of the assessed item using a 0-3 Likert scale, where 0 = lowest level of importance and 3 = highest level of importance; (c) relevance, suitability for the objective of the instrument (yes/no responses); and (d) sufficiency: is the item adequate for assessing the objective, or should it be added or eliminated? (sufficient/add/remove responses). For each item and open-ended question, there was a section for justifying the given score and for taking notes or offering relevant suggestions for improvement. The experts’ assessments were considered for the modification and improvement of the instrument, thus achieving content validation.

In the third phase, the instrument’s inter-observer reliability (Cohen, 1960) was calculated. To do this, two expert dance and body language researchers used the instrument to assess the creativity of twelve dance improvisations performed by individuals without any specific dance training. After observation, agreement was calculated by comparing the data through the weighted kappa (Kw) coefficient, an extension of Cohen’s kappa designed for evaluating the level of agreement between observers in categorical variables with a hierarchical order (Cohen, 1968). Unlike the standard Cohen’s kappa, Kw assigns different levels of varying gravity to disagreements according to their magnitude on the scale (Cohen, 1968). Using it in this research allowed us to more precisely capture the intensity of the disagreements in the assessment of creativity in the dance improvisations, wherein the instrument scores are of an ordinal nature. This provides a nuanced perspective of the level of agreement between the raters. 

In the fourth phase, we calculated the criterion validity of the instrument for the component for which objective data were already available, specifically flexibility. We compared the researchers’ observations from the third phase with the systematic observation conducted in the study by Pérez-Calzado et al. (2024). This research analyzed the flexibility variable (q_stat) using the tool used by Aragonés et al. (2021), which was adapted from the original research by Torrents et al. (2010) and Torrents et al. (2015). This variable systematically and precisely quantifies diversity in the movement patterns (for more information see Hristovski et al., 2013). 

To calculate content validity, we looked to expert opinion, as described in the procedure’s second phase. Interobserver agreement was analyzed in the third phase using IBM Excel software (version 2411) to calculate the Kw (Cohen, 1968) coefficient. To interpret the level of agreement obtained, we used the scale proposed by Landis and Koch (1977),, in which a value of less than 0 indicates “poor agreement,” 0-.2 indicates “slight” agreement, .21-.40 indicates “fair,” .41-.60 indicates “moderate,” .61-.8 indicates “substantial” agreement, and values exceeding .81 represent “almost perfect” agreement. 

Lastly, in the fourth phase of the procedure, we used Pearson’s correlation to determine the criterion validity of the instrument, relating the mean ratings the researchers obtained through DICAT for the flexibility variable with the q_statratings from the systematic observation conducted by Pérez-Calzado et al. (2024).

Results

The results of this research are described below according to the phases described in the study methodology. 

Content Validation 

The results of the content validation by the experts can be seen in Table 3. In terms of clarity, when the experts deemed the initial wording of certain items to be unclear (item 4, item 5, open-ended question – a), those items were modified. All the experts deemed both the items and the open-ended questions to be important to the study subject (M = 2.57; SD = 0.60). In terms of relevance, 100% of the experts deemed items 1 to 4 to be relevant. In addition, 86% of the experts considered item 5 and the open-ended questions to be relevant. Lastly, sufficiency data were considered to modify the number of items. The final instrument can be seen in the article annex. 

Table 3

Results from the panel of experts for each item

See Table

Inter-Observer Reliability

Two researchers used the expert-validated version of the instrument to evaluate the improvised dances of twelve individuals without specific dance training. An optimal Kw index (Kw = .781) was obtained, indicating a substantial level of agreement between the two observers, very close to the level considered “almost perfect,” thereby supporting the reliability of the instrument. 

Criterion Validation

Table 4 shows the researchers’ scores for the observations according to the flexibility section of the DICAT, compared with the q_statvalues from the systematic observation. The parameter q determines the structure of the behavior and its dynamic properties. q_stat is the stationary value obtained by analyzing the dynamics of a time series and is a measure for quantifying exploratory behavior (fluency and variability of behavior). Note that the q_statvalues range from 0 to 1, where 0 represents a completely erratic and diverse behavior and 1 represents completely repetitive behavior. The correlation obtained between them was -.71, indicating a strong correlation.

Table 4

Flexibility scores in the systemic observation and with DICAT

See Table

Discussion

We aimed to design and validate an instrument for the subjective assessment of creativity in dance improvisation tasks performed by individuals without specific dance training and to evaluate their level of creativity across the variables of fluency, flexibility, originality, elaboration, and aesthetics. The resulting instrument, called DICAT, allows expert observers to assess creativity in improvisational dance subjectively.

Creativity in dance has been considered from a quantitative perspective (Aragonés et al., 2021; Pérez-Calzado et al., 2024; Torrents et al., 2010), focusing on objective variables such as the number of times a movement pattern is repeated, the speed of transitions between patterns, or the diversity of patterns. However, dance specifically, and the creative-expressive disciplines in general, promote the constant generation of diverse, unique, and multi-dimensional motor actions (Castañer et al., 2009), which require tools that allow holistic analysis from a qualitative and subjective perspective. 

To guarantee the content validity of the designed instrument, it was necessary to involve experts in the subject matter to stabilize the responses to each item and conduct a proper analysis of them. In this case, the seven experts’ quantitative assessments were accompanied by qualitative contributions, which are considered essential for the development of an instrument (Subramanian and Silverman, 2000).  These contributions were considered when modifying and perfecting the initial version of the questionnaire, with attention to the dimensions of quality, importance, relevance, and sufficiency. 

In addition to content validation, interobserver reliability was calculated. In that sense, we de(Cohen, 1968). In this case, a Kw index of .781 was obtained, indicating good instrument reliability. This index has been used in other studies, in which values between .61 and .80 were considered “good” (Schorer and Weiβ, 2007). The research results indicate that the instrument is valid and ensures optimal reliability for assessing creativity in dance improvisation performed by individuals without specific dance training.

To obtain evidence of criterion validity, we used as external criteria the results for flexibility in movement patterns from the study by Pérez-Calzado et al. (2024). In this case, there was a correlation of -.71, indicating a strong correlation, which supports the instrument’s validity. The innovative aspect of this research is also its main limitation: the inability to validate the criterion for other studied variables (originality, elaboration, and aesthetics) due to the lack of prior research. However, DICAT has emerged as a valid and reliable tool for assessing creativity and opens the door to future research to validate this instrument in other population groups such as experts in dance, similar arts disciplines, or in different stages of the learning process.

Conclusions

DICAT demonstrates optimal validity values, indicating that it is an effective instrument for assessing motor creativity in dance improvisation among individuals without specific dance training. This instrument allows expert observers to subjectively analyze key aspects of improvisation, such as fluency, flexibility, originality, elaboration, and aesthetics. It provides both quantitative and qualitative data, offering a comprehensive and nuanced understanding of creativity.

Validating this instrument has significant implications for both creativity research and for the field of education, as it can be used to assess creativity in training programs across various educational stages. Likewise, it establishes a methodological foundation that can be replicated and that facilitates the development of valid instruments for assessing creativity in different population groups.

Acknowledgements

The authors would like to express their sincere gratitude to the panel of experts who participated in the validation of the instrument, for their time, dedication, and for sharing their expertise.

This work was supported by the scholarship awarded to Elena Pérez-Calzado under the framework of predoctoral contract grants for doctoral research, with international recognition through the 2023 Programa Propio de I+D+i of the Universidad Politécnica de Madrid, in collaboration with Banco Santander Central Hispano SA. Additionally, the research was funded by the National Institute of Physical Education of Catalonia (INEFC) of the Government of Catalonia, within the program of grants for hiring early-career research staff in the fields of physical education, physical activity, sport, and its applied sciences (PINEFC 2023), under Resolution 10/07/2023 and Order PRE155/23/000001. The study is part of the project “Consequences of Participating in a Body Expression and Creative Movement Program on the Creativity and Well-being of Adults” (Project Code: 2022PINEF00016).

Annex

DICAT
Dance and Improvisation Creativity Assessment instrument
Instrument for evaluating dance and improvisation creativity

Guilford (1950) defined creativity as an ability for divergent thinking that can produce innovative, useful, and unexpected ideas. With that definition in mind, Torrance (1966) developed the Tests of Creative Thinking to measure creativity and identify individuals with creative talent. These tests consider four dimensions: fluency, flexibility, originality, and elaboration. 

Building on these authors’ contributions, this instrument was created to facilitate the observation of improvised motor actions and to evaluate the level of creativity from the observer’s subjective perspective. Motor actions are understood as movement of the body with a specific objective in mind (physical, cognitive, and/or socio-affective) within the context of dance and Body Language. For example, if a person spins, takes multiple fast steps while waving their arms, and then jumps with their arms and legs extended, they would be considered three distinct motor actions (a spin, a movement, and a jump). 

The instrument comprises five components corresponding to the following dimensions: fluency, flexibility, originality, elaboration, and aesthetics. All items are measured using a Likert scale from 1 to 5, where 1 = “Very little” and 5 = “A lot.” 

1. Fluency: the number of motor actions a person completes.
2. Flexibility: the diversity of motor actions a person completes, considering the following: 
   a) use of different categories (moving through space, spins or turns in the three axes of space, jumps, balancing, level changes, etc.).
   b) differences between them, even if they are in the same category (use of different body parts, different rhythms, distinct movement qualities, etc.).
3. Originality: uncommon, new, or unexpected movements performed by a person.
   
4. Elaboration: actions with some level of complexity in their construction and/or performance (complexity defined as the number of segments involved, the number of simultaneous actions, or the difficulty or level of expressiveness of said actions).
5. Aesthetics: the artistic value of the composition, degree of sensory and emotional impact generated in the observer

Rate the aspects listed above in the observed improvisation on a scale from 1 to 5, where 1 = “Very little” and 5 = “A lot”:

Respond to the following questions: 

1. If you believe an original action was performed, describe it and explain what made it original.
   I observed multiple original actions…
   
2. From your perspective, evaluate the creative and artistic quality of the improvisation, explaining with as much detail as possible what brought you to that conclusion.