in the cumulative final exam, several weeks after instruction, as a
more distal capture of student meiosis conceptualization. Following
the method used by Kalas et al. (2013), no partial credit was given
for “partially correct” responses on items for which the expert
response required more than one selected choice (items 4 and 17
in our subset). Cronbach’s alpha coefficient was computed as a reliability estimate of internal consistency for the subset of MCI items
using both the pretest and posttest responses. To investigate
changes in performance, paired t-tests matching individual student
pretest and posttest scores were used in analysis of change on average performance across all eight MCI items and also for change in
performance on each of the items individually. Independent-samples t-tests were used to test for differences in performance on
the MCI between lecture section A and lecture section B. Mean normalized change (c) was also used to investigate the change in performance, calculated as described by Marx and Cummings
(2007). Normalized change calculates the mean of the change from
pretest to posttest, rather than the change in the mean performance
from pretest to posttest. Finally, scores from the unit exam for the
unit that included cell division processes were also used as a metric
of performance for comparison between the two lecture sections,
again using an independent-samples t-test. The unit exam did not
contain any items from the MCI.
Multiple regression modeling was used to investigate possible
predictive variables: gender, ethnicity, incoming GPA, and comprehensive ACT, with pre-MCI score as the response variable. For multiple regression modeling with post-MCI as the response variable,
pre-MCI score was also used as a predictive variable.
In addition to student performance data, we collected data
regarding student perceptions of the learning experience in each
modeling scenario through two Likert-scale items and two free-response items. The Likert-scale items were “Please rate how
strongly you agree with the following statement: Modeling mitosis
and meiosis in class helped me understand the processes of cell
division” and “Please rate how strongly you agree with the following statement: Modeling mitosis and meiosis in class helped me
to understand that genes are physical entities located on chromosomes.” The Likert scale used was 1 = strongly agree; 2 = somewhat
agree; 3 = not sure; 4 = somewhat disagree; 5 = strongly disagree.
The responses to Likert-scaled items for each lecture section were
compared using independent t-test analysis. We elected to use
parametric statistical tests for the Likert-scaled data following the
recommendations of de Winter and Dodou (2010) for five-point
Likert scale data analysis.
The free-response item prompts were “If you agreed that
modeling mitosis and meiosis in class was beneficial to your learning, please explain in what ways the modeling activity was helpful”
and “In what ways do you think the modeling mitosis and meiosis
activity is limited, or could be improved?”
Although rigorous analysis of free-response items was not a
primary goal of our investigation, we used content analysis to
help contextualize Likert-scale responses. To identify emergent
themes in student responses to free-response items, content analysis approaches were used. Responses were iteratively read by
two reviewers and assigned a theoretical category code (Maxwell,
2008, pp. 236–238). Some responses included reference to more
than one theme and were included in more than one reported
category.
Results
Demographics of Study Population & Lecture
Sections
Of 92 students enrolled in the course, 86 consented to participate
in the study (participation rate = 93.5%). Of the participants, 78%
identified as female and 22% identified as male; 71.18% identified
as non-URM (White) and 28.34% identified as a URM (Black,
Hispanic, Asian). The mean comprehensive ACT score for the
study population was 24, and the mean incoming GPA was
2.87. There were 43 students in each lecture section (N = 43 for
lecture section A, N = 43 for lecture section B). There was not a
significant difference in gender distribution between lecture sections (two-tailed Fisher’s exact test, p = 0.6040). Similarly, there
was no significant difference with regard to the distribution of
students identified as URM between lecture sections (two-tailed
Fisher’s exact test, p = 0.1535).
Incoming Performance Metrics
The distribution of incoming performance metrics ( i.e., comprehensive ACT score and entering GPA) was analyzed using independent-samples t-tests. There were no significant differences
between the two sections (sections A and B) regarding comprehensive ACT score (t76 = −0.4945, p = 0.6224) or start-of-term
college (GPA t84 = −1.72911, p = 0.0875).
Item Reliability – Alpha Coefficient
Using student responses to the pretest MCI, the Cronbach’s alpha
was 0.1445. Using student responses to the post-assessment MCI
results in a Cronbach’s alpha of 0.5404.
MCI Pre- & Post-performance
Students improved their performance on the MCI items from pre-to post-assessment. Across both sections, there was a statistically
significant increase in performance (t85 = 9.837, p < 0.0001) with
an average normalized change of 28.85% on the posttest compared
to the pretest. A comparison of performance on the MCI between
the two lecture sections revealed no significant differences on the
pre-assessment MCI (t84 = −1.32309, p = 0.1894) or post-assessment MCI (t84 = −1.12013, p = 0.2659). Similarly, no significant
difference was found when comparing the normalized change (c)
between sections (t84 = −0.62974, p = 0.5306).
When performance on the MCI assessment was evaluated
for individual items, there was a significant increase in average
performance from pre- to post-assessment for items 2, 14, 16,
and 17 across the entire study population (Table 2). For item 1,
there was evidence of significant decrease in performance from
pre- to post-assessment (Table 2). Interestingly, this significant
difference was driven by a decrease in performance on this item
for section A (t41 = −2.89239, p = 0.0061) but not for section B
(t39 = −0.90243, p = 0.3724). Performance for the remaining
MCI items (items 4, 13, and 15) showed no significant differences between the pre- and post-assessment. Comparison of performance between lecture sections A and B on the unit exam (which
addressed mitosis and meiosis concepts but did not include any
MCI items) also showed no significant difference (t84 = −1.50442,
p = 0.1362).
