A math game for elementary school students about the decimal points, studying the game in education and improve the current game using the datadriven method.
Collaborate with Prof. Bruce McLaren, Huy Nguyen, and Erik Harpstead
[Paper: Student Agency and GameBased Learning: A Study Comparing Low and High Agency]
[Code for the agency research]
[Code for adaptive game research]
Skills and Tools: Python, rStan, R, CTAT
Introduction
Decimal Points creates a scene of the virtual amusement park, which has different themes (like “Pirate Ship” and so on). This game is a game system, which combines many minigames about decimal points. Here I only analyze the high agency version of the game. There is a welcome interface when the students log into the system at the beginning. On the left, there is a dashboard that is showing the domain knowledge targeted at by the different games (like addition, number line and so on). On the right, there are many games within different contexts and themes (like "haunted house," OK Corral" and so on). When clicking on a specific game, the students start the game, like judging whether the number is smaller or more bigger than the number given. After each minigame, there is a multiple choice to help students reflect on the current problem.
[Paper: A ComputerBased Game that Promotes Mathematics Learning More than a Conventional Approach]
Mechanics
Since it is a system, combining many minigames, the mechanics in different minigames are diverging. In general, the game can be chosen by the preference of the students. Within each game, there are two times of playing the same game to ensure the students get enough practice. When the students finish filling in all the blanks, the border of the blank will become green (correct) or red (wrong). After completing the minigame, the students can get back to welcome interface and select another game as they like. They can stop after they finish half the games or keep going on to do more sets.
Dynamics
Because there is no time limitation for each game, the students can take time to think about the correct answer and the reason for making mistakes if necessary. Also, because of the design of high agency, the students can choose the game as they like on the dashboard.
Aesthetics
Fantasy. The game system is designed as a virtual amusement park, with "Space Adventure," "Haunted House" and so on. So the students will be more motivated and engaged when playing the game. And there are some virtual friends in the system, who will show up when the students finish the game correctly, which also increase the fantasy features of the game.
Instruction Principles
Selfreflection.
There is an interaction at the end of each game. The narratives of the question are "What advice would you give to another student to solve this problem? To find the next two numbers in the pattern, remember that _____”. In this question, the students are required to select the answer which summarizes the pattern of solving this particular problem in the game. By making their own decision, teachers will learn better on their misconception and students will be more clear by reflecting the pattern in general.

Provide immediate feedback on errors. All the feedbacks in this game is immediate. The border of the text box will show the correctness of the answer immediately after the students fill in all the blanks.

Discovery Learning. Even though the feedback is only correct or wrong, the students can know the result at the first time and try to figure out the real answer by themselves. The color of the error shows the students which parts are right and which are wrong.
Synthesis and Critique
According to the MDA framework, the mechanics support the dynamics and the aesthetics of the game. For example, the system is to help students get more accuracy when trying to solve the decimal games. So there is no time limitation. Students can explore by themselves without the pressure of time. What's more, educational objectives are also important for the game design. Since the goals of this game are to reflect on the pattern of solving out the problem, the students get scaffolding by selecting the proper one from the existing options. AAlso, more practice is followed when the students made mistakes on the previous questions, which is the demonstration of testing (instruction principle). And this corresponds to the educational objectives of increasing accuracy and fluency.
I think there are some possible improvements based on the current system. Instead of only offering the right or wrong result, the system can explain the reason for a good timing. For example, students might explore at the beginning. But if the system detects that the students are trying to guess out the answer, which indicates the difficulty level is indeed exceed their ZPD, they probably need more explicit expression on the specific problems.
What's more, the options in the selfreflection question could be updated by collecting more possible misconceptions, if there is one openended question about "why are you wrong about this question at the beginning." As the system running for a period, the answer of students can be collected and updated to the current list of the options.
Agency Study
This is the first project I worked in the lab. A key feature of most computerbased games is agency: the capability for students to make their own decisions in how they play. Involving a total of 158 fifth and sixthgrade students, children played a mathematics learning game called Decimal Point, which helps middleschool students learn decimals. One group of students (79) played and learned with a lowagency version of the game, in which they were guided to play all \minigames" in a prescribed sequence. The other group of students (79) played and learned with a highagency version of the game, in which they could choose how many and in what order they would play the minigames. The results show there were no significant differences in learning or enjoyment across the low and highagency conditions. A key reason for this may be that students across conditions did not substantially vary in the way they played, perhaps due to the indirect control features present in the game. It may also be the case that the young students who participated in this study did not exercise their agency or selfregulated learning. The paper was accepted by the AIED Conference.
My main work in the team is the statistical analysis after the student data was collected. First filtering out the outliers above or below the mean of 2.5 standard deviations. Then ee tried ANCOVA with pretest scores as the covariate, examine the difference of posttest scores and delayed posttest scores for the high agency group and low agency group. Also, we conduct the repeated measurement for each student to assess their learning gaining. What's more, we conducted ANOVA to assess the difference between the high agency and low agency groups about the lesson enjoyment, ease of system using and math efficacy.
Adaptive Game Design
So far we looked into the student agency but does the pure student control always work well? My current research (independent study) is under the advising of Prof. Bruce McLaren. In my study, I will change the fixed order of game into the adaptive version and compare the student control and the system control in the learning effect and game enjoyment.
Exploration and Data Analysis
I use the data collected form the agency study and conducted the data analysis based on that. First, I extracted the knowledge components from the different questions, labeling them with the specific types of questions, like the "addition", "sorting", "sequence" and so on. I used the Learner Factor Analysis to construct the model for the different students and predict the probability of getting a certain problem right.
I set the mastery level (the probability of getting a problem right) to be 85%. And I find that some students still practiced on the questions related to the same knowledge component after they reached the mastery level; while some others went through many questions as the peers do but still below the mastery level.
Adaptive Policy
Since the students are different in the learning status of a certain knowledge component, I am thinking about identifying the knowledge they are not mastering yet and lead the learner to those problems, which will make the learning process more efficient.
To identify the reason that makes the learners result in different performance for different questions, I conducted the Bayesian Factor Analysis and construct the model as follow.
X ~ N (mu, var_p)
mu = b + lam * FS
b ~ N (0.5, 1)
lam ~ N (0, var_p)
FS ~ multi_normal_cholesky(0, Ld)
var_p ~ cauchy (0, 2.5)
Ld = diag(sd_d) * cholesky_decompose(Rho)
* Rho is the corelation matrix between the factors.
I identified five latent factors that influence students learning. Here is the example of the factor scores for ten students.
My hypothesis is the five factors are related to the five misconceptions designed at the beginning. The questions were designed to be targeted at differnet misconceptions. For example, Regz means the decimals treated as regular numbers; the Pegz means two sides of decimals viewed as separate numbers; Megz means the longer decimals are larger; Negz means decimals smaller than 1 are treated as negative numbers; Segz means shorter decimals are longer (more detail please see the previous study here). So I ran a correlation analysis between the scores for the questions related to the five misconceptions (Negz, Megz, Segz, Pegz, Regz) and the factor scores. And I find the positive correlation between them.
Next Step
I will continue the research in the summer semester with professor Bruce. The next step will be the consideration about the adaptive policy not only about the knowledge components but also about the enjoyment of the game. After identifying the policy, the algorithm will be applied into the system and the empirical research about the comparison of the learning effects and lesson enjoyment between the system control (adaptive) and pure learner control.