M-Write, a program built on the premise that students learn complex material by writing, will expand in Fall 2017 to add automated text analysis (ATA) to its digital toolkit.
M-Write helps students develop their conceptual learning and writing skills in large-enrollment gateway courses. It’s another program in the university’s expanding portfolio focused on personalized education.
The first goal of M-Write’s use of ATA is to identify and prioritize the students who need help earlier on, said Chris Teplovs, lead developer at the Digital Innovation Greenhouse in the Office of Academic Innovation.
M-Write was created by Sweetland Center for Writing Director, English Professor, and School of Education Professor Anne Gere and Chemistry Assistant Professor Ginger Shultz.
Writing fellows from the Sweetland Center for Writing discuss the M-Write tool with John W. Sweetland. The corporate leader and former business owner shared with students the importance of writing across professions, which is the basis of the center’s work and is what spurred the M-Write program.
In partnership with the Digital Innovation Greenhouse in the Office of Academic Innovation, and funded by Third Century Initiative, the program is meant to give students in College of Literature, Science, and the Arts and College of Engineering STEM (science, technology, engineering and mathematics) classes a stronger understanding of course concepts that can be applied elsewhere. Students achieve much of this understanding through writing, getting feedback from fellows and faculty on their written pieces, and by conducting peer review of their classmate’s papers.
The program uses writing fellows, students who previously excelled in their given classes. The fellows help students develop initial drafts, make revisions, give feedback to others, and utilize the peer feedback they receive.
New Technology
In Fall 2017, automated text analysis will be used in Statistics 250. ATA will be used to predict the overall score from a student’s piece of writing. After being verified by writing fellows, the ATA-generated score will be distributed to students via ECoach, a personal coaching platform used in large introductory classes.
After students write and submit their essays, ATA is used to evaluate the essay, looking for the qualities of good writing that have been built into the algorithm. These qualities are examined using a variety of text analysis techniques, such as vocabulary matching or topic matching, which the algorithm detects. ATA generates a predicted score which is sent to ECoach for a writing fellow to verify. After this verification, the score will be made available to students.
“The writing fellow will serve as a checkpoint between [ATA] and the students,” said Dave Harlan, principal developer of M-Write at the Digital Innovation Greenhouse.
M-Write will also be integrated into ECoach by sending messages about what makes for a good peer review and what makes a good revision of an essay, and by adding it to the to-do list that is built into the coaching program.
Another key implementation of automated text analysis into M-Write is that the ATA will identify which components of a given rubric the essay submission is strong or weak in. For each section of the rubric, ATA will generate a numeric score, and each of these numeric scores contribute to the overall predicted score.
“We’re hoping to provide a second rater on the essay, providing that other set of eyes on an essay,” Teplovs said. “The verification to us is very interesting because it not only gives the human graders a moment to pause and reconsider their assessments, but more importantly it provides a direct feedback loop to the algorithm development and allows us to creates a better one.”
Not only does M-Write draw from a writing-to-learn pedagogy, but it comes with a potential for graders and professors to identify areas of improvement within their teaching.
With one of the prompts for a course, the Ph.D. students analyzing the essays noticed a jump in essay quality between semesters. Based on the professor’s review of the first set of essays, he modified his approach to teaching the topics. This, Harlan said, may have contributed to the improvement of the next round of essays.
“Overall our goal is to improve student learning and a corollary of that is improving teaching,” Teplovs said.
In the Classroom
Economics Lecturer Mitchell Dudley chose to incorporate M-Write into his Economics 101 course because the writing would allow students to process and assimilate the concepts of economics.
He also noted that M-Write reduces the grading and time commitment issues that come with assigning writing assignments for large classes. Another reason Dudley uses M-Write is because research shows that certain groups of students demonstrate an understanding of concepts more effectively through writing, and he believes economics is a field in which anyone can be successful.
One of the students from Dudley’s class, LSA student Benjamin Rosof, said that the writing complemented the material from class and applied it well to real-world situations.
“Writing has helped me in the past to understand material by allowing me to lay it out in a coherent manner on paper,” Rosof said. “I also think writing is so much more subjective than a class that grades students purely by performance on multiple choice exams, as it allows us to distinguish ourselves and truly show off what we have learned.”
The M-Write program also allows students of the same age—M-Write fellows—to influence their peers. Economics 101 writing fellow and Ross BBA student, Cassandra Wong, noted how this is especially important in large gateway courses.
“As a writing fellow, I hope to influence other students by showing them that huge introductory courses like Economics 101 may seem very daunting—there’s a lot of information to process, and a lot of the concepts are more on the abstract side—but when enough time is taken for understanding the class concepts, it’s really not that bad,” Wong said.
Another Economics 101 writing fellow and Ross BBA student, Brandon Staarmann, explained how this writing represents a key change in how students are exposed to academic concepts.
“Students have to take academic concepts and apply them to real-world situations, and explain the concepts in a way that’s easy for everyone to understand,” Staarmann said. “Doing this helps reinforce the concepts and contributes to long-term retention of the information, and it also prepares students for their careers and life after college.”
A key component of M-Write is giving and receiving peer reviews on work, which is enabled by the automated peer review distribution system. LSA student Elizabeth Stolze said peer reviewing other people’s work allowed her to take another look at concepts and view the same prompts from a different perspective.
Adjustments have been made after each semester to improve M-Write, based off instructor and student feedback. In the long run, program leaders have the goal of reaching more than 10,000 U-M students by 2021.
“The students asked to do more prompts than they were assigned,” said Dr. Laura Olsen, professor of molecular, cellular, and developmental biology (MCDB) and of ecology and evolutionary biology (EEB). “I didn’t have any more prepared, so I asked them to choose their own (difficult) topics and to write a prompt that I could use in the class next year. They were allowed to work in groups to write the prompt, peer review questions, and sample essays. I got 12 new prompts to look at and consider for use in future classes. That is pretty fun for me!”
