At the end of the Fall 2020 semester we asked Yellowdig instructors to share a survey with their students, as well as fill one out for themselves. These analyses are based on over 200 student responses and 30 instructor responses. 88.9% of students were completely remote, 10.5% were hybrid, and less than 1% were in-person. Class sizes of the students surveyed ranged from less than 10 to over 400.
This blog post will be sharing results centered around 3 main topics:
How does Yellowdig compare to other discussion platforms?
For the following 4 statistics, the percentages are based on responses to questions on a scale of 0-6 with answers 4-6 being counted as positive.
92% said Yellowdig’s point system is easy to understand.
76% said that compared to other discussion boards, they find that they’re more likely to post more than is required for them to earn a perfect grade.
84% would rather use Yellowdig for online discussions than other discussion boards they have used.
83% said Yellowdig made them feel more connected to my peers.
We asked students what other discussion platforms they have used for peer-to-peer discussions to understand what what they are comparing their Yellowdig experience to. The majority of students had used Learning Management System discussion boards, with smaller percentages using Facebook Groups, Microsoft Teams, Slack, and GroupMe. With these other discussion experiences in mind the students were asked a number of questions around how their experiences compared with Yellowdig. A summary of the results are below, but I will take a few categories and elaborate on why we believe we see these responses heavily favoring Yellowdig.
Spent More Time Reading Posts
Our platform is designed with a newsfeed style design that encourages students to read their peer’s content. We call this attribute “Listening” and even have a way for instructors to track this in our Community Health Dashboard. Instructors can see their Community’s Listening Score, which allows them to understand how much students are reading, clicking on links, and the conversation ratio. See a sample view below.
Less Likely to Procrastinate Doing
The way we designed Yellowdig, we intentionally reward students for participating early in their point earning window, thus discouraging procrastination. With points for reactions and receiving comments enabled, students understand that they can earn more points from posting earlier in the week with engaging content that others will enjoy reading and want to respond to. We have a visual below that shows how in most discussion platforms students that initially start posting early end up getting discouraged by their peers waiting to respond later and ultimately join the crowd, unlike in Yellowdig, where they stay motivated to start conversations.
Better Interactions with Peers
One of the reasons students experience better interactions with peers in Yellowdig is the way Yellowdig allows students to use Yellowdig for more than just asking questions and sharing content related to class. Yes, this is the main use, but many instructors allow students to have a Topic called something like “Class Lounge” where they are allowed to share about their lives and connect on a more personal channel. In our student panel webinar, Pallas, a public health graduate student at the University of Vermont, shares about her experience with this. You can read about it in our Student Experience blog post.
More Natural Conversations
More Interesting Conversations
Most Likely to Visit Multiple Days
Easier to Use, Better Looking Interface, and Better Overall Experience
Analyzing the Role of the Instructor in Student Satisfaction
We asked students to rate their communities on the amount of structure they used (i.e., prompting and responding to a topic) and the amount of grading or grading-like feedback they received. Then we calculated average net promoter scores for the 4 groups created by taking the cross of responses to those questions. Here are the results:
Instructor net Promoter scores:
6.56 = Highly Structured but Low Grading Feedback N=34 students
8.15 = Unstructured and Low Grading Feedback N=81 students
8.61 = Highly Structured w/ Grading Feedback N=26 students
9.21 = Unstructured w/ Grading Feedback (Probably closest to the “Yellowdig way” if we conceptualize “grading” as giving accolades/comments to provide good feedback) N=53 students
As can be seen from these data, student tend to prefer communities with less structure but still seem to crave feedback. The correlation between how much students endorse their instructor using grading or grading-like feedback and student promoter score is a statistically significant and moderate correlation of r(194)=.32, p<.001. Students do indeed enjoy the experience more and see more value in it when instructors are giving feedback. This mirrors the sentiment echoed by the panelists in our student panel webinar, where they shared that a simple thumbs up reaction from an instructor could go a long way in motivating them and their fellow students.
We believe that this indicates that students like to feel the presence of their instructor on Yellowdig so that they know the time they are putting into the community is being seen and appreciated. Instructors are important in communities and students have to perceive that they care or they are less likely to take this whole thing seriously. Without some additional inputs from instructors we have also seen some student reactions that indicate that they do not feel like Yellowdig is part of the class or that the instructor is just using Yellowdig’s point system to lighten their own workload. These objections disappear with some very small efforts to provide feedback, bring discussions into other parts of the course, or recognize students outside of Yellowdig for their efforts in the community.
When it comes to structure, these data indicate that students don’t necessarily like structure, but they will seemingly tolerate it as long as they feel like they are getting rewarded for the work by getting a good grade and some feedback. Nonetheless, it’s clear that most students don’t necessarily like structure either. Structured discussions remove student agency and self-determination, which is important for motivation.
Ultimately, structure reduces their ability to co-construct their learning experience. As evidence of this, there are a number of studies we have done with partners that prove that structured discussions reduce student participation, limit actual back-and-forth discussion, and yield fewer retention benefits.
Do sizes of communities impact student experience?
From our survey data, there’s no sense among students that Yellowdig is overwhelming the larger the communities get. This comports with behavior data, but there is essentially no correlation between how many students they see in their communities and their net promoter scores (the r value is .008). Given the range of class sizes in the sample, we continue to see no evidence among students to suggest a need to use groups or to try to create more “intimate” conversations.
Actually somewhat surprising is that smaller communities, where more feedback would be easier for an instructor to deliver, are not naturally creating higher promoter scores. I think that’s probably a good indication that there are good (and bad) ways to give student feedback at any scale.
Overall, we learned a lot from our surveys, and look forward to running another survey this semester to compare, learn, and grow even more. If you have any questions, please don’t hesitate to reach out to clientsuccess@yellowdig.com.
Brian Verdine, Ph.D. is the Head of Client Success at Yellowdig. Brian received his Ph.D. in Psychology from Vanderbilt University’s Peabody College of Education and Human Development. He went on to a postdoctoral position in the Education department at the University of Delaware, where he later became, and continues to be, an Affiliated Assistant Professor. His academic research and his now primary career in educational technology have focused on understanding and improving learning outside of classrooms, in less formal learning situations. At Yellowdig, he manages all aspects of Client Success with a strong focus on how implementation in classes influences instructor and student outcomes.
