In summary
- The implementation of an integrated and standardised data eco-system, including incorporating data from student engagement platforms such as Yellowdig, supports critical student success initiatives by breaking down data silos.
- Students observed in this study used Yellowdig Engage to build interpersonal connections that eased their anxieties around their studies, created a sense of community, and set them up for academic success.
- Participation was correlated to increases in student pass and retention rates, making it important to include in our machine learning models to identify at-risk students.
- OES’s bespoke Student Engagement Tool uses data points such as participation in online learning communities to highlight a student’s risk profile in real time, empowering academic and support staff to make direct contact to provide support.
- Data-led interventions enabled by the Student Engagement Tool resulted in an uplift in student success metrics with a 9% increase in pass rates and a 7% increase in students progressing into their next study period, amongst the high-risk students.
Importance of community participation in online learning and the link to advanced learning analytics
At OES we specialise in providing online program management and tailored education solutions, such as learning analytics, to improve experiences and outcomes for students. For over a decade, we have partnered with universities to solve their online, on-campus and blended learning challenges. We utilise leading-edge technologies to improve student performance, progression, and retention. One such technology is Yellowdig; a student engagement platform. It is implemented as an alternative to traditional discussion boards and the user-experience is similar to Facebook. Yellowdig allows instructors to create vibrant learning communities that are more interactive, engaging, and satisfying than traditional discussion boards.
This case study documents the relationship between Yellowdig participation and student outcomes. Student pass rates and retention per study area will be explored for three teaching periods in 2020, for one of our university partners.
Our university partner has delivered online learning for thousands of students, most of whom are mature age and studying part-time. Students at this partner institution use Yellowdig Engage as a safe place to talk about undertaking an online degree, often for the first time. They find support from the other students in the Community who are in similar life stages like being a parent or returning to study after an extended period.
The case study concludes by discussing one of OES’s advanced analytics solutions to improve student performance and retention. We showcase a machine learning propensity model that identifies students at risk of withdrawing, using critical engagement metrics and behavioural indicators, including Yellowdig community participation. We also share the efficacy data of a tool that has been deployed in the Learning Management System (for academic use) and surfaced as a standalone web application (for central student support).
Why do we promote informal online learning communities to our students?
Student-to-student informal learning spaces are used to provide an engaging platform for discussions between students from any course or discipline, creating a sense of belonging to the learning community. It is also used to promote initiatives and provide information to support students outside of the virtual classroom environment.
How are students participating in our informal online learning community?
A key benefit of this Community is that it is student driven; it ebbs and flows with the needs of the students. A ‘typical’ post in this Community is a student introducing themselves. The most used word in these posts is likely “nervous,” as the students consistently share about their anxiety of being an online university student with the pressures of working full time, having children, or other personal challenges faced by students.
The students who are willing to share about their nerves are consistently met with comments riddled with phrases like “I’m in a similar situation,” “I can empathise with how you are feeling,” and “I’m also very nervous.”
The data shows there are 120 views on average per student-authored post, so these conversations are lending comfort not only to the individual students creating the posts, but also to the many students logging on to read the feed.
Some students also took on the role of “Influencers” and would post often with a topic called “Follow my Journey” where they shared about their lives as they were working to get their degree. They encouraged other students to use the “Follow” option, so they would be notified of their posts. They served as inspiration and encouragement for the other students on Yellowdig Engage.
What does participation data reveal?
Participation in the informal learning community is optional and aims to facilitate discussion between students outside of the classroom.
The overall participation rates are around 30% for our university partner, across the three teaching periods of interest as can be seen in Figure 1. The data also shows that students are not only posting, but they are conversing, with a conversation ratio of 3.16. This ratio means that for every post there is on average around 3 comments and further qualitative observation reveals that many students are building strong and meaningful relationships with their peers.
Figure 2 shows the participation rate by study tenure. As can be seen, participation is higher in the first teaching period of study, highlighting the critical need of the student cohort to find a sense of community upon commencement of their degree. More than 25% of students continue to engage with Yellowdig throughout their studies, which showcases the strength of the platform.
Figure 3 shows the participation rates broken down by course area, which highlights the variability of participation across the study areas. At OES we use these granular metrics to identify study areas that will benefit from targeted initiatives to increase participation.
Can online communities uplift student performance?
Our data indicates that there is a positive correlation between participation and student pass rates and retention. However, it is difficult to confirm causation without robust experimentation. It appears that a higher proportion of students that pass and are retained are engaged with Yellowdig communities.
Figure 4 and Figure 5 shows the improved retention and pass rates seen for the student cohort that use Yellowdig.
Data also indicates that there is a correlation between the level of participation and student results. Figure 6 highlights students who are viewing and commenting or posting in the community platform, in general, are achieving better results than students who are passive.
Predicting student performance and providing targeted support
As we discussed above there is correlation between participation and student pass rate and retention. With this knowledge, we have built predictive models to identify students at risk of withdrawing or not performing well using Yellowdig data, integrated with other sources such as the LMS and CRM.
The importance of participation to student performance makes it a vital feature of our machine learning classification models used to predict at-risk students. Figure 7 shows the top 5 features, in order of importance for one of our in-house machine learning risk models, where Yellowdig engagement is a key feature.
To enable access and useability of the predictive modelling by teaching and central support staff, we have developed a Student Engagement Tool that indicates a student’s risk profile in real time and provides functionality to make direct contact with the student (either through a phone call and/or SMS).
The tool is embedded into the LMS (for academic use) and surfaced as a standalone web application (for central student support) with close CRM integration, for the university partner discussed in this case study.
To ensure a successful implementation, teaching and support staff are empowered to take the right type of action, at the right time. Staff were provided training with the tool to decide which students would benefit from direct personal contact and what the best way to reach them would be.
Providing student support through the tool was initially piloted in selected undergraduate study areas across multiple teaching periods. To evaluate the success of the tool, students identified as high-risk in the intervention units were compared with a control group of high-risk students from previous iterations in the same study area and similar classes running within the same teaching period.
In each trial, approximately 25% of the target cohort were defined as high-risk and were candidates for intervention. Amongst the high-risk students, an uplift in student success metrics was observed compared to the control group in the majority of study areas. Overall, in the latest iteration, on average there was a 9% increase in pass rates and a 7% increase in students progressing into their next study period.
Final remarks
We have observed how informal online community engagement is positively correlated with favourable student outcomes and is a key feature in machine learning models we develop. We found that Yellowdig Communities, like our university partner’s, foster genuine relationship building among adult learners and provide students with an outlet to engage with their peers in an informal learning environment. These Communities enable students to connect with students in their study area, and ultimately be more successful.
The predictive and advanced analytics we have created to identify at-risk students require data from the Learning Management System, Customer Relationship Management System, Student Management Systems, and third-party products such as Yellowdig. The combined data provides a holistic view of the student performance enabling the OES Student Engagement Tool’s machine learning modelling.
Our tool delivers the risk profiles of students to academic and student support staff members, for pre-emptive and personalised support, to improve student performance and retention. Overall, informal online communities not only enable our students to feel more connected, but they have provided additional data insights that improve our ability to pre-emptively identify at-risk students.
