Yellowdig Guest Webinar Featuring Mark Rom from Georgetown University

Bob Ertischek, Head of Client Success, Yellowdig 0:07
Hello, everybody. Welcome to our exciting webinar today. I'm here with Dr. Mark Rom. And we'll get started in just a few minutes. What we want to do is let you find your way in and your seats, please be careful as you're going through the aisle. If you have any drinks, keep them covered. And, you know, but we're just gonna let people meet Yes, there you go. Don't show that brandmark Come on. We're just gonna let people come in for just two minutes. And then we will start precisely at 102 Work 1202 or whatever time zone you're in. In the meantime, you'll notice there's a little chat at the bottom of your zoom screen. So if you'd like to share where you're from, and what you do, that would be great. Mark and I will introduce ourselves shortly. Mark, we should have had one of our one of your spin playlists going into while we were starting didn't think of that. I'm still glad to put one up there. If there's time. I can do one of the my latest, but you can absolutely share one or excellent. Yes, yeah. I will vouch for most of them. Okay. Heidi, welcome. Good to see you. Heidi from Regis University. I know Heidi,

Bob Ertischek, Head of Client Success, Yellowdig 1:18
and anyone else who wants to share where they're from? That'd be great. You don't you're not obligated to Nikki from Missouri Southern another great school. All right, one more minute. A Morehead State. Oh, this is great. Grambling

Dr. Mark Rom - Associate Professor, Georgetown University 1:32
So, Bob. You know, I chat. Right, right. So we chat. And one thing I'm noticing that is like how many female faculty or or staff administrators are on the call as compared to males? I mean, this is one of those issues that, you know, we can address perhaps the Yellowdig later on, but Don just find it. Interesting to note.

Bob Ertischek, Head of Client Success, Yellowdig 1:50
Well, it's interesting, Mark, I don't know that. You know, I don't know. All the people may not be chatting, but there. Are there more people that are chatting than that are actually

Dr. Mark Rom - Associate Professor, Georgetown University 1:59
I see. I see. Yeah.

Bob Ertischek, Head of Client Success, Yellowdig 2:01
All right. So oh, hi, Richard. See Richard Wack from Villanova. Hello, Richard, how are you? All right. So I wouldn't Heidi commented on your give us hmm on that thing. All right. So we're gonna get started. I'm going to try to find my screen share button here so that we can get going. And here we are. But welcome again. I'm bombers check. Welcome to our webinar today. And hardest part of my job is sharing the screen. So just give me a second

Bob Ertischek, Head of Client Success, Yellowdig 2:46
shut it down this beforehand. Okay, there we go. Oops, wrong way. I've done this before, I promise.

Bob Ertischek, Head of Client Success, Yellowdig 2:54
You know what, this is the wrong screen. That's why I knew there was something happening. Anyway. I'm Bob Ertischek, I'm Yellowdig is Head of Client Success. I've been with Yellowdig for almost three years. I'm going to introduce my colleague Mark Rom in a minute, my colleague, I suppose that's fair, right, Mark? Sort of, yeah, we've worked together before. And I just want to tell you briefly about me becuase I think it does inform why I'm here talking to Mark today. Prior to coming to Yellowdig, a long time ago before the turn of the century, I was a practising lawyer but I did not enjoy that at all and I got the opportunity to do some teaching in higher education, really liked that. Did that for a while and then early in the century, I started working as an Instructional Technologist Faculty Developer where I helped instructors 20 years ago put their courses online and in that time, I found that I loved the promise of online learning. But I felt like the tools and the pedagogy did not provide the better student experience that I was looking for, or in frankly, the instructor experience. So I got frustrated in that position. I went back to teaching and and like Dr. Rom, I taught American government and political science courses for over a decade. I shouldn't say like Dr. Rom, I'm sure he far exceeds my teaching skills. But in my own courses, I tried to really build a better student experience outside of the learning management system and tried all kinds of wacky things that had their own problems. So when I found Yellowdig, it really had been what I was looking for. And now I'm fortunate enough to work with Yellowdig, and professors like Dr. Rom to help them create better online experiences for their students. And, Mark, if you could tell us a little bit about yourself. That would be lovely, because you've heard enough of me.

Dr. Mark Rom - Associate Professor, Georgetown University 4:44
Great, Bob, thanks for that introduction. You know, it's all you know, I always love to see on the screen. I'd love sitting in person. I've always enjoyed working with you. And I'm delighted to work with Yellowdig. Let me put a just a quick disclaimer here is that I am here, unpaid. I'm not representative of Yellowdig. I speak about my experiences with Yellowdig. But I'm not here to, to sell it or to promote it, but just to talk about it, but I've been excited to use it. I've been using it in the classroom for the last three years or so I've been teaching at Georgetown University for 30 years, courses in American government, US politics and policy, you know, but every year, in fact, every class, it's a new experience, it's always opening night. And you have to figure out, you know, what can I do to engage the students to excite the students to help them embrace the material and embrace the topics. And, Bob, we're gonna talk more about that later today.

Bob Ertischek, Head of Client Success, Yellowdig 5:36
You bet. And just so everybody knows, Mark and I do know each other from prior to either of us using Yellowdig.

Bob Ertischek, Head of Client Success, Yellowdig 5:45
Both of us have been involved with advanced placement and I we work together scoring AP US government exams, that's how we met for about 10 days, every summer, a group of like 700 people get together and score all the essays, the million essays that students write. And we had the great privilege of doing it together and and had a blast, actually, and have been friendly since I think it was like 2012 or 2013, that we we started that mark, does that sound about right when you

Dr. Mark Rom - Associate Professor, Georgetown University 7:54
30, 30 Bob 30

Bob Ertischek, Head of Client Success, Yellowdig 7:58
30? And why don't you tell us a little bit about your courses and the goals that you have for your students when you're teaching.

Dr. Mark Rom - Associate Professor, Georgetown University 8:09
First, also, you know, thanks for taking time out of your day, all the all my colleagues on the call here, because I know that we're all busy, especially the week of Thanksgiving. So I appreciate your presence. And I hope this will be a worthwhile experience for you. So I teach this introduction to US politics course, that's the one I use Yellowdig for. So it's a survey course, it's for first year students, if fulfills a core requirement for anybody in Georgetown college. But it also is a required course for those who want to major in government. And so my classes are typically around 150 students or so between 102 100 It depends on the semester. So big enough that it can be a challenge to get to know this. But small enough that you can still build a community. Although Bob, you'll probably talk later about how this can be used for courses of 500 or 1000. Here or

Bob Ertischek, Head of Client Success, Yellowdig 9:01
30, or 20,

Dr. Mark Rom - Associate Professor, Georgetown University 9:02
Or 30 or 20. Yeah, so let me tell you, here are the two main reasons that I have embraced Yellowdig. And when I say embrace it, you know, I was a skeptical lover, I didn't just decide to do this without hesitation. I was concerned about what it would look like but I decided to try it out with my classes in here are the two purposes it serves. First, you know, there's tons of empirical evidence that demonstrates that students will not remember course content after the semester is over, you know, if you're giving a final exam, you know, they'll study for the final exam and then by March or July they've, they've forgotten it. So there's also I think, a sufficient amount of evidence that students remember the experience of the course like Did they enjoy it? Did it bring some curiosity to them? Did it make them want to learn more about the topic? And just on that note, you know, I took a architecture survey course when I was in college.

Dr. Mark Rom - Associate Professor, Georgetown University 10:00
which, which still influences me today, whenever I look at buildings, I think, okay, now I know something about the buildings, I'm curious about how they're designed. So part of my goal in the course, is not just to provide content, but to excite them about the experience, and leave them with a curiosity and a passion for learning more about the material that I teach. Okay, so that's point number one.

Dr. Mark Rom - Associate Professor, Georgetown University 10:22
Point number two is that I've become increasingly skeptical about giving grades out based purely on performance. The reason for that is students who come into my class with higher performing abilities. And you know, I have evidence for this also, they tend to get higher grades going out, you know, if I just have them write essays at the beginning of the class, in essays at the end of the class, and I compare that to those who perform higher at the beginning, performed higher at the ending, those who performed lower to beginning typically perform lower at the end, there may be improvement for both. But scores are closely correlated to incoming knowledge and abilities, right. So what I want to do is within party VLD, is can I put more weight on effort, so not just how well the students perform, but how frequently they show up, how much they contribute weekly do they engage in, so I trust structure class where they have actually about half their grade, not all on Yellowdig. But other matters, where the showing up every week, and participating engaging matters a lot, in my hope is that sort of balances, the grades between those who come in with loss of skills, and those who come in with loss of desires. Now, I'd like to reward those with more of the desires when they come in. So let me stop there. And let's move on to your next point. But again, engaging them having them enjoying the experience, rewarding effort, in not just performance, those are the things that motivate me to use Yellowdig.

Bob Ertischek, Head of Client Success, Yellowdig 12:07
And through that engagement and that effort, what you're seeing, or at least what you're hoping to see is that provides more connection, ultimately to the material, at least from the perspective of, of interest. And, and, you know, hopefully understanding too, because because of the motivation to engage that builds out of Is that Is that a fair statement?

Dr. Mark Rom - Associate Professor, Georgetown University 12:33
Yeah. And so Bob, you know, the way that Yellowdig is structured and the way that I structured my courses that the students can earn 1000 points week after they can earn 15,000 points over the course of the semester to get the highest grade, they have to do that every week. And this really works against the idea that students have students might have that they can sit back, and then cram to the final, or sit back and write their final paper in the last three days. You know, they have to work every single week, there's no getting scores, if you miss a week, there's no making up for that. And this is a challenge the students, students are not used to this. So the first couple of weeks, especially, I remind the students on a, almost a daily basis. And I can automate these emails, right. So I do this, this is embedded in my Canvas course. So I write them remember to participate in Yellowdig. Today, the example I use is like how often you brush your teeth, you wait, you know, just until you go to the dentist or you wait until the end of the semester, maybe wait till we go to the dentist, but you have to brush every day in order to succeed on that. And so the early engagement and the consistent engagement is something that we really, really focus on.

Bob Ertischek, Head of Client Success, Yellowdig 13:44
And so Mark before Yellowdig. What did you do? How did you manage this?

Dr. Mark Rom - Associate Professor, Georgetown University 13:53
So, I've been a believer in student engagement consistently and routinely across the semester, for many years. It was hard to do before Yellowdig I found the blackboards on canvas or or on Blackboard discussion boards. Those are way too static. They didn't lead to like real student engagement. People would do perfunctory posts. So that, you know, that was not good for me. You know, I'm fortunate to work in campuses, we have a lot of political events, I would require them to go to events in person. And then post about those events, you know, show a screenshot and give a reflection, again, on a static discussion board. So that did not engender additional student engagement. So my students still earn course credit by going to activities like if they go you know, go to a movie or go to a speech, go to a city council, go to a congressional hearing, and there's tons of things they can do. But then they come back and they post about these things on Yellowdig. Again, to share their experience and let other students know what they did and what they learned from that. And so it's much, much more communal than it was through the static discussion boards on Canvas or Blackboard. Again, I don't represent canvas, I don't represent blackboard shieldaig. These are all my experiences a faculty member.

Bob Ertischek, Head of Client Success, Yellowdig 15:16
And Mark. You know, some, I'm pretty much sure that everyone in the audience realizes that when you're talking about an American Government course, there may be some opinions that students have about or that they're developing about American politics, American government in general. And you mentioned to me in our chat little beforehand about equity and and the ability to have a comfortable place for everyone to to be able to share their thoughts in a respectful way. Did you find that in Yellowdig? At all?

Dr. Mark Rom - Associate Professor, Georgetown University 15:53
Yeah, so a couple things about that is that Yellowdig does provide the students an option to flag a post that they believe is, you know, is offensive, offensive, or damaging. And so, I will get an email saying a post has been flagged. So if people are using hostile language, or if they're using sexist or racist language, you know, that can be you know, I will know about that. And then I can go see that post, and to see whether that post is simply, you know, one that you may disagree with, and some of those posts are like that, or whether it truly is damaging to the learning environment. Again, it's my call, then I can do that, you know, this semester with 150 students in multiple hundreds of posts. I've had two posts flagged. And so that's a good sign that there is really engagement with the community that is not, you know, it's not, it's not hostile. It's not it's, it's not racist, sexist, xenophobic, so that that's a good thing. And yeah, on the other hand, you know, it actually, this is really helpful. We had a conversation this morning, you know, if you use the search term disagree, you know, I've got tons of posts, where students are saying they disagree with something. Now, the disagreements are not partisan, balanced, my school tends to tilt pretty heavily, heavily to the liberal side, the Democratic side, so much of the disagreements posed is towards more conservative viewpoints. But the disagreements that are posted are civil and engaging, they're trying to get other students to respond to them, rather than just saying, This person should be canceled, this person should be driven off the platform.

Bob Ertischek, Head of Client Success, Yellowdig 17:32
So they're respectful disagreements about actual things that matter and providing deeper thought and, you know, one thing I we talked about in your communities in discussion boards, you know, usually, like you mentioned, comments, or at least this is our opinion, I think it's supported, perfunctory, and people are just sort of checking checking the box as what's going on. In your community, it seems and I know, actually, it doesn't seem students are having deep conversations, they're going 50 Something comments deep into conversations. So you know, that shows that students are actually reading each other's content, they're not just coming in and getting out there getting out as soon as possible the point system that you're you've employed, you've, you're using the other two point system, which ena points for interactions that they receive from each other. And, you know, for my mind, that is actually driving as it's intended to deeper conversation, students are realizing they're getting points when somebody comments or reacts to their, their post or comment. And that, do you? Are you finding that it is actually driving deeper conversations that students are feeling good about getting those comments and providing better conversations?

Dr. Mark Rom - Associate Professor, Georgetown University 18:50
Yeah, now, a deeper comments Bob, but sort of more consistent comments, like, you know, again, one of the problems with the static discussion boards on Canvas or Blackboard, is that they're usually assigned for a particular week. So people post in a week, and then it's done. And often my students, you know, they, you know, the assignments due on Sunday night, they posted Sunday night, so they may not really be engaged with other students, they're just meeting a deadline. But since the main topics on Yellowdig, they, they persist. I mean, it's like, you know, the Facebook, Twitter where the comments that are sort of most popular still are, they're open for other student comments. So I might have a thread open for like, several weeks, students are still coming on a post, and so doesn't go away the assignment deadline, and that reflects well on on student engagement. Let me let me also just note that I provide topic headings that students must use, so they have to post to a topic. But they do the posts, so I'm not prompting them. So this week in class, we talked about this, you know, what did you know what do you think or what's your assessment or I'm gonna ask you a question. These are student led conversations, student led conversations. I do monitor them. I do weigh in, I post sometimes. But I'm careful to post in ways that are part of the conversation. Rather than driving the conversation, you know, I use accolades I give points out at Bob may want express when accolades are. So, you know, I want to give thumbs up to post a like, or a special award for posts that have lots of developed a lot of traction. So I want to participate. But again, this is a student engagement activity rather than a professor led activity.

Bob Ertischek, Head of Client Success, Yellowdig 20:31
Yeah. So Mark, basically, you know, you're present in the community, and that's vitally important to any instructor using analytics, the students under need to at least initially understand that this is an important part of the course for you, you also make it with 15% of your grade is that you said before?

Dr. Mark Rom - Associate Professor, Georgetown University 20:45
15% semester grade.

Bob Ertischek, Head of Client Success, Yellowdig 20:46
Right? So that extrinsic motivation initially to come into the community along with you know, because the point system goes to transfer centigrade is important, and your presence is there to let students know, to let them know that it is important to you as well, and that you're not just you know, this is not busy work. But beyond that, you know, you've taken your hands off the wheel in this community, you're allowing, you're there, you're going to course correct, if necessary, through a comment or something like that. But mostly, you're there as, as a guide as another citizen of the community shares, right? Yeah. And, you know, do your students feel comfortable? I know you I know, they feel comfortable interacting with you, generally speaking, but are you finding, you know, are you having more value in the community through posting or through commenting? Would you would you do you have an answer to that?

Dr. Mark Rom - Associate Professor, Georgetown University 21:43
You know, I don't I tend to give more accolades to the posts that engender or lots of responses. So, you know, the students are rewarded under the point system, if you get responses to your post, or get accolades that you know, earns you points for your grade. You know, I think that encourages students to give thoughtful posts, and post them engage others. So, Bobby, maybe sharing some slides here. I can't remember. Yeah. So early on in the semester, I asked the students introduce themselves. And so there is one,

Bob Ertischek, Head of Client Success, Yellowdig 22:15
one, Mark, this isn't that I'm sorry about that. I just wanted to show people accolades real quick. Oh, yeah. Okay. Yeah. So accolades are point bearing badges that the instructor, only the instructor or any teaching facility, faculty can award. And it's very simple to do it. It gets it shows your present there tells every student what you think, is important in the community. And they get a point bonus for that, as you can see, they also are highlighted on the side, which can lead them right to those important conversations that you've highlighted. So I just wanted to give people a taste of what those accolades are. And I just also, you mentioned topics, topic tags. These are what guide conversations in Yellowdig, that these are the the course relevant concepts that students talk about Mark in American government, we're talking about the Constitution, we're talking about bureaucracy, we're talking about the judiciary, all that kind of stuff. Those would be examples of topic tags. I see I'm going to stop sharing my screen, but I see that we have a couple of questions in the chat that I may as well just go with right now. Mark, we have Jennifer asks, How would you gauge their critical thinking in the other day communities?

Dr. Mark Rom - Associate Professor, Georgetown University 23:30
You know, that's a great critical, critical thinking question. You know, I don't have a great answer for that. But here's what I tell you is, I give accolades, again, students receive points for accolades. If I believe that the questions they've posed or the top favorites, the issues they've dressed is especially important, interesting, and beneficial to the class. And so, you know, by, you know, sort of selectively giving these accolades. My hope is that students will try to, you know, give posts that are interesting, important and address critical issues class. You know, I don't have a direct measure. In fact, Jennifer, you're getting me to think that, you know, can I measure? Is there other ways that we can think about measuring critical thinking through your day? You know, I don't know the answer that. I mean, yeah.

Bob Ertischek, Head of Client Success, Yellowdig 24:21
To me, I think that we can see it in in, you know, the length, the depth of conversations. And, again, I love the when respectful disagreements occur in communities like they do in yours quite a bit, which allows us to see that students are, you know, sort of looking at what each other say, responding to what each other say, and sort of, perhaps and sometimes even changing their own opinions based on on the kind of input.

Dr. Mark Rom - Associate Professor, Georgetown University 24:49
Bob, Can I ask you a question now? Also, some people in the chat are saying something about power digs, do I I don't, I don't think I know powerdigs.

Bob Ertischek, Head of Client Success, Yellowdig 24:57
No, I that is not I don't think that's something that we Yeah, that must be something that they do at that particular school. And perhaps we can ask Nicole about that another time. But I don't know what that is. Nicole, if you wanted to put it in the chat or something, that's fine.

Dr. Mark Rom - Associate Professor, Georgetown University 25:10
Yeah. So and Robert, so yeah, you know, I, you know, I really want to give accolades, selectively. So I only give out maybe a couple of weeks ago in the class of interviewed students. And it's like, this is a reward to encourage you for especially good post post that engages other students. That's the That's my biggest concern here. I rarely give an accolade although I've gotten a couple accolades to students that get few responses. And occasionally, there is a response that shows deep thinking, but students may have find a hard time responding to so I don't neglect those either.

Bob Ertischek, Head of Client Success, Yellowdig 25:45
Yeah. And you know what, Mark, the accolades are great strategy for that for those lonely posts that people haven't responded to. When you add an accolade to it, or when you comment on it, it'll go right back to the top of the feed. And of course, if they see the accolade they'll realize this is something of value, or if they see you responding to it, they'll realize that's something of value. We're looking at a redacted because we don't want to expose any student information post that a student created in Yellowdig. You want to tell me a little bit about this?

Dr. Mark Rom - Associate Professor, Georgetown University 26:14
Yeah, so no, notice first that it's got several tags at the top. So the students are required to post to a topic, you know, I probably should get rid of that question. Tag because that's too generic. But this person's posting on the presidency, public painting campaigns and elections. So three topics that are covered in the course through our textbook, and his written the question, which is, you know, a real public concerns like, Should Joe Biden read again? Well, the student has got 38 votes cast, and a number of responses to this. And notice the students not just like mouthing off, look at this should bide, run get in question one, then the Democratic Party has more moderate candidates. Question two, do we think Joe Biden Oregon? Question three? Do we think more do question four. So this students asking for questions in the post, which is not just saying, you know, blah, blah, but what do you think?

Bob Ertischek, Head of Client Success, Yellowdig 27:06
And I mean, this is an instructor type of prompt, isn't it? Really?

Dr. Mark Rom - Associate Professor, Georgetown University 27:09
Yeah. Oh, absolutely. Yeah, absolutely. Yeah.

Bob Ertischek, Head of Client Success, Yellowdig 27:12
And by the way to that question tag, and not using it, I have another solution, we now have no point topics which can be used with the point bearing topics which tell contextually, when you're looking at the feed, hey, this is a question about the presidency or something like that. So I'll talk to you about that another time. Mark. Thank you, Bob. Yeah, we also have a question. In the q&a. How many topics do you use? And Mark, tell me forgive me if I'm not correct, but I'm assuming that most of your topics are based around those core concepts that, you know, relate to the your syllabus, those top level syllabus, subject matter questions,

Dr. Mark Rom - Associate Professor, Georgetown University 27:50
like that's right, you know, and I would remove some of the, you know, so some sort of redundant topics that I can, you know, I can clean up. But as this is a survey course, I've got a textbook with 15 chapters. So each chapter has a topic posted. So that, you know, my textbook has a chapter on the presidency, a chapter on public opinion chapter on voting campaigns and elections. So those are the main topics. But then I've also chosen some topics, like, if you want to challenge one of the quiz questions, because they're taking an online quiz, they can do that. Yeah. So here's the, you know, quiz challenges question. And let me tell you, my students will challenge these questions. So that's a topic. I've got a topic about upcoming events.

Bob Ertischek, Head of Client Success, Yellowdig 28:39
Can you can you just go a little bit deeper on the quiz challenges? What what is the goal here? And what's what's happening here? Exactly?

Dr. Mark Rom - Associate Professor, Georgetown University 28:45
Yeah, so the textbook I use, so again, relatively large class. And so I use computer graded multiple choice questions on each chapter of the textbook. This allows me to get a lot of information, a lot of data about how students are doing and keep them on track. So over the course of semester, they have to do quizzes at certain points. I know they're reading the text they're making, they're making progress. But I didn't write these quiz questions. They're written by the textbook publisher. And some of the questions have wrong answers, or it's unclear whether the answer is correct or not. So as a matter of agency, I give my students a chance to challenge the quiz questions. But as you'll see, not in an unlimited way, I tell them if they lose the challenge, they lose additional half point, although I have not enforced that, Bob.

Bob Ertischek, Head of Client Success, Yellowdig 29:33
Well, we'll keep that between us.

Dr. Mark Rom - Associate Professor, Georgetown University 29:36
Thank you very much. And thanks very much. And, this actually gets a lot of engagement students agence quiz question. I believe that I believe the question is wrong. I believe the answer is wrong, or the question is unclear. And so this gives them a chance to sort of live in front of all the other students to challenge this.

Bob Ertischek, Head of Client Success, Yellowdig 29:52
Now he talks about critical thinking. I mean, I think this is it. Isn't it?

Dr. Mark Rom - Associate Professor, Georgetown University 29:55
This is the actually, this is probably the best example of on on Yellowdig because, you know, when students write me and they say, I have a problem with this quiz, I say, don't write me, post it on Yellowdig. Because if you have this question, other students are likely to have that question also. And by the way, you can see at the bottom of the screenshot, I believe that both answers are correct. I have reached out to the publisher to update the quiz. And I do that, you know, I read the publisher, the book and say, you know, we've got a problem, this question, please update that. And so my, I think my students feel validated when I say, Yeah, your answers, you know, it's a good one. So let's give you credit for it.

Bob Ertischek, Head of Client Success, Yellowdig 30:36
Right, I'm just looking through the chat here a little bit.

Bob Ertischek, Head of Client Success, Yellowdig 30:41
So, you know, this slide is interesting to me, because students are helping each other here. Right? Yeah. Some student posted a question, and the others are telling them how to how to study. Now, this might not be this isn't directly related to your course concepts. But is this Do you mind that they're talking about these things that are not exactly on topic?

Dr. Mark Rom - Associate Professor, Georgetown University 31:09
So you know, I gotta tell you, but actually, I'm going to tell you two things about this, okay, is that some of the most popular posts are posts like this, you know, like, I'm falling behind my research paper, can someone help me catch up? And they'll be like, you know, half the class will respond is the class gloves, these kinds of questions. That's great. But he's those build community, they share knowledge among the students. If you can see that, you know, the first answer you get is, you know, here's what I do. And so these are helpful to the students, and they build community, you know, I've questions about whether they should get points towards they're great for these kinds of posts. And you've now shown me that I can have some topics that are on points, and some that are just for community engagement. So next time I do this, I'll probably have, you know, encouraged the points, like, you know, where's the best place to eat on on the campus. And so, you know, I encourage them to write on that. But they should not get points for their grade for that. And so thanks for showing me now how to differentiate between point earning questions and just community building questions.

Bob Ertischek, Head of Client Success, Yellowdig 32:18
Yeah. And by the way, topics is what you mean, as opposed to? That's right. Yeah. And so somebody asked before about the topics that you have, these are the topics that Mark has, and because they are all solid color, you can see that they are all point bearing, if they were nonpoint, bearing, they would have a dotted line, and they would be sort of an x ray version of the regular topics to sort of get there. As far as that goes, Mark, you know, some of the other things that we talked about is, you know, have you been surprised or impressed with any of what you've seen in Yellowdig? From your students, have you learned anything from them?

Dr. Mark Rom - Associate Professor, Georgetown University 33:03
I do, you know, this, this actually gives me a way to sort of understand what the students care about in ways that are hard when you're doing a big lecture class. And you know, I'm not a fan of this class structure, my students meet in an auditorium. So they're tiered seedings. It's like a big movie theater. So I'm down in front, they're all facing me. So it's not the architecture of this class is not designed for engagement. And so this allows the students to engage in ways that they otherwise could not do very easily, very easily in the class. I will know and I think this is important for the faculty are concerned, do use this is that early in the semester, I would spend about 15 minutes at the beginning of the class session, bringing a building posts that were pertinent to the topic. So like, if I'm talking about the Congress this week, I would you know, identify five yielding posts that raised relevant questions. And I enjoyed talking to the class and I wanted to make the students know that yeah, like, I'm reading your posts, and they're beneficial to the class. These are erasing important questions, I want to I want to bring them to class attention. In a sort of a deep dive midterm assessment, done not by me, but by other professionals on campus that do assessment type stuff, the students said, they like Yellowdig a lot, but they want to minimize the class time speaking about it. So rather than spend 15 minutes of class time speaking about y'all dig, you know, maybe five minutes, and I'm glad to have learned that from the students. You know, they think of that, as you know, the class experience is one thing. Yellowdig is a different thing, and they don't necessarily want them to be fully merged. But think about them in some kind of kind of separate ways.

Bob Ertischek, Head of Client Success, Yellowdig 34:43
You know, while I understand exactly what you're saying, I still think it's a good practice to make sure that Yellowdig isn't an island on in into itself, that connection between the course and Yellowdig and other areas of the course.

Dr. Mark Rom - Associate Professor, Georgetown University 34:59
Oh, absolutely. Uh, yeah. And in fact, you know, it's really important for me to weigh in, you know, not let students think that I'm, you know, it's like, I want to let them know that I'm watching and reading these posts, I don't read all of their too many read. But, you know, I can selectively engage in that. So the students know, yeah, this is something that, you know, Professor Ron thinks this is important, not just for the gray, that's for the points, but in terms of the intellectual content, but just not so much for class discussions.

Bob Ertischek, Head of Client Success, Yellowdig 35:24
Yeah, and, you know, we're talking about what your students think Yellowdig I, you know, this may seem a little self serving on the other side, but we did not prompt this. And I just want to share a conversation that a student started in your horse related to Yellowdig if I can here. And maybe it'll show up here. So student asked, What do you think of Yellowdig as a requirement for class, and, you know, I'm unprompted, 52 comments, and, you know, cherry picked here, but most of them are pretty, most of the comments. In fact, I didn't see a comment that was was wholly negative. I mean, we've got, I think everyone is respectful, really helpful, actually learned a lot about revenant news, I assume that's supposed to be relevant news, I don't want to call out that particular student. Since the class is so big, and it is hard to get to, you know, your classroom mates in the actual classroom, I think this is fundamental, felt that Yellowdig is the perfect combination of pure government and political discussion. And to my mind, what that means is the students are not only feeling more connected to the class material, but they're also feeling more connected to the class itself, they're their peers, to you and the institution as a whole. And you know, Georgetown probably doesn't have a retention problem. But some schools have trouble keeping students because those students feel disconnected to the institution. So I think that's a really valuable piece of what's going on there.

Dr. Mark Rom - Associate Professor, Georgetown University 36:53
Super helpful, Bob, and, again, I'm gonna, I'm just want to reiterate, you know, the first three weeks of the class, they basically daily reminders, you need to go to Yellowdig, you have to log in Yellowdig, this is important for your grade as part. And so that was important for students to get in the habit of doing it, you know, to get used to logging in to going there. But after the first three weeks, you know, actually going like once every two weeks and say, here are the people who are, you know, the, you know, the five people in class who have not been participating. So I'm gonna, you know, I send reminders that but aren't sending reminders app to the whole class, but only to the few people, most of the students that are are sort of into it. Yeah.

Bob Ertischek, Head of Client Success, Yellowdig 37:31
You know, I haven't looked, but I assume some of your students exceed their point goals, because they're finding those conversations to be in our data supports that that many students, once they find value in the community, they they jump in on their phones, or whatever, and look at it like they would on some other social network, and just enjoy the feed and participate as as they would otherwise. Excuse me, sorry about that. So, you know, ultimately, have you seen evidence that your students are going beyond their point goals? Or have yet so

Dr. Mark Rom - Associate Professor, Georgetown University 38:04
So. you know, you know, Bob, I need to get some more data from you on this, but because, you know, to earn points, the students have to write certain number of words for a post certain number of words for a comment. And so the question is, how close to those limits are the students writing? Because the closer to the limits, the more it looks like, they're just, you know, punching, you know, checking a box, right? So if you require 20 words, to get points for response, you know, how close are the responses 20 words? Well, a lot more close. 20 words, I mean, students are attentive to points, you know, I understand that, but are much longer than they need to be. And when they're much longer than they need to be. That suggests to me that they're like, they're just writing what's on their, on their minds. And, and not just saying, Okay, I've met my word, goal. And by the way, when you're typing on yellow day, it'll tell you how close you are to the point, the number of words you have to have for a point. So when I'm doing a post, it'll show me you know, like, 75% of the way a percent of the way, right. So, you know, I can get you to that cut off and just stop losses do that. But a lot of them don't, a lot of them post extra material. They don't get credit for, but they posted apparently, because they've got something to say, which is, you know, it's pretty cool.

Bob Ertischek, Head of Client Success, Yellowdig 39:21
And they get reactions, thumbs ups, and things like that for what they do. And, by the way, in case you're new to Yellowdig, we, you know, students can give another student they can't give them accolades, like Mark does, but they can give each other a thumbs up or a smiley face or whatever. And they do not get points for giving those reactions. However, the student who they give them to does get points and what so what does this tell us about the students who are giving the reactions tells us that they're engaged, that they're reading these posts, even if students aren't necessarily always commenting. They are engaged and coming back into the community to read what their peers have done there. And I think that's incredibly valuable, and it's something certainly wouldn't get in a discussion board. Mark, I know you have another class in just a bit. So I want to in the remaining time, see if there are any questions that I missed Bree, or anyone else, if there any questions you want to ask. And we'll, we'll go from there. So is there anything else that anyone wants to ask Professor Rom? Or me? If not, we can keep going a little bit. So

Bob Ertischek, Head of Client Success, Yellowdig 40:32
Mark, you know, government class is 150 students, it's pretty challenging thing to do to keep on top of, of all the students. You know, one of the things that you mentioned in our conversation, is that great, I wanted to go back to that grading perpetuates inequities little conversation. And can you just, you know, how did how did, how would you say the grading does sort of perpetuate those inequities.

Dr. Mark Rom - Associate Professor, Georgetown University 41:03
So, I've got data from other classes in previous semesters. But it's pretty clear that if I designed my class where basically 50% of the points are based on effort, and 50% are based on performance. So performance, you think of quizzes, you think of writing a final paper effort, you think about you doing Yellowdig, coming to class, doing other kinds of events, it's pretty clear that things differentiates the students ultimately, that have higher grades are those that participate all the time, you know, if you can engage participation, Yellowdig going to events coming to classes, those students just do better on the other elements as well. And I don't think it's that, like, the resources that, you know, that sort of the knowledge or skills coming in are driving effort. Again, I don't want to be too hard on this, but that the effort in the class is driving the performance on the on the other measures. So if you can find ways to engage students, you know, get them, you know, get them to come to class every day, you know, participate in Yellowdig, every week, that those things may not directly lead to higher performance, but they, to the extent they lead to higher engagement, higher engagement is likely to lead to higher performance, at least that's sort of the hypothesis that I'm working on here. I've got some evidence that demonstrates that from actually a paper that I've written with some grad students, that's going to be coming out in a journal. Soon, I'm accepting the journal that right higher engagement, just engagement leads to better performance, controlling for these things like SATs scores, and high school GPAs, that, you know, effort matters a lot. Yellowdig enhances effort. Bob, I haven't used this for my smaller courses now. But I think I'm gonna start doing that I think even my courses have, you know, 15 to 25, they may have better conversations in induce effort, and that's something that's, like, super important to me.

Bob Ertischek, Head of Client Success, Yellowdig 42:58
You know, I'm Mark, so the 150, you said, it's a face to face class, right? That's correct. And, and, you know, so no matter how great a professor you are, or how great professor I was, it's impossible to to allow everybody in that 50 minutes three times a week, or whatever your cadence is to fully express themselves or ask the questions that they want to do. And, you know, I mean, I think this really gives students a lot more agency than they would have without it. Is that sort of what you're saying here?

Dr. Mark Rom - Associate Professor, Georgetown University 43:29
Yeah, you know, again, I think that it's pretty clear that, you know, having students raise their hands in a class is, you know, big class can be kind of intimidating, we know that men tend to raise their hands more often than, you know, than others. And so trying to get engagement in class, I do some random calling, and people have a random number generator that I spend the dial, call on students by never, but you know, yielding also gives students to participate in the discussion ways that they might be unwilling to do if they have to raise their hand and say that, I mean, they can post in the students are, you know, I think they're, they seem to be pretty comfortable posting, you know, I don't really know, you know, I, you know, I don't know for sure, there might be some students that say, Look, if something's really on my mind, but I'm just I'm, you know, I'm nervous being called out or, you know, canceled or whatever. But again, only a couple of posts have been flagged for a hostile language. And so, I think that, you know, introverts, shy students, international students have more of a chance to participate in an equitable basis, through Yellowdig, and they would buy, you know, class conversations.

Bob Ertischek, Head of Client Success, Yellowdig 44:37
And then you can, you can think about what you want to say before you before you have to, you know, you can take your time to construct a statement that makes sense.

Dr. Mark Rom - Associate Professor, Georgetown University 44:45
Unlike Twitter, you can edit your posts.

Bob Ertischek, Head of Client Success, Yellowdig 44:48
Let's not go there. I don't know if you want to go there today. Anyway, so, respecting your time any last words that you'd like to share with our people who've joined us here today.

Dr. Mark Rom - Associate Professor, Georgetown University 45:02
Yeah, you know, I just think this is again, I came in as a skeptic, you know, I was, you know, I knew Bob, I liked Bob, I was willing to try this thing, no guarantees, you know, again, I get paid. But now I think that compared to the other learning management systems that I've been exposed to Blackboard and Canvas, those only two, this gives more chance for students to participate in more important ways over the course of the semester. And so, you know, if you're not using it, consider it and I'm glad to, you know, Bob, you'll share my email people out, you know, people want to write me, I'm glad to respond to them or, you know, have other conversations with them about that. But, you know, I think it's something I wish more professors at Georgetown adopted this, I think it's really great for the for the students

Bob Ertischek, Head of Client Success, Yellowdig 45:47
Alright! Thanks, Mark! And, you know, just one last call for any questions or thoughts from the fine people who've decided to join us here today. Otherwise, I want to share just and actually, I also want to say thanks, again, to Dr. Mark Rom, for joining us today. You know, for me, you know, it's a pleasure for me always to talk to you, whether it's about the work we do or just about music or anything else. And I hope I can get down to DC one of these days and actually be great, yeah, meet up in person. Um, but you know, I want to just share some upcoming events, if I can figure out how to share my screen again, that are coming up at Yellowdig.

Bob Ertischek, Head of Client Success, Yellowdig 46:35
Mark, what class do you have after this? What are you teaching today? So the class called politics, morality and policy, it's a senior seminar. So I've got 15 students in it. It's kind of a hybrid hybrid class, they, you know, on Thanksgiving week, I said, we're gonna have class B can do it either in person or you know, if you're traveling at home. We're in we're talking about racial inequality today, and then also defending democracy, Bob, big class. Sounds like a good Yellowdig class mark. Just let me just saying next time next time. Anyway, coming up on December 19. We're sponsoring. We're having a education 3.0 connecting the learning experience. webinar, we actually have a few of them coming up December 5, December 19, Shaunak Roy, Anant Agarwal from 2U, the founder of edX as well. And Steven Kosslyn, who is a longtime instructional designer, and now is the founder and CEO at Foundry College, all talking about connected learning. And you know how you can basically thought leadership on these topics. And we hope you will join us. I believe Bri has posted links in the chat. For those sessions. We'll also be having our Level Up sessions, which I don't know if there's a..oh, there we go! Level up sessions are sessions that my team, Client Success at Yellowdig, we work, we share best practices on using Yellowdig on certain areas, the December 8th session is on the Instructor presence, specifically, we'll be talking about how you can optimize your role in Yellowdig. Mark, come on to this one, you'll have a great time and then I think we have one following that in January on using Yellowdig data to informal teaching. And also, if you don't know much about Yellowdig beyond what you've heard today. We're having a demo on December 13th which will bascially be just an overview of Yellowdig. I want to thank you all again for joining us. I see there's one more message in the chat, so I want to make sure I address that before we. Oh! Thank you for joining us, there we go! And everyone, I hope you've a great thanksgiving or if you're not in the U.S., have a great Thursday and Friday this week and upcoming holiday session and again thanks so much for joining us. If we can do anything for you, if you have any questions about our pedagogy and platform, please feel free to contact me, bob@yellowdig.com and Mark if you want to be contacted, can you put your email in the chat there as well? Thank you all so much for joining us today and I'll be happy to stick around for a few minutes if anyone else has any questions after Dr. Rom goes to teach his class.

Dr. Mark Rom - Associate Professor, Georgetown University 49:44
Again, great to see you buddy. Thanks for coming on. And check it out. It's worth it's worth giving it a try. Thanks, everybody. Happy Thanksgiving.

Bob Ertischek, Head of Client Success, Yellowdig 49:53
Bye