Download transcript PDF

 

00:04

So we're going to talk about how to gather the data from our learners and how to put it into action. So basically what that that's what I'm going to cover today just to give me a little bit of context. Let me know in the poll whether you currently use online learner data to inform your teaching. Okay. Okay. So it's not like you just use the data you gather from your learners for example how much homework they have done but do you use this data to actually change something in your curriculum in your lessons or you don't

00:41

okay I'll give you a couple of minutes you can also pop your answers in the chat box that would be lovely to see as Well, not online data. That's okay because we're going to talk about different kinds of data. And I also try my best to collect the data from very different sources. Not yet. If not yet, I hope it's still going to be quite useful. And if you're using this, okay, that's great. Thank you very much. So the things we're going to cover today um here why data matters what kind of data we can use I will also share

01:23

some classroom examples and we'll take a look at the four steps framework which we can use to um to put this data into practice and just a couple of takeaways to share at the end right so I can see the results of the poll well we're pretty divided I would say that onethird answered each of the options yes a bit not yet okay so let's I hope it's going to be really really practical. Online learning. Yes, statistics. Leela, yes, I'm going to talk about statistics as well. Right. So, what do we mean by

01:53

learner data? It's pretty much any information we can gather during the learning process. And this is the information that shows what learners are capable capable of, what their strengths are, what their weaknesses are, things they need to improve, their performance. And what's most important here is that this kind of data, this kind of information helps teachers adapt their teaching to support their progress. And what's curious here is that for data to be useful, it should be combined with data from many learners. As Kur says,

02:28

the more learners, the more valuable the insights. I would also add here that the more different sources of data you use, the better. So, and this is what we're gonna uh take a look now. So learner data u you can also pop your ideas into the chat box where you get your learner data from. I've got just a few suggestions here like yeah some of you said uh online learning from online platforms. First of all we're talking about scores. So definitely quiz results and these could be quizzes both online

03:00

in a game for instance. You can still check the report and see where learner struggling the most. test grades definitely automark tasks uh for example on an online platform right another interesting kind of data is the time that students spend on task for example how long how long they spend on their listening how long they have spent on their writing error patterns uh is obviously something we do I think pretty much most of the time right we we we notice uh that students uh don't really use present perfect very well or maybe

03:33

they misuse articles And these are the issues that we definitely have to deal with on a daily basis. Success rate, that's one is it that one is interesting. Take a look at the scores from your students or their engagement. Is it the same for different skills? For example, if we're talking about um exam courses, some of the students can have a lot of um a lot of points in their listening like 80% accuracy score whereas reading will take up only 50%. Why is that? Is it because of the tasks or the density of the

04:07

texts? Writing submissions. Oh, definitely falls into the category of learner data can include essays, letters, um teachers feedback. Yes. And also, uh some of you mentioning Martina in the chat box. Yes. Uh individual feedback and writing the diary. That's amazing. I'll talk about some like physical kinds of data as well. Engagement stats. Take a look whether your learners actually skip certain tasks or maybe they leave certain tasks incomplete. Some exercises are not finished. Why is that? And then of

04:40

course the progress tracking for the teacher can come in various shapes as a line graph as a bar charts and maybe it can show improvements in skills scores over time. Let's say you're teaching your students for a year or for for one semester for four to six months. Have the scores changed or not? You're also mentioning classroom discussions. Yes. Reviewing individual feedbacks. Great. Well, I have a link here to mentter. Um, if you feel like you can add two, three different suggestions where you get uh

05:13

your data from, but I also can see a lot of answers in the chat, quizzes and quizlet. And yes and um while you're doing this um let let me talk just very briefly about two different kinds of data in terms of uh how we use this data whether we use it for formative assessment or we use it for summative assessment like Ashan I guess mentioned quizzes quiz list can present data this is uh this can be used both formative so we're just tracking students progress and we're seeing the areas they need help with or that could

05:47

be used as summative assessment Right? For example, if you print out that test and give it to the students and give them a score and say, "Well, that's the end of the term or the end of the of the week, for example, it's a unit test." Definitely, classroom discussions will probably fall into the category of um into the category of uh formative assessment, student discussion boards. Imagine suggesting a topic and students will write down their answers. Okay, let me take a quick look at uh menty if we can actually um share it

06:31

here. Oh, that's interesting. Um so you're talking about Google forms. Yes, I haven't thought about it. Error pattern, scores, tests, quiz. It should be intuitive. Yeah, secondary feedback, cognitive, interesting, formative, class feedback. That's lovely. Padlet. I love Padlet for different kinds of projects, for presentations. Yes. And you can get a lot of data as well because students are writing different posts, right? They're working in groups. You can assess their collaboration. Okay. Lovely. Thank you very much. So, I'm

07:03

going to uh stop sharing and then get back to my slides. Okay. So uh a few examples that I would like to share with you they come from the platform uh by McMillan McMill education everywhere and from uh learning hub uh the bridge council. I will share both digital examples and paper examples. So this is an example of digital assessment. Um here it's done with a demo class obviously for language hub print mediate if you need and if you work for example in a low resource classroom or if your learners don't have

07:40

their own devices you can always print out the test and it's very easy to see how many questions there are in the test right and whether some of uh students writing for example needs um grading uh these are two further examples which can be used well both uh for summative and formative assessment uh of your students and this is where you can generate again a lot of information about your students strengths and weaknesses. Do they struggle with an email? Maybe they struggle with paragraphs or they do uh

08:11

for instance grammar structures really well. Do they have enough words in their email? And the same in the speaking uh task here. Um you can check their accuracy. You can check their inonation. They might be talking like a robot without actually realizing they're doing that. And this is probably what we also need to pay attention to. So they might not receive even a score for this task, but it will give you information on how they are doing. I think this is particularly important when you teach in

08:40

a large classroom and students can do these sort of tasks um on their own at home. Now another example comes from the same platform but if you click click the progress tab for example. So this is the kind of stats I was talking about. You can see the average assessment rate and the homework average. And this you can and this is when you can have a conversation with your students probably asking them well actually um has something going on I haven't seen you really engage with the homework like what's the reason for that? I think it

09:14

is particularly important when we're talking about um high stakes exams and the student definitely need to do their homework um and uh engage with coursework as well. And so again you can see here all all the statistics right and the dashboard can also show you whether it's the student's first attempt last or or their best attempt and this where you can get your data from as you said from different platforms. Um this platform actually shows um how often the student has accessed the system. As you

09:45

can see some of my students they have never been there uh which is a shame. So again we have to discuss together in the classroom maybe try and login in in the classroom and then do a couple of exercises you can see when they access and it's it's really valuable as well and just to on a more practical side I will show you this very quick example from the same platform from uh learning hub uh where students uh in high school upper second or upper intermediate high school students had to write a quick

10:13

assignment about qualities for success. It was not summatively assessed but it can be obviously and so this is the kind of text you can get and this is where it gets really really interesting as uh Jasmine uh discussed in her session. Students often and I remember some of the people sharing that students often just paste the prompt into Chad GPT for example or any other AI tool and they just copy paste it blindly but okay this is my upper intermediate student high school and I pretty much know his

10:44

level. I know he was able to write it perfectly well. But when I see texts like ex with perfect punctuation, paragraphs and uh lack of spelling mistakes, excellent brilliant grammar structures from my pre-intermediate young learner aged 11 for example, I start doubting whether they have actually um done it themselves. Again, you need to be very cautious here and you need to train your students on how to use AI if it's it is uh the task is to be done at home and I think you'll agree with me here and uh a few other

11:19

examples they also come from u McMill education everywhere. So you can see uh all all of the courses which prepare students for exam exams Cambridge exams are available on the platform and this is where you can actually design either a summitative test for example this one you just uh click the unit you want to do and you see it's a pretty much closed end test you can uh assign or set the time how much time students have to complete it and the passing score again maybe to promote achievement even with

11:50

low attainers who actually can struggle a lot with the fact that there is another test, I'm going to get another low score. So, this is maybe where we have to be quite wary and maybe have a quite low grade uh in terms of passing grade uh or score. But at the same time you can use this platform uh for for creating reflective prompts for students like this one where I asked the students to think about their use of English parts uh in the C1 exam and whether they have found this paper really difficult

12:22

and why what's the reason which part of the paper is the most challenging or what strategies they use or they used um when they got stuck on a question what is one thing they improved and uh when you when students go back to the platform when they open it so this is the kind of task they see here and they are asked to write again you can just print it out and that's what I do actually uh when we don't have access to the platform for example or to the device we just use exit tickets you see all sorts of examples uh in different

12:54

colors different promps and here are some uh insights from the students so uh there is quite a lot here uh just just a few interesting uh examples of what actually students can share with you. For example, in this lesson uh to practice our listening exam skills, we we were using a van diagram and I encourage students to take notes during uh using that ven diagram and you see where is where is one student mentioned that he was able to use note takingaking using this diagram and he said it will help me a lot in the future

13:28

listening activities. Another student at the same time said the audio was too clear and the speed was too slow. And this is again this is the kind of data you can get from your students uh in order to kind of tweak your lessons or tweak your your next lesson or maybe the whole syllabus or maybe let's say a unit not the whole syllabus. Um there are some other examples here and students can share a lot with you. They can tell you whether they're struggling with a particular grammar vocabulary structure.

13:57

they can tell you that a certain skill is an essential skill and that you will need to try and implement it many much more in your uh practice. Some students can share with you that they want some they want better man time management skills because they often lose track of whatever they're doing. And uh one final example I have here um except for this one um is that in this project uh we were uh or in this particular series of lessons we're working on the projects about lack of access to education and students had to

14:30

work in teams create their ideas and uh create um and then design um a post on Pedlet and so we we were working in teams and each team was working with a different reason why students don't have access don't have access to education And only then a couple of weeks later following the presentation did I share with the students a prompt a writing prompt where they had to discuss problems and solutions for lack of access of education. And as you can see here students actually commented on how useful the project was because it gave

15:05

them ideas for their writing. they could get suitable vocabulary, grammar structures, and new ideas that help them write down their essay. And so given all of that into account, taking all of this into account, let's take a look at why learner data matters. So maybe you have got some ideas. Maybe you can tell me why do you need to use learner data for your exam courses? Is it just because we need to track down those scores, their progress? But yeah, okay, you can have students a right score in 60, then 65, 70. Okay, he's

15:39

improving, but how do you know they're improving? It's it's um it's a very like tricky ground because we're talking about the exam scores, right? And we need students to take these exams, but how do we help them improve? Well, from my research, from my reading, and from my uh teaching experience, I think it's very important that we use the data that we generate from students for decision- making, for giving them feedback, not just well done, oh, try better or apply yourself, but we're actually sharing

16:11

with them success criteria for a certain task. So the students when they write their next for example they're doing their next reading exercise a listening exercise or their writing and even speaking task they know what they have to pay attention to and by this we're making sure that students are taking ownership of their own learning and this is I think is why it's so important because it absolutely links to assessment for learning which allows teachers to track progress continuously. Assessment for learning. You might have

16:44

heard, you might have seen this concept. It's a it's a great thing uh because it's a tool that helps students and teachers. It helps teachers to support students learning. It's not a tool to measure their scores and their progress, but it helps them to improve. Exactly. Exactly, Kashia. To improve their lessons and make their them more engaging to tailor your teaching, Magdalena. Exactly. To tailor your teaching to the students needs. And this is why we need to first identify the gaps in students knowledge. And when

17:13

it's done in a test like um like an exam, it's a very stressful situation as stressful as as an online presentation like this one, it does not actually I think I help you identify the gaps really well because only when students are really relaxed and they are like sharing you know sometimes in this exit tickets I say what surprised you in this lesson they tell me this exit tickets. Okay. All right. They can share anything about their life, but um if they share something from the lesson, I think it's

17:43

going to be really valuable. And this is uh when you can adjust your teaching based on that evidence, evidence-based learning, as again as Jasmine mentioned in in her wonderful presentation, involving learners again coming back to her talk, uh involving learners in peer and self assessment. In general, our job or maybe our task here is to emphasize progress because imagine imagine you again have low attainers. And when they keep getting the same low score over and over again, they can just question that.

18:15

They can question your teaching and they ask you how can that help me improve. But when you tell them, well, let's try and do this this this is how we can make them improve. Right? So um over uh generally speaking assessment for learning is when assessment is used as a tool to support learning rather than just to measure it. And to be able to implement all of this uh we have kind of designed this uh four steps framework which contains yes as you have guessed four steps four simple things. First we

18:47

analyze patterns and this is what we have been talking about. with spot the patterns and see where the students need help. And then maybe that's one of the most important steps here. You need to adapt. We're not talking here about adapting the whole curriculum, right? It's just probably not really feasible. And how about tweaking a couple of lessons? For example, students have submitted some writing and you can use a modal text, a slightly rewritten version, maybe using even AI or maybe gapful exercises to draw students

19:19

attention to for for instance to those linkers that they are often missing or that because their cohesion suffers and then we need to implement the change. You maybe you can have a quick focus session like five minute session on integrating these changes. you teach the target linkers. But again, we need to draw their attention to the fact that we're not just using linkers mechanically. First, second, third, finally, and uh to summarize, but we need to show them that the cohesion is actually built using all sorts of

19:51

different devices like synonyms. So maybe you focus on one thing at a time and then include cohesion focused tasks and checklists. So for example, you ask students to write a paragraph just a paragraph based on that that lesson or a series of lessons and and they swap their writing and they give each other feedback using a check checklist. There is an example given here in the resources. And finally yes as I uh already as I always say we need to reflect on what was difficult whether they have noticed any improvement or you

20:24

can ask a very direct question. What do you think you should do to improve your cohesion? Right? Or uh take take a look at your first text and the second test. What would you like to change? It can be done via self feedback, can be done via peer assessment, a peer review. I would probably recommend that we do it without scores at first because you know research suggests and research dating back to to the um to the late 80s suggests that when students are given test their um tests back just with

20:59

scores with scores and comments and just with comments guess what students don't pay attention to comments and I'm sure again you will agree with me that whenever you give them scores they just compare themselves to each together. They just think about their status in the classroom. I I I don't speak Vietnamese and I teach Vietnamese learners, but I can hear them discussing together. I know the numbers, you know, and I can hear them discussing together whether they have got five, two or three, etc. 10 depending on on their

21:29

on on their country and they're not interested. Exactly. And so try giving them just scores. And this is what I tried um and I've been experimenting with. You see, you can give them some feedback. It can be much shorter, not like this. ask student to assess themselves and their peers. So this for example and this is like uh the whole version and there is a second part of this version but I'm just focusing on this one here. Okay. Uh organization here is related to the linkers because I was given an example of linkers and

21:59

writing cohesion. um something that some transition could be more natural and then once you have shared and well ideally you will have shared a list of criteria before that with the students so they know that A stands for this B C etc they stand for this and they can and and again as as um as Jasmine mentioned sometimes when you take a look at your own writing and think everything is faulty like I've got only weaknesses but we need to make sure that they can see the strengths as well that they could develop those and to

22:31

make them even better to get even a higher score and then definitely work on their weaknesses. So I have a uh one more example of uh how we can actually implement this um framework just before I finish. Uh it's it's an example of a language lesson or data that we gather from language lessons. In this case, it's vocabulary. Again, you've noticed on the platform, on the quiz, uh in I don't know, on some kind of paperwork, in the speaking, the students really struggle with their collocations and you check the data. You try to

23:07

notice the patterns. For example, you're working on certain unit about failure and achievement, which is quite relevant, I think, given that we're talking about assessment. Try to adapt again a couple of lessons. It would be good maybe to include some vocabulary in lexical sets and building in retrieval practice. So spaced repetition maybe some dictations maybe pair work and during the implementation stage and here again I'm sharing the um uh the uh students book from uh the MI platform because it has

23:39

got this amazing uh downloadable word lists which you can use for those kinds of tasks and you can easily copy paste this and then embed into for example blue or quizlet and it will generate a game straight away and This is how you can implement the changes. Probably think about some other vocabulary games. Maybe even encourage learners to keep a diary. It actually is always a good idea, I think. But once this diary is consistent, then maybe it's going to prove useful at the end and use reflective prompts. Which word do you

24:12

use for this and this? And definitely at the end, we should try to give students some time to think about their progress and the change. Let's take a look at the scores. Now only can you give them the scores for the first um for the first draft probably you see and see whether these scores these scores these grades actually correspond to what to how they graded themselves because now they know the criteria they know the areas where they need to improve and then decide whether uh the vocabulary or the

24:43

cohesion devices they need uh um revision. Again definitely we should try and uh encourage our students to use these words and writing and speaking maybe even sharing the peer checklist uh during the stage where they can tick off you know the units they have they have used um yes as you said Magdalina yes it's also a great idea to ask students to prepare Quizlet themselves for groups of vocabulary yes yes exactly I've tried this as well again in a low resource context you can do it on paper students

25:15

love creating quizzes for for their peers multiple trick quizzes gap fill uh just for a quick practice exactly during the implementation stage like write down five sentences for example with the with these uh like idoms or these collocations in this case and then do not write it the collocation themselves swap with a partner can your partner guess so I I believe that many of you are using this so this is the kind of thing we can try and implement with our students it's just important to be very um firm here I think consistent

25:49

because when it comes to assessment for learning you you may feel that well there is not enough data or I don't see my students improving and if they don't get scores uh we have got uh different stakeholders and they're going to talk to you about the fact that they don't see the results well we need to be able to talk about this how we're doing this and maybe collect samples from students that's what I do as I said I take pictures or screenshots I um uh share students work using portfolio tabs if

26:17

it's available students can do all sorts of challenges for example on the MI platform and yes and also when you do different quizzes online uh they have got reports for you so for a fuller picture maybe try and use both or at least experiment with formative assessment and summative digital tools trying the step work uh the four step framework I'm sorry uh through which you can analyze adapt implement and reflect Because at the end of the day, our main aim or at least that's what I feel my main aim as an as a teacher is to

26:52

develop uh to develop to promote learner autonomy because you should sharing success criteria and you want your learners to be independent, motivated, being able to self assess because this is what will make them um a a a good learner, a good exam taker. stressfree hopefully and also it will prepare them for the for for the workplace if we're talking about uh school students and definitely for um for for anything else in the future I guess motivation and autonomy um I guess are the key and just to finish I think

27:29

that's a that's a really nice quote here from from Craig that digital platforms um that we use should support but not uh definitely replace effective classroom teaching but they are still a very good step towards the ideal of autonomous learners being responsible for their own progress. So these are some of my key uh takeaways. Let me know if you have something else you would like to um to take away today. These are the things we have hopefully covered in today's session and um thank you very much for

28:02

being so engaging and so so interactive. It's been a pleasure. Um there is a link for the feedback. I will also share it later in the in the chat box. And these are the resources. I can't recommend uh Black's book 2003 assessment for learning enough. It's like this pinnacle of of um the whole of the whole research uh around this uh around this concept peer feedback teaches use of data and adaptive learning and also lessons and blended learning. So, take a look and let me know if you have any questions. Thank you very much.