How often do students guess their way through?

When designing online courses, like AlgebraX and GeometryX, a common cause for concern is that students can just "guess their way through" lessons and assessments. They're either totally lost, or not paying much attention, but they can still advance and get credit by brute-forcing their way.

This can be very problematic for the student (who is grinding away without really learning) and the course (which loses credibility when this type of student behavior is common). So here at School Yourself, we've been asking ourselves: 1) How often does this happen in our courses, and 2) What can we do about it?

To figure out how often students guess their way through lessons, our team developed a formula we call a student's guess index. It's a number between 0 and 1, where a greater number means the student is probably guessing. A student's guess index takes into account the question type (guessing on a multiple choice question vs. an algebraic question are handled differently), number of attempts, what students do between attempts, and a few other factors. Over time (and with lots of student data) we have found that a guess index of greater than or equal to 0.4 means a student probably guessed their way through a lesson. An index of 0.2 or 0.3 means they might have struggled through parts of the lesson, but they worked throughout without resorting to guessing. And an index of 0 or 0.1 means they confidently proceeded through the lesson.

So here's the distribution for the most recent 10,000 completed lessons:


Students appear to be guess their way through lessons 2% of the time. And we can do the same analysis for reviews (assessments):


Students seem to guess their way through reviews less than 1% of the time. It's a little comforting to know that students guess less on reviews than they do in lessons, since the reviews are what hold them accountable for their knowledge.

One or two guessing students out of a hundred may not sound too shabby. Indeed, these numbers are likely so low because 1) we prefer asking open-ended questions (with interactive graphs, algebraic expressions, numerical, or short answer) where possible, to ensure students understand what they're doing, and 2) our lessons are built around continuous, step-by-step user interaction, where the questions never get too far ahead of the students. But even though relatively few students are guessing, these are also the very students who need the most help! So now that we can precisely measure guessing behavior, what can we do to mitigate or prevent it?

First, it's helpful to know whether students are guessing at similar rates across all lessons and reviews (indicating a problem with the overall course design, question types, etc.), or if the guessing behavior is concentrated among certain lessons.


Half of the lessons exhibited zero guessing behavior, And guessing is concentrated among a handful of lessons, like Ambiguous case, Multiples, and Unsolvable equations. This is great news, because we can focus our attention on these lessons, see where students are guessing the most, and update the lessons (and add new branches providing feedback for common wrong answers).

When it comes to reviews, students don't guess as much:


The majority (69%) of reviews had zero guessing. And once again, it's a handful of reviews that are the most conducive to guessing. So that's where we'll focus our efforts.

So while students guessing their way through lessons doesn't seem to be a major problem for our content, it's something we can mitigate by focusing our effects on the most guess-prone lessons and reviews.

11 comments:

  1. Is there a way to check and see if my child is guessing he way through? Is there a tracking or other wise way to see his progress through in a teacher mode or something?

    ReplyDelete
    Replies
    1. THE MOST RELIABLE SPELL CASTER THAT CAN HELP YOU GET BACK YOUR LOST LOVE BACK WITHIN 24HRS. (DOCTOR0LOVESPELL@GMAIL.COM)
      My name is Tonyas Mary from Canada, I want to quickly tell the world that there is a real on line spell caster that is powerful and genuine, His name is Dr love0 He helped me recently to reunite my relationship with my husband who left me, When i contacted Dr love0 he cast a love spell for me, and my husband who said he doesn't have anything to do with me again called me and started begging me to come back. he is back now with so much love and care. today i am glad to let you all know that this spell caster have the powers to restore broken relationship back. because i am now happy with my husband. To anyone who is reading this article and needs any help, Dr love0 can also offer any type of help like, Curing of all types of Diseases, Court Cases, Pregnancy Spell, Spiritual protection and lot's more. You can contact him Via email (doctor0lovespell@gmail.com) check his blogs: https://doctorlovespell0.blogspot.com/  or check website:  https://doctor0lovespell.wixsite.com/lovespell or visit his FB page: https://www.facebook.com/Love-Spell-Doctor0-107453264273672/  

      Delete
    2. I have been married with my wife for 6 years and recently she broke up with me and it hurt me deeply when she told me to leave her alone and that she does not love me anymore when i was always faithful and honest to her. I tried all the ways to get her back buying her what she wants like i always did and she still left me heart broken and she even has a new boyfriend which destroyed me even more until a friend of mine from high school directed me to Dr Emu. This man changed my life completely. I followed everything he told me to do and my wife came back. I was stunned everything happened exactly like he told me. I had faith in everything he told me and everything was true. You can contact him via email: Emutemple@gmail.com He will help you in anything you need and quick to answer once you contact him through his whats-app number: +2347012841542
      For more destails about Dr Emu you can visit his Website: http://emutemple.website2.me/

      Delete
    3. I NEED A LOVE SPELL THAT WILL MANIFEST WITHIN 24 HOURS CONTACT (DR MARCUS  ) HE HIS THE BEST LOVE SPELL CASTER WHO HELPED ME RESTORE HAPPINESS BACK TO MY RELATIONSHIP 
      Hello everyone i want to testify of the great and powerful spell caster named Dr MARCUS who brought back my ex who left me and got engaged to another girl,We where happy together when all of a sudden he just change he used to call me every morning and and night before going to bed but all that stopped when i call him he yell at me and told me he didn't want to have anything to do with me anymore i was so sad and confused i didn't know what to do then i went online to search on how to get back my ex then i found  an article where someone was talking about how the great and powerful Dr MARCUS   helped her and she left his email address i took it and contacted him i told him my problem he only smiled and told me to relax everything will be OK i did everything he asked me to do  and he assured me that after 24hrs he will be back,To my greatest surprise the next morning it was my boyfriend he came back knelling and begging for me to accept him back now we are so happy together he can also help you contact him at drmarcushome@gmail.com or drmarcushome@yahoo.com also   directly on whats-app +2347066448668

      Delete
    4. Good day friends. The truth is that GREAT MOTHER IS REAL AND UNIQUELY POWERFUL. Contact the Great Mother today and she will help you. when my husband left me for another woman, i thought that was the end but i contacted many spell casters but there was no positive responses and everything seemed not to be working and i was losing my mind until my cousin told me about the GREAT MOTHER who also helped her to bring back her man and also helped her to cure her herpes disease. I gave it a try and i contacted the Great Mother on her Website and i saw various testimonies about her on her good works how she has helped a lot of people. I explained my problems to her. she laughed and told me that, everything will be okay again and that she will help me bring back my man in 3 days time. I actually thought that it was a joke but since i had no other options, i gave it a try and i did what she instructed me do and to my greatest surprise, my man came back to me begging in 3 days time just as she told me. Contact the Great Mother now on her website:   https://Ourgreatmother1.blogspot.com   and you can also reach her on her email: Greatmotherofsolutiontemple1@gmail.com  or you can equally contact her on her whatsapp number: +17025514367

      Delete
  2. I think this is the best way of the learning online

    ReplyDelete
  3. How I Got My Ex Husband Back...........

    I am Stephine by name also from Canada. Greetings to every one that is reading this testimony. I have been rejected by my husband after three(3) years of marriage just because another woman had a spell on him and he left me and the kid to suffer. one day when i was reading through the web, i saw a post on how this spell caster on this address voodoospelltemple66@gmail.com , have help a woman to get back her husband and i gave him a reply to his address and he told me that a woman had a spell on my husband and he told me that he will help me and after 2 days that i will have my husband back. i believed him and today i am glad to let you all know that this spell caster have the power to bring lovers back. because i am now happy with my husband. Thanks for Dr.voodoo. His email: voodoospelltemple66@gmail.com you can also add Dr.voodoo on whatsApp +2348140120719

    ReplyDelete
  4. Rewriting is nothing else than the modification of a text without changing its context. Many factors make students start seeking a good article rewriting service. The best rewriting services are provided by our writers, so do not hesitate to ask for help.

    ReplyDelete
  5. I NEED A LOVE SPELL THAT WILL MANIFEST WITHIN 24 HOURS CONTACT (DR MARCUS  ) HE HIS THE BEST LOVE SPELL CASTER WHO HELPED ME RESTORE HAPPINESS BACK TO MY RELATIONSHIP 
    Hello everyone i want to testify of the great and powerful spell caster named Dr MARCUS who brought back my ex who left me and got engaged to another girl,We where happy together when all of a sudden he just change he used to call me every morning and and night before going to bed but all that stopped when i call him he yell at me and told me he didn't want to have anything to do with me anymore i was so sad and confused i didn't know what to do then i went online to search on how to get back my ex then i found  an article where someone was talking about how the great and powerful Dr MARCUS   helped her and she left his email address i took it and contacted him i told him my problem he only smiled and told me to relax everything will be OK i did everything he asked me to do  and he assured me that after 24hrs he will be back,To my greatest surprise the next morning it was my boyfriend he came back knelling and begging for me to accept him back now we are so happy together he can also help you contact him at drmarcushome@gmail.com or drmarcushome@yahoo.com also   directly on whats-app +2347066448668

    ReplyDelete
  6. Thank you for sharing useful information with us. please keep sharing like this. You might like the following article also.

    How to Prepare Shahi Paneer
    Recipe for Kheer Kadam

    ReplyDelete
  7. Good day friends. The truth is that GREAT MOTHER IS REAL AND UNIQUELY POWERFUL. Contact the Great Mother today and she will help you. when my husband left me for another woman, i thought that was the end but i contacted many spell casters but there was no positive responses and everything seemed not to be working and i was losing my mind until my cousin told me about the GREAT MOTHER who also helped her to bring back her man and also helped her to cure her herpes disease. I gave it a try and i contacted the Great Mother on her Website and i saw various testimonies about her on her good works how she has helped a lot of people. I explained my problems to her. she laughed and told me that, everything will be okay again and that she will help me bring back my man in 3 days time. I actually thought that it was a joke but since i had no other options, i gave it a try and i did what she instructed me do and to my greatest surprise, my man came back to me begging in 3 days time just as she told me. Contact the Great Mother now on her website:   https://Ourgreatmother1.blogspot.com   and you can also reach her on her email: Greatmotherofsolutiontemple1@gmail.com  or you can equally contact her on her whatsapp number: +17025514367

    ReplyDelete