For the first time on the Tegrity blog, we are giving readers the opportunity to catch up on what they may have missed in recent web seminars. This month, we’re featuring Athens State University, which presented on its use of Tegrity’s remote proctoring feature on July 19, 2012. The webinar was hosted and moderated by University Business. Responses to questions submitted during the live event are provided by webinar presenters Mark Gale and Linda Hemingway, Athens State University; and Erin Blaze, McGraw-Hill Marketing Specialist.
Q: who is reviewing the recording?
A: Faculty or TA who is grading the exam.
________________________________________________________________
Q: how many times does the download fail?
A: Out of 140 students over two semesters, we only had 5 upload failures.
________________________________________________________________
Q: How long does it generally take to grade all the exams especially with having to go through the video of each student?
A: You definitely want to incorporate methods to reduce the time load associated with watching the videos. First, the orientation quiz is the only test where we watch every video. We purposely made that quiz easy and short (5 questions). It takes maybe 30 seconds to watch those and respond with feedback to the student.
Outside of the orientation quiz, we usually scan the thumbnails of each student to look for anomalies such as them getting up from the computer, opening a new browser window, etc. Scanning the thumbnails takes about 30 seconds per student. If we find an anomaly, we will go in and investigate that certain thumbnail in more detail.
We proctored a midterm and final in the two classes and had about 70 students in each class. We randomly looked at about 2/3 of the students on the midterm and 2/3 of the students on the final, making sure every student was viewed at least once. This helped get a good sample and lighten the load as well. This is why we still require Lockdown Browser on the exams even with the Tegrity proctor it gives us the added security.
________________________________________________________________
Q: Did you have any hesitance from the faculty having to offer their courses via this method? Are all distance education courses offered in Tegrity or a hybrid of others?
A: Eventually, we hope to expand this to all distance courses, but for now it is only being used in our accounting classes. With those faculty, there was slight hesitation, but we used one of their old arguments against them. A lot of faculty still are hesitant to do distance because they are worried about test security. So rather than focusing on learning the new technology, we focused on a way to solve their concerns. This made them much more accepting.
________________________________________________________________
Q: Have you caught cheating or questionable behaviors among any of your students?
A: Since this was the first year of using this in one class I think they were too taken back by the experience to cheat. They did not have time to figure it out yet. I have talked with other schools and in one the student was trying to use his smart phone under the desk. They could see his posture and hear the clicking of the phone even though he had turned his volume all the way down.
________________________________________________________________
Q: Do instructors actually watch the recording of individual students taking tests?
A: Yes, but the video is not live. Each student is recorded taking their test and then the instructor reviews the videos as part of the grading process.
________________________________________________________________
Q: Why does the upload process take such a long time?
A: A large portion of our students are from rural Alabama, so they are on dial-up connections. In the metro areas where students have access to high speed, the upload time is pretty quick.
________________________________________________________________
Q: What do you do if students flub an exam and don’t upload or record correctly? What percentage of students have tech problems with proctoring.
A: We give an orientation test and make sure we watch each of the videos from that test and give individual feedback to the students. The test has unlimited tries, so usually if they are doing something wrong in the process, we can catch it there and correct them.
As for the flub, we put a lot of weight behind our threat. Usually, we tell them your test will not be considered for grading if you do not use the proctor system. Chances are that we’d probably let them retry using a new question pool, but as of now, we haven’t had a flub outside of the orientation quiz.
We also have great support and if the video does not upload correctly the students can report that to the systems person and systems person has been able to get the Tegrity to upload.
________________________________________________________________
Q: Why did you implement remote proctoring when you were already using respondus lock down browser?
A: Respondus helped us minimize the possibility of students opening a new web browser for information; however, we were hearing too many stories and possibilities of students using books, having a second laptop, mobile device, or a classmate helping them. Respondus Lockdown couldn’t cover these situations. With Tegrity Remote Proctor, we felt it enhanced the integrity of the test because we could now watch for these areas.
________________________________________________________________
Q: How long does it take for a “typical” exam to upload assuming an average high-speed dsl connection?
A: For an hour exam on high-speed, it took less than 10 minutes.
________________________________________________________________
Q: Can you explain a little more about the “Annotation” option.
A: Annotation is an option on Tegrity Lecture Capture for faculty, not the Remote Proctor Product. Essentially, faculty members can use their mouse to “draw” on their screen. Therefore, they can draw on websites, powerpoints, or any other software they pull up on their screen. Many of our faculty are using tablet computers so they actually have a pen the serves as the mouse.
________________________________________________________________
Q: Does the student’s ability to submit a lecture integrate with Blackboard’s Grade Center? If not, how/where do the students submit their lectures?
A: The student lectures or presentations are stored in Tegrity under the Student Recordings tab. We have gotten around this by creating an assignment in Blackboard and asking a question related to the presentation (such as “Do you have any suggestions on how to improve the instructions on how to record your lecture?”). The answer to the question doesn’t equate into their grade at all, but it creates a column in the gradebook and serves as a reminder to the faculty member that a student has a new recording they need to go view.
________________________________________________________________
Q: Does Blackboard enforce the use of the proctoring software in order to access a test?
A: No. They could still technically start the test without starting remote proctor. We have heavy weight behind our threat though and basically tell the students that your test will be counted as a zero unless it is completed with remote proctor. I don’t know if we’d stick to that because we haven’t had it happen yet, but it seems to encourage most people to do it the right way.
Additionally, having the orientation quiz set up to use the proctoring system reinforces to them the requirement.
________________________________________________________________
Q: Do you have students do any no-stakes “practice” test to make sure they understand how to get through the process once, before it’s graded?
A: We have an orientation quiz that has a minimal point value associated with it (we’ve found a no-stakes test doesn’t get many takers). However, the test has an unlimited number of tries and only accepts the highest grade, so it is essentially a freebie and just there to make sure they get it right.
________________________________________________________________
Q: Is there a problem with student collaboration if the test is available over a period of time, and some students who have taken it cue others prior to their attempts?
A: We have built in a number of questions into our pools to counter this. Therefore if a student wants to memorize 5 questions and answers out of the 500 question pool, it will hopefully not be very effective for his peers.
________________________________________________________________
Q: Is it fairly easy to tell if a student appears to be referring to anyone or anything outside of the camera’s view?
A: Yes. Most of the time, the student will be focused and staring at one area of the camera (because they are focused on the screen). Therefore, you’ll see their head and face peer away from that area they’ve been staring at. You’ll just need to determine if it is a quick stretch or if they are looking away for an extended period of time or the same area a lot in a small period of time.
________________________________________________________________
Q: Did you look at any other proctoring solutions? If so, what was the process for choosing Tegrity over other proctoring solutions?
A: Tegrity Remote Proctor was a free update, so we gave it a try first because of that. It met all of our expectations, so at that price, we didn’t feel we needed to look elsewhere.
________________________________________________________________
Q: Does the “Form Better Relationships” methodology include the student perplexity factor (which I do by seeing student faces and watch the eyes fro that glazed over look).?
A: Unfortunately, as far as we can tell, they’ll be watching your recording and if they are confused, you won’t know. You’ll need to incorporate other mechanisms in the course such as small quizzes or discussions or something similar to make sure they are understanding the content and then intervene.
________________________________________________________________
Q: How do you ensure the student does not copy from the internet?
A: We incorporate Respondus Lockdown Browser as well, so they literally cannot open another internet window. However, if you don’t use Respondus, Tegrity Remote Proctor records both their webcam video and a video of their screen at the same time. So if they open Google, you’ll see it.
________________________________________________________________
Q: So an instructor needs to verify the ID of each student? Wouldn’t this preclude giving a test simultaneously to a large class (say of 25 students?)
A: The online classes, even if there are 25 students, will take their tests on their own schedule within a block of days. If you are going to use it in a face-to-face scenario, the faculty member would check IDs as the students came in the door, eliminating the need to check the IDs later with the proctor system.
________________________________________________________________
Q: you mentioned BlackBoard. We have Desire to Learn…is it compatible with Tegrity? What about Moodle?
A: Both Desire2Learn and Moodle are compatible with Tegrity.
________________________________________________________________
Q: When do you use the Respondus Lock-Down Browser since you use the Remote Proctor for tests?
A: We use both on the tests. Using Respondus actually helped manage the amount of time required to view the video of the students when we had larger classes. With Respondus running too, we no longer had to scan the video of the student’s screen, we could solely focus on the webcam video since we knew Respondus wouldn’t let them open any new windows.
________________________________________________________________
Q: I know you may have mentioned this, but is the remote proctor mandatory or can students still use live proctors at testing facilities if they wish?
A: In the classes it is implemented in, it is mandatory. The student is still allowed to come to campus to take the test, but they will be placed in a lab that has the remote proctor on it and they will still have to go through the process of using remote proctor – just now on our campus instead of using it at home.
________________________________________________________________
Q: Lets say you have a class of 150 students. Does that faculty member review in detail all 150 recordings?
A: You definitely want to incorporate methods to reduce the time load associated with watching the videos. First, the orientation quiz is the only test where we watch every video. We purposely made that quiz easy and short (5 questions). It takes maybe 30 seconds to watch those and respond with feedback to the student.
Outside of the orientation quiz, we usually scan the thumbnails of each student to look for anomalies such as them getting up from the computer, opening a new browser window, etc. Scanning the thumbnails takes about 30 seconds per student. If we find an anomaly, we will go in and investigate that certain thumbnail in more detail.
We proctored a midterm and final in the two classes and had about 70 students in each class. We randomly looked at about 2/3 of the students on the midterm and 2/3 of the students on the final, making sure every student was viewed at least once. This helped get a good sample and lighten the load as well.
________________________________________________________________
Q: can the faculty contact the remote student during a test? (what are you looking, can you turn up your volume, adjust your webcam, etc.)?
A: No. The recording is not synchronous. The instructor watches the video as part of the grading process after completion of the test.
________________________________________________________________
Q: Have you had any privacy issues come up with the remote proctor?
A: No. We have a limited sample since we are still piloting it, but for the most part, students seem more than happy to give up any privacy issue for the convenience of not having to drive to campus or find a live proctor.
________________________________________________________________
Q: The LMS doesn’t matter does it when using this proctoring format with Tegrity?
A: No, LMS does not matter. If you are using Tegrity, you can use the remote proctor feature.
________________________________________________________________
Q: Do you dedicate technical support people to online classes and if so, how many courses per staff member?
A: We have two people supporting most of the online courses right now as well as a 24/7 help desk outsourced to Blackboard for generic questions. Our staff is 1 system admin and me – instructional design by trade.
________________________________________________________________
Q: How can you ensure that cheating does not occur if you do not review all the recordings in detail?
A: It is a balance between integrity and time management of the faculty member. We make sure that at least on recording of the student is watched throughout the semester outside of the initial orientation video.
That is why we view the orientation video as essential for reviewing everyone and giving individualized feedback. When the student gets that note about the instructor seeing something (like a pet or child) in the video on the orientation video, it leaves that sense in a students mind that they MAY watch me again.
________________________________________________________________
Q: how long will tegrity store these videos for? 1 week after the test? 30 days after the term?
A: We have control of how long we store them. We hold on to videos for two semesters after the end of a course.
________________________________________________________________
To watch the presentation on-demand and hear the questions answered during actual Q&A, click here!
by Dr. Carter and Dr. Alice Yeager, the webinar addresses possible implementation strategies as well as best practices for using Tegrity to create content for the flipped classroom model.
