u2: Video Case Study

Unit 2: Video Case Study

Next up…
That’s all for Unit 2. When you’re ready, and it doesn’t have to be now unless you’re really pumped about learning, head over to Unit 3: ADDIE Process.


Video Transcript
DR. CHOW (VO): Hi. I’m Dr. Anthony Chow, and welcome to the Unit 2 Case Study: Preliminary User Testing. In this module we’ll share with you direct user feedback and how we responded to it.

Let’s begin. In the spirit of practicing what we preach, we brought together a user group, although not reflected of our target group. User feedback of any kind certainly was better than nothing. We did a focus group along with individual interviews. We first walked through the site for the users and had an open conversation. Lots of excellent input and discussion occurred. There were also exchanges with the development team about preferred refinements. We had an excellent dialogue about what we all wanted and expected. Let me introduce you to our wonderful user group members.

BECKY: My name is Becky Croxton. I’m currently a second year doctoral student in the UNCG School of Education with a background in library and information studies.

EMILY: I’m Emily. I am a grad student to Dr. Chow. And I’m pursuing my — to Dr. Chow. And I’m pursuing my Masters in Library and Information Science.

DIANE: My name is Diane Barber. And I’m originally from Houston, Texas. I moved here to Greensboro in the fall of 2010 to start my MLS degree. I’ve really enjoyed it. I also work full-time on campus in a variety of positions. First, in a few of the residential colleges on campus and then later in the registrar’s office.

DR. CHOW (VO): Each of our three users were interviewed separately. Our first interview question was, what is your overall impression of the MOOC so far? The feedback was excellent and overall positive with some issues we needed to address.

DIANE: I really liked the MOOC’s topic. I’m really interested in usability and design and just how users interact with interfaces, and what makes it a more immersive experience for the user, and what are they really going to get out of it in the learning objectives and keeping those in mind as you’re designing the MOOC. I think it’s really a great process to learn about.  I like being able to look at the videos and read the content. And just being able to take the short test and getting the feedback immediately is really nice.

BECKY: Information is very user-friendly. The information presented on the screen and in the videos is very clear to understand. I liked it a lot. My only — my primary concern at this time was I think the navigation could be a little more intuitive. As I was testing the site I didn’t always know where to go next and what to click on.

EMILY: My overall impression, I think it’s good. I never expected to be doing web usability when I, you know, perceived [sic] my library degree, but I think it’s definitely an essential thing that you need to know in today’s world of Internet. And who knows what else is coming? So if you’re going to be pursuing some sort of web design career or if you just want to make your own website or whatever, I think definitely usability is — it’s an important thing to know. So I think the MOOC is definitely going to steer people in the right direction.

DR. CHOW (VO): The second interview question was, what do you think of Unit 1 specifically?

EMILY: It’s working for me. I’m used to really technical kind of dry, academic things. So it’s kind of nice that it’s a little more personal, and there’s a little more — with the tiny quizzes it’s less . . . You know, like, okay, you have to be studying for this thing. It’s more like, okay, I’m going to read this and I’m going to listen to this, and I’ll just naturally absorb the answers to these questions.

BECKY: Overall, I would say the content works well. I managed to get through all the material. It didn’t take me very long. Again, I think there could be more of a welcome, more of an introduction of what to expect as I go through the module or through — you know, like if each module or each unit is going to be structured the same way, I think it would be helpful for a new user to know what to expect each time. And if it was consistent from unit to unit, I think on the . . . maybe there could be a sidebar, you know, just with links to reading discussion board activity.

DR. CHOW (off-screen): So you can jump to that anytime.

BECKY: So you could jump to that. Or, you know, if you really didn’t care about going through the whole course from start to finish, that you could jump into any of the units and kind of have the same expectations from unit to unit.

DIANE: I like it. I especially liked that it has a story throughout that you’re kind of following along with, and you also get to watch the videos, and thinking about how the usability not only applies to the class but also to your daily life, and how can you integrate it into your work or your personal life or all your volunteer activities, like just looking at how I can integrate it on a daily basis.

DR. CHOW (VO): Again, the feedback was excellent and very helpful and understanding how the users engage with the module, navigation, content, and media.

The third interview question was, [was] the way the information was presented to you effective? We found that our video snippets were in fact too short and needed to be merged together.

EMILY: Well, Spiro didn’t work for me. But nothing . . . again, it’s a generational thing I think. It’s a fine balance between, you know, like, I want this to be a game and I want to be treated like an adult, especially for people in my generation and younger. And you know, you absorb more, just more easily if it’s like a game. But then once you get to that point where you think oh, this is really kind of treating me like a little bit like a child, that just jars you and you’re like, I don’t think I want to do this anymore.

BECKY: I think the navigation from slide to slide or from component to component could be a little more — you can make it a little more obvious. I think that would be easier for the user. Keep most of the information above the fold that’s critical to your site visitors.

DR. CHOW (VO): The next question was specifically about the video. How was the video with text presentation?

BECKY: You know, if I were a student in the course, I think what I’d prefer is to have right now — I think there’s seven or eight videos, little snippets — I think you could either do one full video or chunk it up into two or three. You know, I was testing the site remotely on a slower connection. You know, not hardwired. And I had a lot of wait time as each video loaded. So if they were chunked into one, two, or three, you know, I’d only have to wait a couple of moments for each one versus seven or eight times. And I think the script could just be a link or an attached item that you could click on if you wanted to read the script. You know, make it obvious. Some people would prefer to read the script and not even watch the video. And if you could have animated bullets of the key points as you’re talking, that would be great. I think users would like that. I would like that.

EMILY: I think definitely having it in one would be better. And I think not having, you know, what is being said on the side of the screen, because I’m the type of person, I can’t absorb anything unless I read it. So I definitely want the script somewhere. But I think it would be better to have just bullet points of what’s being discussed.

DR. CHOW (off-screen): And the summary.

EMILY: And then you know, I’m just kind of that person that if I don’t absolutely pay attention, I will tune out. So I would go with just the longer video and the option to read it. So you know, if you don’t want to watch the video, then it won’t hinder you in any way.

DIANE: I like that they just have the combination of being able to read on your own and watching a video and — than hands-on exercises.

DR. CHOW (VO): Finally, we wanted general strengths and weaknesses.

DIANE: So far I like the strengths of the MOOC as far as that it says right out, like the learning outcomes, what you should expect to learn from covering this content and what I will get out of it by investing my time. I also liked that the content is just something that I’m interested in. Like, that’s one I would pick, you know, out of a list. Usability and design is something that I can use not only like in my coursework but also at my workplace, and I enjoy that.

The weaknesses? I really want the left-hand navigation. I don’t know if it’s just happened but I really look to the left. Whenever I try to navigate somewhere, I want to see that next button. And I think that would help me, like, just navigate it a little bit better.

The initial — I got used to it after a while. I had to go up to the top and look at the subheadings, but that was my initial hesitation of, like, where do I go next?

EMILY: I don’t think there’s any really huge weaknesses, other than just the confusing menus, and that’s probably pretty easily rectified. The strengths I guess would be — though I don’t like Spiro or the Spartan — I do like the human element and the personal connection and the idea that you are learning this from one person rather than it’s just, you know, some team that put something together and didn’t really think about you as a person.

BECKY: I think a strength — one of the strengths is that you bring the personality of the instructor into it. It’s much — I guess I had a warmer feeling having watched the video and knowing there was this instructor behind it all versus just reading a bunch of text and doing a quiz, et cetera. So I like the personality. I liked the — everything was very readable, clear, clearly — clear to read and very understandable. I liked that too. Just the navigation needs to be more obvious and having a little more instruction what to expect each time. You know, here’s what we’re going to cover and here’s what I’d like you — here are the four major things we’re going to do.

DR. CHOW (VO): From the user group meeting there was a long list of potential action items we needed to address. As expected, it was very helpful and provided us with a lot of excellent feedback and action items for refinement. As expected, some users helped a lot and provided a very useful perspective. Nielsen found that five representative users found approximately 85 percent of all usability problems. And, although our user group may have not been exactly representative, they gave us a long but manageable list of suggested refinements. The next module we will take a closer look at some of our design and development decisions for the MOOC.