ADDIE Week 5: Evaluation

This is the last post in a series. If you did not read the other posts in the series, start here.

The E in ADDIE stands for Evaluation, which I will also use interchangeably with assessment in this post.

I have an admission to make. I may be the ideas queen, but I am not yet the assessment queen. The fact of the matter is, traditional methods of assessment is tough in the library setting. In a one-shot class, they are not your students. The librarian does not give out grades; in fact, we only see them for 50 minutes once, maybe twice. If you embed a three-minute library tutorial at the point of need, why would a student take the time to give feedback unless they had very strong feelings about it? That being said, there are many steps in the ADDIE process where you can do assessment of yourself. Perhaps the best place to start is to describe a couple of kinds of assessment.

Formative assessment allows students and their instructors to check in and monitor their understanding. An example of a formative assessment would be a research proposal, a PollEverywhere survey, or a quick verbal check of understanding of main points during a class. This allows the instructor to come back and reteach the topic another way, if necessary.

Courtesy of Hey Girl blog

Summative assessment allows an instructor to see how much a student learned. The most common example of a summative assessment is a midterm or final exam.

Formative assessment is the type of assessment that I feel best fits library instruction. As I mentioned before, we are not the ones giving the grades (thankfully!). Quick spot checks are the best way to figure out if students are learning what they need to know. Some examples of old-school library formative assessment is the 3-2-1 Assessment, the minute paper, or the muddiest point exercise.

The issue here is that formative assessment is hugely important--but it does not unequivocally show student success, the kind of thing where you can equate how well someone learned something. This has become what I call "the Goose that Laid the Golden Egg" for the library. If the library could find a way to prove their impact on each student, this would mean we could fight things like budget cuts and get new positions more readily. Formative assessment is also not the type of thing that you want to use to assess whether someone is a good teacher or not. My feeling is that it is mainly an ethereal kind of assessment--temporary and made to incite change in the student or instructor.

Another form that assessment takes is self-assessment during the ADDIE process. Some examples of this are to check back to make sure that the lesson or tutorial that you are designing teaches the learning objectives you hope the students will master. You may wish to do a usability test to try out the website, tutorial, or instruction on a learner or two and see if they understand the learning objectives after having gone through the instructional intervention.

I am excited to learn more about assessment and I have made this an area of growth for myself personally and for our department. To do my part, I am going to the ARL Assessment Conference in October. The past proceedings are published up until 2014 and these are a treasure trove of ideas that I have yet to dive into. I also worked to reorganize our department to make room for a Coordinator of Instructional Assessment, who just started in March. Through these methods, I hope to become an assessment guru!

This was the last blog post in the ADDIE series. If you have questions or comments, or wish to know more about a specific part of the process, please let me know!

ADDIE Week 4: Implementation

This blog post is part of a series. If you did not read the previous posts, you may wish to start here.

The Implementation Phase of ADDIE is usually a lot shorter than the other phases that you have tackled up until now. Back in the Analysis phase, you should have determined what the best delivery method would be for your project. Now is the time to put that plan in to motion. Did you make a video? Perhaps you should upload the video to YouTube or Teacher Tube. Are you delivering an in-person instruction session? Put the final touches on it, show up to the class, and teach it. If you made something online, the implementation phase also includes marketing the new video or tutorial, putting it on your LibGuides Template guide, embedding it on your library's tutorials webpage, and making sure that other librarians know of its existence. If it is a large-scale comprehensive tutorial like an undergraduate research tutorial, you might want to send emails to your composition instructors or teach an info session for your university's office of faculty excellence. Make sure to update all the links for your tutorial if anything has changed.

Our next post is about evaluation, but don't forget assessment at this stage, either. See you next week!

ADDIE Week 3: Development

This is the third part in a five-part series. If you have not read the previous weeks' posts, you may want to start reading at ADDIE for Librarians.

We have been talking about the basic instructional design (ID) process called ADDIE. Week 3 of this blog series is about the Development phase, which is the phase most people think of as ID. Development includes the creation of learning objectives and creating the tutorial itself. This can take a lot of forms, including a lesson plan, a video, a LibGuide, or an interactive HTML tutorial. It has been my experience that the development phase can take the longest to complete. Instead of telling you step-by-step how to create a video or make an interactive tutorial, I want to give you two ideas of innovative software products to try.

ThingLink: http://thinglink.com/

An old friend told me about ThingLink recently. It is a free product, with an upgrade available for $35 a year. It creates a tutorial that allows the user to click around on the screen, revealing information bubbles. I have used it to make a virtual tour of a journal article that helps the student locate citation information. Here is an example:





This could also be used for more graphical information, if a photograph was used as the background image.

Timeglider: https://timeglider.com

This is a timeline app. It is free of charge, with an upgrade available. I have used it to create an example of the information timeline, which describes to students how long it takes for items to become published. Here is an example of a timeline using the Benghazi attack as an example:






How do you learn how to use these (and other) software packages? I recommend just playing around with them. Most are pretty user friendly. If the software package you would like to learn requires more experience, you can also try accessing tutorials on Lynda.com, or many software packages come with tutorials, as well.

ADDIE Week 2: Design

This week's discussion on ADDIE brings us to the Design phase. The Design phase is often confused with the Development phase because most people think that this is when the tutorial or instruction is created. In a way, it is when it is created, but not as far as putting the tutorial together. Think of the Design phase as an advanced planning stage when learning objectives are created, as well as scripts and storyboards. Like the Analysis phase, you will have to think things through in greater detail before moving on to the creation of the tutorial. A good script is key for creating a strong video tutorial, but before you create that, create 2-3 learning objectives that you would like to teach in the video or instruction. Keeping the number of learning objectives to 2 or 3 will help you in creating a video no longer than 5 minutes. Here are some examples of things you will create during the Design phase:

Learning Objectives

Here are some examples of learning objectives from Joyner Library's Library 101 tutorial.

1) Students will be able to describe the research process and selecting a topic.

2) Students will be able to illustrate the information timecycle.

 
Script

You will create a script of everything you will say in a video. This script will usually be about a page long for a 5 minute video. Make sure to try to read the writing out loud. Sometimes you will find that certain phrases sound funny when you are speaking them in a normal speaking voice and need to be changed.

Storyboard

Using Microsoft Word or even pen and paper, you can create a storyboard that will allow you to start to plan out the video. Use your script and break it up with still images and direction for the video. I have included an example of a video I made about our Pull and Hold service. You can also google storyboard templates and use a pen and paper.

Image Credit: Katy Kavanagh Webb

 PowerPoints, Still Shots, and Intro/Outros

During the Design phase, I like to decide what visual language I will convey in the video. Sometimes I make a PowerPoint that acts as a backdrop for the video. I also may make an introduction clip with some audio and text, or use one that I have used in previous videos. This will make it look like a more professional product at the end.

ADDIE Week 1: Analysis

For this first week of the ADDIE series, I will be talking about the A in ADDIE: Analysis. It probably goes without saying, but before you make a tutorial or any kind of instruction, it is best to think through what you will be making and why. In this phase, you should think about your learning goals, the target students or group, and where the tutorial will be embedded or used. Some instructional designers suggest conducting an in-depth learner analysis, which can include doing interviews, surveys and a background study. For most academic librarians, it is clear that you will be teaching information literacy to undergraduate and graduate students, so it is not as critical to do this legwork. It does help to have done it once or twice if you are new to an institution or to the field of study you are serving. And let's be realistic--you may differentiate the instruction you are providing to a freshman English composition class versus a capstone Engineering class. You may also want to think about what tools you have at your disposal this point in the process, as well as the time it will take you to learn the software. You can make a timeline for your project during the Analysis phase, too.

Here are some examples of things you might come up with during the Analysis phase and examples of them. Instead of using an information literacy skills example, I am going to use the example of training new graduate student workers to work at the RIS desk. I will focus on just one aspect of this training: using the printer/scanner/copier in the workroom. In this case, I will pretend I am making a training binder with step-by-step instructions, not a video.

Learner Analysis

Without doing an in-depth analysis, I can think about what I do know about my learners. I know that my learners are graduate students, that they work approximately 12 hours a week on the desk, and that they are concerned about getting some homework done while on the desk. They often work without direct supervision on nights and weekends. Since I was part of the hiring process, I can say that 2 have experience working in an office prior to coming to the library and the others do not. I also know their majors. This information tells me that they are going to be motivated, but they may also not remember how to use the copier when it comes time to use it, and that I may not be there to show them what to do. From this analysis, I have decided to create a pamphlet with images on it so they can follow along with my instructions when I am not available to help. This is another part of the Analysis phase--determining the best course of instruction for the instructional problem.

Instructional Goal

You will want to craft an instructional goal that the instruction hopes to solve. Here is an example of an instructional goal:

As part of their office skills training, Graduate Assistants will complete the process of collating and stapling a copied document using the Toshiba e-Studio 356 copier in the Research and Instructional Services Workroom.

Performance Objectives

You may want to think through what success would look like if the learners complete the goal. Here are some sample objectives related to the graduate student example:

Objective 1: Graduate Assistants should be able to define the terms collate and staple in relation to copying and explain the use of collation and stapling documents without error.

Objective 2: With the use of the training pamphlet, Graduate Assistants should demonstrate the use of the touch panel and options on the Toshiba copier to collate and staple a photocopied document.

Sub-objective 2a: Using the graphic on page 3 of the training pamphlet as a guide, Graduate Assistants should be able to identify the location of the collate option on the Toshiba touch control panel and set the collate option to Rotate Collate.

Sub-objective 2b: Using the graphic on page 4 of the training pamphlet as a guide, Graduate Assistants should be able to identify the location of the staple option on the touch control panel and set this option to staple 1 staple in the upper left of the copied document.

Project Needs and Timeline

Another thing to think through in the Analysis phase is how long it will take to create the tutorial. In this case, I can create this instruction in about 3 hours using a still image camera and Microsoft Word. I know how to do this already, so the instruction will not take long to create. In the case of a video, you may have to learn to use Camtasia or a video editing suite. You may have to get that software installed on your computer from an IT department. Think through what you will need to do before you get started.

Many times, I find myself underestimating the Analysis phase, but as you can see, if you start out with a strong analysis of your needs and your learners' needs, you will have fewer surprises later.

Join me next week for a breakdown of the Design phase!

ADDIE for Librarians

I have had this idea for a while to put together some library training or MOOCs. One of my ideas is to do the LibGuides Summer Project (see last week's post) for a larger audience of librarians at other institutions. The other idea that I have been kicking around with a few people was for an ADDIE online workshop or MOOC where people complete an entire tutorial or instructional design project over the course of 6 weeks over a summer. If you are not familiar with it, ADDIE is a general instructional design process. The letters of ADDIE stand for analysis, design, develop, implement, and evaluate. These are the basic stages that a person or team will go through as they are creating a tutorial, planning a class, or otherwise creating instruction. It looks a little something like this:

Image credit: Wikipedia

ADDIE requires evaluation throughout, so it is not a linear, step-by-step process. Each week for the next 5 weeks, I will look at the different stages of ADDIE in this blog by following a project through the process. Here is a preview of what you will read about every week.

Analyze: determining the audience, format, timeline, etc. Basically, your project planning phase.
Design: writing learning objectives, scripts, and storyboards.
Develop: this is the actual creation of tutorials or instruction.
Implement: rolling the instruction out to the actual users.
Evaluate: assessment of the students; additionally can be used throughout to fix issues in the instruction.

This is a whirlwind overview of ADDIE, but I look forward to going into greater detail about it with you over the coming weeks.

LibGuides Summer Project

Last Friday, I was fit to burst because I found out that my LOEX lightning talk was accepted. If you have not attended a Library Orientation Exchange (LOEX) conference and you are an instructional librarian, you should consider it, since the conference focuses only on library instruction. The tongue-in-cheek title for my 7-minute presentation is "From Garbage to Garden: Composting Instructional Concepts to Improve LibGuides." I picked the idea of composting because the theme of the conference is recycling and repurposing and I felt like the ideas of composting and gardening were better suited to a summery theme. I have written and presented on various themes on this topic a few times already, so in this blog post, I'll give you a run down of what it is (should take you less than 7 minutes to read!) and then let you read my article and view a poster I created on the topic. Without further ado, here's a description:

LibGuides--we've all got them. The biggest issue with LibGuides is that almost as soon as we put them on the web, they get out of date. Most librarians make an effort to review course guides when a class comes over for instruction, but subject guides and LibGuides made for classes that have not come over in a while are not updated or overhauled frequently. Since these are on the open web, broken links, out-of-date databases, and other errors can lead patrons to dead ends. Sometimes a cursory glance to make minor changes is OK, but new initiatives can mean a total overhaul of how LibGuides are presented or organized (for instance, a change in catalog, updates to tutorials, and more). Instead of leaving our garbage lying around, librarians can use make use of commonly known instructional concepts to plan a summer-long team project. Some commonly known instructional concepts that are used for the project are scaffolding, modeling, lesson planning, office hours, and peer review. One librarian acts as the organizer of the project by creating a lesson plan at the beginning of summer that lays out the LibGuides updates for the entire summer in two- to three-week increments. In a short departmental meeting, the librarian models the action that needs to be taken during that two-week period by bringing up a LibGuide on the screen and each module builds on the last (scaffolding). Depending on the LibGuides knowledge/tech level of the librarians on your team, you can offer an office hour for people to stop by to get help. By the end of the summer, each individual's LibGuides are updated. From my recent poster, here are the steps:

Image Credit: Katy Kavanagh Webb

Since the project is team-based, everyone on the team is in it together and can help each other, if needed. Peer review also ensures that the work gets done. If you are looking for ideas for updates to include in your LibGuides Summer Project, here are some ideas.

Image Credit: Katy Kavanagh Webb

Want to know more? To read my College and Research Libraries' News article on the LibGuides Summer Project, click here.

To view a poster I presented at the NCLA Biennial Conference about the Summer Project, click here. This is a link to the ECU Institutional Repository, so you will have to click on the PDF at the top to view the poster.

There is a zip file with planning documents if you would like to do this at your own institution here.

Hope to see some of you in Pittsburgh in May! If so, you'll already have an idea of what I am talking about.