Library Assessment: 2 Assessment

Assessment refers to the methods chosen to measure the success of outcomes and to the collection of data by those methods.

• Establish measurement methods and collect data
  • Designate who is responsible for measurement methods or data collection
  • Determine what is an acceptable level of "success" for the data 
  • Data collection and measurement methods should be part of regular duties
  • Data collection and measurement methods should not be unduly burdensome

Assessment methods should be selected that are appropriate to measure a designated outcome.

The library should do an assessment audit to determine what work is already being done and what gaps exist.

What are you doing now? Take advantage of the data you are already collecting...if it fits your outcomes.

A common mistake is to add assessments and data collection methods to an assessment plan before the assessment audit has been completed. Many librarians are quick to suggest additiional surveys and data collection before outcomes have been decided and agreed or existing assessments have been inventoried.

The danger is that assessments may be added that are unnecessary, that are not appropriate to an outcome, that do not map to any existing outcome, that duplicate existing assessments, or that add excessively to existing workloads.

• Just counting or measuring without providing a framework doesn’t say much.The assessment plan may require a reevaluation of the plan itself.
• Measurements and data collected should connect back to the goals and outcomes. If you are collecting data that does not measure your specified goals or outcomes, maybe you should not bother collecting that data, or maybe you need to change your stated goals and outcomes to include that data.
• Measurement methods should be appropriate to your stated outcomes. Don't measure whether students can create a successful search by gving a multiple choice test (which simply requires students to recognize the right answer).
• Just because you can count it, doesn’t mean you should!
• Snapshot counting can be as successful as bean counting.
• Assessment and evaluation need not be done on every outcome every year. Data collection and analysis takes time. Changes take time to see results.
• If you tell someone, it is an anecdote. If you write it down, it is assessment.

The best assessments are a combination of direct and indirect measures.

Inputs are resources your library makes available and contributes to provide services. The number of books held, the number of subscriptions, the number of events offered are all examples of inputs.

Outputs measure the work done. Use of materials, number of reference questions received, reference questions answered satisfactorily, visits per university population, etc. are examples.

Direct measures are types of measurement that directly measure the outcome. Assignments that require students to demonstrate that they know the difference between a scholarly journal and a popular magazine are a form of direct measure. Quizes which require students to identify scholarly and popular publications are types of direct measures. A report of observations of what users are actually doing in the library is a direct measure.

Indirect measures are types of measurement that indirectly measure the outcome. Distributing a survey which asks students if they know the difference between a scholarly journal and a popular magazine is a form of indirect measure. Surveys which ask students if they learned anything during a library instruction session are types of indirect measures.

Libraries have already done assessment but they tend to focus on inputs and what they are doing rather than on the effect they are having on their users and their community. Libraries face challenges in trying to assess their student learning because they do not teach traditional credit bearing courses and they cannot give traditional tests. However, some libraries have managed to come up with creative ways to get direct measures. Look for ideas from other libraries and use them if they are appropriate for your library.

Search a library database for examples of what otehr libraries are doing to measure success of their library services and student learning of information literacy. Try variations of the search phrases below to get you started.

(Assess* OR evaluat*) AND circulation
(Assess* OR evaluat*) AND reference
(Assess* OR evaluat*) AND catalog*
(Assess* OR evaluat*) AND archiv*
(Assess* OR evaluat*) AND “information technology”
(Assess* OR evaluat*) AND purchasing or acquisition*

Examples of some interesting assessments follow. Evaluate whether they might be right for your assessment plan.

• Request copies of final papers (no student names) from faculty along with assignment guidelines (3 scholarly articles, etc.) from a capstone course. Evaluate those papers according to your outcomes. (Do students select quality sources, do students write an articulate thesis statement identifying their information need, etc.)
• From a department, request copies of bibliographies of papers (no student names) from faculty along with assignment guidelines. Note quality of resources by number, choice, etc
• Is the university or a department conducting exit interviews with graduating seniors? Look at the interview questions and request copies of answers to questions relevant to the library outcomes.
• Students fill out worksheet listing research topics and requiring a draft of Boolean search phrases. Collect worksheets and analyze for precision and ingenuity.
• Students fill out worksheet listing number of results of specified Boolean searches. Collect worksheets  and note percentage of successful searches.
• At the Reference Desk, record the number of reference desk inquiries "I can't find anything on my topic. Where have I looked? ProQuest and the Internet." Or record other relevant data for your outcomes.
• At the Reference Desk, casually ask questions about research queries concerning what class, who is instructor, did you receive a library instruction session in this class or any other class, etc. After student leaves, record on form by librarian.