Information Literacy Assessment:
Distinguishing Different Levels of Understanding

Though there is some variation, information literacy programs at every level (elementary school through college) have very similar educational objectives, as these samples show:

K-12
The Big6 (More information about the Big6)
1.Task definition Determine the nature of the information problem and define the purpose for the information search
2.Information seeking strategies Determine the types of sources and strategies for acquiring the resources required to meet the information task
3.Resource location and access Find sources; fine pertinent information within the resources
4.Using the information Engage with the information source (read/view/use); extract relevant information
5.Synthesis Integrate, structure, and repackage the information to meet the defined task
6.Evaluation Judge the information problem-solving process and whether the information need was met

Information Power from American Association of School Librarians (AASL)
The student who is information literate:
1. Accesses information efficiently and effectively.
2. Evaluates information critically and competently.
3. Uses information accurately and creatively.
4. Pursues information related to personal interests.
5. Appreciates literature and other creative expressions of information.
6. Strives for excellence in information seeking and knowledge generation.
7. Recognizes the importance of information to a democratic society.
8. Practices ethical behavior in regard to information and information technology.
9. Anticipates effectively in groups to pursue and generate information.

AASL's Web site features an article by Carol C. Kuhlthau ("Implementing a Process Approach to Information Skills: A Study Identifying Indicators of Success in Library Media Programs" <http://www.ala.org/aasl/SLMR/slmr_resources/select_kuhlthau1.html>) which emphasizes the constructivist approach.

College/University Levels
Association of College & Research Libraries
1. Determine the extent of information needed
2. Access the needed information effectively and efficiently
3. Evaluate information and its sources critically
4. Incorporate selected information into one’s knowledge base
5. Use information effectively to accomplish a specific purpose
6. Understand the economic, legal, and social issues surrounding the use of information, and access and use information ethically and legally

A lengthy set of Standards, Performance Indicators, and Outcomes supports these objectives.

California Community Colleges
1. Recognize the need for information.
2. State a research question, problem or issue.
3. Determine information requirements in various disciplines for the research questions, problems or issues.
4. Use information technology tools to locate and retrieve relevant information.
5. Organize information.
6. Analyze and evaluate information.
7. Communicate using a variety of information resources and technologies.
8. Understand the ethical and legal issues surrounding information and information technology.
9. Apply the skills gained in information competency to enable lifelong learning.

CSU Information Competence Project <http://www.lib.calpoly.edu/infocomp/project/outline.html>
List of Core Competencies
1.Define the research topic
2.Determine the information requirements for the research question, problem, or issue.
3.Locate and retrieve relevant information.
4.Use the technological tools for accessing information.
5.Evaluate information.
6.Organize and synthesize information.
7.Communicate using a variety of information technologies
8.Understand the ethical, legal, and socio-political issues surrounding information and information technology.
9.Use, evaluate, and treat critically information received from the mass media.
10.Judge the product and the process.

Efforts at various levels pitch information and concepts differently. Just as we teach English composition to 10 year olds, 10th graders, and college students, so also is information literacy taught at various levels of sophistication.

Information Literacy: Learning Outcomes
Cabrillo College has a very active information literacy program. A one-unit Information Research course is a co-requisite to the college's English 1A course; about 850 to 900 students a semester take this course. In addition, instructors bring between 90 and 100 classes every semester for course-related instruction, for which a librarian prepares special instruction. (To sample Web-based exercises prepared for course-related instruction, from the Cabrillo College Library homepage, select Internet Links, click on the name of a Department and look for "Course-Related Materials.")

There are many information literacy instruction efforts; anecdotal evaluations tell us that they improve students' abilities to do research. On a more formal basis, a learner outcomes study done Fall 1999 with an anthropology class demonstrated that instruction sessions, in addition to the class coming regularly to the library, improved students' abilities to do research.

Information Literacy: Assessing Different Levels of Understanding
To begin to get a grasp on different levels of understanding (especially high school vs. college), I developed an experimental Information Literacy Assessment instrument. I put it together fairly late in this sabbatical project (early April 2000), and, by the time I shared it with high school librarians, and librarian colleagues at community colleges and four year institutions, it was relatively late in the semester. Thus, at this moment, I have just 31 responses so far, an insufficient number from which to draw substantial conclusions.

That being said, it is nonetheless instructive to look informally at the data collected so far, and make some observations.

Who were the respondents?
20 high school students (8 in 9th or 10th grades; 12 in 11th or 12th grades)
11 college students (8 lower division; 3 upper division)

Had they had training in how to do research?
Of the high school students, 9 had learned on their own; 11 had had training in high school
Of the college students, 3 had had training in high school; 8 had had training in college

Did they consider themselves computer literate?
Of the high school students, 13 (more than half) were "pretty comfortable" with computers (all but 1 of the 9th/10th graders fell in this category); 6 were "moderately comfortable; only 1 indicated "I haven't learned enough about computers as I should."
Of the college students, 5 (about half) were "pretty comfortable;" 3 were "moderately comfortable;" and 3 indicated "I haven't learned enough."

Commentary Familiarity with the Internet is a forward-pushing phenomenon. Younger students more than older ones have learned on their own, and nearly all of the younger students feel comfortable with computers. All of the college students have had some training in how to do research in high school or college, but not all of them rate their computer literacy skills as high as the younger students do (or, perhaps, age brings with it a bit of modesty).

Forming a research question from a broad topic.
1. Let's say that you wanted to research the topic computer and Internet crimes. Given that broad area of interest, what might be a research question to investigate -- for example, "Should governments get involved in regulating use of the Internet?

Commentary Nearly all students came up with a topical research question. The interesting difference was that the high school students most often asked a who/what/where type of question. Examples: What types of Internet crimes exist? Where have computer and Internet crimes been occurring the most? How many crimes are done every year? How do Internet crimes get solved? The college students were fairly consistent with asking a social issues-type question, or were probing some more theoretical construct. Examples: Should there be legislation regulating the Internet? How have Internet crimes influenced the way we think about free speech and the computer? Who should control the Internet? (An alternative explanation might be that the college students took the example research question more strictly as a model to follow, whereas the younger students felt more comfortable branching out with their own types of questions.)

Identifying keywords
2. Using the research question you drafted above, what keywords might be good to use in the first stages of research?

Commentary Every student responded with something, so the assumption is that everyone was familiar with the concept "keyword." However, a number of high schoolers responded with full phrases, with no quotes around them (e.g., crime rate on the Internet; different types of Internet crimes; crimes that were committed on the computer and the Internet), which would indicate that they did not really understand the concept. The keywords the college students chose were either single words, or phrases (e.g., "computer hackers;" "Internet policies,") indicating a better grip on the idea.

In responding to this question, keywords and words in phrases were often misspelled (or mistyped), a more frequent occurrence among the younger students. Examples: accur (for occur); electonic (for electronic); pornagraphic (for pornographic); miner (for minor); etc. If students went on to use search statements in computer searching based on these spellings, their search results would be greatly affected. A handy dictionary and/or thesaurus would be a great assist in any computer search mode. Surely, someday, everytime you use a library catalog, library-accessible fulltext database, or even the Web, there will be a small window on the top righthand side of your screen that will suggest spelling corrections and synonyms.

Thinking about resources
3. Using the research question, what kinds of resources do you think you would use to find answers to your question?

Commentary High school students and college students listed a wide variety of resources, e.g., books, magazines, online periodical databases, newspapers, etc. But, invariably, they all mentioned the Internet. As you would expect, the younger high school students would specifically mention very commonly known tools, such as Yahoo! and Ask.com. Whereas, if the college students mentioned specific resources, they were more sophisticated, e.g., CQ Weekly and the California Digital Library.

Distinguishing popular from scholarly periodicals
4.Popular periodicals...are written to appeal to the general public. What are some of their other characteristics?
Scholarly periodicals...are written to appeal to researchers and students in the field of study. What are some of their other characteristics?

Three of the high school students did not respond at all to these questions. The most frequent response from both groups to the question about popular periodicals was that they carried advertisements, wrote about fashions, and had pictures. Among the college students, the most frequent response (5 of the 11) to the question about popular periodicals was that they include documentation (references).

Commentary Whereas students at all levels are aware that periodicals are important information resources (15 of the 31 students specifically mentioned periodicals in the "kinds of resources" question, no. 3), substantial uncertainly exists as to what distinguishes popular from scholarly. This might be something to emphasize with students at all levels.

Distinguishing primary from secondary resources
5. An example of a primary resource would be:____________________________
An example of a secondary resource would be:_____________________________

Two of the high school students did not respond at all; four gave acceptable responses (e.g., "witness to a murder/ law book telling how case was won in a similar situation;" "an autobiography/ someone else writing or telling you about some else's childhood"). Three of the 11 college students gave acceptable responses (e.g., "from the source itself/ a report about the source;" "direct statement from horse's mouth/ summary of information from source that heard the horse")

Commentary The distinction between primary and secondary resources is an important one in information gathering, analysis, and problem solving. In today's world of standards-based education, the distinction between primary and secondary resources is part of the analysis skills taught in K-5. [See Historical and Social Science Analysis Skills, K-5 -- scroll down just a bit to "Historical Research, Evidence, and Point of View" for chapter/verse of the standards] For K-12 Web examples, see K-12 Lessons: Using Primary Sources - Overview and Primary and Secondary Sources of Information; for a college example, see About Primary Resources. The distinction between primary and secondary resources is generally an aim of information literacy efforts (for example, see Western Washington University Libraries Information Literacy Learning Outcomes). Yet the concept isn't all that clear to many individuals. This would be another fundamental area to emphasize with students at all levels.

Using the World Wide Web for research
6. Let's say you begin your research on the topic...using the World Wide Web. What are the first couple of steps you would take?

Nearly every student indicated that they would go to a search engine and type in keywords. If specific search tools were mentioned, high schoolers tended to mention Yahoo! and Ask.com. College students tended to mention Google or Melvyl. No one mentioned selecting a search tool that would be most appropriate to searching the particular topic.

Commentary As far back as the Lawrence and Giles 1998 Science article(1), we have known (as opposed to surmised) that although the various search engines call themselves complete and comprehensive, they are far from that ideal state (and will probably never get there). The "standard" search engines operate under different search protocols, and vary in other ways. Indexing of purely commercial sites is on the upswing. For certain subject fields, subject gateways are coming to play larger and larger roles. In other words, "going to a search engine" is a risky proposition these days. Students at all levels need to know more about selection of search tools, their use, and how to plan a search strategy to include the usual search engines and additional, more specialized tools.

Evaluating Web sites
7. How do you know whether a site you have found on the World Wide Web includes information that is reliable and appropriate for academic research?

At all educational levels, students can use help in evaluating the credibility and reliability of Web resources. Of the high school students, 4 mentioned carefully reading the words returned in the search result by the search engine; two said that if it were a commercial site, you could trust it (perhaps they were joking); one person said that he/she didn't evaluate Web pages: "I just trust it." College students responded more with comments about trusting sites that included citations; or, if site is associated with a university, then it's reliable; or, check to see if it's a personal Web page; or, check the authors' credentials.

Commentary Being able to evaluate Web resources is becoming a critical survival skill, not just for school, but in on-the-job situations. It would be useful to emphasize evaluation at all educational levels. This might include teaching K-12 teachers more about evaluation, since most of them have picked up Internet skills on their own and may not have been formally instructed in how to critically analyze and evaluate Web resources.

Plagiarism
8. Plagiarism is presenting the work of others as though it were your own; it is a form of theft. Which of the following is an example of plagiarism? (check all that apply) 1. Using phrases and sentences of others as if they were your own without giving credit 2. Using ideas of others as if they were your own without giving credit 3. Copying text written by someone else and using it with no quotation marks 4. Rewording someone else's information and using it without giving credit 5. Using copyrighted images from the Web without giving credit

Of the high school students, only 3 of the 20 checked all five options as examples of plagiarism (the correct response); 7 of the 10 college students checked all five options. This is a significant difference in levels of understanding. The fourth option ("Rewording someone else's information and using it without giving credit") was far and away the option most frequently not selected (13 of the 20 high school students didn't check it; 3 of the 10 college students didn't select it). Seven of the high school students (but only 1 college student) did not identify "Using copyrighted images from the Web without giving credit" (no. 5) as plagiarism. Four of the high school students did not identify "Using phrases and sentences of others as if they were your own without giving credit" (no. 1) as plagiarism, whereas all of the college students did.

Commentary College students seem to have a significantly better understanding of what plagiarism is than do high school students. Social and ethical issues associated with copyright, and plagiarism could certainly be emphasized at all levels.

Using different types of resources
9. Information you find on a research topic may come from: (1) Printed materials and electronic resources made available through libraries (for example, databases that provide fulltext copies of periodical articles) OR (2) Materials you find using Web search engines. What are some of the major differences between these two sets of materials?

Half the students in each group mentioned that materials available through libraries (the category 1 materials) were likely to be more reliable. Several students in both groups mentioned that Web materials could be more up to date. A number of students did not understand the question.

Commentary There is a significant lesson here. More and more, libraries are offering quality, value-added databases (e.g., fulltext periodicals) and other resources (e.g., encyclopedias, Grove's Dictionary of Art) via their Web sites. It's important that students understand the differences between these types of resources and the world of Web sites.

Organizing your information
10. Once you start researching a topic, how do you keep the information you gather organized? For example, do you take notes by hand, using paper (or cards)? Or, do you take notes using a computer (switching back and forth between wordprocessing software and the Internet)? Or? We'd appreciate a brief description.

Students at every level are learning to interweave information and materials electronically. A quarter of the high school students, and half of the college students, indicated that they switch back and forth between wordprocessing and the Internet. The second most frequently mentioned approach was to printout materials and to highlight.

Commentary The approach in the high schools (e.g., those who have been hit by the Digital High School initiative) is to teach students to move electronically back and forth between applications, e.g., Internet to Word documents. College libraries -- where use of computers for information access but not word processing is emphasized -- may begin to experience some student pressure for more options.

 

(1) Steve Lawrence and C. Lee Giles, "Searching the World Wide Web," Science vol. 280, 3 April 1998, p. 98-100.

last rev. 4/13/01

Investigating Information Literacy Efforts

Sabbatical Project Report homepage