Metronet Information Literacy Project Evaluation

All of the evaluative instruments and tests were developed for Metronet by an outside expert in educational assessment.  All of the data gathered by these instruments were analyzed by this consultant and this analysis is the basis of this report.

Finally, the project’s partners – St. Paul Public Library and Metropolitan State University Library – provided assistance to the students involved in the project through the St. Paul Public Library’s Homework Help Centers, while Metropolitan State’s library provided an instructor in information literacy.  Reference specialists from the MINITEX Library Information Network also supported this project.

SURVEY RESULTS:

I.    Participating Teachers

Seventeen high school teachers from four St. Paul high schools participated in the Metronet Information Literacy Project (MILP) and every one of them submitted an end-of-project survey.  The first question addressed their expectations for the project.  Most answers focused on becoming more familiar with online resources, how to use instructional technologies more effectively, how to teach the research process, evaluate websites, and take greater advantage of digital resources.  Twelve of the teachers reported that their expectations were met, while five indicated they were “somewhat” met.

When asked to elaborate on why only some of their expectations were met, respondents indicated that implementing information literacy instruction is difficult, that more one-on-one help with equipment is needed, and that they needed more “how to” help in integrating MILP concepts with their classes.

In response to the question on the number of hours of information literacy instruction provided during the year, eight of the teachers indicated one to seven hours, five said six to ten hours, and three reported more than 10 hours.  This instruction was provided in most cases by the teacher partnering with the library media specialist, with the teacher being the primary instructor.

When asked which specific aspects of information literacy received greater or lesser attention, the table below indicates teacher responses:

Following this question, teachers were asked to indicate which of the MILP concepts were the most challenging for students.  These responses are displayed below:

All of the teachers reported that the MILP had enhanced their ability to teach the research process more effectively, as well as helped them identify and use reliable resources.

While most of the participating teachers reported progress in helping students use resources and technology more responsibly, some felt that more work needed to be done in this area.  Students need more help in learning how not to plagiarize, how to cite sources correctly, etc.

When asked whether they would participate in the project were it to be offered again, the great majority answered affirmatively and all of the participants indicated that they would recommend it to their teacher colleagues.  In fact, one teacher stated that all of the St. Paul public school teachers should participate!

A final question asked how the MILP might be improved.  Many excellent suggestions were made, including:

• that the MILP start at the 9^th grade;
• that tests be required throughout the project so that students could monitor their progress;
• that the monthly meetings of teachers be organized by discipline rather than by high school;
• that more instruction be provided on how to use equipment more effectively;
• that Metronet provide more information literacy lessons and exercises for possible use by teachers;
• that more be demanded of teachers, for example, develop student assignments using the “Research Calculator.”

II.   Participating Library Media Specialists

All of the library media specialists from the four St. Paul high schools (Como, Central, Highland and Harding) responded to the end-of-project survey.  Their expectations for the project ranged from having the time and the opportunity to teach information gathering skills and assisting teachers with the integration of information literacy skills with the curriculum, to realizing the benefit of collaborating with teachers involved in this project, becoming more aware of online resources, and becoming more knowledgeable in using available equipment for teaching.  When asked whether these expectations were met, two media specialists said that they were while two indicated that they were met to some degree.  Those falling in the “somewhat” category explained that they really needed more time to work with the teachers on the MILP objectives.

The library media specialists were asked to estimate how many information literacy sessions they taught during the year.  All four respondents indicated more than eight sessions, sessions taught by both teachers and librarians in all cases.  This is somewhat at variance with teacher responses to this question.  This instruction took place in classrooms as well as in library media centers.

A question on the specific areas of instruction which were emphasized by the library media specialists resulted in the responses displayed below.  These results generally coincide with the responses of teachers to a similar question.

In teaching these information literacy concepts, the library media specialists were of the opinion that the students had an average to low-level of interest.

Metronet provided each media specialist with 12 hours of assistance each week, in the form of library media interns from the College of St. Catherine’s graduate program in school library media studies.  The role of the interns was to support the work of the media specialist by performing clerical duties, helping students with their research questions, maintaining the library’s website, and other duties.  According to the library media specialists, the interns were an extremely important part of the project and enabled the media specialists to have the time to build relationships with the teachers.

As was the case with the participating teachers, the library media specialists were asked how the MILP could be improved and whether they would be interested in participating if the project were offered again.  All four respondents answered affirmatively to this question, with one describing the experience as being “fabulous” and another stating that the program was “most beneficial” to her.

Suggestions for improvement included:

• the development of more lesson plans and a common information literacy curriculum;
• having more “hands-on” activities at the monthly meetings of teachers and librarians;
• opening the project to all high school teachers, regardless of the grade of their students; and
• developing and using a standard vocabulary for the project from its inception.

III. Student Responses

A total of 792 students took the 20 question pre-test on information literacy, while only 386 took the post-test.  Responses by high school and grade level are as follows:

There are other significant problems with these data, however.  At each school, some classes took the pre-test but not the post-test, while others took the post-test but not the pre-test.  It was only at Harding Senior High School that most students took both tests.  Because of this situation, it is not possible to infer from survey responses whether students did in fact improve with respect to becoming more information literate.

One reason for the disparity in the data is that two of the high schools organized their class schedules on a trimester system, while the others operated on a semester system.  Some classes in the semester-based schools were one semester in length while others extended for both semesters.  It was difficult for project staff and the teachers to schedule pre-testing and post-testing not only for individual schools, but also for individual classes.  This is an area that will be given much closer attention in the second year of the literacy project.

With regard to the pre-test, it is interesting to note that of the 22 questions that were designed to test students’ knowledge of information literacy, on average 58% of the questions were answered correctly.  See the table below:

CONCLUSIONS:

The Metronet Information Literacy Project was well-received by administrators at the St. Paul Public Schools, by participating teachers, and by the library media specialists.  All of the participants felt that the project addressed some very critical needs and served as an important professional development program for both teachers and librarians.  Perhaps the best gauge of the project’s success is the commitment of the St. Paul Public Schools to continue it for a second year.

Despite the problems with the student survey, a considerable amount of anecdotal information suggests that students did learn some essential information literacy concepts and skills such as steps in conducting research, how to cite resources used, how to evaluate websites, and why databases and information in printed sources often contain information that is more reliable than web-based sources of information.  Classroom visits by Metronet staff confirmed that these concepts were being taught and interviews (which were filmed) with a small group of students also indicated that they understood these concepts.

The project also served to raise the visibility of the library media centers and enabled the library media specialists to become more engaged in teaching.  They were able to build partnerships with the participating teachers and by attending the monthly project meetings were able to share ideas with each other.  The project also demonstrated the important role played by library media specialists in coaching teachers on how to teach information literacy, and teaching students the steps in the research process.  Library media specialists also reinforced the instruction students received from their teachers.

The project also demonstrated the value of collaboration and the sharing of expertise across library systems.  The participation of the St. Paul Public Library – administrators and Homework Help Center staff – was very important, particularly their support of the MILP students after school.  Similarly, the support provided by the Dean of Libraries at Metropolitan State University and the Library’s Information Literacy Specialist added an important dimension to the project.

This project was Metronet’s first attempt to promote information literacy in a group of schools.  To a certain extent, Metronet designed the program while it was being implemented.  As a result of lessons learned, Year II of the partnership with the St. Paul Public Schools is certain to have an even greater impact on student learning.

________________

7/16/07