STUDENT SERVICES PROGRAM REVIEW PROJECT

FINAL REPORT 

 THEY SAID IT COULDN’T BE DONE!  

Volume 1
(Volume 2 is not available on the web)

Santa Ana, California
October 1986

 

FOREWORD

The Student Services Program Review Project (SSPRP) was a unique, voluntary effort on the part of many California community colleges. Its purpose was to develop evaluation models to measure the efficacy of the various student services programs provided by the colleges.

The SSPRP had the board support of the colleges and districts involved, and of many statewide organizations and agencies. They were generous in the commitment of both personnel and resources. Over 1,000 persons were involved in the Project during the three years of the Project. Those participating colleges, organizations and contributors are acknowledged in the appendices in Volume 2.

The uniqueness of the Project was in its goals as well as in its broad-based foundation. As a result of the efforts of the participants, all community colleges now have evaluation designs – including goals, criteria, measures, and methods – which were field-based and field-produced, and with which the colleges will be able to evaluate their student services programs.

Since the purpose of the Project was the development and refinement of evaluation designs, colleges participating in the field tests of the evaluation models were not asked to share evaluation results, but only that information necessary to produce and refine the designs. Standards against which to measure program success were developed by the participating colleges for their own use.

The final products of the Project are contained in this Report and are intended for use by colleges. It is anticipated and hoped that the design will be constantly reviewed and improved by frequent use.

The Steering Committee and Staff

 

INTRODUCTION

BACKGROUND
Accountability and reform for the California Community Colleges represent current and urgent topics of discussion and items for action on the agendas of major public policy making bodies including the legislature.

The mission and funding of the California community colleges continue to undergo close scrutiny. All programs in community colleges are vulnerable in a period of financial crisis, but student services programs seemed particularly subject to reduction or elimination. One of the reasons for that vulnerability was the lack of reliable, verifiable information which described and evaluated student services programs. The information that did exist was anecdotal or difficult to aggregate, and therefore, often not usable as support for the continuation of programs.

It was apparent that if student services programs were to continue to be supported as an essential part of the mission and functions of California community colleges, an effort would have to be made to systematically assess the programs’ contributions to student outcomes. In response, the Commission on Student Services of the California Association of Community Colleges (CACC) and the Northern California Cooperative Institutional Research Group (NORCAL) agreed that the colleges must become active in evaluating student services and in using the results of these evaluations to underscore successes and to modify services as necessary for improvement.

 

PROJECT BACKGROUND

The Student Services Program Review Project (SSPRP) was developed, therefore, with the intention of developing and testing evaluation approaches and criteria for the various areas of student services. The original group of participants, headed by Dr. Robert Jensen, then President of NORCAL, and Peter Hirsch, then Associate Executive Director of CACC, included both student services professionals and persons skilled in research. A Steering Committee was formed and continues to direct the Project. To facilitate the implementation of Project activities, Project Directors (See Appendix B) and Research Coordinators were also named.

The Project goal was to develop and pilot test evaluation designs in order to assist colleges in the implementation of program evaluation of selected programs of student services on their campuses.

Several assumptions were made at the inception of the Project. These were: (1) the Project was to be a "grass roots" activity involving volunteer participation by colleges; (2) the Project was to be a coalition effort by and for the participating colleges; (3) all California Community Colleges were to be given the opportunity to participate; (4) financial assistance was to be requested from outside sources to support Project coordination and development; (5) ongoing operational support was to be generated through fees from participating colleges.

The Project objectives were to:

  1. Develop evaluation models;
  2. Develop data collection, data analysis and information-reporting procedures;
  3. Pilot test evaluation models and procedures;
  4. Widely disseminate models and procedures; and,
  5. Develop support materials and services to assist community colleges in implementing program evaluations appropriate to their institutions.

A sequence of activities intended to achieve these objectives was established and revised periodically by the staff and steering committee (Appendix D).

 

ACTIVITIES OF THE PROJECT: PHASE I

It was agreed that the first phase of the Project would focus on those areas of student services selected by the participating colleges as having the highest priority for review and evaluation. To identify the programs to be evaluated during Phase I, several surveys of California’s community colleges were conducted. In addition, a number of statewide student services organizations provided guidance and information. Based on this review process, which occurred over a six-month period, the following areas were selected for evaluation by Northern and southern colleges for the first phase: (1) Admissions and Records; (2) Counseling; (3) Financial Aid; (4) Student Affairs. In addition, the participating northern colleges elected to review the area of Job Placement.

 

CHARRETTES

To develop concepts essential to the conduct of the Project and to begin the foundation work leading to development of evaluative criteria for each program, two charrettes were held, one in the north at De Anza College, and one in the south at Mt. San Antonio College. Over three hundred people participated in these two activities.

The term "charrette" is from the French Parisian architectural students, preparing for the final defense of their work and their rights to graduation, entered into intensive development of their last designs and drawings. When this occurred, colleague-students would pick up the student who was preparing for the examination in a cart known as a "charrette." They would load the students’ drawings and designs onto the cart, and as they traveled through the streets of Paris, the student would finish her/his work. Commonly, the student would call for her/his colleagues to review the final work. Critique and revision would follow; consequently, the final drawing or design would often be the best in the student’s portfolio.

The charrette concept as applied to issue resolution describes an intensive, group-oriented, planning and development process. People with different backgrounds, different orientations, and different perceptions, but all allied by a common interest in resolving the issues under consideration, meet together to analyze issue components and develop consensus resolutions. The SSPRP Charrettes resulted in the development of a mission statement for Student Services, goals for each program under study, and lists of suggested evaluative criteria and methods for their use in the development of evaluation designs.

Writing teams worked with the results of the charrettes to develop consensus statements. Drafts of these statements were first reviewed by all charrette participants and other student services personnel. Their reactions were then used by the writing team and the Project Steering Committee to prepare a final draft of evaluation models. The attempt during these workshops was to maintain the sense of the consensus of both charrettes in developing measurable goals, suggested criteria and evaluation methods. The Charrette Report was distributed in June 1984.

 

 CHARRETTE OUTCOMES
The mission statement for student services was jointly developed by the more than 300 charrette participants. It conveys the critical nature of student services programs in community colleges and specifies they key goals which these services are designed to accomplish.

MISSION
Student services provide comprehensive programs and services which are an integral part of the educational process. These programs and services promote equal access and retention, and enable students to identify and achieve their educational and career goals.

 

Goals were also developed by the Charrette participants for each of the five program areas: Admissions and Records, Counseling, Financial Aid, Student Affairs and Student Employment Services.

The initial identification of evaluation criteria and methods for each goal was begun by the Charrette participants. These were not meant to be final products, but rather guides for further development of the actual criteria, methods, and measures for use by the pilot colleges.

In June 1984, an intensive writing workshop was held at Cabrillo College, Aptos, California. Participants included members of the Steering Committee, persons representing the professional organizations of the student services areas under study, researchers from Northern and Southern California, and key California community college leaders. For two-and-one-half days, writing groups developed criteria, measures and methods for every goal in the five areas of student services. The results of the writing workshop were then reviewed by the participants and field reviewers recommended as representative of the five program areas.

 

PILOT TEST AND RESULTS

Colleges participating in the Project began to pilot test the evaluation designs in the fall of 1984. Workshops provided assistance to the participating colleges, including an orientation to the procedures, evaluation instructions, and guidelines.

The pilot testing of the evaluation models was conducted by participating colleges from October 1984 through the spring semester 1985. The results of the pilot (critiques of the models) were reviewed by a team of practitioners and researchers, and the goals, criteria, measures, and methods were refined as recommended by the participating colleges. The final evaluation models are provided in this Report for use by colleges.

 

ACTIVITIES OF THE PROJECT: PHASE II

In fall 1985, following a process similar to that of Phase I, Phase II of the SSPRP began with the completion of a survey by all California community colleges. The colleges were asked to select other areas of student services having high priority for review and evaluation. Three additional areas were selected: Assessment Services, Career/Life Services, and Tutorial Services.

Twenty-three colleges and one hundred twenty-five student services practitioners and researchers participated in charrettes held at the College of San Mateo and at Rancho Santiago College in April 1985. The purpose of the charrettes was to produce goal statements and evaluative criteria for the three areas. The recommended goals and criteria were subsequently reviewed by a writing team resulting in the development of a charrette report. This report was disseminated to all community colleges for comments and suggestions.

In August 1985, a writing workshop was conducted during which Student Services Program Review staff, Steering Committee members, and practitioners from each of the program areas reviewed the charrette report and field responses. The writing workshop produced the goals, criteria, measures, and methods to be used in the pilot tests which began in fall 1985. Participating colleges conducted pilot testing of the evaluation models of one or more of these areas. Using critiques from the colleges’ pilot tests, a final review and writing workshop was held in June, 1986, resulting in the production of revised criteria, measures and methods for the three Phase II areas. These designs are also part of this Report and are now available for use by colleges.

 

IMPLICATIONS OF THE PROJECT
The Student Services Program Review Project has made significant progress toward the goal of enabling colleges to develop the information necessary for the support and improvement of their student services programs. With the information gathered as a result of systematic program review, Student Services can be shown to be an integral – not peripheral – part of the education process.

The Project has implications for many other efforts currently under way in the community colleges in California. Consider, for example, differential funding. In that funding proposal, Student Services has been identified as one possible "cost Center." Since both qualitative and quantitative measures will be required at the point of determining what will be funded and in what amounts, it is clear that having a systematic way of reviewing student services programs could be of great advantage to the colleges. Other examples include the fact that the Accrediting Commission for Community and Junior Colleges may us SSPRP results to review and revise its Student Services Standards. Many of the pilot colleges used the evaluation results as part of their self-studies. This liaison between the Project and the Accrediting Commission should serve to further encourage evaluation and coordinate efforts to improve student services.

The SSPRP, a joint effort on the part of California’s community colleges, statewide organizations, and student services personnel, has given the student services an advantage: a head start in determining their own fate. It is essential that California’s community colleges have the major responsibility for their own futures. If they do not, those futures are less likely to reflect the needs of millions of citizens seeking educational opportunities in the colleges, and are more likely to be the myopic vision of various distant policy making groups.

 No one is apathetic except in the pursuit of someone else’s goals.
(Anonymous)

Clearly, the Student Services Program Review Project , with its board-based participatory and voluntary foundation, has involved colleges in the development of their own goals.


 

EVALUATION DESIGNS

STUDENT SERVICES PROGRAM REVIEW PROJECT
CRITERIA, MEASURES, METHODS

 

Admissions and Records

GOAL 1: To Provide Clear and Concise Information to All Members of the Community

Criteria

Measures

Methods

E/A*

 a) Availability of information

 

1. Evidence of each Admissions & Records information item (written and non-written).

1.1 Provide example or documentation of each.

 

E

b) Accessibility of information

1. Evidence of distribution to service area and target groups.

1.1 List the distribution locations, description of distribution method, targeted group, and date of availability of each item listed above.

 

E

2. Evidence of diverse distribution locations.

2.2 Indicate hours of operation of distribution centers.

E
3. Ease of obtaining information.  Level of community aware-ness of information distributed. 3.3 Survey awareness and satisfaction

& (students and non-students)

4.4 Survey could include written questionnaire or interview of sample or entire population.

 

E

c) Readability and accuracy of information.

1. Evidence of clear, concise, accurate, and complete information.

1.1 Measure reading grade level of all information provided.

 

A

2. Evidence of appropriateness in reading special target group populations.

2.1 Third party (selected individuals from outside institutional A&R staff) examination and analysis of information to determine clarity, accuracy, conciseness, and completeness.

2.2 Indicate the appropriateness of language for targeted groups.

 

A

 

 E

d) Timeliness of information distribution.

1. Evidence of appropriate re-lationship between timing of information distribution and educational and student services provided.

1.1 Demonstrate inter-relationship between information provided and services. (Indicate actual dates of information distribution)

1.2 Survey users of information to determine level of satisfaction with timing.

 

E

Indicates which methods are essential ("E") for program review and which provide additional ("A") information and/or insight.

 

Admissions and Records
GOAL 2: To Admit and Register All Students in a Timely and Accurate Manner

Criteria

Measures

Methods

E/A*

a) Admit and register students in a timely manner.

1. Amount of time required to be admitted and/or registered.

1.1 Conduct test sample during admissions and registration processes.

1.2 Survey students to determine whether they were admitted and registered in a timely manner.

E

 

E

2. Hours and modes of Admissions & Records service.

2.1 Review and analyze hours of operation and hours of services available.

2.2 Provide evidence of alternative methods of admissions and registration.

E

 

E

b) Coordination, admiss-ions and registration of students with other campus service units.

1. Evidence of coordination efforts between campus service units.

1.1 Interview representatives from campus service units.

1.2 Provide and review formed plan for coordination efforts.

E

c) Ease of application and registration process

1. Evidence of simple and efficient forms and process

1.1 Third party review of forms and processes.

1.2 Staff/student survey to determine simplicity and efficiency

E

 

E

d) Accuracy of data collected

1. Level of accuracy of registration and enrollment data.

1.1 Internal audit

1.2 Third party review

E

E

e) Accuracy of students’ schedules of classes

1. Consistency between students’ class schedule and roll sheet.

1.1 Test sample for consistency

A

2. Existence of errors due to Admissions & Records processing

2.1 Monitor number and type of student/staff/faculty complaint

2.2 Identify and analyze errors to determine cause and remedy.

2.3 Survey staff/faculty and students to determine level of accuracy.

E


 

E

Indicates which methods are essential ("E") for program review and which provide additional ("A") information and/or insight.

 

Admissions and Records

GOAL 3: To Provide Information and Supportive Contacts with Students, Faculty, Administration, and the Community

THIS GOAL HAS BEEN ACCOMPLISHED IN GOALS 1 & 2 & 8.

 

Admissions and Records 
GOAL 4: To Store, Maintain, and Retrieve Records in an Efficient, Accurate, and Secure Manner

Criteria

Measures

Methods

E/A*

a) Effective and efficient design of records collection instruments (Forms).

1. Evidence of:
Completeness

Ease of use

Utility for efficient integration into the information system

1.1 Third party review by knowledgeable source.

1.2 Survey users.

E

 

A

b) Effective and efficient storage of A&R records

1. Evidence of adequate capacity for present and future records.

1.1 Provide or demonstrate a plan and procedures.

E

2. Evidence of backup system for records.

2.1 Document back-up system.

E

3. Evidence of security measures including other areas having access to computer data base.

3.1 Analysis of current and projected storage use in relationship to capacity.

3.2 Review and assess adequacy of records contingency plan.

3.3 Visual inspection of storage system.

3.4 Field test back-up system.

3.5 Review and assess policy and regulations to verify compliance requests.

3.6 Field test security.

3.7 Review and assess security protection of student confidentiality.

E

 E

 

E

E

E

 

E

E

c) Efficient and effective maintenance of A&R records.

1. Evidence of updating records in a timely and accurate manner.

1.1 Conduct test sample to test accuracy and timeliness of record updates.

1.2 Audit computerized record system for timely updating of records.

1.3 Survey of A&R staff by third party to determine whether records are updated in a timely and accurate fashion.

1.4 Analyze time taken for required record changes.

1.5 Review and analyze policy and regulations in regard to purging (retirement) of A&R records and compliance requirements.

E

 A

A

 

 

E

 

E

Indicates which methods are essential ("E") for program review and which provide additional ("A") information and/or insight.

 

Admissions and Records

GOAL 4: To Store, Maintain, and Retrieve Records in an Efficient, Accurate, and Secure Manner (continued)

Criteria

Measures

Methods

E/A*

d) Effective and efficient retrieval of student and instructor records.

1. Evidence that records are secure in accordance with stated policy and regulations.

1.1 Review and analyze policies and regulations for compliance requests.

1.2 Field test security system.

1.3 Analyze time to retrieve student and instructor records.

E

 

E

E

2. Evidence of timely and easy access to and retrieval of student records.

2.1 Survey A&R staff and other users of system.Field test ease and timeliness of access.
E

 

E

Indicates which methods are essential ("E") for program review and which provide additional ("A") information and/or insight.

 

Admissions and Records
GOAL 5: To Evaluate and Distribute Student Records in a Timely and Accurate Manner

Criteria

Measures

Methods

E/A*

  •  

    a) Efficient, accurate, and regular evaluation.

    1.  
    2. Evidence of efficient, accurate, and timely evaluation of:
    • transcripts
    • graduation
    • residency
    • certification
    • degree or certificate requirements
    1.  
    2. Test sample of each to determine time taken and accuracy.
    3. Third party review.
    4. Survey students and staff users.

     

    E

    E

    A

  •  

    b) Efficient and timely distribution of student records.

  •  

    1. The length of time taken for the recipient to receive document.

    1.  
    2. Conduct test sample of the time taken.
    3. Survey recipients to determine how long it took them to receive documents and how accurate the records were.

     

    E

    E

     

     

     

    Admissions and Records

    GOAL 6: To Certify and Report Attendance Data to Appropriate Agencies

    Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Submission of attendance data in an accurate and timely manner.

  •  

    1. Extent of compliance with agency requirements.

  •  

    1.1 Review and analyze agency audit.

     

    E
  •  

    2. Extent of compliance with agency timeliness.

    1.  
    2. Review and analyze college audit.
    3. Comparison of each report deadline with each report submission data. Review and analyze any discrepancies and reasons.

     

    E

    E

     

    Admissions and Records

    GOAL 7: Report Student Characteristics and Enrollment Data as Requested or Required

    Criteria

    Measures

    Methods

    E/A*

  •  

    a) Availability of adequate and efficient character-istics and enrollment data.

  •  

    1. Evidence of the existence of student characteristics and enrollment information.

    1.  
    2. List data elements available.
    3. List all information reports.
    4. Survey data users to determine satisfaction and accuracy of information provided.
    5. Document ease of retrieval of data elements and information accessibility of users.

     

    E

    E

    E

  •  

    b) Coordination of collect-ion and dissemination of student characteristics and enrollment data with other campus units.

  •  

    1. Evidence of the existence of a plan for coordination of collection and dissemination of information.

    1.  
    2. Provide a plan for defining responsi-bilities and outlining coordination for the collection and reporting of data.
    3. Survey of information producers and users to determine the efficiency and timeliness of the coordination.

     

    E

     

    E

     

     
    Admissions and Records

    GOAL 8: To Ensure that A&R Functions are Performed in an Efficient and Effective Manner

    Criteria

    Measures

    Methods

    E/A*

  •  

    a) Existence of A&R systems (operations) Manual.

  •  

    1. Examination of Manuals for Operations and Procedures.

  •  

    1.1 Review of Systems (operation) Manual for: Recency; Completeness; Back-up System (contingency plan).

  •  

    E
  •  

    b) Cost effective.

  •  

    1. Evidence of on-going analysis for cost effective-ness.

  •  

    1.1 Determine cost per student and make comparisons internally and externally (other comparable colleges), (e.g., year to year, system to system, etc.).

  •  

    E
  •  

    c) Competent and trained staff.

  •  

    1. Evidence of implementation of plan for inservice and staff development.

  •  

    1.1 List of events and dates for each activity.

  •  

    E
  •  

    d) Effective coordination of A&R functions with other campus service units.

    1.  
    2. Evidence of a plan or formal procedure/activities for coordination between A&R and other service units.
  •  

    1.1 Survey campus service units to determine extent and quality of coordination.

  •  

    E

     

     Admissions and Records

     

    GOAL 9: To Conduct On-Going Evaluation of Admissions & Records, Service and Programs

    THIS GOAL IS ACCOMPLISHED IN EARLIER GOALS.


      

    STUDENT SERVICES PROGRAM REVIEW PROJECT
    CRITERIA, MEASURES, METHODS

    Assessment Services

    GOAL 1: To Coordinate Assessment Services with other Student and Instructional Services, Feeder High Schools, and Four-Year Institutions

    Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Are assessment services coordinated with appropriate instructional staff, departments, and programs?

  •  

    1. Evidence of communication structure for the purpose of assessment and placement planning.

  •  

    1.1 Review documentation regarding existence of communication structure (e.g., minutes, membership lists, meeting schedule).

  •  

    E
  •  

    2. Satisfaction with above communication structure.

  •  

    2.1 Survey appropriate staff and departments to assess satisfaction.

  •  

    E
  •  

    b) Are assessment services coordinated with appropriate student personnel services and special programs?

  •  

    1. Evidence of communication structure for purpose of assessment and placement planning.

  •  

    1.1 Review documentation regarding existence of communication structure.

  •  

    E
  •  

    2. Satisfaction with above communication structure.

  •  

    2.1 Survey appropriate staff and departments.

  •  

    E
  •  

    c) Are assessment services coordinated with appropriate administrative support services (e.g., data processing, scheduling)?

  •  

    1. Evidence of a communi-cation structure for the purpose of assessment and placement planning?

  •  

    1.1 Review documentation regarding existence of communication structure.

  •  

    E
  •  

    2. Satisfaction with above communication structure.

  •  

    2.1 Survey appropriate staff and departments.

  •  

    E
  •  

    d) Is there college-wide organizational structure (steering committee) and designated individual(s) responsible for the development of the assessment and placement program?

  •  

    1. Evidence of an organi-zational structure and/or designated responsible individual(s).

  •  

    1.1 Review documentation regarding existence of organizational structure (e.g., membership lists, minutes, etc.).

  •  

    E
  •  

    2. Satisfaction with organizational structure.

  •  

    2.1 Survey appropriate staff and departments.

  •  

    E
  •  

    e) Are assessment services coordinated with feeder high schools, four-year institutions, and other regional post-secondary institutions.

  •  

    1. Evidence of communication structure for purpose of assessment & planning.

  •  

    1.1 Review documentation regarding existence of communication structure.

  •  

    E
  •  

    2. Satisfaction with above communication structure.

  •  

    2.1 Survey appropriate staff of feeder high schools, four-year institutions, and other post-secondary institutions.

  •  

    A

     

    Assessment Services
    GOAL 2: To Provide a Comprehensive and Systematic Assessment Program Including Identification of Student Skills, Needs, Interests, Goals, and Abilities

    Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Is there an institutional policy regarding assessment?

  •  

    1. Evidence of institutional policy.

  •  

    1.1 Review policy.

  •  

    E
  •  

    b) Is the policy consistently implemented?

  •  

    1. Evidence of consistent implementation.

  •  

    1.1 Comparison of actual number of students tested versus number of students in testing group (by policy).

  •  

    E
  •  

    c) Is there an adequate delivery system for administering assessment?

  •  

    1. Evidence of availability of frequent testing.

  •  

    1.1 Review testing schedules.

  •  

    E
  •  

    2. Evidence of appropriate testing environment.

  •  

    2.1 Observe testing environment.

  •  

    E
  •  

    3. Evidence of adequate security and confidentiality.

  •  

    3.1 Observe security and confidentiality safeguards.

  •  

    E
  •  

    4. Evidence of qualified testing personnel.

    1.  
    2. Review qualifications of staff.
    3. Survey students to determine satisfaction of assessment delivery.

     

    E

    A

  •  

    d) Is individual assessment available for specific interests and needs of students?

  •  

    1. Evidence of individualized assessment capabilities.

  •  

    1.1 Examine testing inventory and availability of staff to administer and interpret tests.

  •  

    E
  •  

    2. Evidence of adequate referral.

  •  

    2.1 Review referral system.

  •  

    E
  •  

    e) Are the assessment instruments valid and reliable in deter-mining students’ skills, aptitudes, and goals?

  •  

    1. Validity and reliability of assessment instruments.

  •  

    1.1 Review norms of test used.

  •  

    E
  •  

    f) Does the assessment system provide compe-tency/proficiency testing (e.g. course

  •  

    1. Evidence of competency testing.

  •  

    1.1 Review competency/-proficiency testing.

  •  

    E
  •  

    2. Satisfaction with competency/proficiency testing availability.

  •  

    2.1 Survey staff/departments to determine level of satis-faction.

  •  

    A
  •  

    g) Is there adequate publicity to inform students about the assessment process?

  •  

    1. Evidence of publicity materials.

  •  

    1.1 Review inventory of materials.

  •  

    E
  •  

    2. Dissemination of materials.

    1.  
    2. Review dissemination schedule.
    3. Survey students to determine adequacy of materials.

     

    E

    A

     

    * Ind Indicates which methods are essential ("E") for program review and which provide additional ("A") information and/or insight.

     

    Assessment Services
  • GOAL 2: To Provide a Comprehensive and Systematic Assessment Program Including Identification of Student Skills, Needs, Interests, Goals, and Abilities

     

    (continued)

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
    1.  
    2. Are adequate records maintained on assessment results?
    1.  
    2. Evidence of accurate and complete record keeping.
    1.  
    2. Review system for information storage.

     

    E
  •  

    2. Evidence of easily retrievable evidence.

  •  

    2.1 Conduct test sample of assessment records accessibility.

  •  

    E

     

     

     

    Assessment Services
  • GOAL 3: Using Assessment Information to Provide Interpretation and Advisement Regarding Assessment Results, Appropriate Course Selection, Education and/or Career Planning, and Referral Services to Students
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Is assessment information dissemin-ated in a timely and efficient way to users?

  •  

    1. Evidence of timely dissemin-ation of assessment information to users.

  •  

    1.1 Survey counselors, instructors, and students to determine efficiency and timelines.

  •  

    E
  •  

    b) Are clear and concise assessment results provided to students?

  •  

    1. Evidence of clear and concise assessment results.

    1.  
    2. Staff or third party review degree of clarity and conciseness.
    3. Survey students to determine satisfaction with clarity and conciseness.

     

    E

    E

  •  

    c) Are assessment results used to refer students to appropriate special student services (e.g. handicapped, EOPS)?

  •  

    1. Evidence of referrals.

  •  

    1.1 Review sources of student enrollment in special support services.

  •  

    E
  •  

    d) Are assessment results used for career planning advisement?

  •  

    1. Evidence of results being used.

    1.  
    2. Staff or third party review.
    3. Review advisement procedures.
    4. Survey counseling/advisement staff.

     

    E

    E

    E

  •  

    e) Are student career, educational and personal goals considered in the assessment/advisement process?

  •  

    1. Evidence of existence of goal information collected from students and used for advisement.

  •  

    1.1 Review student goal information collection instruments and related advisement procedures.

  •  

    E
  •  

    f) Are enough qualified staff available for interpretation and advisement regarding course selection, educational and/or career planning and referral services?

  •  

    1. Comparison of requirements and staff qualifications.

  •  

    1.1 Review same.

  •  

    E

     

     

     

    Assessment Services
  • GOAL 4: To Place Students in Courses for Which They are Prepared
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Are assessment results used to place students in appropriate classes?

  •  

    1. Evidence of course placement advisement using assessment results.

  •  

    1.1 Review procedures for course placement advisement.

  •  

    E
  •  

    2. Comparison of students’ assessment results and course placement.

  •  

    2.1 Random review of correlation between scores and placement.

  •  

    E
  •  

    b) Do course offerings, and curriculum match students’ needs identified in the assessment process?

  •  

    1. Comparison of aggregated assessment scores with enrollments in courses.

  •  

    1.1 Review statistics for congruence.

  •  

    E
  •  

    2. Evidence of sequential courses to accommodate students’ basic skills needs.

  •  

    2.1 Review same.

  •  

    A
  •  

    c) Are students’ test scores matched with course enrollment prerequisites?

  •  

    1. Comparison of students’ course and program enrollments with their assessment score and advisement.

  •  

    1.1 Review random sample of assessed students’ records.

  •  

    E

     

     

    Assessment Services
  • GOAL 5: To Provide Assessment Information to Staff and Administration for the Purpose of Planning, Research, and Evaluation
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Do you monitor the progress of those students who were assessed (e.g., for retention rates, performance)?

  •  

    1. Evidence of a system monitoring student progress and performance.

  •  

    1.1 Review student follow-up system.

  •  

    E
  •  

    2. Comparison of retention and completion rates of assessed and non-assessed students.

  •  

    2.1 Review statistics.

  •  

    A
  •  

    b) Do you disseminate summarized assessment program results (e.g., skill levels, aggregate student performance)?

  •  

    1. Evidence of dissemination to appropriate staff.

  •  

    1.1 Review dissemination process.

  •  

    E
  •  

    2. Satisfaction of accuracy and completeness of information.

  •  

    2.1 Survey appropriate staff.

  •  

    E
  •  

    c) Is assessment information used for planning of student services programs and curriculum?

  •  

    1. Evidence of interpretation of information for use in planning.

  •  

    1.1 Review procedures and activities.

  •  

    E
  •  

    2. Evidence of use of student services personnel and instructional staff in developing new programs and services.

  •  

    2.1 Review procedures and activities.

  •  

    E


     

     STUDENT SERVICES PROGRAM REVIEW PROJECT

    CRITERIA, MEASURES, METHODS

    Career/Life Services

    GOAL 1: To Develop a District-Wide (College-Wide) Philosophy to Support and Implement a Career Life Program(s)

    Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Is a Career/Life Program philosophy written and available to the public?

  •  

    1. Evidence of philosophy written in college-wide literature.

  •  

    1.1 Check college literature (e.g., catalog, brochures).

  •  

    E
  •  

    b) Was the philosophy developed and updated with wide participation?

  •  

    1. Evidence of wide partici-pation and input into developing philosophy.

    1.  
    2. Review minutes and membership of related meetings.
    3. Interview staff to determine level of awareness of philosophy statement.
    4. Demonstrate evidence of wide participation and dissemination.

     

    E

     

    E

     

    E

  •  

    c) Do you have an annual operational plan which includes goals and objectives?

  •  

    1. Evidence of operational plan.

    1.  
    2. List annual goals and objectives.
    3. Review year-end report.

     

    E

    E

     

     

     

    Career/Life Services
  • GOAL 2: To Assist Students in Developing Career/Life Planning Skills Including Areas Such as Self Assessment, Occupational Search, Decision-Making, Goal Determination
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Is a complete range of services available (e.g., career orientation, assessment, goal setting, skill development)?

  •  

    1. Evidence of all services available.

  •  

    1.1 Inventory of all services.

  •  

    E
  •  

    b) Do students avail themselves of Career/ Life Services?

  •  

    1. Level of service utilization.

  •  

    1.1 Count numbers of clients using each type of service.

  •  

    E
  •  

    c) Are adequate professional staff and budget available?

  •  

    1. Evidence of professional staff available.

    1.  
    2. Record staff currently available.
    3. Compare adequacy of staffing to client demand.

     

    E
  •  

    2. Evidence of adequate budget available.

    1.  
    2. Review budget allocation.
    3. Compare adequacy of budget to that required by client demand.

     

    E

    E

  •  

    d) Are Career/Life Services located in an easily accessible appropriate facility?

  •  

    1. Accessibility of services to clients.

  •  

    1.1 Review campus traffic flow to assess accessibility.

  •  

    E
  •  

    2. Proximity to related services.

  •  

    2.1 Review site facilities plans to determine proximity to related services.

  •  

    E
  •  

    3. Adequacy and quality of space.

    1.  
    2. Determine if space is sufficient to meet client demand.
    3. Survey opinions of staff and clients to determine satisfaction with space allotment.
    4. Determine if relationship of materials to facilities is appropriate.

     

    E

     

    E

     

    E

     

     

     

    Career/Life Services
  • GOAL 3: To Assist the Client in Developing a Process for Career/Life Decision-Making and Serve as a Clearing House for Information
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Are skill develop-ment experiences provided?

  •  

    1. Evidence of skill development experience.

  •  

    1.1 Inventory skill development activities.

  •  

    E
  •  

    b) Do students partici-pate in skill develop-ment experiences?

  •  

    1. Number of students utilizing service.

  •  

    1.1 Count number of clients involved in skill development experiences.

  •  

    E
  •  

    c) Are skill develop-ment experiences delivered in a variety of formats?

  •  

    1. Evidence of a variety of delivery modes.

  •  

    1.1 Inventory formats for presentations (e.g., classes, seminars, individual contacts).

  •  

    E
  •  

    d) Do skill development experiences meet student need?

  •  

    1. Evidence of adequacy of skill development experi-ences to meet student need.

  •  

    1.1 Survey clients to determine relevancy of skill development experiences.

  •  

    E
  •  

    e) Are Career/Life materials available, relevant, compre-hensive, and current?

  •  

    1. Evidence that Career/Life materials meet the educational career/ occupational and other personal needs of students.

    1.  
    2. Survey clients, staff, and advisory committees, and other experts in the field.
    3. Inventory client self-assessment instruments, (e.g., placement tests, interest inventories, value clarification).
    4. Evaluate materials in terms of:
    1. industry professional publications
    2. business and industry visitations
    3. copyright dates of materials
    4. gender equity
    5. labor market statistics
    6. other relevant measures

     

    E

     

     

    E

     

    E

  •  

    2. Evidence that Career/Life materials are provided to other services.

  •  

    2.1 Interview staff to determine whether appropriate materials provided.

  •  

    E
  •  

    3. Evidence of state of the art hardware and software.

  •  

    3.1 Inventory computerized equipment and software library.

  •  

    E

  •  

    * *Indicates which methods are essential ("E") for program review and which provide additional ("A") information and/or insight.

     

     
  • Career/Life Services
  • GOAL 3: To Assist the Client in Developing a Process for Career/Life Decision-Making

    and Serve as a Clearing House for Information

     

    (continued)

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    f) Is the community aware of your Career/Life Services?

  •  

    1. Evidence of developing awareness of Career/Life Services.

    1.  
    2. Inventory public information resources regarding Career/Life Services.
    3. List all activities relating to services (e.g., Career days, college days, guest speakers, seminars).
    4. Count number of non-students (community members) who use career assessment services.

     

    E

     

    A

     

     

    E

  •  

    * *Indicates which methods are essential ("E") for program review and which provide additional ("A") information and/or insight.

  •  

    Career/Life Services
  • GOAL 4: To Coordinate Career/Life Services with Other Student Services/Instructional Programs
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Are there cooperative activities with other services and departments.

  •  

    1. Evidence of cooperative activities.

    1.  
    2. Document organizational structure that facilitates coordination.
    3. Review master calendar of all coordinated activities and scheduled meetings.
    4. Review publicity regarding activities (e.g., classroom visits, joint projects, division meetings).
    5. Review coordination and referral efforts regarding job placement and work experience.

     

    E

     

    E

     

    E

     

    E

  •  

    b) Are advisory committees broadly based?

  •  

    1. Evidence of broadly based committee involvement.

    1.  
    2. List composition of occupational planning committees.
    3. Demonstrate representation from faculty, community practitioners, student services, and students.

     

    E

     

    E

     

     

     

    Career/Life Services
  • GOAL 5: To Provide Career/Life Services for Special Populations (i.e, Older Adults, EOPS, Disabled and Re-Entry)
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Are information and materials for special populations available and accessible?

  •  

    1. Evidence of availability and accessibility of information and materials.

    1.  
    2. Inventory material for special population.
    3. Survey special groups regarding availability and accessibility.

     

    E

     

    E

  •  

    b) Are there Career/Life Services activities avail-able for special populations?

  •  

    1. Evidence of special activities.

    1.  
    2. Review master calendar of planned activities for special populations.
    3. Record number of participants.

     

    E

     

     

     

    Career/Life Services
  • GOAL 6: To Provide Staff In-Service and Educational Upgrading Opportunities for Skills and Knowledge in Career/Life Areas
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Do in-service training activities meet staff needs?

  •  

    1. Evidence of staff needs assessment.

    1.  
    2. Complete staff needs assessment.
    3. Review planned in-service activities and agendas.
    4. List participants in in-service activities.
    5. Identify changes which resulted as a result of the in-service.

     

    E

    E

     

    E

    E

  •  

    b) Are educational upgrading opportunities provided?

  •  

    1. Evidence of educational upgrading opportunities.

    1.  
    2. Identify programs available (e.g., conferences, seminars, workshops, on and off campus classes, skill training).
    3. Survey satisfaction with in-service activities.

     

    E

     

     

    E

     

     

    STUDENT SERVICES PROGRAM REVIEW PROJECT

    CRITERIA, MEASURES, METHODS

     

    Counseling
    GOAL 1: To Conduct Student Orientation About College Curricula and Services

    Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Orientation includes complete information on curricula and services.

  •  

    1. Orientation script and materials

  •  

    1.1 Read/observe script.

     

    E
  •  

    b) Availability of orientation.

  •  

    1. Evidence of appropriate orientation for all students (printed schedules, etc.)

  •  

    1.1 Interview services personnel.

     

    E
  •  

    2. Frequency of orientation times (day/eve/other) & locations.

  •  

    2.1 Read printed materials.

     

    E
  •  

    c) Student participation

  •  

    1. Percent of students participating.

  •  

    1.1 Count total number of student & compare to total enrollment

  •  

    E
  •  

    2. Demographic information on participating.

  •  

    2.1 Count students (or sample) by sex, age, ethnicity, status.

  •  

    E
  •  

    d) Effectiveness of student orientation program.

  •  

    1. Use of student services.

  •  

    1.1 Survey of participants.

  •  

  •  

    2. Retention of orientation information.

  •  

    2.1 Survey students completing orientation (population or sample).

  •  

    E
  •  

    e) Degree of student satisfaction with orientation.

  •  

    1. Evidence of student satisfaction.

  •  

    1.1 Survey students at end of orientation and end of semester.

  •  

    E

     

     

     

    Counseling
  • GOAL 2: To Articulate with Schools, Business, Industry, and Appropriate Agencies for the Purpose of Identifying Potential Students and Assisting Them to Enter the College
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Counselor visits to appropriate/locations (schools, businesses, etc.).

  •  

    1. Number of visits.

  •  

    1.1 Count number of visits.

     

    E
  •  

    b) Distribution and use of written material about college and programs.

  •  

    1. Evidence of distribution of materials.

  •  

    1.1 Determine number and location of distributed materials.

  •  

    E
  •  

    2. Evidence of utilization of materials.

  •  

    2.1 Survey population or sample of high schools, businesses, or industries to determine extent of use.

  •  

    E
  •  

    c) Awareness of college programs and services.

  •  

    1. Percentage of population indicating awareness of programs.

  •  

    1.1 Survey population or sample of high schools, businesses, industries, etc.

  •  

    E
  •  

    d) Reason for selecting college.

  •  

    1. Percent of students for each reason.

  •  

    1.1 Survey (sample) students enrolling.

  •  

    A
  •  

    e) Satisfaction with articulation activities.

  •  

    1. Percentage of respondents indicating satisfaction with activities and agreements.

  •  

    1.1 Survey personnel who are part of articulation process.

  •  

    A
  •  

    2. Suggestions from respondents.

  •  

    2.1 Survey personnel who are part of articulation process.

  •  

  •  

    f) Similarity of demo-graphics of service area and student population.

  •  

    1. Census data.

  •  

    1.1 Examine latest census.

  •  

    E
  •  

    2. Feeder high school demographics.

  •  

    2.1 College/high school documents.

  •  

    E
  •  

    3. Student data base (Chancellor’s Office).

  •  

    3.1 Procure information.

  •  

    A
  •  

    4. Chamber of Commerce data.

  •  

    4.1 Procure information.

  •  

    A
  •  

    5. College/district student demographics.

    1.  
    2. Use data base or survey sample of students.
    3. Compare student profile information to district service area demographics.

     

    E

    E

  •  

    g) College enrollment rates from feeder high schools.

  •  

    1. Percentage of most recent college enrollees from high school graduating classes.

  •  

    1.1 Count size of feeder high school graduating class. Count number of those who entered college directly.

  •  

    E

     

    Counseling
  • GOAL 2: To Articulate with Schools, Business, Industry, and Appropriate Agencies for the Purpose of Identifying Potential Students and Assisting Them to Enter the College

     

    (continued)

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    h) Establish reasons for non-enrollment.

  •  

    1. List of reasons for non-enrollment.

    1.  
    2. Sample a list of high school graduates who did not enroll.
    3. Have high schools conduct exit surveys of their students.

     

    A
  •  

    i) Enrollment rates from targeted businesses, industries, and social agencies.

  •  

    1. Percentage of enrollees compared to potential enrollees.

    1.  
    2. Establish target populations.
    3. Determine potential enrollment.
    4. Determine percentage of enrollment.

     

    E
  •  

    j) Extent of difficulties in transferring coursework into this institution.

  •  

    1. Evidence of difficulties.

  •  

    1.1 Survey in-coming students after transcript/experience evaluations have been made.

  •  

    E
  •  

    2. Evidence of procedures for evaluation of prior learning.

  •  

    2.1 Check policy and procedure manuals and other documents.

  •  

    A
  •  

    3. Evidence of procedure for alleviating problems/ grievances.

    1.  
    2. Check policy and procedure manuals and other documents.
    3. Survey students with problems/ grievances.

     

    E

     

     

     

    Counseling
  • GOAL 3: To Provide Academic, Career, and Personal Counseling to Assist Students with

    Course and Program Selection, Career Selection, and the Identification of

    Personal and Special Needs

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of counseling services.

  •  

    1. Official list.

  •  

    1.1 Review list of counseling services and validate by interviews.

  •  

    E
  •  

    b) Variety of service delivery methods.

  •  

    1. List of methods available for each service.

  •  

    1.1 Validate by interview/observation or survey.

  •  

    E
  •  

    c) Student utilization of counseling services.

  •  

    1. Number of duplicated and unduplicated counseling contacts.

  •  

    1.1 Count counseling contacts.

  •  

    E
  •  

    d) Student satisfaction with counseling services.

  •  

    1. Degree of student satisfaction with counseling services.

  •  

    1.1 Survey of participating and non-participating students.

  •  

    E
  •  

    e) Student course completion.

    1.  
    2. Student course pass rates.
    3. Percent of students with educational plan.
    4. General knowledge of requirements to meet educational goals.
    5. Number of program changes.
    6. Evidence of assisting in the resolution of personal problems.
    7. Use of referral services.
    8. Degree of student satisfaction with counseling.
  •  

    1-7 Sample to obtain equivalent groups of students who took guidance course(s) and those who did not.

  •  

    E
  •  

    f) Availability of services for students with special needs.

  •  

    1. Published listings of counseling services for students with special needs.

  •  

    1.1 Review listings of counseling services and validate by interviews/ surveys.

  •  

    E
  •  

    g) Sufficient number of credentialed counselors.

  •  

    1. Evidence of students to counselor appropriate ratios.

    1.  
    2. Calculate student to counselor ratio.
    3. Compare ratio to accepted standards (e.g., - APGA)

     

    E

    A

     

     

     

    Counseling
  • GOAL 4: To Provide Students with Information About Their Skills and Abilities and About Program and Course Expectations to Assist Them in Achieving Their Academic Goals
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Relationship between course success rates and assessment scores.

  •  

    1. Letter grades, assessment scores.

  •  

    1.1 Determine rate of course success among qualified vs. under-qualified students.

  •  

    E
  •  

    b) Availability of assessment program.

    1.  
    2. Evidence of appropriate assessment program for all students (printed schedule, etc.)

     

    1.1 Interview Student Services personnel and read printed material.

     

    E

     

    Frequency of assessment program day/eve/other location.

  •  

    2.1 Interview Student Services personnel and read printed material.

  •  

    E
  •  

    c) Student participation.

  •  

    1. Percent of students participating in assessment.

  •  

    1.1 Count number of participating students, compare total enrolled.

  •  

    E
  •  

    2. Demographic information on participating and non-participating students.

  •  

    2.1 Count students (or sample) by sex, age, ethnicity, status and compare to those enrolled.

  •  

    E
  •  

    d) Student satisfaction with assessment/placement procedure.

  •  

    1. Degree of student satisfaction in assessment/ placement procedure.

  •  

    1.1 Survey students at the end of session/year.

  •  

    E
  •  

    e) Instructor satisfaction with assessment/ placement procedure.

  •  

    1. Degree of instructor satisfaction in assessment/ placement procedure.

  •  

    1.1 Survey instructors at the end of session/year.

  •  

    E
  •  

    f) Demonstrate reliability and validity of assessment instrument.

  •  

    1. Reliability and validity coefficients.

  •  

    1.1 Compute reliability and validity coefficients.

  •  

    E
  •  

    g) Distribution of aggregated results from assessment.

  •  

    1. Published aggregated results.

  •  

    1.1 Review final product.

  •  

    A
  •  

    2. Extent of distribution.

  •  

    2.1 List of recipients of these results.

  •  

    E

     

    Counseling
  • GOAL 5: To Provide Students with Information Which Will Assist Them to Identify and Achieve Their Career Goals
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of career information program.

    1.  
    2. Evidence of descriptive written material.
    3. Availability of career inventories.
    4. Evidence of computerized information system.
  •  

    1-3 Interview staff, examine literature and materials.

  •  

    E
  •  

    b) Utilization of career information.

  •  

    1. Amount of use.

  •  

    1.1 Examine records.

  •  

    E
  •  

    c) Degree to which students identify career goals.

  •  

    1. Evidence of increase in clarity of career goals.

    1.  
    2. Post or pre/post survey of participants.
    3. Compare career assessed students with non-assessed students.

     

    E

    A

  •  

    d) Distribution of analysis of career results.

  •  

    1. Evidence of analytical reports.

  •  

    1.1 Examine records.

     

    E
  •  

    2. Evidence of distribution.

  •  

    2.1 Examine list of distribution locations. Note distribution methods and frequency.

  •  

    E
  •  

    e) Student satisfaction with career services.

  •  

    1. Percent of respondents indicating satisfaction.

  •  

    1.1 Survey students (population or sample).

  •  

    E

     

    Counseling
  • GOAL 6: To Articulate with Education Institutions, Business, Industry and Appropriate Agencies for the Purpose of Providing Necessary Planning Information to Students
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Currency and accuracy of articulation agreements with education and industry.

  •  

    1. Evidence of articulation agreements.

    1.  
    2. Examine documents; note dates.
    3. Verify with appropriate institutions/ organizations/etc.

     

    E

    E

  •  

    b) Accessibility of agreements.

  •  

    1. Number and location of articulation agreements.

  •  

    1.1 Determine number and location.

     

    E
  •  

    2. Counselor and student knowledge of articulation documents.

  •  

    2.2 Interview, survey counselors and students.

  •  

    E
  •  

    c) Degree of satisfaction with articulation agreements.

  •  

    1. Percentage of respondents indicating satisfaction.

  •  

    1.1 Interviews with appropriate personnel.

  •  

    E
  •  

    d) On and off campus contacts with business and industry.

  •  

    1. Nature and number of contacts.

  •  

    1.1 Interview college and business personnel.

  •  

    A
  •  

    e) College and business participation of vocational advisory committees.

  •  

    1. Nature and number of meetings.

  •  

    1.1 Review advisory committee minutes.

  •  

    E
  •  

    2. Attendance reports.

  •  

    2.1 Check attendance.

  •  

    E
  •  

    f) Evidence of joint college/community/ business activities (i.e., Career Day, Transfer Day)

  •  

    1. Nature and number of joint functions.

  •  

    1.1 Review published documents.

  •  

    E
  •  

    2. Attendance at joint functions.

  •  

    2.1 Count attendance.

  •  

    E

     

     

     

    Counseling
  • GOAL 7: To Provide Needed Instruction In Counseling-Related Courses
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Provision of counseling related courses.

    1.  
    2. Report of student needs.
    3. Evidence of courses.
    4. List of unmet needs.
  •  

    1&3 Students needs assessment; compare needs assessment to courses offered.

  •  

    E
  •  

    b) Student course success.

  •  

    1. Achievement of student and course objectives.

  •  

    1.1 Tests, interviews, surveys, and longitudinal information.

  •  

    E
  •  

    2. Course completion rates.

  •  

    2.1 Examine grade records.

  •  

    E
  •  

    c) Satisfaction with counseling-related courses.

    1.  
    2. Percentage of respondents indicating satisfaction.

     

    1.1 Survey students and personnel who are part of counseling related courses.

     

    E

     

    Suggestions from respondents.

  •  

    2.1 Survey students and personnel.

  •  

    E

     

     

     

    Counseling
  • GOAL 8: To Coordinate Counseling Services with Other Student Services and Instructional Programs
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
    1.  
    2. Nature and frequency of cooperative activity between counseling and:
    3. Instructional areas,
    4. Counseling aspects of special programs,
    5. All other student services.
    1.  
    2. Records of meetings.
    3. Memos.
    4. Reports.
    5. Counselor agreement.
  •  

    1-4 Examine records and interview personnel.

  •  

    E
    1.  
    2. Satisfaction with and effectiveness of joint activities.
  •  

    1. Nature and number of joint programs and activities.

  •  

    1.1 Examine records.

  •  

    E
  •  

    2. Expressed satisfaction.

  •  

    2.1 Interview staff.

  •  

    E
  •  

    c) Student intra-institutional referrals.

  •  

    1. Number of referrals.

  •  

    1.1 Check referral records.

  •  

    E
  •  

    d) Counselor participants in instructional planning.

  •  

    1. Records of counselor participation, e.g., membership on instructional committees.

    1.  
    2. Interview appropriate personnel.
    3. Review minutes and records.

     

    E

    E

     

     

     

    Counseling
  • GOAL 9: To Monitor Student Progress for the Purpose of Assisting Students to Achieve Their Goals
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Plan for monitoring student progress.

  •  

    1. Evidence of written document.

  •  

    1.1 Examine document.

  •  

    E
  •  

    b) Utilization of monitoring process.

  •  

    1. Number of student contacts attributable to monitoring process.

  •  

    1.1 Examine records.

  •  

    E
  •  

    2. Records of actions (interventions).

  •  

    2.1 Examine records.

  •  

    E
  •  

    3. Number of referrals to other agencies, services, etc.

  •  

    3.1 Examine records.

  •  

    E
  •  

    c) Effectiveness of monitoring process.

  •  

    1. G.P.A., retention, academic progress toward goal.

    1.  
    2. Analysis of student records.
    3. Comparison of students receiving intervention with those needing but not receiving intervention.

     

    E

    E

     

     

     

    Counseling
  • GOAL 10: To Prepare Students for a Successful Transition Beyond the Community College
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
    1.  
    2. Identification of:
    3. transfer students
    4. vocational students
  •  

    1. Evidence of written operational definition of transfer student.

  •  

    1.1 Examine written definition.

  •  

    E
  •  

    2. Evidence of written operational definition of vocational student.

  •  

    2.1 Examine written definition.

  •  

    E
  •  

    3. Identify numbers of transfer and vocational students.

  •  

    3.1 Count students so identified.

  •  

    E
    1.  
    2. Transition activities for:
    3. transfer students
    4. vocational students
  •  

    1. Evidence of transition activities (including on-campus visits, etc.).

    1.  
    2. Examine documents.
    3. Interview staff.

     

    E

    E

    1.  
    2. Effectiveness of transition activities.
  •  

    1. Transfer rate and student success at transfer institutions.

  •  

    1.1 Examine transfer institutions’ records.

  •  

    E
  •  

    2. Vocational placement and success.

  •  

    2.1 Examine community college records.

  •  

    E
  •  

    3. Evidence of student articulation problems.

    1.  
    2. Interviews
    • Students
    • Community College staff
    • Transfer institutions staff
    • Business – Industry supervisors, personnel, officers, etc.

     

    A

     

    Counseling
  • GOAL 11: To Provide In-Service Training and Other Opportunities for Staff Development
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Determination of in-service training and professional growth needs.

  •  

    1. List of such needs.

  •  

    1.1 Survey staff, management, etc.

  •  

    E
  •  

    b) Provisions of in-service programs.

  •  

    1. Funding.

  •  

    1.1 Examine budget allocations.

  •  

    E
  •  

    2. List of programs.

  •  

    2.1 Examine program records. Note number, types, etc.

  •  

    E
  •  

    3. Availability of resource personnel.

  •  

    3.1 Examine program records. Note number, types, etc.

  •  

    E
  •  

    c) Effectiveness of in-service programs.

  •  

    1. Attainment of objectives.

  •  

    1.1 Survey of participants.

  •  

    E
  •  

    2. Participant satisfaction.

  •  

    2.1 Survey of participants.

  •  

    E
  •  

    3. Evidence of institutional change as a function of in-service training.

  •  

    3.1 Examine documents (e.g., staff evaluations, planning documents, etc.).

  •  

    E

     

     STUDENT SERVICES PROGRAM REVIEW PROJECT

    CRITERIA, MEASURES, METHODS

    Financial Aid
  • GOAL 1: To Seek Financial Aid Funding From All Available Sources
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Division of funding among grants, work and loan resources.

  •  

    1. Institutional policy regarding financial aid.

  •  

    1.1 Examine written institutional policy or interview panel.

  •  

    E
  •  

    2. Evidence of applications for funding at federal, state, and local levels.

  •  

    2.1 Check financial aid office for applications and program participation agreement.

  •  

    E
  •  

    b) Proportion of applications funded.

  •  

    1. Established student financial need as documented in applications.

    1.  
    2. Examine reports.
    3. Track number of applications denied for lack of funding.

     

    E

     

  •  

    2. Level of funding awarded from federal, state, local, and other sources.

  •  

    2.1 Review allocation letters; compare awards to application requests.

  •  

    E
  •  

    c) Scholarship program development.

  •  

    1. Number of scholarships available.

    1.  
    2. Count scholarships.
    3. Review the number and quality of contacts with donors (including follow-up reports).

     

    E

    A

  •  

    2. Dollar amounts available.

  •  

    2.1 Count dollars received.

     

    E
  •  

    3. Number of students applying.

  •  

    3.1 Count number of student applications.

  •  

    E
  •  

    4. Number & type of students awarded scholarships.

    1.  
    2. Count the number & type of awards.
    3. Compare donor criteria to potential applicant pool.

     

    E

    E

  •  

    d) Extent of lobbying/ political efforts to obtain funding.

  •  

    1. Evidence of contacts made and time spent at local, state, and federal levels.

  •  

    1.1 Examine the nature and number of contacts made.

  •  

    E
  •  

    2. Level of involvement.

  •  

    2.1 Check records or interview staff for information. Review the types of contacts made.

  •  

    E
  •  

    e) Extent of work professional organizations.

  •  

    1. Evidence of time spent working with organizations.

  •  

    1.1 Interview staff to determine time spent.

  •  

    E
  •  

    2. Level of involvement.

  •  

    2.1 Review quality of involvement (e.g., office holder, active participant, etc.).

  •  

    E
  •  

    f) Level of participation.

  •  

    1. Number and scope of programs.

    1.  
    2. Count number of programs.
    3. Review types of programs.

     

    E

    E

     

    Financial Aid
  • GOAL 2: To Award Funds in a Manner Appropriate to Student Needs and in Accordance with Institutional Policy
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Written institutional awarding policies.

  •  

    1. Evidence of written policies adopted by Board.

  •  

    1.1 Examine college publications for statements of financial aid policies.

  •  

    E
  •  

    b) Advisory group participation.

  •  

    1. Evidence of advisory committee meetings.

    1.  
    2. Count number of meetings.
    3. Review the minutes regarding the nature of issues addressed.
    4. Review minutes for decisions made, actions taken.

     

    E

    A

  •  

    2. Number of participants.

    1.  
    2. Count number of participants.
    3. Review participants attendance at meetings.

     

    E
  •  

    3. Breadth of advisory group.

  •  

    3.1 Review list of members to ascertain representation.

  •  

    E
  •  

    4. Evidence of formal charge to committee.

    1.  
    2. Examine FAO advisory group records.
    3. Survey staff and committee members.

     

  •  

    c) Financial aid recipient demographics.

  •  

    1. Degree to which recipient demographics conform to institutional policy or goals.

    1.  
    2. Examine formal and/or informal policies.
    3. Compare recipient demographics with policy statements.
    4. Review recipient demographics for income levels, etc.

     

    E

    E

  •  

    d) Nature of disburse-ments.

    1.  
    2. Number and frequency of disbursements.

     

    1.1 Count number and frequency.

     

    E

     

    Timeliness of disbursements.

    1.  
    2. Measure turnaround time.
    3. Random survey of students to determine timelines.

     

    E

     

    Proportion of eligible students served.

    1.  
    2. Compare recipient numbers with eligible student numbers.

     

    E

     

     

     

    Financial Aid
  • GOAL 2: To Award Funds in a Manner Appropriate to Student Needs and in Accordance with Institutional Policy

     

    (continued)

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    e) Effectiveness of awarding policies.

  •  

    1. Percentage of recipients in the total enrollment.

  •  

    1.1 Compare total enrollment to recipients records.

  •  

    E
  •  

    2. Retention, GPA of recipients.

  •  

    2.1 Compare retention rates and GPA of recipients to non-recipients.

  •  

    A
  •  

    3. ADA generated by aid recipients.

  •  

    3.1 Count ADA generated by aid recipients.

  •  

    A
  •  

    4. Number of applicants compared to number of recipients.

    1.  
    2. Compare number of applicants to number of recipients.
    3. Identify reasons/categories of applicants not funded.

     

    E

    A

     

     

     

    Financial Aid
  • GOAL 3: To Develop and Disseminate Information to Targeted Segments of the Community About Financial Aid Programs and Services
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Informing targeted segments of community.

  •  

    1. Evidence of institutional policy.

  •  

    1.1 Review policy statement or interview staff.

  •  

    E
  •  

    2. Evidence of list of targeted segments.

  •  

    2.1 Review FAO records.

  •  

  •  

    3. Evidence of awareness of financial aid services.

  •  

    3.1 Survey community to determine awareness.

  •  

    E
  •  

    b) Provision of financial aid workshops.

  •  

    1. Number of workshops.

  •  

    1.1 Count workshops.

  •  

    E
  •  

    2. Quality of workshops.

    1.  
    2. Review participants’ evaluations of workshops.
    3. Review content of workshops.

     

    E

    E

  •  

    3. Location, time, and attendance.

  •  

    3.1 Review records.

     

    E
  •  

    c) Extent of media coverage.

  •  

    1. Number and scope of new items prepared for publication.

  •  

    1.1 Evaluation of the quantity and quality of media coverage; develop survey.

  •  

    E
  •  

    d) Extent of student contacts.

  •  

    1. Composition of student contracts (with whom, when, where, and how).

  •  

    1.1 Review records and/or interview financial aid staff.

  •  

    E
  •  

    e) Applications distributed.

  •  

    1. Number of applications (with whom, when, where, and how).

    1.  
    2. Review records and/or interview financial aid staff and students.
    3. Annual comparison of number of applicants.

     

    E

    E

  •  

    f) Nature of financial aids publications.

  •  

    1. Quality of publication in terms of content and readability.

    1.  
    2. Review publications for compliance with local, state, and federal regulations.
    3. Review publications for readability/ assess reading level required.

     

    E

     

    E

  •  

    2. Availability of publications.

    1.  
    2. Check numbers and types of publications available.
    3. Check distribution locations.

     

    E

    A

  •  

    3. Budget available to FAO to produce publications.

  •  

    3.1 Review FAO publications budget.

  •  

    E

     

     

     

    Financial Aid
  • GOAL 4: To Provide Assistance in the Financial Aid Application and Awarding Process
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Financial aid workshops.

  •  

    1. Number of workshops.

  •  

    1.1 Count workshops.

  •  

    E
  •  

    2. Number attending workshops.

  •  

    2.1 Review records.

  •  

    E
  •  

    3. Timelines of workshops.

  •  

    3.1 Review schedule.

  •  

    E
  •  

    4. Availability of workshops.

  •  

    4.1 Review geographic/demographic considerations.

  •  

  •  

    b) Quality of appli-cation.

  •  

    1. Completeness of applications.

  •  

    1.1 Review student files.

  •  

    E
  •  

    2. Types of assistance.

  •  

    2.1 Interview students and staff.

  •  

    E
  •  

    3. Quality of assistance.

  •  

    3.1 Survey students.

  •  

    E
  •  

    4. Number of students complaints.

  •  

    4.1 Interview staff and students.

  •  

    E
  •  

    5. Availability of financial aid counseling services.

  •  

    5.1 Examine staffing (numbers, bilingual) and office hours.

  •  

    E
  •  

    6. Evidence of cycles, deadlines, priorities for awarding grants.

  •  

    6.1 Review published policies and procedures.

  •  

    E

     

     

     

    Financial Aid
  • GOAL 5: To Provide Counseling and Referrals in Matters related to Financial Aid
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Extent of counseling service.

  •  

    1. Availability.

  •  

    1.1 Examine staffing and office hours.

  •  

    E
  •  

    2. Number of contacts, establish referral tracking system.

  •  

    2.1 Examine records for count, average length, content.

  •  

    E
  •  

    3. Types of contacts (e.g., group, individuals).

  •  

    3.1 Interview staff and students.

  •  

    E
  •  

    4. Number of student referrals.

  •  

    4.1 Interview staff and students.

  •  

    E
  •  

    5. Number of staff.

  •  

    5.1 Examine personnel records.

  •  

    E
  •  

    b) Effectiveness of counseling services.

  •  

    1. Debt management instructional activities.

    1.  
    2. Interview staff and students.
    3. List types of activities.
    4. Examine exit/entry interview records.

     

    E

    E

    E

  •  

    2. Retention rates.

  •  

    2.1 Examine student records.

     

    E
  •  

    3. Number and nature of award revisions.

  •  

    3.1 Examine student records.

     

    E
  •  

    4. Results of referrals.

  •  

    4.1 Interview other services providers and students.

  •  

    E
  •  

    5. Confidentiality.

    1.  
    2. Examine facilities.
    3. Interview staff and students.

     

    E

    E

     

     

     

    Financial Aid
  • GOAL 6: To Monitor the Academic Progress or Financial Aid Recipients to Comply with

    Federal, State, and Institutional Guidelines

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Academic progress policy.

  •  

    1. Evidence of an academic progress policy and follow-up.

    1.  
    2. Review policy and procedures for adherence to federal and state guidelines.
    3. Compare procedures with established policy.
    4. Check records or interview financial aid personnel and students for information.

     

    E

     

    E

     

    E

  •  

    2. Evidence that program participation agreement is enforced.

  •  

    2.1 Check FAO records for follow-up contacts, student agreements, provision for follow-up, counseling, etc.

  •  

    E
  •  

    b) Recipients on probation or disqualified.

  •  

    1. Number of recipients on probation who remain enrolled.

  •  

    1.1 Examine student records.

  •  

    E
  •  

    c) Recipients on probation who remain enrolled.

  •  

    1. Number of recipients on probation who remain enrolled.

  •  

    1.1 Examine student records.

  •  

    E
  •  

    2. Number of recipients who graduate, transfer, obtain honors, etc.

  •  

    2.1 Examine student records.

  •  

    E
  •  

    d) GPA and units attempted/completed by recipients.

  •  

    1. Number of units attempted/ completed, and GPA of recipients.

  •  

    1.1 Examine student records.

  •  

    E

     

     

     

    Financial Aid
  • GOAL 7: To Certify and Report Financial Aid Date to Appropriate Agencies
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Evidence of completion of required reports.

  •  

    1. Accuracy

  •  

    1.1 Audit and records review.

  •  

    E

     

    2. Timelines.

  •  

    2.1 Audit and records review.

  •  

    E

     

    3. Audit trail.

  •  

    3.1 Interview auditors.

  •  

    E
  •  

    4. Evidence of conformity with Institutional Guide for Financial Aid Self-Evaluation.

  •  

    4.1 Review reports according to Guide.

  •  

    E
  •  

    b) Audit exceptions.

  •  

    1. Number of audit exceptions.

    1.  
    2. Review annual audit and response to resolution of audit citings.
    3. Exit interview with auditors.

     

    E

     

    A

  •  

    c) Overawards.

  •  

    1. Number of overawards.

    1.  
    2. Review student files.

     

    E
  •  

    d) Defaults.

  •  

    1. Number of defaults.

  •  

    1.1 Review student files.

     

    E
  •  

    e) Collections.

  •  

    1. Number of collections.

  •  

    1.1 Review student files.

     

    E

     

    Financial Aid
  • GOAL 8: To Report Student Data to the College Community and to Outside Organizations
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Nature of publicity.

  •  

    1. Number and content of newspaper articles and other media prepared for publication.

  •  

    1.1 Check files and/or interview public information officer.

  •  

    E
  •  

    b) Informational reports generated.

  •  

    1. Timeliness, number, and accuracy of reports.

  •  

    1.1 Review reports and interview staff.

  •  

    E
  •  

    2. Circulation (to whom, how, when, and where).

  •  

    2.1 Interview staff and those receiving reports and check distribution list.

  •  

    E
  •  

    3. Evidence of responses to inquiries.

  •  

    3.1 Compare number of requests to number of responses.

  •  

    E

     

     

     

    Financial Aid
  • GOAL 9: To Work with Other College Officers to Ensure that All Financial Aid Functions are Performed in an Efficient and Effective Manner
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Inter-related activi-ties with other college offices.

  •  

    1. Evidence of inter-related activities.

  •  

    1.1 Interview personnel and examine records.

  •  

    E
  •  

    2. Number of meetings held between offices.

  •  

    2.1 Count number of meetings.

  •  

    E
  •  

    3. Number of informal contacts.

  •  

    3.1 Interview personnel.

  •  

    E
  •  

    4. Timeliness of inter-related activities.

  •  

    4.1 Examine records of activities relative to college calendar.

  •  

    E
  •  

    5. Nature and frequency of meetings and informal contacts (include in-service meetings).

  •  

    5.1 Interview staff in financial aid and other offices.

  •  

    E
  •  

    6. Degree of cooperation with other offices.

  •  

    6.1 Interview staff in FAO and other offices.

  •  

    E
  •  

    b) Effectiveness of inter-related activities.

  •  

    1. Appraisal of financial aid by other college staff.

  •  

    1.1 Interview and survey staff in other college offices.

  •  

    E
  •  

    c) Inter-office functions and procedures.

  •  

    1. Evidence of inter-office functions and procedures.

  •  

    1.1 Review records and procedures.

  •  

    E
  •  

    d) Coordination of funding services.

  •  

    1. Evidence of coordination.

  •  

    1.1 Review records.

  •  

     

     

     

    Financial Aid
  • GOAL 10: To Administer Programs in Compliance with Appropriate Program Regulations
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Compliance with all regulations.

  •  

    1. Evidence of compliance.

    1.  
    2. Review audit reports.
    3. Review program review results.
    4. Review data validations-EOPS.
    5. Review accreditation report.

     

    E

    E

    E

    A

  •  

    b) Loan billing and collection.

  •  

    1. Evidence of procedures.

    1.  
    2. Review default rate.
    3. Review audit exception.

     

    E

    E

  •  

    c) Security.

  •  

    1. Evidence of security and retention of files.

    1.  
    2. Inspection of facilities.
    3. Interview staff re: procedures

     

    E

    E

  •  

    2. Evidence of policy of student rights and responsi-bilities.

    1.  
    2. Review written policy.
    3. Interview students.

     

    E

    A

     

    Financial Aid
  • GOAL 11: To Conduct On-Going Evaluation of Financial Aid Programs and Services
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Extent of self-evaluation.

  •  

    1. Use of self-evaluation guide.

    1.  
    2. Review files.
    3. Check records.

     

    E

    E

  •  

    2. Number and scope of self-evaluation activities.

  •  

    2.1 Review results of evaluation activities.

  •  

    E
  •  

    3. Evidence of FAO goals and objectives for improvement of service.

    1.  
    2. Review FAO records.
    3. Interview FAO office and staff.

     

    A
  •  

    b) Effectiveness of self-evaluation.

  •  

    1. Uses of evaluation.

    1.  
    2. Examine number of changes in policies and procedures as a result of evaluation.
    3. Interview staff re: evaluation results.

     

    E

     

    E

     

      STUDENT SERVICES PROGRAM REVIEW PROJECT
    CRITERIA, MEASURES, METHODS

    Job Placement

  • GOAL 1: To Develop or Identify Employment Opportunities Which are Appropriate to Student Needs and to the College’s Programs
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of job listings.

  •  

    1. Evidence of listings initiated by college staff.

  •  

    1.1 Weekly review of job listings to determine number.

  •  

    E
  •  

    2. How often listings are updated and currency of listings.

  •  

    2.1 Check job listings for recency of dates.

  •  

    E
  •  

    3. Accuracy of listings.

  •  

    3.1 Check with employers (random).

  •  

    E
  •  

    4. Types of job listings and sources of listings.

  •  

    4.1 Check job listings for types of jobs and sources for each type.

  •  

    A
  •  

    b) Extent of employer participation.

  •  

    1. Number of unduplicated employers represented on listings.

  •  

    1.1 Unduplicated number of employers on listings.

  •  

    E
  •  

    2. Number of employers brought on campus (job fairs, etc.).

  •  

    2.1 Count number of employers brought on campus.

  •  

    E
  •  

    3. Types of employers vs. community employment base.

  •  

    3.1 Count number of employers in each category vs. number in employment base.

  •  

    A
  •  

    4. Frequency and timing of employer participation.

  •  

    4.1 Count how often and note when employers participate.

  •  

    E
  •  

    c) Extent of employer initiated contacts.

  •  

    1. Number of contacts with Job Placement Office.

  •  

    1.1 Count number of contacts within a specified time interval.

  •  

    E
  •  

    2. Frequency and timing.

  •  

    2.1 Count how often and note when contacts are made.

  •  

    E
  •  

    3. Types of employers making contacts.

  •  

    3.1 Count number in each occupational category vs. number in employment base.

  •  

    E
  •  

    d) Staff contacts with employers.

  •  

    1. Number of staff contacts with employers.

  •  

    1.1 Review records to count number of contacts.

  •  

    E
  •  

    2. Types of contacts (e.g., phone, letter, involvement in professional organizations).

  •  

    2.1 Review of records to count types of contacts within a specified interval.

  •  

    E
  •  

    3. Frequency and timing of contacts.

  •  

    3.1 Count how often and note when contacts are made.

  •  

    E
  •  

    4. Types of employers (industry) contacted.

  •  

    4.1 Define employment base to be used (i.e., corporate guide).

  •  

    E

     

    Job Placement
  • GOAL 1: To Develop or Identify Employment Opportunities Which are Appropriate to Student Needs and to the College’s Programs

     

    (continued)

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    e) Degree of employer satisfaction.

  •  

    1. Level of satisfaction with program.

  •  

    1.1 Employer survey.

  •  

    E

     

  •  

    2. Level of satisfaction with contacts.

  •  

    2.1 Employer survey.

  •  

    E

     

  •  

    3. Continuing employer requests for student employees.

  •  

    3.1 Count number of repeated employer requests.

  •  

    E

     

     

     

    Job Placement
  • GOAL 2: To Develop and Disseminate Information About Employment Trends and Job Opportunities to Students and College Program Staff
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of information.

  •  

    1. Evidence of types of information (written and non-written, on-campus/off-campus coverage, etc.).

  •  

    1.1 List and provide examples of each.

  •  

    E
  •  

    b) Accessibility of information.

    1.  
    2. Evidence of distribution to students, staff, and community.
    3. Evidence of diverse distribution locations.

     

    1&2 For each item listed, list distribution locations, description of distribution method and indication of targeted group; indicate when each is available.

     

    E

     

    Ease of obtaining information.

     

    3.1 Consumer satisfaction survey (students and non-students).

     

    A

     

    Evidence of reaching targeted group population.

  •  

    4.1 Survey of targeted groups.

  •  

    E
  •  

    c) Readability, accur-acy, and complete-ness of information.

  •  

    1. Evidence of clear, concise, and complete information.

    1.  
    2. Measure reading grade level of all information provided and compare to population reading level.
    3. Review by independent observers (media experts).

     

    E

     

     

    A

  •  

    d) Recipient awareness of employment trends and job opportunities.

  •  

    1. Evidence of how the student learned about job opportunities.

  •  

    1.1 Ask the question on student intake form.

  •  

    E
  •  

    2. Evidence of how staff learned about employment trends and job opportunities.

  •  

    2.1 Staff survey.

  •  

    E
  •  

    e) Degree of recipient satisfaction.

  •  

    1. Level of satisfaction with materials.

  •  

    1.1 Survey recipients.

  •  

    E
  •  

    2. Continuing requests from recipients for services.

  •  

    2.1 Count number of service requests from recipient groups.

  •  

    E

     

     

     

    Job Placement
  • GOAL 3: To Disseminate Information About Student Employment Services to Students, Staff, and Community
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of information.

  •  

    1. Evidence of types of information (written and non-written)

  •  

    1.1 List and provide example of each.

  •  

    E
  •  

    b) Accessibility of information.

    1.  
    2. Evidence of distribution to students, staff, and community.
    3. Evidence of diverse distri-bution locations.

     

    1&2 For each example list distribu-tion locations, description of distribution method, and indication of targeted group; indicate when available.

     

    E

     

    Ease of obtaining information.

     

    3.1 Consumer satisfaction survey (students and non-students)

     

    A

     

    Evidence of reaching targeted group population.

  •  

    4.1 Survey of targeted groups.

  •  

    E
  •  

    c) Readability and accuracy of information.

  •  

    1. Evidence of clear, concise and complete information.

    1.  
    2. Measure reading grade level of all information provided and compare to population reading level.
    3. Review by independent observers (media experts).
    4. Compare number of service requests from targeted groups to representation of targeted groups in the community.

     

    E

     

     

    A

     

    A

  •  

    d) Recipients’ aware-ness of student employment services.

  •  

    1. Evidence of how the student learned about employment services.

  •  

    1.1 Ask the question on student intake form.

  •  

    E
  •  

    2. Evidence of how staff learned about employment services.

  •  

    2.1 Survey staff to determine extent of knowledge of job placement services.

  •  

    E
  •  

    3. Evidence of how community learned about employment services.

    1.  
    2. Count the number and types of business, industries, community agencies, churches, etc. receiving information.
    3. Compare the numbers and types receiving the information to the distribution of these groups in the community.

     

    E

     

     

    A

     

     

     

    Job Placement
  • GOAL 3: To Disseminate Information About Student Employment Services to Students, Staff, and Community

     

    (continued)

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    e) Degree of recipient satisfaction.

  •  

    1. Level of satisfaction with materials.

  •  

    1.1 Survey/interview recipients.

  •  

    E
  •  

    2. Level of satisfaction with distribution methods.

  •  

    2.1 Survey/interview recipients.

  •  

    E
  •  

    3. Continuing recipient requests for services.

  •  

    3.1 Count number of service requests from recipient groups.

  •  

    E

     

     

     

    Job Placement
  • GOAL 4: To Assist Students to Acquire Job Search and Job Retention Skills
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of activities.

  •  

    1. Evidence of types of activities.

  •  

    1.1 List and provide examples of each type.

  •  

    E
  •  

    b) Accessibility of activities.

  •  

    1. Location of activities.

  •  

    1.1 For each activity, indicate list of locations.

  •  

    E
  •  

    2. Frequency of activities.

  •  

    2.1 For each activity, indicate how often offered.

  •  

    E
  •  

    3. Times activities are offered.

  •  

    3.1 For each activity, indicate when offered (e.g., days/eve).

  •  

    E
  •  

    4. Number and demographics of participants.

    1.  
    2. Record number and characteristics of student participants.
    3. Compare participants’ characteristics to total student population characteristics.

     

    E
  •  

    c) Degree of student learning in work-shops/classes/indi-vidual sessions.

  •  

    1. Evidence of skills learned resulting from studies.

  •  

    1.1 Pre/post evaluation examination (e.g., written, oral, third party).

  •  

    E
  •  

    d) Degree of employer satisfaction with students’ general job skill preparation.

  •  

    1. Level of employer satisfaction.

  •  

    1.1 Survey/interview employers.

  •  

    E
  •  

    e) Degree of student satisfaction with preparation.

  •  

    1. Level of student satisfaction.

  •  

    1.1 Survey/interview students who were placed.

  •  

    E
  •  

    f) Student job retention and effectiveness at students’ job search skills.

  •  

    1. Number of students retained in jobs.

  •  

    1.1 Count number of students retained compared to number placed.

  •  

    E
  •  

    2. Characteristics of students retained as compared to students not retained.

  •  

    2.1 Survey/interview employers and students to identify reasons for leaving; i.e., technical/general job skills, factors unrelated to skills.

  •  

    A

     

     

     

    Job Placement
  • GOAL 5: To Assist Students to Acquire the Skills Needed for Professional Growth and

    Transition

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of activities.

  •  

    1. Evidence of types of activities.

  •  

    1.1 List and provide examples of each type of activity.

  •  

    E
  •  

    b) Accessibility of activities.

  •  

    1. Location of activities.

  •  

    1.1 For each activity, indicate list of locations.

  •  

    E
  •  

    2. Frequency of activities.

  •  

    2.1 For each activity, indicate how often offered.

  •  

    E
  •  

    3. Times activities are offered.

  •  

    3.1 For each activity, indicate when offered (e.g., day/eve).

  •  

    E
  •  

    4. Number and demographics of participants.

  •  

    4.1 Compare characteristics of participants to characteristics of targeted student populations.

  •  

    E
  •  

    c) Degree of student learning in workshops/classes.

  •  

    1. Evidence of skills learned.

  •  

    1.1 Pre/post evaluation/examination (e.g., written, oral, third party).

  •  

    E
  •  

    d) Change in student job status.

    1.  
    2. Number of students changing jobs:
    • receiving promotions
    • lateral transfers within field
    • changing job fields
    1.  
    2. Survey/interview students and employers to identify the number of students changing jobs.
    3. Compare the number of students changing jobs to the number of students served by the Program.

     

    E

     

     

    E

    1.  
    2. Characteristics of students changing jobs, compared to characteristics of students not changing jobs.
    1.  
    2. Survey/interview students and employers to identify characteristics of students served by the Program who changed jobs.

     

    E
  •  

    e) Degree of student and employer satis-faction with preparation.

  •  

    1, Level of student and employer satisfaction.

  •  

    1.1 Survey students and employers to determine level of satis-faction.

  •  

    E

     

     

     

    Job Placement
  • GOAL 6: To Identify Qualified Applicants and Refer Them to Prospective Employers
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of policies and procedures.

  •  

    1. Evidence of policies and procedures regarding applicant identification and referral to employers.

  •  

    1.1 Examine policy manuals, procedure manuals, and other documents.

  •  

    E
  •  

    b) Extent of student referrals to employers.

  •  

    1. Number of students sent to employers for job interviews (including on-campus, off-campus).

    1.  
    2. Count student referral:
    • total
    • by job category
    • by employer category

     

    E
  •  

    c) Match between applicant qualification and job-hire requirements.

  •  

    1. Evidence of skills training and experience for the jobs to which students are referred.

  •  

    1.1 Examine student qualifications and compare to job requirements.

  •  

    E
  •  

    2. Number of students placed.

    1.  
    2. Count student placements:
    • total
    • by job category
    • by employer category

     

    E
  •  

    3. Characteristics of students placed compared to characteristics of students not placed.

    1.  
    2. Use student applications to identify characteristics of students placed and not placed
    • total
    • by job category
    • by employer category

     

     

     

    E

    A

    A

  •  

    4. Employer evaluation of qualification of referred students.

    1.  
    2. Survey/interview employers to identify characteristics of students placed and not place
    • total
    • by job category
    • by employer category

     

     

     

    E

    E

    A

  •  

    5. Student evaluation of their qualification for the jobs to which they were referred.

    1.  
    2. Survey/interview students to learn their perceptions about their preparation—comparing students placed and not placed
    • total
    • by job category
    • by employer category

     

     

     

    E

    A

    A

  •  

    d) Availability of employment counseling services.

  •  

    1. Evidence of employment counseling services in the Program.

    1.  
    2. Validate by interview of observation
    • students
    • staff

     

    E

     

     

     

    Job Placement
  • GOAL 7: To Gather Information About Job Performance and Satisfaction from Students and

    Employers

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of information on job performance and satisfaction.

  •  

    1. Evidence of available information.

  •  

    1.1 Examine the Program records to determine information collected.

  •  

    E
  •  

    b) Degree of employer satisfaction.

  •  

    1. Level of employer satisfaction with student job performance (including potential for promotion).

  •  

    1.1 Employer survey.

  •  

    E
  •  

    2. Level of employer satisfaction with student job performance (including potential for promotion).

  •  

    2.1 Employer survey.

  •  

    E
  •  

    3. Continuing employer requests for students as employees.

  •  

    3.1 Count number of repeated requested from employers.

  •  

    A
  •  

    c) Degree of student satisfaction.

  •  

    1. Level of student satisfaction with preparation.

  •  

    1.1 Student survey.

  •  

    E
  •  

    2. Level of student satisfaction with job situation and their performance on the job.

  •  

    2.1 Student survey.

  •  

    E

     

     

     

    Job Placement
  • GOAL 8: To Report Student Employment Services Data to the College Community and Other Appropriate Agencies
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of policies and procedures.

  •  

    1. Evidence of policies and procedures regarding dissemination of data.

  •  

    1.1 Examine policy/procedures manuals and other documents.

  •  

    E
  •  

    b) Availability of needed data.

  •  

    1. Evidence of availability of needed data.

  •  

    1.1 Examine student employment services records and reports and other documents.

  •  

    E
  •  

    c) Extent of requests for data and responses to requests.

  •  

    1. Evidence of requests.

    1.  
    2. Examine requests for data to determine:
    • number
    • when received
    • when needed
    • types of data requested
    • who made request(s)
    • legality and appropriateness of request(s)

     

     

    E

    E

    E

    E

    E

    E

  •  

    2. Evidence of responses to requests for data.

    1.  
    2. Examine responses to requests for data to determine:
    • number of responses (compared to number of requests)
    • deadline met
    • appropriateness of responses to requests

     

     

    E

  •  

    d) Degree of recipient satisfaction with responses to data requests.

  •  

    1. Level of recipient satisfaction with responses to data requests.

    1.  
    2. Number accepted by agency and group compared to number sent.
    3. Survey receiving agencies/ groups.

     

     

    E

    A

     

     

     

    Job Placement
  • GOAL 9: To Work Effectively with the College Community
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Cooperation with other college offices.

  •  

    1. Evidence of cooperation with other college offices (e.g., Career handling, Instructional Dept./Division).

    1.  
    2. Document:
    • number of joint meetings.
    • number of positive responses to requests for assistance or information
    • number of requests made by Program for assistance/information

     

    E
  •  

    2. Evidence of results of cooperative efforts.

    1.  
    2. For each item identified above document:
    • who participated
    • when and how
    • results of efforts

     

    E
  •  

    b) Degree of satis-faction with cooperative efforts.

  •  

    1. Level of satisfaction within the Program.

    1.  
    2. Survey student employment services group.
    3. Continuing requests for cooperative efforts from student employment services staff.

     

    E

     

    E

  •  

    2. Level of satisfaction within groups with which coopera-tive efforts were made.

    1.  
    2. Survey groups.
    3. Continuing requests for cooperative efforts from outside groups.

     

    E

    E

     

     

     

    Job Placement
  • GOAL 10: To Conduct On-Going Evaluation of Student Employment Services
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of an evaluation plan.

  •  

    1. Evidence of appropriate planning.

    1.  
    2. Examine documents for evidence of planning:
    • progarm level
    • student services area level
    • college level
    • community level

     

    E
  •  

    b) Degree of on-going implementation of plan.

  •  

    1. Evidence of on-going imple-mentation of plan.

    1.  
    2. For each appropriate level identified, document::
    • evaluation dates and content
    • evaluation report dates and content
    • who participates
    • when and how they partici-pate
    • program and service notifications resulting from evaluations (include staffing and funding)

     

    E

     

    STUDENT SERVICES PROGRAM REVIEW PROJECT

    CRITERIA, MEASURES, METHODS

    Student Affairs

    GOAL 1: To Provide Information About Student Activities, Programs, and Services

    Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Availability of information.

  •  

    1. Number of informational items available.

  •  

    1.1 Count number of various informational items available by category.

  •  

    E
  •  

    2. Frequency location and manner of distribution.

    1.  
    2. Check publication schedule.
    3. Check location and manner of distribution.

     

    E

    E

  •  

    b) Effectiveness of information.

  •  

    1. Level of community awareness.

  •  

    1.1 Community survey.

     

    E
    •  
    • Accuracy.
    1. Timeliness.
    2. Appropriateness of publications to service population.
    3. Level of student awareness.
    1.  
    2. Student and staff surveys.
  • 1-5 Third party review (person or persons not involved in student affairs program).
  •  

    E

    A

  •  

    c) Type of information.

  •  

    1. Evidence of information re: student due process rights and responsibilities.

  •  

    1.1 Check Student Affairs or other publications.

  •  

    E

     

  •  

    2. Evidence of information re: student affirmation action and Title 1X.

  •  

    2.1 Check Student Affairs or other publications.

  •  

    E

     

     

     

    Student Affairs
  • GOAL 2: To Provide for Student Involvement in Student Government and Institutional Governance
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Opportunity for student participation in institutional governance.

  •  

    1. College commitment to student involvement in governance.

  •  

    1.1 Check college/district policy statement.

  •  

    E
  •  

    2. Variety of governance opportunities.

  •  

    2.1 Number of separate college committee with student members.

  •  

    E
  •  

    3. Number of student positions in governance.

  •  

    3.1 For each committee, count number of committee memberships allocated to students compared to total number of committee members.

  •  

    E
  •  

    b) Effectiveness of student involvement on governance.

  •  

    1. Number attending and participating in committee activities.

    1.  
    2. Count students attending.
    3. Check committee minutes for student participation.

     

    E

    A

  •  

    2. Evidence of orientation to governance.

    1.  
    2. Check record or survey students.
    3. Enrollment in orientation program.
    4. Check course outline and constitution for unit credit of student affairs course.

     

    E

     

     

    E

  •  

    c) Opportunities for students to participate in student governance.

  •  

    1. Number and profile of students running for office (e.g., day, evening).

  •  

    1.1 Check number and profile in election results.

  •  

    E
  •  

    2. Number and profile of students applying for appointive positions.

  •  

    2.1 Check number and profile of students applying.

  •  

    E
  •  

    3. Number and profile of students voting in elections.

  •  

    3.1 Count number and profile of students voting compared to total enrollment.

  •  

    E
  •  

    d) Institutional support for student government.

  •  

    1. Number & type of staffing.

  •  

    1.1 Check staff pattern.

  •  

    E
  •  

    2. Nature & amount of funding.

  •  

    2.1 Examine SA budget (fees, expenses).

  •  

    E
  •  

    3. Adequacy of facilities.

  •  

    3.1 Check facilities master plan and/or interviews.

  •  

    E

     

     

     

    Student Affairs
  • GOAL 2: To Provide for Student Involvement in Student Government and Institutional Governance

     

    (continued)

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    d) Institutional support for student government. (continued)

  •  

    4. Staff involvement in encouraging student participation.

  •  

    4.1 Interview.

  •  

    E
  •  

    e) Student satisfaction with opportunities for participation in governance.

  •  

    1. Evidence of student satisfaction with range and quality of opportunities.

  •  

    1.1 Student survey.

  •  

    E

     

     

     

    Student Affairs
  • GOAL 3: To Provide Opportunities for Student Involvement in Campus and Community

    Activities Which Foster Cultural and Citizenship Enrichment and Volunteer

    Service

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Existence and avail-ability of campus and community activities (e.g., student clubs).

  •  

    1. Evidence of a wide range of activities designed to reach the maximum number of students.

  •  

    1.1 Check records or activities calendar.

  •  

    E
  •  

    2. Evidence of process for developing a new activity.

  •  

    2.1 Check policy manual and department goals and objectives.

  •  

    E
  •  

    3. Evidence of location suitable for activities.

  •  

    3.1 Check facilities provided and facilities plan.

  •  

    E
  •  

    b) Student participation in these activities.

  •  

    1. Number of students participating.

  •  

    1.1 Count number in each activity.

  •  

    E
  •  

    2. Demographic information on participants.

  •  

    2.1 Count students by appropriate categories and compare to demographics of total enrollment.

  •  

    E
  •  

    c) Student satisfaction with activities.

  •  

    1. Degree of student satis-faction with existing activities.

  •  

    1.1 Student survey.

  •  

    E
  •  

    2. Student satisfaction with range and quality of activities.

  •  

    2.1 Student survey.

  •  

    E
  •  

    d) Staff and community satisfaction with activities.

  •  

    1. Degree of satisfaction with activities.

  •  

    1.1 College staff and community surveys.

  •  

    E
  •  

    e) Existence of activities recognizing student contribu-tions: academic, service, leadership.

  •  

    1. Evidence of activities.

  •  

    1.1 Review records or activity calendar.

  •  

    E

     

     

     

    Student Affairs
  • GOAL 4: To Provide Opportunities for Students to Participate in Intercollegiate and Intramural Athletic Competition
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Existence of inter-collegiate athletic programs.

  •  

    1. Evidence of appropriate level of intercollegiate program.

  •  

    1.1 Check schedule and records.

  •  

    E
  •  

    2. Compliance with state and federal regulations.

  •  

    2.1 Check compliance with regulations.

  •  

    E
  •  

    b) Campus and com-munity involvement in planning and eval-uation of inter-collegiate programs.

  •  

    1. Evidence of campus and community involvement in planning and evaluation.

    1.  
    2. Records and minutes of meetings.
    3. Interview students and other involved.
    4. Examine campus policy.

     

    E

     

    E

    E

  •  

    c) Institutional support for intercollegiate program.

  •  

    1. Number and type of staffing.

  •  

    1.1 Check staffing patterns.

     

    E
  •  

    2. Nature and amount of funding.

  •  

    2.1 Examine budget.

     

    E
  •  

    3. Adequacy of facilities.

  •  

    3.1 Check facilities master plan and/or interviews with students/ staff.

  •  

    E
  •  

    d) Existence of intramural athletic programs.

  •  

    1. Evidence of appropriate level intramural program.

  •  

    1.1 Schedules and records.

  •  

    E
  •  

    2. Compliance with state and federal regulations.

  •  

    2.1 Check compliance with regulations.

  •  

    E
  •  

    e) Campus community involvement in planning and evaluation of intramural programs.

  •  

    1. Evidence of campus community involvement in planning and evaluation.

    1.  
    2. Records and minutes.
    3. Interviews.
    4. Examine campus policy.

     

    A

    E

    E

  •  

    f) Institutional support for intramural program.

  •  

    1. Number and type of staffing.

    1.  
    2. Check staffing patterns.

     

    E
  •  

    2. Nature and amount of funding.

  •  

    2.1 Examine budget to determine nature and amount of funding.

  •  

    E
  •  

    3. Adequacy of facilities and/or interview.

  •  

    3.1 Check facilities master plan.

  •  

    E

     

     

     

    Student Affairs
  • GOAL 4: To Provide Opportunities for Students to Participate in Intercollegiate and Intramural Athletic Competition

     

    (continued)

  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    g) Existence of pro-grams and services that enhance and support the educa-tional process for student athletes.

  •  

    1. Evidence of such programs and services.

    1.  
    2. Interview.
    3. Check student handbook and other printer materials.
    4. Identify services.
    5. Check academic progress of student athletes.

     

    E

    E

    E

    E

  •  

    h) Student satisfaction with intercollegiate program.

  •  

    1. Degree of student satisfaction with program.

  •  

    1.1 Survey students.

     

    E

     

  •  

    2. Student satisfaction with educational support system.

  •  

    2.1 Student survey.

     

    E
  •  

    i) Student/community attendance at athlete events.

  •  

    1. Total attendance.

  •  

    1.1 Count those attending.

     

    E

     

     

     

    Student Affairs
  • GOAL 5: To Educate the Campus Community About the Value of Student Affairs Programs and Services
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Awareness, under-standing, and acceptance of the value of student activity programs and services by the campus community.

  •  

    1. Degree of awareness, understanding, and acceptance by college students and staff.

  •  

    1.1 Survey campus personnel.

  •  

    E
  •  

    2. Extent of staff participation.

  •  

    2.1 Number of staff involved compared to total number.

  •  

    E
  •  

    3. Extent of student participation.

  •  

    3.1 Number of students involved compared to total number.

  •  

    E
  •  

    4. Degree of institutional support.

  •  

    4.1 Check facilities, funding history, and staffing pattern.

  •  

    E
  •  

    b) Campus-community involvement in student activities planning and evalu-ation.

  •  

    1. Evidence of an involvement process.

    1.  
    2. Interview appropriate personnel.
    3. Examine appropriate records, policies, and/or minutes.

     

    E

    E

  •  

    2. Number of participants involved.

  •  

    2.1 Count number involved.

     

    E
  •  

    3. Demographic information on participants.

  •  

    3.1 Count by appropriate categories and compare to demographic of total enrollment and community.

  •  

    E
  •  

    c) Interaction and coor-dination between curricular and co-curricular programs.

  •  

    1. Evidence of scheduled interactions.

  •  

    1.1 Count number.

  •  

    E
  •  

    2. Development of jointly sponsored programs.

    1.  
    2. Check appropriate records, count number of programs.
    3. Survey appropriate personnel.

     

    E

    E

  •  

    d) Existence of instruction available to students involved in student activity programs.

  •  

    1. Types of courses in student leadership and development.

  •  

    1.1 Count number of classes in schedules and catalog.

  •  

    E
  •  

    e) Long-range benefits to individuals parti-cipating in student affairs.

  •  

    1. Evidence of long-range benefits.

    1.  
    2. Longitudinal studies.
  • 1.2 Post-hoc studies of former students.
  •  

    A

    E

     

     

     

     

    STUDENT SERVICES PROGRAM REVIEW PROJECT

    CRITERIA, MEASURES, METHODS

    TUTORIAL SERVICES

     

     

     

    Tutorial Services
  • GOAL 1: To Promote Individual Student Success and Retention
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Do you develop written definitions of success for each student?

  •  

    1. Evidence of written definitions.

  •  

    1.1 Examine written definitions.

  •  

    E
  •  

    b) Is there evidence that tutored students are successful?

  •  

    1. Course grades.

  •  

    1.1 Examine student transcripts for improvement.

  •  

    E
  •  

    2. Pre-post changes in relation to success definition.

  •  

    2.1 Examine student performance.

  •  

    E
  •  

    3. Degree of student satis-faction with their progress.

    1.  
    2. Exit interview.
    3. Formal/informal survey.

     

    E

    E

  •  

    4. Difference in course grades between tutored and non-tutored students (who need tutoring).

  •  

    4.1 Establish equivalent groups. Give one group tutoring and compare course grades of groups.

  •  

    A
  •  

    c) Do tutored students complete courses, programs, and objectives?

  •  

    1. Course completion rates.

  •  

    1.1 Examine transcripts.

  •  

    E
  •  

    2. Educational objectives completion rates.

  •  

    2.1 Follow-up interview/ questionnaire.

  •  

    E
  •  

    3. Continuation rates in program as per objective.

  •  

    3.1 Evidence of enrollment in program per transcripts.

  •  

    E

     

     

     

     

    Tutorial Services
  • GOAL 2: To Assure that Students and Community Receive Appropriate Information About Tutorial Services
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Are students, staff and appropriate community people aware of tutorial services?

  •  

    1. Evidence and source of awareness.

  •  

    1.1 Survey students, staff, and community (e.g., high school students).

  •  

    E
  •  

    b) Is information about tutorial services available and widely distributed?

  •  

    1. Amount of information available.

  •  

    1.1 Count number.

  •  

    E
  •  

    2. Variety of types of information available.

  •  

    2.1 Count number of types.

  •  

    E
  •  

    3. Frequency of distribution.

  •  

    3.1 Tally frequency of distribution.

  •  

    E
  •  

    c) Is information appro-priate for intended audience (e.g., read-ability, language used)?

  •  

    1. Opinion of staff and audience.

  •  

    1.1 Interview/survey appropriate audience and staff.

  •  

    E

     

     

     

     

    Tutorial Services
  • GOAL 3: To Help Identify, Refer, and Determine Students’ Tutorial Needs
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Are effective procedures in place whereby students can be identified and referred to tutorial services?

  •  

    1. Scope of referral network (e.g., referred by counselors, self, instructors, assessment center, peers, other support services, by academic standing, etc.).

  •  

    1.1 Examine procedure and count students by type of referral.

  •  

    E
  •  

    b) Are students’ tutorial needs accurately identified?

  •  

    1. Verification of students’ needs with staff statements, intake interview/referral forms and assessment data.

    1.  
    2. Interview staff and students.
    3. Compare student, staff, and assessment information.

     

    E

    E

     

     

     

     

    Tutorial Services
  • GOAL 4: To Provide Effective Training for Tutors
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Are training activities provided for tutors?

  •  

    1. Evidence of types of training activities.

  •  

    1.1 List activities.

  •  

    E
  •  

    b) Are all tutors provided training?

  •  

    1. Evidence of completion of training activities.

  •  

    1.1 Count number of tutors and number of tutors trained.

  •  

    E
  •  

    c) Is tutor training effective?

  •  

    1. Degree of tutor satisfaction with training.

  •  

    1.1 Survey trained tutors.

  •  

    E
  •  

    2. Degree of staff satisfaction with training.

  •  

    2.1 Survey staff.

  •  

    E

     

     

     

    Tutorial Services
  • GOAL 5: To Provide Tutorial Assistance to Students in Specific Areas.
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Is content tutoring provided in a diverse scope of content area?

  •  

    1. Evidence of broad scope of content available.

  •  

    1.1 Compare listings of tutoring available to class schedule, student requests, and faculty referrals.

  •  

    E
  •  

    b) Is an adequate number of content tutors available?

  •  

    1. Evidence of adequate number of tutors per subject.

  •  

    1.1 List tutors by subject and compare to need.

  •  

    E
  •  

    c) Is an adequate budget for content tutoring available?

  •  

    1. Evidence of adequate budget for tutor salaries.

  •  

    1.1 Compare budget to student request and faculty referral needs.

  •  

    E
  •  

    d) Do students avail themselves of content tutoring.

  •  

    1. Number of students served.

  •  

    1.1 Count number of students.

  •  

    E
  •  

    e) Is content tutoring effective?

  •  

    1. Student satisfaction with content tutoring.

  •  

    1.1 Survey student opinion.

  •  

    E
  •  

    2. Improvement in classroom performance.

    1.  
    2. Compare students’ course performance before and after receiving tutoring.
    3. Survey faculty regarding improvement.

     

    E

     

    E

     

     

     

    Tutorial Services
  • GOAL 7: To Provide Specialized Tutorial Assistance to Targeted Groups
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Are specialized tutorial services provided for targeted groups?

  •  

    1. Evidence of specialized tutorial services.

  •  

    1.1 List services by type/target group.

  •  

    E
  •  

    2. Number of students tutored from targeted groups.

  •  

    2.1 List and count number of students from targeted groups.

  •  

    E
  •  

    b) Is funding available for specialized tutorial services for targeted groups?

  •  

    1. Evidence of budget accounts and audit reports.

  •  

    1.1 Examine budget and reports.

  •  

    E
  •  

    c) Is specialized tutorial assistance effective?

  •  

    1. Difference in academic performance between members of targeted groups not receiving special assistance.

    1.  
    2. Compare grade performance of groups.
    3. Compare "time of task" of groups.
    4. Compare student interest levels of groups.

     

    A

    A

    A

     

     

     

    Tutorial Services
  • GOAL 8: To Coordinate Tutorial Services, Including Referrals, with other Student Services and Instructional Programs
  • Criteria

     

    Measures

     

    Methods

     

    E/A*
  •  

    a) Is an effective coordinating network in place which includes tutorial services, student services, and instruc-tional problems?

  •  

    1. Evidence of an effective coordination plan.

    1.  
    2. Review plan.
    3. Check plan for inter-relationship of all services.

     

    E

    E

  •  

    2. Satisfaction with existing coordinating network.

  •  

    2.1 Survey concerned parties from support services and instruction.

  •  

    E

     

     

     

     

     

     

     

    STUDENT SERVICES PROGRAM REVIEW PROJECT

    PILOT COLLEGES AND AREAS TESTED

     

    PARTICIPATING COLLEGES WHO COMPLETED PILOT TESTING OF SELECTED EVALUATION MODELS

     

     

     

     

    COLLEGE

     

    ADMISSIONS & RECORDS

     

    ASSESSMENT SERVICES

     

    CAREER CENTER SERVICES

     

    COUNSELING

     

    FINANCIAL AID

     

    JOB PLACEMENT SERVICES

     

    STUDENT AFFAIRS

     

    TUTORIAL SERVICES

     

    American River

     

     

     

     

    x

     

    x

     

    x

     

    x

     

     

    Butte

     

    x

     

     

     

    x

     

    x

     

    x

     

    x

     

     

    Cabrillo

     

     

     

    x

     

     

     

     

     

    x

     

    Cañada

     

    x

     

     

     

    x

     

     

     

     

     

    Chabot

     

     

     

    x

     

    x

     

     

     

     

     

    College of San Mateo

     

    x

     

    x

     

    x

     

    x

     

     

     

     

     

    College of the Siskiyous

     

     

    x

     

    x

     

     

     

     

     

    x

     

    Cosumnes River

     

     

    x

     

    x

     

     

     

     

     

     

    Foothill

     

    x

     

    x

     

     

    x

     

    x

     

     

     

     

    Grossmont

     

     

     

     

     

     

     

    x

     

     

    Merced

     

    x

     

     

     

     

     

     

     

     

    Mira Costa

     

     

     

     

    x

     

     

     

     

     

    Modesto

     

     

     

    x

     

    x

     

    x

     

    x

     

     

     

    Monterey Peninsula

     

    x

     

     

     

    x

     

     

     

     

     

    Moorpark

     

     

    x

     

    x

     

     

     

     

     

    x

     

    Mt. San Antonio

     

     

    x

     

    x

     

     

     

     

     

     

    Napa Valley

     

    x

     

     

     

    x

     

     

     

    x

     

     

    Ohlone

     

     

    x

     

    x

     

     

     

     

     

    x

     

    Palomar

     

     

     

     

     

     

    x

     

     

    x

     

    Rancho Santiago

     

    x

     

    x

     

     

    x

     

     

     

     

     

     

    PARTICIPATING COLLEGES WHO COMPLETED PILOT TESTING OF SELECTED EVALUATION MODELS

     

    (continued)

     

     

     

    COLLEGE

     

    ADMISSIONS & RECORDS

     

    ASSESSMENT SERVICES

     

    CAREER CENTER SERVICES

     

    COUNSELING

     

    FINANCIAL AID

     

    JOB PLACEMENT SERVICES

     

    STUDENT AFFAIRS

     

    TUTORIAL SERVICES

     

    Rio Hondo

     

     

    x

     

    x

     

    x

     

     

     

     

     

    Sacramento City

     

    x

     

     

     

    x

     

     

     

     

     

    Saddleback

     

     

     

    x

     

     

     

     

     

     

    San Bernadino Valley

     

     

     

     

     

     

     

    x

     

     

    San Diego City

     

     

    x

     

     

     

     

     

     

     

    Santa Barbara City

     

    x

     

     

    x

     

     

    x

     

    x

     

    x

     

     

    Santa Rosa

     

    x

     

     

     

    x

     

     

    x

     

     

     

    Solano

     

     

    x

     

    x

     

     

     

     

     

    x

     

    Taft

     

    x

     

     

     

     

     

     

     

     

    TOTAL

     

    12

     

    11

     

    13

     

    14

     

    5

     

    6

     

    6

     

    6

     

     

     

     

     

    COLLEGE REPORT ON

    SSPRP EXPERIENCES

     

    THE COLLEGES REPORT ON THEIR SSPRP EXPERIENCES

     

    At the close of each SSPRP phase, "process surveys" were distributed to the colleges which implemented the pilot models. The purpose of the survey was to identify the environmental and process characteristics that were most conducive to successful implementation in order to assist colleges using the models in the future. We found the following:

     

     

    WHY DID THE COLLEGES JOIN SSPRP?

    In all cases, the decision to participate was made and/or supported by either the college president or the dean or vice-president of student services.

     

    The colleges reported that the need to evaluate student services was a pressing one and models available are limited. The SSPRP provided the instruments and the opportunity to evaluate through use of a common model, providing a common database among colleges.

     

     

    HOW DID THE COLLEGES BEGIN?

    Most of the colleges formed broad-based SSPRP steering committees with strong administrative participation, and they appointed a SSPRP Coordinator who was usually a student services administrator. Most attended SSPRP Orientation Workshops.

     

     

    DID THE COLLEGES HAVE RESEARCH SUPPORT?

    Colleges did not all have institutional research offices but most involved an individual on staff with interest and expertise who assisted with the technical portions of the research (questionnaires, statistics). Computer assistance was helpful.

     

     

    HOW DID SSPRP HELP THE COLLEGES?

    More than half of the implementing colleges used the models for the student services portion of their self-studies for accreditation, as recommended by WASC. Colleges reported that the evaluation process provoked healthy discussion among staff and often resulted in needed changes.

     

     

    DID YOU HAVE ANY PROBLEMS?

    Some colleges found the complete implementation of a model too time consuming and selected only the goals to evaluate which would provide the most valuable information to the college.

     

     

     

     

     


     

    PROGRAM EVALUATION:

    PRINCIPLES, PURPOSES, AND PROCEDURES

     

    The Student Services program Review Project (SSPRP) was created by colleges who made the assumption that program review or evaluation is a healthy and necessary practice for improving the services they provide to students. To assist those embarking upon the program review process, the following summarizes the main tenets of evaluation agreed on by experts in the field. In addition, possible purposes of evaluation are considered, and procedural steps are outlined.

    Principles

    It is generally agreed upon among all evaluators, whatever methodology or model they may espouse, that there are certain criteria which should be the mainstay of any evaluation.

    The evaluation should result from information which is useful to the program participants, administrators, and other intended users.

    The program evaluation should be developed so that it can be done within the resources of the college or district – that is, it should be possible for the college to accomplish the evaluation with current staff or be able to provide adequate outside resources to reach the goal.

    The evaluation being conducted should be appropriate to the institution and to its purposes; the college should be asking questions to obtain the appropriate information.

    The evaluation should be conducted with concern for attention to validity and reliability of the data being collected. Clearly, evaluation reports based on inaccurate data would be of no use to the college or to program practitioners.

    There are other basic principles which should be considered before starting an evaluation.

    Program evaluations and research are not synonymous; collecting data is not the "end" of program evaluation. Judgements made from collected data, as well as other considerations make up the sum of a program evaluation project.

    Desired changes resulting from program activities should be defined. These become the performance standards against which the success of program services may be measured.

    Before decisions are made about the purposes of the evaluation or the procedures, another essential step is the development of a working relationship among the participants in the process. Evaluation is a group process like any other and must be based on a solid cooperative foundation between the evaluator(s) and the other program participants.

    Before initiating a program evaluation, the college should consider the "audience" and possible uses of the evaluation results. Being clear about the intended uses helps determine which alternatives for information collection are implemented.

     

    Purposes

    Why bother to evaluate something? Since there are many possible answers to this question, it is essential that a college consider specifically why an evaluation is being done.

    In most cases, people do evaluations in order to improve programs. The stated purpose of the Student Services Program Review Project was to develop designs for evaluation which can be used by colleges for their own purposes, with the assumption that the results would be used to validate or improve services to students.

    More specifically, some of the questions which can be answered by conducting program evaluation, are:

    1. What is a program intended to accomplish and what are the results of the program practices?  
    2. Are the program’s objectives realistic, appropriate to the institution, and important to the participants?

    3. Are the resources available to the program adequate to carry out the program objectives?

    4. Are resources being used effectively? (accountability)

    5. How can the program be changed or improved? (formative evaluation)

    6. Should the program be continued? (summative evaluation)

    Findings from an evaluation may also reveal, in addition to answers to the questions asked, other relevant (but unexpected) information. It may reveal, for example, that further research is necessary to determine whether the apparent differences between goals reached and goals intended are a result of program practices or other influences.

    Because of the many possible purposes and outcomes of program evaluation, the consideration of purposes must be a deliberate, carefully planned step in the program evaluation process. Once, these purposes are clear to all concerned, the process of actually "doing" and evaluation begins.

    Procedures
    The Student Services program Review Project has involved colleges in the development of objectives for eight areas of student services. For each of these objectives, teams of writers have identified criteria, measures, and methods which are used to determine whether the objectives are being met. A college may evaluate as many of the programs as they wish. Colleges may select objectives to evaluate from within each program as appropriate to their institutions.

    The question, next, is HOW TO GET STARTED? Eight steps which normally are included in an evaluation plan are described in Program Evaluation Skills for Busy Administrators developed by the Northwest Regional Educational Laboratory. The steps are summarized as follows.

     
    Evaluation Plan Outline
    Step Question
    1. Objectives/Issues What questions are being asked?
    2. Information Requirements What information is needed to answer the questions?
    3. Information Source Form whom can the necessary information be secured?
    4. Instruments What can be used to find the answer?
    5. Design Who will complete the instruments and what comparisons may be made?
    6. Time Schedule When will the information be collected, analyzed and reported?
    7. Analysis What do we do with the data?
    8. Report Who needs to know about it?
     

    Once an evaluation has been formulated (steps 1-5), a timeline for implementation should be developed including the target completion dates for all steps in the evaluation process. In addition, specific action plans should be outlined and persons responsible for implementation should be identified.

    The final step in an evaluation will be the analysis, reporting, and dissemination of data. This is the final and most important step.

     
    GETTING ORGANIZED

    This section provides information designed to assist you in organizing the evaluation activities that will take place as part of your involvement in the Student Services Program Review Project. The ideas that are presented have proven themselves in prior evaluation projects and represent a kind of mini-catalog of possibilities.

    Although recommendations are made concerning the specifics of the ideas presented, these should be viewed as suggestions. You should feel free to add-to, delete-from, or otherwise modify these suggestions as appropriate to your own college.

     

    1. Evaluation Steering Committee

    It is always a good idea to establish a steering committee. Although roles tend to vary from project, in general, a steering committee can provide the following: Membership on a steering committee should be determined by the nature of the project-related activities and general college/district policies. As an underlying principle, it is appropriate to have each major constituency which will be affected by the outcome of projects activities represented on the Steering Committee. This includes administrators, faculty, trustees, students, classified staff, and community representatives.

    A second principle recognizes outcomes which are expected to have implications for budget allocations. Persons who are responsible for making budget allocation decisions should be included in the membership of the Steering Committee. This serves the dual purpose of keeping the project within reasonable college resources limits, and keeping college budget decision-makers directly aware of project developments.

    Typically, steering committees have the following project-related responsibilities:

    a. Develop consensus on goals and general objectives for the project
    b. Develop and implement the process by which evaluation criteria, methods, and measures will be selected and implemented in each aspect of the project
    c. Establish the calendar for accomplishing project-related activities
    d. Establish and implement mechanisms for keeping all members of the college community informed of project progress. Such mechanisms might include a project newsletter, presentations at faculty and other college meetings, distribution of project progress reports, and the like.
    e. Establish and implement communications appropriate to managing the project (e.g., reminder notices about approaching deadlines, information about who is available to provide what kind of assistance to project teams.)
    f.

    Serve as resource persons for project teams as they implement those aspects of the project for which they are responsible.

       
       
       
       
    SSPRP Colleges have used steering committees to oversee one or more program area evaluations and to coordinate project teams.

     

    2. Project Teams

    The establishment of project teams provides several advantages in implementing the project:

    Within the guidelines established by the project Steering Committee, each project team would have the following responsibilities.
    a. Develop the specific objectives which will attain the project goals in the team’s area of responsibility.
    b. Select and implement the methods and measures which provide the information necessary to respond to the evaluation criteria relevant to the team’s area.
    c. Establish a team calendar for accomplishing project-related task.
    d. Develop progress and final reports.
       
       
       
    SSPRP Colleges have used project teams to implement each program area evaluation or evaluation of objectives within the program area.
     
    EVALUATION METHODS
    The common activities of most student services programs make it possible to articulate evaluation methods that can be generally applied. However, the diverse nature of the same programs will require local adaptation to these methods. For this reason, a "boiler-plate" evaluation model has been provided for the following evaluation methods, which are frequently recommended in the models:

    A. Record Examination

    B. Listing Data

    C. Third Party Review/Report

    D. Simulation

    E. Surveys

    Each evaluation method will be presented with essential steps clearly identified. Each step will be further described with the key points brought to the attention of the users. Because each method is presented in general sense, college personnel will need to exercise discretion to adapt the method to fit their selection. 

     

    A. RECORD EXAMINATION
    This evaluation method involves the examination of records to ascertain needed information about the nature of student services programs. To adequately conduct a records check, the following steps should be followed:
    Step 1      
    Determine purpose of records check Who will use it? For what purpose? When needed?    
           
    Step 2      
    Determine what information will be gathered? In what form should data be collected?    
           
    Step 3      
    Estimate time, revenue and staff needed. How long will it take? Estimate expenses Who will do it? What are responsi-bilities?
           
    Step 4      
    Locate records/ collect data. Identify keepers of records. Establish records keeping if none exists.  
           
    Step 5      
    Document/report results.      
           

    Step 1: Determine purpose of records check.

    It is important for the evaluator to know for what purpose the information will be used. In general, data of this type is used to:

    a. Determine the efficacy of a program

    b. Acquire more knowledge about a program

    c. Make decisions about the future status of a program

    If the results of the records check are to be used for decision-making, then it is imperative for the evaluator to know what decisions will be made. Without this knowledge, the process may yield information that is unnecessary for decision-making purposes. Questions to be answered at this stage include the following:

  • What decisions will be made as a result of this examination?
  • Who will use the data gathered?
  • Are there federal, state, or local mandates that govern the acquisition of these data?
  •  

    Step 2: Determine what information will be gathered.

    To make sure the final report contains the desired information in the correct format, the evaluator must determine with the users of the report what information will be needed. The pertinent questions for this purpose are as follows:
  • Will data on al students served by the program be needed, or is a sample sufficient?
  • Are there any constraints on how data can be gathered?
  • Can data be collected at any time? If not, what are the crucial dates?
  •  

    Step 3: Estimate the time, revenue, and staff needed.

    The evaluator will need to determine the timeline for the completion of the records check. Establishing a timeline will assist in determining what staff can be assigned and what resources will be available. To proceed with this estimate, the evaluator will need to respond to the following:

    • How long will this evaluation process take to complete? What are the crucial deadlines of the project? Beginning? Staff assignments? Due date?
    • Will the evaluation project have a budget? Will any of the following items need to be included?
    Staff salaries/benefits
    Consultants
    Travel
    Supplies and printing
    Telephone
    Equipment

     

    Step 4: Locate records/collect data.

    The actual finding and recording of information may be a simple one day or hour task, or it may require numerous days to ascertain the necessary information. However, the process cannot proceed unless records are identified and made available.

    Records can be kept in various locations; however, the evaluator must begin the search in the most logical places. To this end, identify the supervisor of the student services program under examination, and begin the search seeking answers to the following questions: 

    • Who and where are the record keepers, and what kind of information do they maintain?
    • Are records maintained by two or more people? Who will inform them of your needs and coordinate collection of materials, data, etc.?
    • Who is responsible for record updating?
    • Who maintains the historical records files? Where?
    • If records do not exist, who should maintain them? How should that process be initiated?
    • Can data be collected electronically? Who will write programs to assist such a process?

     

    Step 5: Document/report results.

    The evaluator should maintain files of each record examination project. These files can be used as archives and as backup if ever needed. It is strongly suggested that archives such as these be kept electronically and on hard copy. Thus, the use of a computer is highly recommended.

    To determine how these files will be maintained, the evaluator should answer the following questions:

    •  Will this information be used for other reports? By other evaluators? If so, can it be easily retrieved?
    • Is the information confidential in nature? What type of security precautions will be needed?
    • Will the information or final reports be distributed to sources outside of the college or be used for audits? If so, what procedures will be established to prepare reports for dissemination or audit reviews?

     

    The report writing is the final stage of this evaluation process. The evaluator should strive to create an evaluation report which summarizes the information that has been gathered in a clear concise manner.

     In general, each evaluation report should have:

       
    a. A title and author identification
    b. An introduction which highlights evaluation purpose and key questions
    c.

    A description of the methodology

    d. A description of costs, timeline, staffing
    e. A summary of general findings
    f. A conclusion – an analytical statement based upon the evaluation findings and a critique of the evaluation process
    g. Recommendations – if required

     

    B. LISTING DATA
    This evaluation method entails the development of data lists to be used as evidence of existing services, materials, or other elements vital to the success of a program. This process differs from records examination in that these lists must be created.

    Step 1      
    Determine purpose of data list.      
    Step 2      
    Determine what information will be needed.      
    Step 3      
    Estimate time, revenue and staff needed. How long will it take? Estimate expenses. Who will do it? What are responsibilities
    Step 4      
    Determine procedure to develop list.      
    Step 5      
    Develop list.      
    Step 6      
    Document/report results      

     

    Step 1: Determine purpose of data list.

    As with the records check method, it is important for the evaluator to know the purpose for developing a data list. The projected use of the list might affect the process to be used. For a general description of this step, refer to #1 under records check.

     

    Step 2: Determine what information will be needed?

    The evaluator should identify the source and nature of the information to be used in developing a list. These may include:

    a. Student bulletins
    b.Course schedules

    c.Student handbooks

    d.School calendars

    e.Interviews

    f. Orientation schedules
    g. Recruitment materials
    h.Staff memorandums


    For example, to confirm the availability of activities to assist students to acquire skills needed for professional growth, the evaluator might want to create a list of seminars provided by student employment services staff. If accessibility of information is to be confirmed, the evaluator may want to develop a list of locations to which information was distributed.

    To proceed through this step, the following questions need to be answered:

    • What administrator or administrative unit is responsible for maintaining the needed materials? In what form are the materials? NOTE: Once determined, the evaluator should meet with the administrator in charge to see where and how the information is maintained. If the information is to be gleaned from individuals, the evaluator will need to establish interview schedules.

       

    • What types of data will be placed on the list? Define the data elements clearly (i.e., locations, orientation times, etc.)

       

    • In what manner will the list need to be constructed? Electronically? Hard copy?

       

    • Will certain staff have knowledge of schedule activities? If so, who are they?
     

    Step 3: Estimate time, revenue, and staff needed.

    The evaluator will need to determine the timeline for the completion of the records check. Establishing a timeline will assist in determining what staff can be assigned and what resources will be available. To proceed with this estimate, the evaluator will need to respond to the following:

    Staff salaries/benefits

    Consultants

    Travel

    Supplies and printing

    Telephone

    Equipment

    • Who will be the key staff assigned to this process? Who will be designed as backup staff? What are staff responsibilities? Who will be in charge?

     

    Step 4: Determine procedure to develop list.

    The evaluator will need to determine if the list can be constructed from printed materials, personal interviews, on-line computer screens, or whatever. If the desired information is not available, then some record keeping process may need to be developed.

    In any event, the evaluator should answer the following questions to proceed through this step:

    • Will the materials be available at certain times? If so, when and where?
    • At what time and locations will materials be available?
    • Who will construct the list? What are their respective responsibilities?

    Step 5: Develop list.

    This step involves the act of constructing the list and should occur without any problems if the previous steps have been addressed adequately. As the process unfolds, the evaluator will need to be cognizant of:

    • Will deadline be met? If not, inform the appropriate people.
    • Are any modifications needed? If so, what are they? Why? Are additional resources needed?

    Step 6: Document/report results.

    The evaluator should maintain files of each records examination project. These files can be used as archives such as these being kept electronically and on hard copy. Thus, the use of a computer is highly recommended.

    To determine how these files will be maintained the evaluator should answer the following questions:

    • Will this information be used for other reports? By other evaluators? If so, can it be easily retrieved?
    • Is the information confidential in nature? What type of security precautions will be needed?
    • Will the information or final reports be distributed to sources outside of the college or be used for audits? If so, what procedures will be established to prepare reports for dissemination or audit reviews?

    The report writing is the final stage of this evaluation process. The evaluator should strive to create an evaluation report which summarizes the information that has been gathered in a clear concise manner.

     

    In general, each evaluation report should have:

    a. A title and author identification

    b. An introduction which highlights evaluation purpose and key questions

    c. A description of the methodology

    d. A description of costs, timeline, staffing

    e. A summary of general findings

    f. A conclusion – an analytical statement based upon the evaluation findings and a critique of the evaluation process

    g. Recommendations – if required

     

    C. THIRD PARTY REVIEW/REPORT

    This procedure requires the use of non-program personnel as evaluators of a program. This method is desirable when an objective assessment is important. This method also decreases the potential of program employees’ bias influencing the outcome of the evaluation.

    The role of the third part evaluator must be clearly defined and articulated. To define the evaluator’s role, program personnel may want to consult with the evaluator. A conjointly approved role definition will eliminate confusion, and will make it easier for the evaluator’s performance to be measured. Also, this procedure will reduce the potential of role conflicts. Remember the evaluator will be sharing turf with program employees and participants. If the third party evaluator activities are met with resistance, the effectiveness of this method is severely affected.

    Step 1    
    Determine purpose of third part review.    
    Step 2    
    Select third party evaluator.    
    Step 3    
    Define evaluator role. Clearly state role Sign contract if appropriate.
    Step 4    
    Estimate time, revenue and staff needed.    
    Step 5    
    Conduct evaluation.    
    Step 6    
    Document/report results    

     

    Step 1: Determine purpose of third part review.

    The purpose will vary depending on the evaluation task at hand. In some cases the evaluator will determine the appropriateness of written materials for certain target groups, or she/he may be asked to confirm the simplicity and effectiveness of application forms. Regardless of the task, the purpose must be stated clearly. Questions that should be answered at this stage are as follows:

    • What is the purpose of this evaluation?
    • Who will use the results?
    • What materials, procedures, or practices will be examined?
    • What impact will the results of this evaluation have?

    Step 2: Select third part evaluator.

    There are no widely accepted guidelines for this process. However, some basic common sense consideration should be addresses. To develop a potential list of qualified candidates, be sure to query campus resources, CACC, research associations, and community resources. Questions to be answered are as follows:

     

    • What special expertise must the evaluator possess? Specific program knowledge, cultural experience, community awareness, reading expertise?
    • Are there individuals on campus with the expertise? What references can be checked?
    • If the evaluator has to be from outside the college, does he/she have demonstrated expertise that fulfills your need? What references can be checked?
    • Can the evaluator commit the necessary time to complete the project?
    • Will the evaluator require additional resources or support staff?

     

    Step 3: Define evaluator’s role.

    The role of the third party evaluator is a critical link in this process. The role should coincide with the prescribed methods of measuring the criteria in question. For example, to measure the readability of printed materials, a third party evaluator’s role would be to determine the level of the printed material. The evaluator would need access to the materials, and would need to apply some objective method of measuring readability to the materials. In summary, the features of that role would be:
    • To measure the level of readability of printed materials.
    • To report the results to the program administrators.
    • To make recommendations for improvement.

     

    This example is indicative of what needs to be done regardless of the evaluation objective or method. Questions to be answered during this stage are:

    • What is the role of the external evaluator?
    • To whom will she/he report?
    • Will program employees work with the evaluator? If so, have they been informed, and do they understand the evaluator’s role?

     

    Step 4: Estimate time, revenue, and staff needed.

    The evaluator will need to determine the timeline for the completion of the records check. Establishing a timeline will assist in determining what staff can be assigned and what resources will be available. To proceed with this estimate, the evaluator will need to respond to the following:
    • How long will this evaluation process take to complete? What are the crucial deadlines for the project? Beginning? Staff assignments? Due date?
    • Will the evaluation project have a budget? Will any of the following items need to be included?

    Staff salaries/benefits

    Consultants

    Travel

    Supplies and printing

    Telephone

    Equipment

    • Who will be the key staff assigned to this process? Who will be designated as backup staff? What are staff responsibilities? Who will be in charge?

     

    Step 5: Conduct evaluation.

    The third party evaluator will need to be provided all the necessary resources to complete the project. If support staff, materials, or other assistance are to be provided, then these items should be available at the proper time. Following the timeline suggested in Step 3 will keep the project on track.

    During the course of the evaluation, some adjustments might be required; if so, all staff affected should be so informed. Additionally, the staff and program participants with whom the third party review will interface also should be informed before the evaluation commences. Please remember that the potential for role conflict is always present; therefore, you must be careful to "pave the way" for the evaluator. Questions to be answered are:

    • Were affected staff/program participants alerted about the evaluation?
    • Will the evaluator have access to everything needed to complete the task? What is needed? What is available?
    • Will adjustments to the original plan/timeline be needed? If so, who should be alerted?

     

    Step 6: Document/report results.

    The evaluator should maintain files of each records examination project. These files can be used as archives such as being kept electronically and on hard copy. Thus, the use of a computer is highly recommended.

    To determine how these files will be maintained, the evaluator should answer the following questions:

    • Will this information be used for other reports? By other evaluators? If so, can it be easily retrieved? Is the information confidential in nature? What type of security precautions will be needed?
    • Will the information or final reports be distributed to sources outside the college or be used for adults? If so, what procedures will be established to prepare for dissemination or audit reviews?


    The report writing is the final stage of this evaluation process. The evaluators should strive to create an evaluation report which summarizes the information that has been gathered in a clear and concise manner. In general, each evaluation report should have:

    a. A title and author identification
    b. An introduction which highlights evaluation purpose and key questions

    c. A description of the methodology

    d. A description of costs, timeline, staffing

    e. A summary of general findings

    f. A conclusion – analytical statement based upon the evaluation findings and critique of the evaluation process
    g. Recommendations – if required

     

     

    D. SIMULATION
    This evaluation method will be useful in an "in vivo" example if the effect of certain procedures is desired. Simulation would entail having either non-program personnel or program personnel emulate the activities of program participants for the purpose of gauging the effectiveness of admissions procedures, registration procedures, transcript request/response procedures, etc. It is necessary that performance standards be used as the benchmark for the simulation. Thus, if the staff desires to admit and register students in a span of forty-five minutes, then this performance standard becomes the benchmark to which the simulation results are compared. To guide users in the development of simulation, two examples will be presented.

    Criteria "a" under objective 2 for Admissions and Records reads, "Admit and register students in a timely manner." It is suggested that the method of measuring the amount of time it would take to "admit and register student" would be a simulation. This simulation could be done with program staff. For example, let us assume that forty-five minutes is our desired performance standard.

    Program staff could "pose" as students and actually walk through the various steps of admission and registration while being timed by observers. In so doing, the actual time would take to complete this process could be ascertained. Furthermore, the staff could note those points during the process that are particularly time consuming. To ensure reliability, several simulations could be conducted with different staff in the roles of students. The average amount of time for all simulations could be compared to the desired performance standard. If the average time was more than desired, staff could examine what modifications could be made to improve the process.

    A second example of a simulation would be to attempt to breech the security of A & R records. The desired performance standard would be that no authorized persons would be able to gain access to the student record system. To simulate the intrusion, several situations could be enacted by student confidants or program staff. For example:

    1.  An advanced student from the data processing department could attempt to break the on-line or batch student record system for the purpose of changing a grade.
    2. Students could attempt to enter the records sections of the office without being observed.

    3. Staff could attempt to produce official transcripts without proper authorization.

    .

    These attempts to breech security of the A&R system could be orchestrated without the knowledge of program staff responsible for protecting that security. Notes could be taken on easy access points, and recommendations for improving the security made.

    The simulation method could prove an effective way by which to evaluate some aspects of the programs. It will require some creativity on the part of users, and will definitely assume local characteristics as the situation dictates. The following question should be addressed

    • What is the purpose of this simulation?
    • Have staff or volunteers been identified for the simulation? Where are they?
    • If the simulation will be conducted with limited staff awareness, who will be informed? Have they been informed?

    .

    E. SURVEY TECHNIQUES

    "Survey" can refer to either written questionnaires or interviews. Interviews, furthermore, can take place either over the telephone or face to face and can be either formal, requiring a written instrument, or informal not requiring a written instrument. In some cases, the SSPRP models specify which survey method is preferred. A matrix is provided in the appendix of this handbook which identified each criterion population, whether or not a structured instrument is recommended, when the survey should take place, and other descriptive information. (Appendix D in Volume 2)

    While it is not within the scope of this handbook to provide an in-depth discussion of survey methods, steps to include in your approach are outlined below. Many good reference books are available on this topic should you need further assistance (Bibliography, Appendix E, Volume 2). Your campus institutional researcher or similar resource person can provide the expertise necessary to conduct surveys.

    Survey Steps

    1.  Clarify objectives of the survey, preferably in writing.
    2. Identify target population, and select sampling method and size and sample, if necessary. (See Appendix D, Volume 2)

    3. Construct instrument. (See below)

    4. Pre-test the instrument on a small sample and revise questions, where necessary.

    5. Administer the survey.

    6. Analyze the response rate and reliability of the sample and conduct a follow-up, if necessary.

    7. Tabulate the results.

    8. Analyze results in terms of original purpose and questions being asked.

     

    Constructing a Survey Instrument
    1.  Design the questionnaire attractively.

    2. In the design, keep in mind how you will analyze the responses. Does it need to be computer-ready.

    3. Be very brief. Ask only questions directly relating to the purpose. Lengthy surveys discourage response and are more difficulty to analyze.

    4. Multiple choice, quantitative responses (vs. qualitative) are preferable. In multiple choice response questions, be certain that responses options are directly related to the question, are exhaustive, and are mutually exclusive. For example:

    5. How satisfied are you with service provided by financial aid?
      They are conveniently located.
      They are not conveniently located?

       How satisfied are you with the service provided by the Financial Aid Office?
      I am very satisfied with the service.
      I am satisfied with the service provided.
      I am not satisfied with the service.

      Do you enjoy your job?
      Yes
      No

      Do you enjoy your job?
      Yes, always
      Sometimes
      No, never
      Not employed

    6. Group similar-topic questions and use a logical progression of questions.

    7. Place sensitive questions at the end of the survey.

    8. Phrase questions specifically; don’t be vague.

    Do you think you will ever use financial aid?
    Will you use the Financial Aid Office this semester?

    8. Make questions easy to answer.

    What was your family’s gross income for 1971?
    In 1983, what was the approximate total income of your Household?
    Less than $10,000 $20,00 to $29,000
    $10,000 to $19,000 $30,00 or more
    Don’t know

    9. Be objective.

    Don’t you just love our school?
    Have you been pleased with your experiences at this college?
    Yes Mixed
    No Don’t know

    10. Questions should be non-threatening and non-humiliating to answer.

    1.  Responses should be anonymous, if possible.
    2. Provide specific instructions for easy return of the questionnaire.

    3. Provide respondents with the option of writing comments regarding the questionnaire topic.

    4. You may want to collect demographic (age, gender, ethnicity, socio-economic) variables so that you may analyze the results of these variables. In that way, you may be able to determine if a How well did you understand the information presented about (fill in specifics)?
      Did you receive all of the information you needed to ?
      Were all of your questions answered regarding ?
      Did you receive the information you needed in time to ?
      Did the _____________________listen to your concerns?service is serving particular groups better than others.

    5. Special care should be taken in writing student satisfaction questions. Students who have the greatest need for a service may lack experience with effective educational services; therefore, they often lack a standard to measure satisfaction. Students will be able to answer (with greater precision and accuracy) questions which ask for information regarding their own needs. For example:

       

     


    GLOSSARY

    ACCCA
    The Association of California Community College Administrators

    Accountable
    I
    n general, to be obliged or required to account for, to explain, to provide reasons for, to describe or report the worth or importance of something. More specifically, the California taxpayers want Community Colleges to deliver evidence that they are giving the society its money’s worth. Institutional Research and program evaluation are characteristically called on to supply this evidence.

    ADA
    Average Daily Attendance

    Aggregated Group (or Aggregated Results)
    Clustered together to form one mass, or unit – a particular collection of data in summary form.

    Applicant Pool
    The collection of people who are applying for something; e.g., a loan, a job.

    Archives
    Records or a place for keeping records.

    Articulation
    State of being jointed – in agreement – a process or plan of transfer of course credit from one school or college to another; e.g., the local community college has an articulation agreement with UCLA by which the completion of the college’s English 1A class will be recognized as transferable to UCLA.

    Audit
    In general, a formal, often periodic checking of records or reports to verify their accuracy. Also, a review by an outside person or persons to verify program information.

    Audit ExceptionsIn finance, the existence of an audit exception indicates a problem; either the documentation is not sufficient or it is sufficient but shows that a violation of current guidelines has occurred.

    Audit Trail
    The existence of documentation from the beginning of a financial transaction to its termination; e.g., in finance, from the receipt of an invoice to the printing of a check to make required payment.

    Backup System
    A secondary method or set of procedures that provides insurance against loss by the primary method – a contingency plan.

     Boiler Plate Evaluation Strategies
    A detailed flow chart and step-by-step description of procedures that are necessary to be able to utilize the following five evaluation methods: A records-examination, a listing of data, a third party review/report, a simulation, and a survey.

    CACC
    California Association of Community Colleges

    CCCCSSAA
    The California Community College Chief Student Services Administrators Association

    CEOs
    The chief executive officers; the college; presidents, or the district chancellors or superintendents.

    Charrette
    An intensive, goal directed, consensus developing group experience within which something is accomplished (e.g., a document is produced) after much give and take by the members of the group. Charrette is a French word that originally referred to a cart in which Parisian architectural students worked on their designs in final preparation for graduation. Colleagues of the student were expected to assist by reviewing and critiquing the student’s work.

    Clearing House
    A central place where information is gathered and from which it is disseminated.

    Co-curricular
    Activities that accompany instruction, or co-exist alongside instruction.

    Communication Structure
    A system within which information is shared.

    Competency Testing
    Testing of knowledge or a particular skill to establish whether a pre-established minimum standard has been met.

    Confidentiality
    Keeping certain information secret; e.g., the identity of students on probation.

    Content Tutors
    Student tutors (or teachers) of a specific subject matter; e.g., a reading tutor, a mathematics turor.

    Contingency Plan
    A backup or secondary system or set of procedures designed to provide insurance against loss by the primary system.

    Control Group
    Is a group of people who are similar to the experimental group in all respects except one: They are not exposed to the intervention or experimental treatment condition; e.g., students who did not receive specialized counseling.

    CPEC
    The California Postsecondary Education Commission

    Criteria
    Standards for forming a judgment; e.g., in order to determine whether the admissions and records functions were performed in an efficient and effective manner. (Admissions and Records Program Goal 8) the level of cost effectiveness (Criteria B) can serve as a standard or "Criterion."

    Database
    A repository of information. A collection of characteristics about a population or sample.

    Data Elements
    A generic term referring to any item or variable already measured or to be measured. A database is made up of data elements; e.g., the student database is made up of the student’s age, sex, G.P.A., etc.

    Debt Management
    Deals with financial obligations; e.g., management of student loans.

    Default
    A default has occurred when a financial aide recipient has failed to make timely repayment.

    Demographic
    Characteristics or social statistics of a group; e.g., age, gender, ethnicity, socioeconomic status; but usually not test scores.

    Donor Criteria
    Standards established by a person(s) donating a gift to be given to someone else; e.g., characteristics of a student that are essential before he can be selected to receive a scholarship.

    Duplicated Count
    A total number based on a way of numbering that allows for repeats such that a single individual may be counted more than once; e.g., repeated counseling visits by one person might be included in a summary statement that "counselors had 9,500 counseling contacts last semester."

    EDD
    Employment Development Department

    EOPS
    Equal Opportunity Programs and Services

    Equivalent Groups
    Groups of people which are seen as alike in characteristics judged to be important to the outcome of the study; e.g., age, gender, academic load, total number of hours working, etc. Equivalent groups are often formed when random assignment of students to experimental and control groups are difficult or impossible.

    Evaluation
    In general, evaluation is the collection and use of information to make decisions about all or part of an educational program. More specifically, evaluation is a process for determining the value, worth or merit of all or part of an educational program. The beginnings of evaluation as an area of specialization are often dated from the late 1960’s when Great Society programs infused large sums of money into education and the Federal Government demanded program evaluation reports.

    Evaluation Model
    An approach, viewpoint, framework, design, set of procedures or guidelines which are meant to be useful to community college student service practitioners as they plan, develop and conduct evaluations of their local college programs. More specifically, a set of student service area specific goals, criteria, methods, measures, and other materials developed to assist in this way. Eight evaluation models have been developed during the Student Services Program Review Project. The eight evaluation models do not include a listing of measurable objectives; these must be written by the student service practitioners at each college. Only they can decide who should do what by when and to what level of performance.

    Evaluation Plan
    A written plan by which an evaluation of an educational program will be done. At a minimum, it includes program goals, objectives, and a time line. Also, an eight step plan for conducting an evaluation. (Please see "Goal," "Objective," Operational Plan," and "Evaluation Plan Outline.")

    Evaluation Plan Outline
    A series of eight steps and corresponding questions to be answered when planning the conduct of a program evaluation.

    Exhaustive
    A term indicating that all possible responses are included or incorporated; e.g., all categories of attitude are to be represented on a multiple-choice item. The catch-all response category of "other" is often added for this reason.

    Experimental Group
    Is a group of people who are exposed to the intervention or experimental treatment condition; e.g., students who received specialized counseling.

    External Comparison
    Judging the degree of similarity of characteristics of something internal to the system to something outside the system; e.g., the student/counselor ratio at our college to the student/counselor ratio at other colleges.

    F.A.
    Financial Aide. Money provided to qualifying students.

    FAO
    Financial Aide Office

    Field-Based
    Methods established for use by practitioners of the student service.

    Field-Produced
    Developed by practitioners of the student service.

    Field-Test
    A practice or trial run of a set of procedures undertaken by those individuals likely to use them.

    Formative Evaluation
    Takes place during the development, implementation, or refinement of a program. Provides for midcourse corrections. Takes place during the time when program activities are occurring and provides guidance on how to improve the program.

    Gender Equity
    The degree to which fairness and equality has been or will be shown to both sexes.

    Goal
    A statement, general and abstract, of desired states in human conditions or social environments; e.g., "Students will be satisfied with counseling related courses." (Compare to the definition of "objective.")

    Governance
    Formal system of administration.

    GPA
    Grade-point-average.

    Hard Copy
    A record that exists in printed form (in a book, report, computer printout; etc.) as contrasted to existing electronically on a computer disk or tape.

    HSPS
    Handicapped Student Programs & Services

    In-service Training
    Training taken while the participants are employed.

    Intake Form
    A questionnaire or survey instrument completed before the student receives a student service.

    Intercollegiate
    Between different colleges.

    Internal Audit
    A formal or official examination and verification of records produced, procedures followed, and accomplishments made by individuals within the system.

    Internal Comparisons
    Judging the degree of similarity of characteristics within the same system; e.g., the attrition rate last fall quarter at College A to the rate of each of the fall quarters over the last five years, at College A.

    Inter-segmental
    Between the three segments of three types of institutions that comprise California higher education. There are the California Community Colleges, the California State Universities and Colleges (CSUC), and the Universities of California (UC).

    Intervention
    Any planned treatment or exposure to an event designed to produce intended changes in a target population; e.g., tutoring provided to students on probation. Sometimes called a strategy or described as being composed of a set of strategies.

    Interview
    A method of survey directly asking an individual for information.

    Intra-institutional
    Within or inside the institution.

    Intramural
    Between or among the members of a single college.

    Inventory
    An itemized list or count of things. Also, a psychological measuring instrument; e.g., a personality questionnaire.

    In Vivo
    In life or as in real life.

    Longitudinal
    Measurement of something over a period of time, usually years.

    Measures
    Those variables that can be observed, reviewed or appraised to determine whether an objective developed from a goal has been attained; e.g., to gather information on the level of student involvement in institutional governance (Student Affairs Program Goal 2), the number of student positions in governance (Measure #3) can be reviewed.

    Method
    In general, an orderly procedure, process, or manner, or way of doing anything. More specifically, an action that will provide the evidence called for in the measure (see above) and which will also answer the explicit or implied criteria question given in each evaluation model shown in this document. For example, for the measure that calls for "clear and concise assessment results" (Assessment Services Program Goal 3), two methods which are given are: A third party review of the degree of clarity and conciseness (Method 5.1), and a survey of counseling advisement staff (Method 5.3).

    Mode
    Method or manner of doing something. In statistics the most frequent score or value.

    Model
    In general, an initial form, a plan, a pattern, a structure, an approach, a representation, a design or standard for imitation or comparison. (See "Evaluation Mode")

     Mutually Exclusive
    A phrase indicating that items in two separate sets do not overlap; e.g., items on two questionnaires do not overlap.

    Needs Assessment
    The process, or method by which the needs of an individual or group are identified; e.g., the process by which the special needs of the students seeking jobs are identified.

    Objectives
    Very specific, operational statements. Statements made in measurable or behavior terms. At a minimum an objective states: Who will do what, by when, and to what level of performance. (In addition, many evaluators insist that an objective also include a description of how the task accomplishment will be determined; i.e., what evidence will be reviewed before the evaluator decides whether or not the task was completed satisfactorily.) For example, a program objective for the counseling area could be, "upon completion of the spring semester counseling-related classes entitled, ‘Educational and Vocational Planning;’ at least 85% of the students will, on questionnaire, rate the class as either ‘Satisfactory’ or ‘Very Satisfactory.’" (The evaluator could examine the completed questionnaires and a summary sheet that showed the number and percentage of student answering "Satisfactory" and "Very Satisfactory.") Objectives can be categorized as either product or process. Product objectives focus on the primary outcomes, results, or products of program activity, whereas process or activity objectives focus on the activities and procedures needed to achieve the desired outcomes. For example, the counseling objective just given would be a product objective since it describes terminal student behavior; i.e., it is focused on a "Student Outcome." An example of a related process or activity objective would be, "By November 15, 1986, all counselors will order all interest inventories (Kuder, Strong, etc.) needed for the spring "Educational and Vocational Planning" classes. Measurable objectives are not provided in the eight evaluation models given in this publication. Local college personnel are encouraged to use the student service area goal, criteria, measures, and methods as input to develop their college’s objectives. Only they can decide who should do what by when, and to what level of performance. (Compare to the definition of "Goal" and "Evaluation Plan.")

    Open-Ended Question
    A question that allows or requires the respondent to formulate their answer in their own words. Words like "How," "What," and "Why" are often used in open-ended questions; e.g., "Why did you come in for counseling?" Open-ended questions are different from "focused choice" questions in which response categories have already been chosen and the respondent must make choices from among those.

    Operational Definition
    Description of something expressed according to the actions (operations) necessary to make the observation fit the definition or description; e.g., for a given study the operational definition for a "full-time student" could be, "A student who is enrolled in 12 or more units as of first census."

    Operational Plan
    An implemented plan or one capable of being implemented; a specific, detailed description of what will be done, usually with a time horizon of one year or less. Normally includes at least goals, objectives, and a timeline. Frequently includes a budge.

     Over-awards
    A student has received or is about to receive a sum of money, an award, that exceeds the amount to which he is entitled.

    Pilot Test
    A pre-trail or preliminary use of a method or testing instrument.

    Population
    The entire group about which one wishes to draw some conclusion or make some inference. (Compare to the definitions of "sample" and "random sample.")

    Post-hoc
    After the fact; e.g., a research study of students’ community college experiences after their graduation from the community college.

    Practitioners
    People involved with providing a student service.

    Pre-Post Survey
    Measurement taken before and after an event or intervention.

    Proficiency Testing
    (See "Competency Testing")

    Purge
    To eliminate or destroy.

    Qualitative Response
    A judgement of quality as opposed to pure quantitative evaluation; e.g., a determination of how good a service is judged to be.

    Quantitative
    Numerical index; e.g., how many nurses graduated, the mean reading level of entering freshmen.

    Questionnaire
    An instrument designed to measure something; e.g., a survey form for use in polling student attitudes.

    Random Assignment
    In basic social science research, a method of assigning subjects to either an experimental group (that receives the special treatment or intervention) or a control group (that does not receive the special treatment). The assignment is random if it occurs purely by chance.

    Random Sample
    A sample selected in such a way that every case in the population has an equal probability of being included in the sample and the selection of one case has no influence on the selection of any other case. (See the definitions of "sample" and "population.")

    Range
    The difference between the lowest and highest values; e.g., if the lowest value is 20 and the highest is 80, the range is 60.

    Readability
    Degree of ease or difficulty in reading certain written material.

    Reliability Coefficient
    An index or number that varies between -1 and +1 (including 0) that indicates the degree of consistency of measurement; e.g., the consistency with which the same students get the same or nearly the same test scores on the same test on repeated trails.

    Research
    A careful, systematic, patient study in a field of knowledge (education) undertaken to discover or establish facts or principles. A comparison is often drawn between research and program evaluation. Popham (1975) noted: (1) That evaluation focuses on decisions and research on conclusions, (2) That generalizability to other situations is low in evaluation and high in research, and (3) The value emphasis in evaluation is on a program’s worth or value, whereas in research which is focused on the search for truth. Institutional Research is that research which is focused on one institution or one kind of institution; e.g., California Community Colleges.

    Researcher
    One who does Community College Institutional Research and Program Evaluation. A number of such specialists assisted the student service practitioners in the development of the eight student service area program evaluation models presented here.

    SA
    Student Affairs or Student Activities

    Sample
    A subset (at least one less) of the entire population. To measure a smaller group than the population. The purpose of drawing a sample is to make an inference or draw a conclusion about the population. (See the definitions of "Random Sample" and "Population.")

    Sampling Method
    The method by which a sample will be or has been drawn from a population. The sampling may be random, systematic, or stratified. In random sampling every member of the population has an equal chance of being selected. Systematic sampling could involve drawing every tenth person from an alphabetized list of everyone in the population. A stratified sample is drawn when there are two or more major ways of classifying data (age, gender, ethnic membership) and if it is important to insure that each category is proportionately represented in the sample.

    Script
    A document specifying what is to be said or done.

    SES
    Student Employment Services. Socioeconomic status.

    Stimulation Study
    To act out or approximate the activities under investigation.

    SSPRP
    The Student Services Program Review Project

    Statistics
    A collection of methods for numerical analysis. Also the summary values derived by the analysis; e.g., the arithmetic average or mean, the standard deviation, the value of t, F, Chi-square, etc.

    Strategy
    As used in "boiler plate evaluation strategy," a detailed step-by-step plan of action that is designed to help student service practitioners utilize five frequently recommended evaluation methods. Also used interchangeably with intervention, or to mean a part of an intervention. (See "Boiler Evaluation Strategies" and "Intervention.")

    Student Profile
    Summary information (can include demographics and test scores) about students.

    Summative Evaluation
    Takes place at the end of a program; after program activity has ceased. Is concerned with the success or failure of operational procedures to attain their desired effects. Provides guidance on whether or not to continue the program. A third party review sometimes takes the form of a summative evaluation simply because the cost of having an outsider do both formative evaluation and summative evaluation is often prohibitive.

    Survey
    A data gathering instrument; e.g., a written questionnaire or interview. A method of polling individuals.

    Target Population
    Persons or groups to which interventions are directed; e.g., students seeking information and skills that will help them make better career choices.

    Testing Environment
    The place and circumstances under which tests are administered.

    Third Party
    An external, disinterested, or objective person or persons.

    Underqualified Student
    A student whose present skill or attribute level falls below the minimum criteria for qualification.

    Unduplicated Count
    A total number not permitting repeats. An individual or an action may be counted only once.

    Validity Coefficient
    An index or number that varies from -1 to +1 (including 0) that indicates the degree that something is genuine; e.g., a valid reading test measures reading skills, not mathematics skills; it measures what it is supposed to measure.

    Variable
    Any stimulus, event, or characteristic which may influence an outcome and which, when measured, gives a range of values; e.g., the age of students.

    BACK TO TOP || OIR HOME PAGE