Student Services Evaluation

The Student Services Program Review Project (SSPRP) was a unique, voluntary effort on the part of many California community colleges. Its purpose was to develop evaluation models to measure the efficacy of the various student services programs provided by the colleges.

The SSPRP had the board support of the colleges and districts involved, and of many statewide organizations and agencies. They were generous in the commitment of both personnel and resources. Over 1,000 persons were involved in the Project during the three years of the Project. Those participating colleges, organizations and contributors are acknowledged in the appendices in Volume 2.

The uniqueness of the Project was in its goals as well as in its broad-based foundation. As a result of the efforts of the participants, all community colleges now have evaluation designs – including goals, criteria, measures, and methods – which were field-based and field-produced, and with which the colleges will be able to evaluate their student services programs.

Since the purpose of the Project was the development and refinement of evaluation designs, colleges participating in the field tests of the evaluation models were not asked to share evaluation results, but only that information necessary to produce and refine the designs. Standards against which to measure program success were developed by the participating colleges for their own use.

The final products of the Project are contained in this Report and are intended for use by colleges. It is anticipated and hoped that the design will be constantly reviewed and improved by frequent use.

BACKGROUND
Accountability and reform for the California Community Colleges represent current and urgent topics of discussion and items for action on the agendas of major public policy making bodies including the legislature.

The mission and funding of the California community colleges continue to undergo close scrutiny. All programs in community colleges are vulnerable in a period of financial crisis, but student services programs seemed particularly subject to reduction or elimination. One of the reasons for that vulnerability was the lack of reliable, verifiable information which described and evaluated student services programs. The information that did exist was anecdotal or difficult to aggregate, and therefore, often not usable as support for the continuation of programs.

It was apparent that if student services programs were to continue to be supported as an essential part of the mission and functions of California community colleges, an effort would have to be made to systematically assess the programs’ contributions to student outcomes. In response, the Commission on Student Services of the California Association of Community Colleges (CACC) and the Northern California Cooperative Institutional Research Group (NORCAL) agreed that the colleges must become active in evaluating student services and in using the results of these evaluations to underscore successes and to modify services as necessary for improvement.

The Student Services Program Review Project (SSPRP) was developed, therefore, with the intention of developing and testing evaluation approaches and criteria for the various areas of student services. The original group of participants, headed by Dr. Robert Jensen, then President of NORCAL, and Peter Hirsch, then Associate Executive Director of CACC, included both student services professionals and persons skilled in research. A Steering Committee was formed and continues to direct the Project. To facilitate the implementation of Project activities, Project Directors (See Appendix B) and Research Coordinators were also named.

The Project goal was to develop and pilot test evaluation designs in order to assist colleges in the implementation of program evaluation of selected programs of student services on their campuses.

Several assumptions were made at the inception of the Project. These were: (1) the Project was to be a "grass roots" activity involving volunteer participation by colleges; (2) the Project was to be a coalition effort by and for the participating colleges; (3) all California Community Colleges were to be given the opportunity to participate; (4) financial assistance was to be requested from outside sources to support Project coordination and development; (5) ongoing operational support was to be generated through fees from participating colleges.

A sequence of activities intended to achieve these objectives was established and revised periodically by the staff and steering committee (Appendix D).

It was agreed that the first phase of the Project would focus on those areas of student services selected by the participating colleges as having the highest priority for review and evaluation. To identify the programs to be evaluated during Phase I, several surveys of California’s community colleges were conducted. In addition, a number of statewide student services organizations provided guidance and information. Based on this review process, which occurred over a six-month period, the following areas were selected for evaluation by Northern and southern colleges for the first phase: (1) Admissions and Records; (2) Counseling; (3) Financial Aid; (4) Student Affairs. In addition, the participating northern colleges elected to review the area of Job Placement.

To develop concepts essential to the conduct of the Project and to begin the foundation work leading to development of evaluative criteria for each program, two charrettes were held, one in the north at De Anza College, and one in the south at Mt. San Antonio College. Over three hundred people participated in these two activities.

The term "charrette" is from the French Parisian architectural students, preparing for the final defense of their work and their rights to graduation, entered into intensive development of their last designs and drawings. When this occurred, colleague-students would pick up the student who was preparing for the examination in a cart known as a "charrette." They would load the students’ drawings and designs onto the cart, and as they traveled through the streets of Paris, the student would finish her/his work. Commonly, the student would call for her/his colleagues to review the final work. Critique and revision would follow; consequently, the final drawing or design would often be the best in the student’s portfolio.

The charrette concept as applied to issue resolution describes an intensive, group-oriented, planning and development process. People with different backgrounds, different orientations, and different perceptions, but all allied by a common interest in resolving the issues under consideration, meet together to analyze issue components and develop consensus resolutions. The SSPRP Charrettes resulted in the development of a mission statement for Student Services, goals for each program under study, and lists of suggested evaluative criteria and methods for their use in the development of evaluation designs.

Writing teams worked with the results of the charrettes to develop consensus statements. Drafts of these statements were first reviewed by all charrette participants and other student services personnel. Their reactions were then used by the writing team and the Project Steering Committee to prepare a final draft of evaluation models. The attempt during these workshops was to maintain the sense of the consensus of both charrettes in developing measurable goals, suggested criteria and evaluation methods. The Charrette Report was distributed in June 1984.

CHARRETTE OUTCOMES
The mission statement for student services was jointly developed by the more than 300 charrette participants. It conveys the critical nature of student services programs in community colleges and specifies they key goals which these services are designed to accomplish.

Goals were also developed by the Charrette participants for each of the five program areas: Admissions and Records, Counseling, Financial Aid, Student Affairs and Student Employment Services.

The initial identification of evaluation criteria and methods for each goal was begun by the Charrette participants. These were not meant to be final products, but rather guides for further development of the actual criteria, methods, and measures for use by the pilot colleges.

In June 1984, an intensive writing workshop was held at Cabrillo College, Aptos, California. Participants included members of the Steering Committee, persons representing the professional organizations of the student services areas under study, researchers from Northern and Southern California, and key California community college leaders. For two-and-one-half days, writing groups developed criteria, measures and methods for every goal in the five areas of student services. The results of the writing workshop were then reviewed by the participants and field reviewers recommended as representative of the five program areas.

Colleges participating in the Project began to pilot test the evaluation designs in the fall of 1984. Workshops provided assistance to the participating colleges, including an orientation to the procedures, evaluation instructions, and guidelines.

The pilot testing of the evaluation models was conducted by participating colleges from October 1984 through the spring semester 1985. The results of the pilot (critiques of the models) were reviewed by a team of practitioners and researchers, and the goals, criteria, measures, and methods were refined as recommended by the participating colleges. The final evaluation models are provided in this Report for use by colleges.

In fall 1985, following a process similar to that of Phase I, Phase II of the SSPRP began with the completion of a survey by all California community colleges. The colleges were asked to select other areas of student services having high priority for review and evaluation. Three additional areas were selected: Assessment Services, Career/Life Services, and Tutorial Services.

Twenty-three colleges and one hundred twenty-five student services practitioners and researchers participated in charrettes held at the College of San Mateo and at Rancho Santiago College in April 1985. The purpose of the charrettes was to produce goal statements and evaluative criteria for the three areas. The recommended goals and criteria were subsequently reviewed by a writing team resulting in the development of a charrette report. This report was disseminated to all community colleges for comments and suggestions.

In August 1985, a writing workshop was conducted during which Student Services Program Review staff, Steering Committee members, and practitioners from each of the program areas reviewed the charrette report and field responses. The writing workshop produced the goals, criteria, measures, and methods to be used in the pilot tests which began in fall 1985. Participating colleges conducted pilot testing of the evaluation models of one or more of these areas. Using critiques from the colleges’ pilot tests, a final review and writing workshop was held in June, 1986, resulting in the production of revised criteria, measures and methods for the three Phase II areas. These designs are also part of this Report and are now available for use by colleges.

IMPLICATIONS OF THE PROJECT
The Student Services Program Review Project has made significant progress toward the goal of enabling colleges to develop the information necessary for the support and improvement of their student services programs. With the information gathered as a result of systematic program review, Student Services can be shown to be an integral – not peripheral – part of the education process.

The Project has implications for many other efforts currently under way in the community colleges in California. Consider, for example, differential funding. In that funding proposal, Student Services has been identified as one possible "cost Center." Since both qualitative and quantitative measures will be required at the point of determining what will be funded and in what amounts, it is clear that having a systematic way of reviewing student services programs could be of great advantage to the colleges. Other examples include the fact that the Accrediting Commission for Community and Junior Colleges may us SSPRP results to review and revise its Student Services Standards. Many of the pilot colleges used the evaluation results as part of their self-studies. This liaison between the Project and the Accrediting Commission should serve to further encourage evaluation and coordinate efforts to improve student services.

The SSPRP, a joint effort on the part of California’s community colleges, statewide organizations, and student services personnel, has given the student services an advantage: a head start in determining their own fate. It is essential that California’s community colleges have the major responsibility for their own futures. If they do not, those futures are less likely to reflect the needs of millions of citizens seeking educational opportunities in the colleges, and are more likely to be the myopic vision of various distant policy making groups.

Clearly, the Student Services Program Review Project , with its board-based participatory and voluntary foundation, has involved colleges in the development of their own goals.

Criteria	Measures	Methods	E/A*
a) Availability of information	1. Evidence of each Admissions & Records information item (written and non-written).	1.1 Provide example or documentation of each.	E
b) Accessibility of information	1. Evidence of distribution to service area and target groups.	1.1 List the distribution locations, description of distribution method, targeted group, and date of availability of each item listed above.	E
	2. Evidence of diverse distribution locations.	2.2 Indicate hours of operation of distribution centers.	E
	3. Ease of obtaining information. Level of community aware-ness of information distributed.	3.3 Survey awareness and satisfaction & (students and non-students) 4.4 Survey could include written questionnaire or interview of sample or entire population.	E
c) Readability and accuracy of information.	1. Evidence of clear, concise, accurate, and complete information.	1.1 Measure reading grade level of all information provided.	A
c) Readability and accuracy of information.	2. Evidence of appropriateness in reading special target group populations.	2.1 Third party (selected individuals from outside institutional A&R staff) examination and analysis of information to determine clarity, accuracy, conciseness, and completeness. 2.2 Indicate the appropriateness of language for targeted groups.	A E
d) Timeliness of information distribution.	1. Evidence of appropriate re-lationship between timing of information distribution and educational and student services provided.	1.1 Demonstrate inter-relationship between information provided and services. (Indicate actual dates of information distribution) 1.2 Survey users of information to determine level of satisfaction with timing.	E

Criteria	Measures	Methods	E/A*
a) Admit and register students in a timely manner.	1. Amount of time required to be admitted and/or registered.	1.1 Conduct test sample during admissions and registration processes. 1.2 Survey students to determine whether they were admitted and registered in a timely manner.	E E
a) Admit and register students in a timely manner.	2. Hours and modes of Admissions & Records service.	2.1 Review and analyze hours of operation and hours of services available. 2.2 Provide evidence of alternative methods of admissions and registration.	E E
b) Coordination, admiss-ions and registration of students with other campus service units.	1. Evidence of coordination efforts between campus service units.	1.1 Interview representatives from campus service units. 1.2 Provide and review formed plan for coordination efforts.	E
c) Ease of application and registration process	1. Evidence of simple and efficient forms and process	1.1 Third party review of forms and processes. 1.2 Staff/student survey to determine simplicity and efficiency	E E
d) Accuracy of data collected	1. Level of accuracy of registration and enrollment data.	1.1 Internal audit 1.2 Third party review	E E
e) Accuracy of students’ schedules of classes	1. Consistency between students’ class schedule and roll sheet.	1.1 Test sample for consistency	A
e) Accuracy of students’ schedules of classes	2. Existence of errors due to Admissions & Records processing	2.1 Monitor number and type of student/staff/faculty complaint 2.2 Identify and analyze errors to determine cause and remedy. 2.3 Survey staff/faculty and students to determine level of accuracy.	E A E

Indicates which methods are essential ("E") for program review and which provide additional ("A") information and/or insight.

Criteria	Measures	Methods	E/A*
a) Effective and efficient design of records collection instruments (Forms).	1. Evidence of: Completeness Ease of use Utility for efficient integration into the information system	1.1 Third party review by knowledgeable source. 1.2 Survey users.	E A
b) Effective and efficient storage of A&R records	1. Evidence of adequate capacity for present and future records.	1.1 Provide or demonstrate a plan and procedures.	E
	2. Evidence of backup system for records.	2.1 Document back-up system.	E
	3. Evidence of security measures including other areas having access to computer data base.	3.1 Analysis of current and projected storage use in relationship to capacity. 3.2 Review and assess adequacy of records contingency plan. 3.3 Visual inspection of storage system. 3.4 Field test back-up system. 3.5 Review and assess policy and regulations to verify compliance requests. 3.6 Field test security. 3.7 Review and assess security protection of student confidentiality.	E E E E E E E
c) Efficient and effective maintenance of A&R records.	1. Evidence of updating records in a timely and accurate manner.	1.1 Conduct test sample to test accuracy and timeliness of record updates. 1.2 Audit computerized record system for timely updating of records. 1.3 Survey of A&R staff by third party to determine whether records are updated in a timely and accurate fashion. 1.4 Analyze time taken for required record changes. 1.5 Review and analyze policy and regulations in regard to purging (retirement) of A&R records and compliance requirements.	E A A E E

Criteria

Measures

Methods

E/A*

d) Effective and efficient retrieval of student and instructor records.

1. Evidence that records are secure in accordance with stated policy and regulations.

1.1 Review and analyze policies and regulations for compliance requests.

1.2 Field test security system.

1.3 Analyze time to retrieve student and instructor records.

2. Evidence of timely and easy access to and retrieval of student records.

2.1 Survey A&R staff and other users of system.Field test ease and timeliness of access.

Indicates which methods are essential ("E") for program review and which provide additional ("A") information and/or insight.

Criteria

Measures

Methods

E/A*

a) Efficient, accurate, and regular evaluation.

Evidence of efficient, accurate, and timely evaluation of:

transcripts
graduation
residency
certification
degree or certificate requirements

Test sample of each to determine time taken and accuracy.

Third party review.

Survey students and staff users.

b) Efficient and timely distribution of student records.

1. The length of time taken for the recipient to receive document.

Conduct test sample of the time taken.

Survey recipients to determine how long it took them to receive documents and how accurate the records were.

Criteria

Measures

Methods

E/A*

a) Submission of attendance data in an accurate and timely manner.

1. Extent of compliance with agency requirements.

1.1 Review and analyze agency audit.

2. Extent of compliance with agency timeliness.

Review and analyze college audit.

Comparison of each report deadline with each report submission data. Review and analyze any discrepancies and reasons.

Criteria	Measures	Methods	E/A*
a) Availability of adequate and efficient character-istics and enrollment data.	1. Evidence of the existence of student characteristics and enrollment information.	List data elements available. List all information reports. Survey data users to determine satisfaction and accuracy of information provided. Document ease of retrieval of data elements and information accessibility of users.	E E E
b) Coordination of collect-ion and dissemination of student characteristics and enrollment data with other campus units.	1. Evidence of the existence of a plan for coordination of collection and dissemination of information.	Provide a plan for defining responsi-bilities and outlining coordination for the collection and reporting of data. Survey of information producers and users to determine the efficiency and timeliness of the coordination.	E E

Criteria	Measures	Methods	E/A*
a) Existence of A&R systems (operations) Manual.	1. Examination of Manuals for Operations and Procedures.	1.1 Review of Systems (operation) Manual for: Recency; Completeness; Back-up System (contingency plan).	E
b) Cost effective.	1. Evidence of on-going analysis for cost effective-ness.	1.1 Determine cost per student and make comparisons internally and externally (other comparable colleges), (e.g., year to year, system to system, etc.).	E
c) Competent and trained staff.	1. Evidence of implementation of plan for inservice and staff development.	1.1 List of events and dates for each activity.	E
d) Effective coordination of A&R functions with other campus service units.	Evidence of a plan or formal procedure/activities for coordination between A&R and other service units.	1.1 Survey campus service units to determine extent and quality of coordination.	E

Criteria	Measures	Methods	E/A*
a) Are assessment services coordinated with appropriate instructional staff, departments, and programs?	1. Evidence of communication structure for the purpose of assessment and placement planning.	1.1 Review documentation regarding existence of communication structure (e.g., minutes, membership lists, meeting schedule).	E
	2. Satisfaction with above communication structure.	2.1 Survey appropriate staff and departments to assess satisfaction.	E
b) Are assessment services coordinated with appropriate student personnel services and special programs?	1. Evidence of communication structure for purpose of assessment and placement planning.	1.1 Review documentation regarding existence of communication structure.	E
	2. Satisfaction with above communication structure.	2.1 Survey appropriate staff and departments.	E
c) Are assessment services coordinated with appropriate administrative support services (e.g., data processing, scheduling)?	1. Evidence of a communi-cation structure for the purpose of assessment and placement planning?	1.1 Review documentation regarding existence of communication structure.	E
	2. Satisfaction with above communication structure.	2.1 Survey appropriate staff and departments.	E
d) Is there college-wide organizational structure (steering committee) and designated individual(s) responsible for the development of the assessment and placement program?	1. Evidence of an organi-zational structure and/or designated responsible individual(s).	1.1 Review documentation regarding existence of organizational structure (e.g., membership lists, minutes, etc.).	E
	2. Satisfaction with organizational structure.	2.1 Survey appropriate staff and departments.	E
e) Are assessment services coordinated with feeder high schools, four-year institutions, and other regional post-secondary institutions.	1. Evidence of communication structure for purpose of assessment & planning.	1.1 Review documentation regarding existence of communication structure.	E
	2. Satisfaction with above communication structure.	2.1 Survey appropriate staff of feeder high schools, four-year institutions, and other post-secondary institutions.	A

Criteria	Measures	Methods	E/A*
a) Is there an institutional policy regarding assessment?	1. Evidence of institutional policy.	1.1 Review policy.	E
b) Is the policy consistently implemented?	1. Evidence of consistent implementation.	1.1 Comparison of actual number of students tested versus number of students in testing group (by policy).	E
c) Is there an adequate delivery system for administering assessment?	1. Evidence of availability of frequent testing.	1.1 Review testing schedules.	E
	2. Evidence of appropriate testing environment.	2.1 Observe testing environment.	E
	3. Evidence of adequate security and confidentiality.	3.1 Observe security and confidentiality safeguards.	E
	4. Evidence of qualified testing personnel.	Review qualifications of staff. Survey students to determine satisfaction of assessment delivery.	E A
d) Is individual assessment available for specific interests and needs of students?	1. Evidence of individualized assessment capabilities.	1.1 Examine testing inventory and availability of staff to administer and interpret tests.	E
	2. Evidence of adequate referral.	2.1 Review referral system.	E
e) Are the assessment instruments valid and reliable in deter-mining students’ skills, aptitudes, and goals?	1. Validity and reliability of assessment instruments.	1.1 Review norms of test used.	E
f) Does the assessment system provide compe-tency/proficiency testing (e.g. course	1. Evidence of competency testing.	1.1 Review competency/-proficiency testing.	E
	2. Satisfaction with competency/proficiency testing availability.	2.1 Survey staff/departments to determine level of satis-faction.	A
g) Is there adequate publicity to inform students about the assessment process?	1. Evidence of publicity materials.	1.1 Review inventory of materials.	E
	2. Dissemination of materials.	Review dissemination schedule. Survey students to determine adequacy of materials.	E A

* Ind Indicates which methods are essential ("E") for program review and which provide additional ("A") information and/or insight.

Criteria	Measures	Methods	E/A*
Are adequate records maintained on assessment results?	Evidence of accurate and complete record keeping.	Review system for information storage.	E
Are adequate records maintained on assessment results?	2. Evidence of easily retrievable evidence.	2.1 Conduct test sample of assessment records accessibility.	E

Criteria	Measures	Methods	E/A*
a) Is assessment information dissemin-ated in a timely and efficient way to users?	1. Evidence of timely dissemin-ation of assessment information to users.	1.1 Survey counselors, instructors, and students to determine efficiency and timelines.	E
b) Are clear and concise assessment results provided to students?	1. Evidence of clear and concise assessment results.	Staff or third party review degree of clarity and conciseness. Survey students to determine satisfaction with clarity and conciseness.	E E
c) Are assessment results used to refer students to appropriate special student services (e.g. handicapped, EOPS)?	1. Evidence of referrals.	1.1 Review sources of student enrollment in special support services.	E
d) Are assessment results used for career planning advisement?	1. Evidence of results being used.	Staff or third party review. Review advisement procedures. Survey counseling/advisement staff.	E E E
e) Are student career, educational and personal goals considered in the assessment/advisement process?	1. Evidence of existence of goal information collected from students and used for advisement.	1.1 Review student goal information collection instruments and related advisement procedures.	E
f) Are enough qualified staff available for interpretation and advisement regarding course selection, educational and/or career planning and referral services?	1. Comparison of requirements and staff qualifications.	1.1 Review same.	E

Criteria	Measures	Methods	E/A*
a) Are assessment results used to place students in appropriate classes?	1. Evidence of course placement advisement using assessment results.	1.1 Review procedures for course placement advisement.	E
	2. Comparison of students’ assessment results and course placement.	2.1 Random review of correlation between scores and placement.	E
b) Do course offerings, and curriculum match students’ needs identified in the assessment process?	1. Comparison of aggregated assessment scores with enrollments in courses.	1.1 Review statistics for congruence.	E
	2. Evidence of sequential courses to accommodate students’ basic skills needs.	2.1 Review same.	A
c) Are students’ test scores matched with course enrollment prerequisites?	1. Comparison of students’ course and program enrollments with their assessment score and advisement.	1.1 Review random sample of assessed students’ records.	E

Criteria	Measures	Methods	E/A*
a) Do you monitor the progress of those students who were assessed (e.g., for retention rates, performance)?	1. Evidence of a system monitoring student progress and performance.	1.1 Review student follow-up system.	E
	2. Comparison of retention and completion rates of assessed and non-assessed students.	2.1 Review statistics.	A
b) Do you disseminate summarized assessment program results (e.g., skill levels, aggregate student performance)?	1. Evidence of dissemination to appropriate staff.	1.1 Review dissemination process.	E
	2. Satisfaction of accuracy and completeness of information.	2.1 Survey appropriate staff.	E
c) Is assessment information used for planning of student services programs and curriculum?	1. Evidence of interpretation of information for use in planning.	1.1 Review procedures and activities.	E
	2. Evidence of use of student services personnel and instructional staff in developing new programs and services.	2.1 Review procedures and activities.	E

Criteria	Measures	Methods	E/A*
a) Is a Career/Life Program philosophy written and available to the public?	1. Evidence of philosophy written in college-wide literature.	1.1 Check college literature (e.g., catalog, brochures).	E
b) Was the philosophy developed and updated with wide participation?	1. Evidence of wide partici-pation and input into developing philosophy.	Review minutes and membership of related meetings. Interview staff to determine level of awareness of philosophy statement. Demonstrate evidence of wide participation and dissemination.	E E E
c) Do you have an annual operational plan which includes goals and objectives?	1. Evidence of operational plan.	List annual goals and objectives. Review year-end report.	E E

Criteria	Measures	Methods	E/A*
a) Is a complete range of services available (e.g., career orientation, assessment, goal setting, skill development)?	1. Evidence of all services available.	1.1 Inventory of all services.	E
b) Do students avail themselves of Career/ Life Services?	1. Level of service utilization.	1.1 Count numbers of clients using each type of service.	E
c) Are adequate professional staff and budget available?	1. Evidence of professional staff available.	Record staff currently available. Compare adequacy of staffing to client demand.	E
c) Are adequate professional staff and budget available?	2. Evidence of adequate budget available.	Review budget allocation. Compare adequacy of budget to that required by client demand.	E E
d) Are Career/Life Services located in an easily accessible appropriate facility?	1. Accessibility of services to clients.	1.1 Review campus traffic flow to assess accessibility.	E
	2. Proximity to related services.	2.1 Review site facilities plans to determine proximity to related services.	E
	3. Adequacy and quality of space.	Determine if space is sufficient to meet client demand. Survey opinions of staff and clients to determine satisfaction with space allotment. Determine if relationship of materials to facilities is appropriate.	E E E

Criteria	Measures	Methods	E/A*
a) Are skill develop-ment experiences provided?	1. Evidence of skill development experience.	1.1 Inventory skill development activities.	E
b) Do students partici-pate in skill develop-ment experiences?	1. Number of students utilizing service.	1.1 Count number of clients involved in skill development experiences.	E
c) Are skill develop-ment experiences delivered in a variety of formats?	1. Evidence of a variety of delivery modes.	1.1 Inventory formats for presentations (e.g., classes, seminars, individual contacts).	E
d) Do skill development experiences meet student need?	1. Evidence of adequacy of skill development experi-ences to meet student need.	1.1 Survey clients to determine relevancy of skill development experiences.	E
e) Are Career/Life materials available, relevant, compre-hensive, and current?	1. Evidence that Career/Life materials meet the educational career/ occupational and other personal needs of students.	Survey clients, staff, and advisory committees, and other experts in the field. Inventory client self-assessment instruments, (e.g., placement tests, interest inventories, value clarification). Evaluate materials in terms of: industry professional publications business and industry visitations copyright dates of materials gender equity labor market statistics other relevant measures	E E E
	2. Evidence that Career/Life materials are provided to other services.	2.1 Interview staff to determine whether appropriate materials provided.	E
	3. Evidence of state of the art hardware and software.	3.1 Inventory computerized equipment and software library.	E

Criteria	Measures	Methods	E/A*
f) Is the community aware of your Career/Life Services?	1. Evidence of developing awareness of Career/Life Services.	Inventory public information resources regarding Career/Life Services. List all activities relating to services (e.g., Career days, college days, guest speakers, seminars). Count number of non-students (community members) who use career assessment services.	E A E

Criteria	Measures	Methods	E/A*
a) Are there cooperative activities with other services and departments.	1. Evidence of cooperative activities.	Document organizational structure that facilitates coordination. Review master calendar of all coordinated activities and scheduled meetings. Review publicity regarding activities (e.g., classroom visits, joint projects, division meetings). Review coordination and referral efforts regarding job placement and work experience.	E E E E
b) Are advisory committees broadly based?	1. Evidence of b