Objectives

(a) The number of students completing associate degrees or completing credits toward transfer to a baccalaureate degree program in STEM fields will increase by 50% over the course of the grant. We will also double the number of Transfer Admission Guarantees in STEM majors.

See Fig. 1.

(b) Students who participate in the Precalculus Readiness Seminar will experience an average

success rate in Precalculus that is 20% higher than comparable math students who do not participate. In this way a success rate of 60% would increase to one of 80%.

(c) Students who participate in the Energy Laboratory Academy will have an average success rate in physics laboratory courses that is 20% higher than comparable non-participating students.
(d) Of the non-declared STEM majors in each of the two summer courses, 35% will subsequently declare themselves STEM majors.

(e) Over the course of the grant, the number of students from underrepresented groups who enroll in STEM programs will double from the level in 2006-07, by maintaining a steady increase of about 20% per year improvement from the previous year.

2006-07	2007-08	2008-09	2009-10	2010-11	2011-12
384	390	462	547	648	768

Fig. 7 Underrepresented STEM Majors by Academic Year: actual for 06-07; projected for subsequent years

(f) Students participating in STEEP will be more likely to persist in a STEM pathway after transfer than other STEM transfer students who were not in STEEP. This objective will be measured using data from the California Partnership for Achieving Student Success (Cal-PASS) data system.

MEASURING PROGRAM OUTCOMES

Our goal is to develop a succinct panel of key performance indicators that will allow us to assess the effectiveness of our outreach strategies, our student engagement strategies and our student learning & progress.

Indicator	Areas Assessed	Time Frame	Goal
Track STEM majors	Outreach	Annual	Increase number of participating students declaring a STEM major
Number of Transfer Admission Guarantees	Progress	Annual	Increase number
Number of STEEP students in each cohort	Outreach	Annual	Increase STEM majors, especially within underrepresented population
Focus groups	Engagement	Annual	Increase reported student engagement
Skills assessments	Learning	Twice per semester	Increase level of student mastery of course materials
Survey of student engagement & procedural knowledge	Engagement	Annual	Establish baseline and improve student engagement
Number of STEM units	Progress	Each semester	Increase average number of lifetime units and the rate of unit accumulation
Track number of transfers	Progress	Annual	Use the NSC (National Student Clearinghouse) database to identify former student participant destinations

Fig. 9 Key Performance Indicators for Summer Energy Lab Academy & Precalculus Preparedness Seminar

Assessment and Evaluation: Figure 9 gives a nice overview of what outcomes you plan to measure but little indication of how you will do so. How will you carry out and respond to formative assessment of the project; assess each of the activities as you go along so that they can be amended as needed?

The skills assessments, focus groups, survey of student engagement and tracking of the number, retention and persistence of student cohorts will all provide opportunities for multiple updates during the course of the year. It is anticipated that these data updates will provide formative information that may allow for mid-year adjustments and fine tuning of services.

Will all your assessment and evaluation be quantitative in nature?

The summative evaluation is largely quantitative but not all of the evaluation will be quantitative. For instance, there will be multiple opportunities for analyzing and responding to data about the processes during the academic year, before any formal annual, summative report. For instance, as annual cohorts are developed, descriptive statistics will be analyzed and reported. Any patterns of interest (e.g., a gender imbalance) could be pursued programmatically (e.g., increased targeted outreach efforts).

Will there be qualitative methods used as well to enable you to determine some of the reasons for increase or decrease in student recruitment, retention or graduation (including transfer to four year institution prior to earning the AA degree)?

The student focus groups and survey of student engagement and procedural knowledge will provide qualitative information that will shed light on student perspectives, experiences and rationales.

Also please clarify the roles of the Director of Research and Planning and of the independent evaluator.

For quantitative data, the Director of Research, Planning, and Knowledge Systems (DRPKS) will perform database queries and extracts as well as an initial review of descriptive statistics for key variables. The independent evaluator will receive the extracted flat files, review the data for accuracy, and complete the analysis with input and review from the DRPKS.

For qualitative data, the DRPKS and the Principle Investigator will design and implement surveys and focus groups, conducting any data input and screening, and perform initial analyses. The independent evaluator will consult on the design of qualitative assessments, assist with data screening, and finalize analyses.

The independent evaluator will attend periodic meetings and be available for phone and email consultation as the need arises. The independent evaluator will produce an annual report evaluating the data and information collected in support of grant objectives.