Finally, Affordable Objective program Evaluation
For more than a decade Creating Change has visited and evaluated literally hundreds of OST programs. We have taken that vast knowledge and created a highly objective and thorough process of evaluation that is simple to administer and powerful in the data provided. You will find three options in using this process from the simple raw data to through analysis and reporting and technical assistance. With strapped budgets and limited time, this is a very effective method that you can choose.
The process of evaluation is critical to
the foundation and continuation of program success in any endeavor, but
especially where the number one task that can be performed under the program is
academic progress. Under the guidelines
of the 21st CCLC program, the top priority is academic
enrichment. The federal documentation is
clear that the purpose of academic enrichment is to support the regular school
day. The purpose of the regular school
day is to help children achieve proficiency academically. To best assist students with academic
success, 21st CCLC programs can focus on a singular item that is
highly identifiable and concentrate the precious academic time on only this
item to maximize the opportunity.
The second federal priority of 21st
CCLC is additional educational opportunities.
It may most practical to consider this as everything except curricular
math and reading, and that includes everything morally acceptable with
students. The primary purpose of
additional educational opportunities is to attract and retain students to this
voluntary student opportunity that is 21st CCLC. Experience demonstrates that students will
not want to attend a program whose apparent purpose is to extend what students perceive
as school. These enrichment
opportunities are the things students want to do.
The third priority is family programming
and it is given a very broad definition.
The purpose here is to engage parents in the educational process with
their children with the research supported understanding that the more family
involvement - the better the student performs academically and socially.
All aspects of the program are focused on
these priorities, with additional efforts made toward other state or local
priorities that are centered in identified need.
Process - Standard fee of 2% ($1,500 minimum with no travel, $2,500 minimum with 1 site visit included) per grantee
per program year
Effective program evaluation then focuses
all efforts toward these established priorities. Effective programs (characterized by student
academic and social achievement) have evidence of vision, processes, procedures
and results that support the priorities.
These are categorized into 6 areas and evaluated:
These 6 areas are evaluated internally and
externally, subjectively and objectively, formatively and summatively - for one
singular purpose - program improvement.
While evaluation may also serve to determine funding levels and continuation
processes, those are really consequences of evaluation, not the purpose. Evaluation with the purpose of program
improvement should be highly objective and specific to identified tasks or
opportunities. If the purpose of
evaluation is program improvement, then expectation should accompany what is
done with evaluation by the recipient.
Expectation for the purpose of improvement should also be highly
objective and established by those who have stewardship for the support
provided to administer the program.
Internal evaluation will contain
self-assessment and survey data that will inform performance on a quarterly and
annual basis. The quarterly report
process will provide an external evaluator a method whereby to assess the
collected performance data against the reported process data. The combination of this information will
inform progress both summative and more importantly formatively.
Each of the six areas will be self-assessed
either annually, quarterly or monthly, on various elements from the list of
measured & weighted variables.
Directors will be encouraged to utilize their own process to gather
information so that they can answer the periodic self-assessments
accurately. This information will be
analyzed with data that will be reviewed. For OSTLMS clients the data is in the system
and review is done with no additional effort on the part of the director.
For all programs, the following will be
schedule including both academic and non-academic offerings
courses are offered?
is the balance of academic and enrichment?
there appear to be alignment to objectives?
attendance by course
might attendance rates tell us?
achievement data and student goals (number of identified students with a
student goals established with students working clearly toward those goals?
for the next month (indication of changes to be made; why, what & when)
there evidence that data gathered and analyzed to date is used to alter
programming toward improvement?
For all programs the following will occur
of monthly reviews
responded program self-assessment
review - draw downs and expenditures
audit against objectives - do courses align to objectives and to state
objectives and are attendance trends identified
operations - is staff status identified and are there plans to address what is
- assessment of the relationships between the program and school staff
For all programs, the following will occur
Assistance (Coaching Option - Cost is 2% of grant [$1,500 minimum]/program year plus travel if used)
of quarterly reports
From the data collected on the schedule
provided above, the areas of technical assistance will be identified and
provided to those in need. Where the
need is established a plan will be presented to the director for this topic to
be addressed either in a webinar, site visit or another effective forum. The process of support is evolutionary, specific
to each grantee and follow a process of effectiveness. For each grantee, the areas of need should be
identified, prioritized by the weights established by the state director, and
addressed accordingly. This option includes OSTLMS (our data management system for OST programs) along with the support you need ongoing to really optimize the program. Using OSTLMS also include the PPICS reporting process that we do for you.
Addressing needs that exists will include:
- Identification of the need
- Communicate this need with the program
director with the intent to assist in the development of a plan to fill the
- Establish the plan, outcome, benchmarks and timeline
- Provide needed support in accordance with the plan
- Monitor the progress according to the timeline and benchmarks
If you would like to use our Evaluation Process, please contact Stacey @ 208-467-6545 or email Stacey@creatingchange.com. We look forward to serving you.