Mission Statement

The first stage of the program is to understand the history, current position and future outlook relating to collaborative evaluation, not just for the organization as a whole, but for each individual department, including: customer service; e-business; finance; globalization; human resources; information technology; legal; management; marketing; production; education and logistics. This will be achieved by implementing a process within each department, enabling the head of that department to conduct a detailed and thorough internal analysis to establish the internal strengths and weaknesses and the external opportunities and threats in relation to collaborative evaluation and to establish a MOST analysis: Mission; Objectives; Strategies; Tasks, enabling them to be more proactive about the way in which they plan, develop, implement, manage and review collaborative evaluation, within their department.

 

---

Objectives

01. Obtain a clear understanding of the core objective of Workshop 1. Time Allocated: 1 Month
02. Analyse the history of Collaborative Evaluation within your department. Time Allocated: 1 Month
03. Analyse the current position of Collaborative Evaluation within your department. Time Allocated: 1 Month
04. Analyse the future outlook of Collaborative Evaluation within your department. Time Allocated: 1 Month
05. Analyse internal strengths and weaknesses, relating to Collaborative Evaluation, within your department. Time Allocated: 1 Month
06. Analyse external opportunities and threats, relating to Collaborative Evaluation, within your department. Time Allocated: 1 Month
07. Identify and engage up to 10 Key Stakeholders within your department. Time Allocated: 1 Month
08. Identify a process that would enable your stakeholders to decentralize Collaborative Evaluation. Time Allocated: 1 Month
09. Estimate the likely costs and the ongoing financial budget required for this process. Time Allocated: 1 Month
10. Estimate the likely hours and the ongoing time budget required for this process. Time Allocated: 1 Month

 

---

Strategies

01. Each department head is to personally set aside time to study Workshop 1 content thoroughly.
02. List the key projects that have been undertaken historically within your department and analyse how and if Collaborative Evaluation was used and where it was successful.
03. List the key projects that are currently being undertaken within your department and analyse how and if Collaborative Evaluation is being used and where it is being successful.
04. List the key projects that are scheduled to be undertaken in the future within your department and analyse how Collaborative Evaluation can be used in order to ensure success.
05. Research internal strengths and weaknesses, relating to Collaborative Evaluation, within your department.
06. Research external opportunities and threats, relating to Collaborative Evaluation, within your department.
07. Review the files and resumes of employees within your department in order to identify those with a collaborative nature.
08. Research and identify a process that would enable your stakeholders to decentralize Collaborative Evaluation.
09. Liaise with the Finance department to evaluate the likely costs and the ongoing financial budget required for this process.
10. Liaise with the Human Resource department to evaluate the likely hours and the ongoing time budget required for this process.

 

---

Tasks

01. Read through the entire workshop content while making notes including: Profile; MOST; Introduction; Executive Summary; Curriculum; Distance Learning; Tutorial Support; How To Study; Preliminary Analysis; Course Manuals; Project Studies; Benefits.
02. Create a task on your calendar, to be completed within the next month, in order to list and analyse historical projects.
03. Create a task on your calendar, to be completed within the next month, in order to list and analyse current projects.
04. Create a task on your calendar, to be completed within the next month, in order to list and analyse future projects.
05. Create a task on your calendar, to be completed within the next month, in order to research and analyse internal strengths and weaknesses, relating to Collaborative Evaluation, within your department.
06. Create a task on your calendar, to be completed within the next month, in order to research and analyse external opportunities and threats, relating to Collaborative Evaluation, within your department.
07. Set up interviews with employees within your department in order to identify those with a collaborative nature.
08. Implement a process that will enable your stakeholders to decentralize Collaborative Evaluation.
09. Set up an appointment with the Finance department to evaluate the likely costs and the ongoing financial budget required for this process.
10. Set up appointment with Human Resource department to evaluate the likely hours and the ongoing time budget required for this process.

 

---

Introduction

Workshop Objective

The first stage of the program is to understand the history, current position and future outlook relating to collaborative evaluation, not just for the organization as a whole, but for each individual department, including: customer service; e-business; finance; globalization; human resources; information technology; legal; management; marketing; production; education and logistics. This will be achieved by implementing a process within each department, enabling the head of that department to conduct a detailed and thorough internal analysis to establish the internal strengths and weaknesses and the external opportunities and threats in relation to collaborative evaluation and to establish a MOST analysis: Mission; Objectives; Strategies; Tasks, enabling them to be more proactive about the way in which they plan, develop, implement, manage and review collaborative evaluation, within their department.

 

---

Collaborative Evaluation systematically invites and engages stakeholders in program evaluation planning and implementation. Unlike “distanced” evaluation approaches, which reject stakeholder participation as evaluation team members, Collaborative Evaluation assumes that active, on-going engagement between evaluators and program staff, result in stronger evaluation designs, enhanced data collection and analysis, and results that stakeholders understand and use. Collaborative Evaluation distinguishes itself in that it uses a sliding scale for levels of collaboration. This means that different program evaluations will experience different levels of collaborative activity. The sliding scale is applied as the evaluator considers each program’s evaluation needs, readiness, and resources. While Collaborative Evaluation is a term widely used in evaluation, its meaning varies considerably. Often used interchangeably with participatory and/or empowerment evaluation, the terms can be used to mean different things, which can be confusing. The processes use a comparative Collaborative Evaluation Framework to highlight how from a theoretical perspective, Collaborative Evaluation distinguishes itself from the other participatory evaluation approaches.

Collaborative processes are being promoted as an alternative decision-making process for managing. This is a relatively recent phenomenon, and, given its growing popularity, it is important to develop and apply methods and criteria for evaluation, to determine strengths and weaknesses, and to identify best practices for effective use of the collaborative model. Evaluation based on multiple criteria and at several points in time can assist those involved in designing and organizing collaborative processes to ensure the process is responsive to stakeholders’ and achieves its objectives. The success of both the process and the outcome of collaborative processes can be effectively appraised using participant surveys.

Evidence from case studies of collaborative approaches show these processes can generate higher quality, and more creative and durable agreements that are more successfully implemented due to increased public buy-in and reduced conflict. Collaboration can generate social capital, by facilitating improved relationships between stakeholders, generating new stakeholder networks, enhancing communication skills, and co-producing new knowledge with stakeholders. However, collaborative processes are a relatively recent phenomenon, particularly when compared with historical planning and decision-making processes.

“Is our program working?” This is a key question in education today, particularly in this era of heightened accountability. A collaborative program evaluation model is an extremely useful way to answer this question when education organizations want to find out if their initiatives are achieving the intended outcomes, as well as why this is the case. In the collaborative program evaluation model, the client (e.g., districts, states, public and independent schools, non-profits, and foundations) works with the external evaluator to determine the questions that will be explored through the evaluation. They continue to work collaboratively to ensure that the context is understood, that multiple stakeholder perspectives are taken into account, and that data collection instruments are appropriate in content and tone. The model produces data that can proactively inform program implementation, provide formative information that supports program improvement, and offer summative information on the effectiveness of the program.

Collaborative evaluation is a proactive evaluation model that enables program staff to engage in continuous program improvement. Specific benefits of the model include

A customized evaluation design that reflects the nuances of the program being evaluated.

An evaluation design that is flexible and adaptable to the purposes of the evaluation and to changes in program implementation over time.

Increased reliability of results.

Greater buy-in among stakeholders with both the data collection process and the evaluation findings.

Development of program staff’s capacity to continue to monitor their progress toward program goals beyond the duration of the evaluation.

Development of a culture of inquiry among program staff.

Potential cost efficiencies.

Each of these benefits is described in detail below:

Address program nuances

All evaluators should tailor evaluation services to the needs of each client (Patton, 2002). In the collaborative evaluation model, this is accomplished by evaluators working closely with program staff to identify evaluation questions and engage in an evaluation process that is attuned to the needs of program staff and stakeholders. As a result of the close knowledge built through collaborative program evaluations, such studies also guide program staff to identify and capitalize on external and internal program networks that they can tap to help them to achieve program goals (Fitzpatrick, 2012).

Flexible design

In a collaborative evaluation, continuous communication at the outset between program staff and the evaluation team is essential for laying the groundwork for mutual understanding. Ongoing communication is also a key ingredient for ensuring that the evaluation plan continues to be relevant to the program. By communicating regularly about program developments and context, evaluators can make adjustments in the evaluation plan to accommodate changes in the program.

Increased reliability of results

Another benefit of working collaboratively with program staff in developing the evaluation is increased reliability of the study. Because the evaluation team develops a high level of understanding of the program, data collection can be designed to accurately capture aspects of interest, and appropriate inferences and conclusions can be drawn from the data that are collected.

Greater buy-in for results

Engaging an experienced outside evaluator alone increases the reliability of the study and the credibility of the findings. The use of a collaborative program evaluation also improves buy-in for the study’s results from a variety of stakeholders. Staff members who actively participate in the evaluation better understand how the results can be used to facilitate program improvement, while administrators and other decision makers are more likely to have confidence in the results if they are aware that program staff helped inform elements of the evaluation study (Brandon, 1998).

Increased ability to monitor progress

The evaluation team works with program staff to develop tools to measure desired outcomes of the program. Because tools are designed in collaboration with program staff, staff are better able to understand the purpose of the tools and what information can be gleaned from each. This makes it more likely that staff will feel comfortable with and use the instruments to collect data in the future to monitor ongoing progress, an added benefit to the client.

Development of a culture of inquiry

Because use of evaluation results is a primary goal of collaborative evaluation, the evaluation team may also facilitate a process in which practitioners examine data on program implementation and effectiveness throughout early stages of the evaluation. This process of reviewing evaluation results can foster the development of a culture of inquiry among program staff and support the goal of continuous improvement.

Potential cost efficiencies

There are several ways that a collaborative program evaluation can reduce costs in the short term and over time. There can be immediate cost savings because evaluation resources are tightly coupled with the program’s stage of development. The model can help avoid costly data collection strategies and analytic approaches when there is little to measure because the project is in a nascent stage of implementation. Cost savings may also emerge over time because of program improvements based on formative feedback. Additional savings may be found as the evaluation team develops the internal capacity of program staff through their active participation in the design and execution of the evaluation. With increased capacity, the program staff can then continue the progress monitoring process by themselves.

The collaborative evaluation process incorporates four phases: planning; implementation; completion; and dissemination and reporting. These complement the phases of program development and implementation. Each phase has unique issues, methods, and procedures.

 

---

Collaborative Evaluation – History

Most companies or organizations want to attain sustainable, successful goals for themselves, their employees and communities of interest. But more often than not, they have difficulties achieving them. It’s the “getting there” that’s hard. This corporate training program features a process-driven system that can help. If one is committed to and serious about reaching their goals, there is almost an inevitability about succeeding. It is based on effi