What Evaluation Services Does BIG Offer?

Dr. Beth Covitt, Head of Science Education Research & Evaluation at BIG, can provide support in developing evaluation plans for grant proposals and in conducting project evaluation. Common evaluation types are described below. Other types of evaluation and evaluation of project elements other than broader impacts can be provided as well. Contact Beth Covitt to discuss evaluation services and options.

We can develop and implement evaluation…

• Plans and methods
• Logic models
• Timelines and scopes of work
• Data collection instruments and data collection
• Data analysis, recommendations, and reporting

What Are Common Types of Broader Impacts Evaluation?

Formative and summative are two big umbrellas in evaluation. Formative evaluations provide information that can support changes in the ongoing operation and implementation of projects. Summative evaluations focus on documenting and reporting project products, results, and impacts. Many projects utilize both formative and summative evaluation in concert. Some types of formative and summative evaluations include the following:  

Formative Evaluation                                     

• Implementation evaluation examines the process of implementing the project and whether the project is operating as planned.

• Progress evaluation assesses progress in meeting the project’s goals. It involves ongoing collection of information to measure the extent to which benchmarks for progress are being met and providing information for improving effectiveness during the life of the project.

Summative Evaluation

• Outcome or impact evaluation examines to what extent the project achieves its ultimate intended outcomes. Examples of outcome or impact evaluation questions can include things like:

• Did greater numbers of students from diverse backgrounds express interest in studying STEM disciplines in college?

• Did participants report increased STEM knowledge and skills?

• Did participants report increased engagement with STEM issues and activities in their communities?

How Much Do BIG’s Evaluation Services Cost?

If you propose to use BIG services for project evaluation, we can assist with creating the evaluation plan for your grant proposal at no cost. The budget for an evaluation will vary depending on the scope of work involved, but a basic package spanning from planning to reporting for a broader impacts project evaluation starts at about $6,500/year. Assuming similar services across years, budgets will change annually consistent with ORSP budget development guidelines.   

Evaluation Stories From BIG

Bitterroot Fisheries Program implemented by Dr. Andrew Whitely with 5^th grade students in Corvallis, MT. The Bitterroot Fisheries Biology Program is a broader impacts activity that Dr. Andrew Whiteley (Franke College of Forestry) has designed for Corvallis Middle School students as part of his NSF Career Grant studying fish genetics. The Bitterroot Fisheries Program provides fifth graders in Corvallis, MT with an introduction to the biology of local fish populations and methods used to study them, an opportunity to observe electrofishing, and a river snorkeling and aquatic insect sampling experience. Iterative, annual evaluation of this program includes: (1) pre and post surveys of students designed to measure their knowledge, motivations, connections to nature, and interests in STEM studies and careers; (2) interviews with teachers and other adult participants; and (3) a report summarizing findings and recommendations. Dr. Whiteley has used the evaluation information for NSF reporting and to refine the program, including through expanding to serve all fifth-grade students instead of a select group participating in an elective course. PHOTO: UM Wildlife Biology Associate Professor Andrew Whiteley (far right) leads a snorkeling field trip with 5th graders from Amanda Bestor's Corvallis Middle School class. The students donned dry suits to explore the underwater world of the Bitterroot River as part of this NSF-funded outreach educational program.

UM BRIDGES Project implemented by Dr. Laurie Yung and Dr. Andrew Wilcox with UM graduate students. UM BRIDGES is an NSF-supported Research Trainee (NRT) Project led by Dr. Laurie Yung (Franke College of Forestry) and Dr. Andrew Wilcox (Geosciences). This project trains UM graduate students as future leaders to advance societally relevant science toward more sustainable food-energy-water systems. Evaluation of this program includes multi-year formative and summative evaluation elements that track the experiences and project outcomes for graduate student cohorts and participating faculty. Evaluation data collection methods include surveys, interviews, and focus groups. Information provided by the evaluation has supported ongoing refinements to the project and is used in required NSF reporting. PHOTO: University of Montana BRIDGES students visit Grand Coulee Dam during a food, energy, water field trip in June 2018.

Bitterroot Summer of Science, Heman Foundation.  For over a decade, spectrUM Discovery Area has partnered with communities in the Bitterroot Valley to deliver STEM programming in settings both within and outside of K-12 schools. Evaluation of the Bitterroot STEM Education Partnership, which is supported in part by the Jane S. Heman Foundation, has been designed to provide both formative and summative findings. Evaluation questions focus on eliciting the experiences and perceptions of partners who make the programs possible and audiences who participate. Combinations of interviews and surveys are used to collect data, and each year a report is written to (1) articulate successes and challenges associated with the collaboration and programming, and (2) provide suggestions for changes, which emerge from evaluation responses. In addition, each year, evaluation questions are updated and revised to realign with the most current issues and questions facing the partnership. A guiding principle is to make sure that voices representing the spectrum of project stakeholders are accurately and adequately represented. PHOTO: Community members create structures and creatures with cardboard and makedo tools at spectrUM's Summer of Science at the Darby Public Library.