Toolkit – Activation Lab

ACTAPP: THE ACTIVATION LAB EVALUATION TOOLKIT

This page will take you through the Activation Lab tools that you can use to evaluate your learning programs. We call this toolkit the “ActApp.” Go through our four steps to design your study and access the tools:

Step 1: Decide if the tools align with your evaluation questions
Step 2: Explore our Tools
Step 3: Using the Toolkit: a User’s Guide
Step 4: Use the Toolkit Now

What is the ActApp Toolkit?

The ActApp Toolkit has been designed to share tools from the Learning Activation Lab. The Activation Lab tools were designed to enable those who offer science, technology, engineering, art, and math (STEAM) learning experiences to evaluate and improve their programs using research-based tools.

I can create my own tools. Why should I use the ActApp Toolkit?

As an evaluator, or as a program leader who understands evaluation, you know that it is very time-consuming and challenging to design, test, and refine instruments. The ActApp Toolkit provides validated, research-based instruments ready for use.

Is the Toolkit right for my project?

Learning activation describes a state composed of dispositions, practices, and knowledge that enables success in future learning experiences.

Step 1: Decide if the Toolkit aligns with your evaluation questions

1. Description of the Tools

All of the tools are appropriate for pre/post designs, post-only designs, within-group, and comparison group designs. See the technical reports for information about its fit for different design types.

The contents of this Toolkit include tools that can produce quantitative and qualitative data. They may be used in a variety of combinations, depending on your specific evaluation questions and needs.

Surveys measure the dimensions of activation, engagement in activities, youth background or experience with STEM learning, and other critical elements that can measure or explain your program outcomes.

Observation forms track aspects of engagement (affective, behavioral, and cognitive) and can be used alone to get a sense of participant engagement, or in conjunction with survey data to measure observed engagement against self-reported engagement.

Interview guides can be used to understand the program from the perspective of the administrators, educators, and participants. Information gathered during interviews can help you ensure that program elements designed to activate youth were implemented as intended.

Using the tools alone or as an integrated suite of tools can help you tell a story about your program and make meaningful changes.

Step 2: Explore Our Tools

Now that you know more about the dimensions of activation, use the table below to identify which tools are right for your program. The tools can be used in any combination, or in standalone administrations (with the exception of the demographics survey, which is intended to accompany other survey instruments).

Available instruments include:

Science Fascination Survey
Science Values Survey
Science Competency Beliefs Survey
Scientific Sensemaking Survey
STEM Fascination Survey
STEM Values Survey
STEM Competency Beliefs Survey
STEM Innovation Stance Survey
Perceived Success Survey
Engagement Survey
Emerging Activation Survey
Choice Preference
Engagement Observation Form
Simplified Engagement Observation Form
Group Activity Observation Form
Demographic Survey

Step 3: Using the Toolkit: A User’s Guide

Each of the phases below are part of a research project. Now that you have identified if the tools are right for you, review the following phases to plan your research study.

Activation Lab staff are available to consult with you in multiple ways through the process. Contact us at actapp@activationlab.org to discuss options.

Phase I – Design Data Collection

First, you need to decide what evaluation question(s) you will ask, what population will be measured by the tools, when you will administer the tools, and any other logistics about collecting data.

Phase II – Deploy the Tools

Once you have your data collection plan lined up, you can begin collecting data by using the tools.

Phase III – Get data

Your data collection is complete! Now you need to compile your data to begin scoring and analysis.

Phase IV – Scoring

When you have your data in hand, they are ready to be scored. “Scoring” refers to getting the data in shape for analysis.

Phase V – Analysis

Your data collection is complete, and now you are ready to use it to better understand and effect change in your program.

Step 4: Use the Toolkit Now

You are about to enter a system that will allow you to access tools for your evaluation. Before you begin this process make sure you have reviewed the information above about each of the tool choices.

Before using the materials, you will need to know:

Which tool(s) you will need
If you are collecting information pre and post
Basic information about your study design, program content, program location, and program timing

Observation Forms

Engagement Observation Form and Instructions
Simplified Engagement Observation Form and Instructions
Group Activity Observation Form and Instructions

Need Help?

This material is based upon work supported by the National Science Foundation (Award No. 1337186). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.