Doing Improvement Research

Adaptive Integration
Learning how to integrate a change package into different settings. This may require significant changes in the intervention, and in the work settings into which it is being introduced, for the intervention to fit and achieve quality outcomes locally.

PDSA (Plan-Do-Study-Act)
A PDSA cycle is the basic method of inquiry in improvement research. It's a pragmatic scientific method for iterative testing of changes in complex systems. Each cycle (Plan-Do-Study-Act) is a mini-experiment in which observed outcomes are compared to predictions. Discrepancies between predictions and observed outcomes become a source of learning.

Process Map
A tool for visualizing the steps in a process that can assist an improvement team in identifying gaps, strengths, and opportunities for improvement.

Run Chart
A graphical display of some measured characteristic over time. Data from PDSAs are often displayed in run charts.

Solution System
An intervention that consists of multiple interrelated components that must mesh well together. Formally, such interventions have a systems character. For positive effects to occur reliably, coordinated improvements need to occur across all of the drivers that comprise the solution system. A material weakness in any one driver can undermine the efficacy of the overall solution.

Driver Diagrams & Working Theories of Practice Improvement

Aim Statements
An Improvement Aim is a goal for an improvement effort that answers the question, "What are we trying to accomplish?". Improvement aims should clearly specify how much, for whom, and by when. They sit at the far left end of a driver diagram.

Change Idea
A change idea is an alteration to a system or process that requires testing through a PDSA cycle. The PDSA results examine the change idea's efficacy in improving one or multiple drivers in a working theory of improvement.

Concept Framework [also may be referred to as design principle]
An account that provides conceptual detail and relevant research findings that form design principles for key drivers and change ideas. It also provides a conceptual basis for the development of practical measures.

Driver Diagram
A driver diagram visually represents a group's working theory of practice improvement. It creates a common language and coordinates efforts among many different individuals joined together in solving a shared problem.

Pareto Principle
A principle anchored in a long history of organizational studies that 80 percent of the variability in organizational performance is often associated with only 20 percent of the possible causes. It aids in the selection of primary, and when needed, secondary drivers.

Primary Driver
A primary driver is a representation of a community's hypothesis about the main areas of influence necessary to advance an improvement aim.

Secondary Driver
A secondary driver is a system component that is hypothesized to activate each primary driver. It is the "how" of change.

Working Theory of Practice Improvement
A small interrelated set of hypotheses about key drivers necessary for achieving an improvement aim and specific changes associated with each driver. It requires a creative blending of observations arising from the causal system analysis with relevant research that bears on this problem together with wise judgments from expert educators.

Evidence for Improvement

Evidence-Based Practice
Tools, materials, or sets of routines, typically grounded in theoretical principles, that have been subject to rigorous empirical study. Their use is warranted by results from a rigorous field trial that demonstrated the intervention can work because it has somewhere.

Know-How
The detailed practical knowledge necessary to get good ideas actually to work in classrooms, schools, and districts.

Level-A, -B, -C Learning
Three interrelated levels of learning, developed by Douglas Engelbart, that together form a schema for individual and organizational learning. Level-A learning represents the knowledge acquired by front-line workers as they engage in their practice. Level-B learning occurs across individuals within an organization. Level-C learning is orchestrated by a network hub and coupled with appropriate technologies to support rapid communications across distributed sites. This confluence is what enables the network to accelerate how it learns to improve.

Practice-Based Evidence
Evidence that grows from practice and can be used to improve it. This evidence, emerging from improvement research, demonstrates that some process, tool, or modified staff roles and relationships works effectively under a variety of conditions and that quality outcomes will reliably ensue. It is the evidence of know-how.

Improvement Science & Quality Improvement

Continuous Improvement
Improvement research that involves multiple iterative cycles of activity over extended time periods.

Improvement Research
Particular acts of inquiry, or projects, that aim for quality improvement.

Improvement Science
The methodology that disciplines inquiries to improve practice. Undergirding it is an epistemology of what we need to know to improve practice and how we may come to know it.

Quality Improvement
An effort to increase the capacity of an organization to produce successful outcomes reliably for different subgroups of students, being educated by different teachers, and in varied organizational contexts.

Solutionitis
The tendency to jump quickly on a solution before fully understanding the actual problem to be solved. This behavior results in incomplete analysis of the problem to be addressed and fuller consideration of potential problem-solving alternatives. It is siloed reasoning—seeing complex matters through a narrow-angle lens—that can lure leaders into unproductive strategies.

Measurement

Balancing Measure
A measure that helps improvers keep an eye on the other parts of the system that are not currently the target of improvement but nevertheless may be affected by the changes being pursued.

Lagging Outcome Measure
A measure that is available only well after an intervention has been initiated.

Leading Outcome Measure
A measure that predicts the ultimate outcome of interest but is available on a more immediate basis.

Measurement for Accountability
A broad, general measure that aims to sort individual or organizational units into performance categories. This use introduces increased formality into the measurement process and places primacy on the reliability of individual scores. Often used as a measure for improvement aims.

Measurement for Improvement
A measure that links directly to the specific drivers and work processes that are the object of change. It provides evidence for testing changes and examining hypothesized causal connections in the working theory of improvement.

Measurement for Research
A detailed measure developed by researchers to represent particular theoretical constructs. It is used in academic research to test relational propositions among key constructs that form a theory. It is often useful as a basis for developing practical measures.

Outcome Measure
A measure that operationalizes the aim statement in the driver diagram. These data provide a way of assessing whether progress is being made on the specific problem to be solved. Accountability measures are often used here.

Practical Measurement
Data to inform improvement that is embedded in regular work. Since the intent is to inform continuous improvement, practical measures are collected frequently to assess whether positive changes are in fact occurring. Since the focus is on specific populations and contexts, the measures are framed in a language that is natural and comprehensible to those asked to respond to them.

Primary Driver Measure
A measure associated with primary drivers. Since these drivers are intermediate outcomes in the working theory of improvement (i.e., intermediaries between process changes and leading and lagging outcomes), they play a key role in the testing of a working theory of improvement.

Process Measure
A measure that feeds back valuable information about how specific processes being tested are performing under different conditions.

Networked Improvement Communities

Networked Improvement Community (NIC)
A networked improvement community is an intentionally designed social organization with a distinctive problem-solving focus; roles, responsibilities, and norms for membership; and the maintenance of narratives that detail what it is about and why affiliating with it is important. A NIC is marked by four essential characteristics. It is:

• Focused on a well-specified common aim
• Guided by a deep understanding of the problem, the system that produces it, and a shared working theory to improve it
• Disciplined by the methods of improvement research to develop, test, and refine interventions
• Organized to accelerate diffusion of interventions out into the field and effective integration into varied educational contexts

Colleagueship of Expertise
A colleagueship of expertise is a community of academic, technical, and clinical experts deliberately assembled to address a specific improvement problem. All involved are improvers seeking to generate strong evidence about how to achieve better outcomes more reliably.

Meta-Network
A Networked Improvement Community formed from many other Networked Improvement Communities, each one with its own aim statement and driver diagram, all working towards some larger aim (e.g., 'improve literacy') from a variety of fields.

NIC Charter
A NIC charter is a document that provides sustaining guidance to the distributed efforts of NIC members. It is typically composed of the network's aim, causal system analysis (often consisting of both fishbone diagrams and a system improvement map), and the working theory of improvement (typically represented in a driver diagram).

Network Hub
A network hub is a core group formed either as a single organization or distributed across network members that carry out critical functions necessary for the support and effective operations of a networked improvement community. These functions include, but are not limited to, improvement science expertise, analytics, knowledge management, convenings, communications, and technological support.

Network Initiation Team
A network initiation team is a team that accepts responsibility for the formation of a networked improvement community. It leads a set of processes that articulate the problem to be solved, analyzes the system that produces current undesirable outcomes, and develops the aim statement and initial working theory of practice improvement. The initiation team also takes the lead in securing the necessary supports for the network (both political as well as material), recruiting initial members into the community, and engaging the academic and technical expertise relevant to the specific problem to be solved.

NIC of NICs
A single Networked Improvement Community formed from many other Networked Improvement Communities, all working on a unified aim statement from a shared driver diagram.

NIC as a Scientific Community
An organization ordered around a shared theory and shared measures for its aim and primary drivers. Participants engage in disciplined inquiries using established inquiry methods such as PDSA cycles. Promising results are subject to replication across the network to warrant claims that changes are improvements.

Standard Work & Processes

High-Leverage Process
A process that has the following properties: (1) it consumes substantial resources, especially teacher or student time; (2) its execution and outcomes may vary considerably; and (3) there are reasons to believe that changes to it might improve resource efficiency and effectiveness.

Macro Process
A process for which the execution typically entails a sequence of more discrete micro processes.

Micro Process
An elemental activity or segment of work taken to achieve a particular end.

Standard Work
Regularly occurring processes that are amenable to formulation as best practice routines within a networked improvement community. The purpose of these routines is to assist educators in carrying out their work by reducing the cognitive load associated with the performance of complex tasks. Such processes reduce undesirable variation in performance and free educators to better focus their attention, thereby advancing quality outcomes more reliably. High-leverage processes are attractive candidates for standard work.

Systems

Causal System Analysis
An analysis that directs attention to the question, "Why do we get the outcomes that we currently do?" In working through this analysis, participants develop a shared understanding of the specific problem(s) they are actually trying to solve. The process also provides a first test as to whether a team seeking to initiate a NIC can engage together productively as a focused improvement community.

Fishbone Diagram - Ishikawa Diagram, Cause-and-Effect Diagram
A tool that visually represents a group's causal systems analysis (sometimes known as a cause-and-effect diagram or an Ishikawa diagram).

System
An organization characterized by a set of interactions among the people who work there, the tools and materials they have at their disposal, and the processes through which these people and resources join together to accomplish goals.

System Improvement Map
An analytic tool that represents what we learn through the causal system analysis about the different organizational levels (e.g., classrooms, schools, and districts) and key organizational subsystems (e.g., human resources, finance, instruction) relevant to solving the identified problem.