Step 1: Maintaining a Computerized Data System
The importance of gathering, reviewing, and reporting program data cannot be overemphasized. Policymakers, program administers, advocates, families, and funders will want data such as how many families are using information and referral resources; how many families are consequently receiving services; and what types of services are being received. A single-point-of-entry system should therefore collect the following:
- demographic information;
- how callers heard about the system;
- what concerns callers presented;
- what action(s) were taken;
- where referrals were made; and
- outcomes of the cases.
| The demographic information collected on Help Me Grow provides examples of data that are available through a single-point-of-entry client-tracking system. |
Step 2: Evaluating Impact
In addition to program tracking data, states should use other strategies for monitoring the extent of developmental surveillance and referral by health care providers, families, child care centers, and agencies. One strategy is to perform a periodic analysis of who is calling the access point and why they are calling. This analysis can help determine needs for ongoing community education and outreach efforts. Another strategy is to review State Children's Health Insurance Program (SCHIP) and Early Periodic, Screening and Diagnosis and Treatment (EPSDT) data to see the extent to which children are receiving well-child services that can serve as the starting point to their connection to community-based developmental support services. Child health care practices and other sites should be encouraged to review their own performance in monitoring development and referring children. Discussion of these strategies can be part of the training.
The ideal data set for evaluating the impact of a Help Me Grow–type of program would include long-term outcomes for children connected to services through the system. One goal of early intervention is to decrease the number of children requiring—and the intensity of—special education services in schools, and to improve school performance overall. To measure long-term success, states are also encouraged to monitor the number of individuals in need of social services that address emotional and behavioral well-being throughout the lifecycle.
In addition to gathering information on program processes and outcomes, states should collect data on training and outreach. Doing so helps to determine the extent of contacts with providers throughout the state and the development of a distribution list for program materials and updates. This database is essential for implementing a coordinated outreach effort to the state's child health care providers.
Prior to developing a training program, child health care providers and other target audiences should be surveyed to determine perceived barriers to monitoring development and making referrals to early intervention services. National data exist for pediatricians, but barriers reported by family physicians and mid-level providers should also be considered. State data also inform the development of training content by targeting issues specific to each state.
Survey data also should be collected at or shortly after each training session. Participants should be asked to complete questionnaires about their satisfaction with the training and the extent to which they found it useful in their practices. These data give the training staff real-time feedback, allowing them to refine the presentation and materials to meet the needs of target audiences.
The ongoing monitoring of data from these surveys provides an important framework for continuous quality improvement and general training evaluation. In Connecticut, the training program was pilot-tested in a few practices. Data about its impact and information gleaned from follow-up phone calls were used to revise the program prior to the statewide rollout.
Step 3: Using Data for Advocacy and Planning: Legislative Support
State agencies and other governmental bodies will want information on the use of the Help Me Grow system. Denise Merrill, co-chair of the Connecticut legislature's Appropriations Committee, explains: "Appropriations Committees deal primarily in numbers. The best testimonies are those that employ accurate and meaningful data....Statistics spell out an argument in black and white. Make sure that statistics used are...applicable [and] to check where the studies were done, when they were completed, and by whom. The most accurate and admissible information comes from non-partisan organizations. Government publications, academic journals, and university-sponsored research are good sources to cite. I cannot stress enough the power of numbers."
Step 4: Continuous Quality Improvement (CQI)
A mechanism and supporting data are needed to ensure that the system remains responsive to families, service providers, and program staff. CQI is an approach to quality management that builds upon traditional quality assurance methods by emphasizing the organization and systems. It focuses on "process" rather than the individual, recognizes both internal and external "customers," and promotes the need for objective data to analyze and improve processes.
The following links offer additional information on CQI: elements of CQI, and commonly used CQI tools and methods.
