Introduction

Definition of Coordinated School-linked Services

Purpose and Method

Results

Discussion and Conclusions

Tables

References

The research reported herein is supported in part by the Office of Educational Research and Improvement (OERI) of the U.S. Department of Education through a grant to the Mid-Atlantic Laboratory for Student Success (LSS) at the Temple University Center for Research in Human Development and Education (CRHDE). The opinions expressed do not necessarily reflect the position of the supporting agencies, and no official endorsement should be inferred.

Public schools in the 1990's are challenged to serve large numbers of children in a variety of at-risk circumstances. Poverty, lack of employment, disorderly and stressful environments, poor health care, children born to children, violence, substance abuse, and highly fragmented public services threaten the quality of life of these children and their families. Despite, or perhaps because of, these immense challenges, the educational reforms of the 1990's have been marked by more coherence than earlier waves of school reform. National education goals, content and school performance standards, state-level curriculum frameworks, and new forms of assessment have all served as systematic tools to reinforce reform efforts and improve our nations schools. No tool, however, has promoted systemic reform more completely than coordinated school-linked services. Coordinated school-linked services require service providers to harness the resources of the family, school, and community to create an environment that supports students learning by meeting the physical and social wellness needs of students and their families. Inherent in the concept of school-linked services is a new perspective that recasts childrens "academic," "physical," or "psychological" problems into "cases" of children and families with a variety of needs. Labels and artificially separate categories of problems are replaced with a broad-based framework that redefines client needs and increases interprofessional collaboration. Students and families facing adversity benefit from increased access to the range of services provided in collaborative arrangements. Public schools, especially those serving families in at-risk circumstances, are candidates for school-linked services. When school-linked services are successfully implemented, communitarian values replace concerns for bureaucratic expediency, which have been the predominant focus of efforts to improve social service delivery and school operations. School improvement efforts of the past three decades, for example, have bureaucratized public schools, which, in turn, has contributed to excessive regulation and a sense of isolation among teachers. Traditionally, teachers have been cut off from other teachers and certainly from other professionals in detecting student problems and offering solutions. While teachers understandably place students educational problems at the top of their list of concerns, other issues such as students medical, financial, and legal needs often do not receive attention.

Several terms have been coined over the past decade to refer to programs that bring together the resources of the community, family, and school to meet the needs of children and their families. Among these terms are:

• school-linked services
• coordinated services
• collaborative services
• integrated services
• school-family-community connections
• school-family-community partnerships.

In this research, school-linked services are defined as: a system of inter-related resources that links schools, families, and public and private service agencies. These programs may engage health, social service, housing, and law enforcement agencies; transportation services; local businesses; institutions of higher education; libraries; museums; and religious institutions. The programs are designed to:

• increase access and delivery of school and community resources to students and families
• radically improve students learning and social and physical development
• enhance professionals skills in assisting families to identify and use available resources, and
• promote resilience among children and families in adverse life circumstances.

Many professional groups agree that the problems of at-risk children and their families cannot be tackled by our schools alone (Council of Chief State School Officers, 1989). Kirst (1991a) admonishes that schools no longer are able to exist in splendid isolation relying on their school boards and separate property taxes to guarantee the well-being of students and programs. Rather, broader social policies must be established to protect the nations at-risk children and their families. One response to the call for broad social policies has been the establishment of interagency, coordinated programs that link schools and human service agencies. This is not a new response.

Since the 1890's, improving the plight of at-risk children has been one of the goals of school systems in the United States (Tyack, 1992). For over 100 years reformers have advocated using schools as a base from which a number of social ills could be remedied. School-linked services have waxed and waned in popularity as a means of meeting the needs of at-risk students and their families. Reformers in the 1890's campaigned for medical and dental examinations, school lunches, summer academic programs, recreational activities, and school-based child welfare officers. Many of the health-oriented programs of the 1890's were based on a philosophy of improving the human capital of the nations children and ensuring equal educational opportunity. However, because reformers doubted the capacity of immigrant parents to provide for all their childrens needs, they rarely sought input from parents as they designed and implemented these new services.

By the end of the 1930's, school-linked, coordinated services were entrenched in our nations public schools. Over the next 30 years, successful service programs received support from influential community groups, meshed with prevailing instructional approaches, and met some of the needs of poor, at-risk children. However, services were delivered best in wealthy communities with large property tax bases. Thus, both the children of the wealthy and the poor became recipients of coordinated interagency services originally intended only for students in adverse circumstances that placed them at risk.

During the late 1970's and early 1980's the role of the schools shifted toward producing students who could compete in the global marketplace and maintain the nations competitiveness. This shift combined with significant budget cutbacks reduced some of the social services provided. In spite of these reduced services, Tyack (1992) reports that teachers accounted for 70% of all school employees in 1950 but only 52% of school employees by 1986, indicating schools had become multipurpose institutions that looked beyond only the academic performance of their students. Modern day reformers, like their earlier counterparts, look to school-linked services as an avenue to desperately needed resources, especially for children in depleted urban and rural neighborhoods.

Lisbeth Schorr (1988), in her book Within Our Reach: Breaking the Cycle of Disadvantage, took up the argument for coordinated school-linked services. She argued that complex social problems could be ameliorated through collaborative social programs. Schorr presents research supporting the use of intensive, comprehensive services that address the needs of the "whole" child and the community in which the child lives as a means to breaking the cycle of disadvantage. She argues that the risks besetting many of our nations children require a societal response and not simply a response from the child or family placed at-risk.

Schorrs book was one of the first of many calls for coordinated school-linked services to supplement the schools role in society. (See Behrman, 1992; Hodgkinson, 1989; Levy, Kagan, & Copple, 1992; National Association of State Boards of Education, 1991; and National Commission to Prevent Infant Mortality, 1991.) Further evidence of collaboratives rising popularity was reported in the 24th annual Gallop Poll which found that seventy-seven percent of adults polled favored using schools as centers to provide health and social welfare services by various government agencies ("Public in poll backs," 1992). The compelling appeal for school-linked services from child advocates, in tandem with the publics enthusiasm for these reforms, propelled the educational research and evaluation communities into addressing the effectiveness of these programs using a limited database. As Knapp (1994) points out, eagerness to implement new social interventions often overrides available evidence.

The knowledge base on school-linked services is growing, but the number of rigorous research results available is still small. Narrative reviews describing the status of existing programs have been compiled by Crowson & Boyd (1993), the United States General Accounting Office (1993), and Gomby (1992). Rigsby, Reynolds, & Wang (1995) described the policy debate, organizational issues, social context, and results of several studies of school-linked services. Surveys of programs in action have described the prevalence, scope, nature, and characteristics of services rendered (Kagan, Rivera, & Lamb-Parker, 1990; Chang, Gardener, Watahara, Brown, & Robles, 1991; American Public Welfare Association, 1992; and Driscoll, Boyd, & Crowson, 1994). Knapp (1994) identified several evaluations of state-wide initiatives (State Reorganization Commission, 1989; Wagner, Golan, Shaver, Newman, Wechsler, & Kelley, 1994) and a multiple program comparison study (Marzke, Chimerine, Morrill, & Marks, 1992). Much of the extant knowledge base, however, is composed of case studies, single project evaluations, and descriptions of demonstration projects (Arvey & Tijerina, 1995; Mickelson, Yon, & Carlton-LaNey, 1995; Zetlin, Ramos, & Valdez, in press). The complexities of evaluating the programs have been addressed by Gomby & Larson (1992), Gray (1993), Family Impact Seminar (1993), Knapp (1994), Stake (1986), Weiss & Jacobs (1988), and the U.S. Department of Education and American Educational Research Association (1995). Because the knowledge base describing the effectiveness of school-linked services is limited, this paper presents descriptive and explanatory findings about the implementation and effects of school-linked services.

Kirst (1991b) expresses concern that many of the innovative programs spawned during the 1980's and 1990's have not provided evidence of replicable, long-term, beneficial effects for students. As Knapp (1994) points out, eagerness to implement new social interventions often overrides available evidence.

Purpose

While there is a paucity of longitudinal studies to document the effects of innovative programs, "snapshot" studies, attempting to describe particular educational programs or interventions, have been overused. The lack of robust, empirical results documenting the immediate, intermediate, or long-term impact of these innovations has produced an information gap for educators and policy makers. Little systematic, empirical information has been collected depicting the range of characteristics, practices, and relative effects of coordinated school-linked services for children and their families.

This paper integrates results of a cross-program analysis examining six coordinated school-linked service programs and results of a quantitative synthesis of 44 studies of the effects of school-linked programs on childrens cognition, affect, and behavior.

Method

The literature searches and procedures used to conduct the cross-program analyses and the quantitative synthesis are described below.

Cross-Program Analyses. A literature search was conducted to identify case studies, program descriptions, and evaluations of K-6 coordinated school-linked service programs. In order to obtain information on current practices, the search was focused on publications between 1988 and 1993. Twelve cases were located, and six contained data of sufficient detail to permit analysis of features contributing to program success or failure. The six cases became the analytic sample.

Elementary, school-linked services programs selected for inclusion represented single versus multi-site programs and private versus publicly sponsored programs. Of the six cases selected, one was a school district sponsored program which had a single site; two were state-sponsored, multi-site programs; the fourth was based at an alternative school, which was a single site. The remaining two cases were privately sponsored, one being a single-site program and the other a multi-site program.

Data on the six programs were organized using the seven dimensions listed in Table 1. The dimensions were identified from a review of selected background articles on collaboration and features of effective organizations (see Table 2). Cross-program analyses were carried out to determine commonalities and unique features of the programs along each dimension. The cross-program analyses were designed to reveal the characteristics and implementation processes that lead to program success.

Quantitative Synthesis. The purpose of the quantitative synthesis was to identify key characteristics of coordinated school-linked services and their effects on student outcomes. A comprehensive literature search was conducted of researcher and practitioner journals in education, psychology, public health, public policy, and social services. In addition to the literature search, 45 organizations were contacted to identify fugitive documents that were available only from the agency sources and were not yet available in libraries. A final corpus of 44 studies of coordinated school-linked services were identified for inclusion for the research synthesis study, based on three basic criteria: (a) studies reporting programs implemented between 1983-93, (b) programs that served preschool through high school students, and (c) reported student outcome data.

Table 3 identifies the ten features that were coded for each study. Based on the information coded, some general characteristics of studies of school-linked services were drawn.

To describe the current state of practice in school-linked service programs, information was integrated from the quantitative synthesis and the cross-program analysis. The characteristics, daily practices, effectiveness, and evaluation practices of coordinated school-linked programs are presented below.

Characteristics and Scope of Coordinated School-linked Service Programs

In order to describe current coordinated service programs, this research examined the 44 quantitative studies of school-linked service programs and six cross-program case studies.

Programmatic areas. The 44 studies in the quantitative synthesis and the six cases used in the cross-program analyses were categorized into one of six programmatic areas. Table 4 presents the six programmatic areas, the number of studies in each, and the number of programs for which results were reported. Many of the coordinated service programs in these 44 studies focused on particular goals, such as substance abuse or parent involvement, rather than addressing all the students and families needs. In contrast, the six cross-program case studies depicted programs that address the full range of students and families needs. This paper presents findings about both types of school-linked services those that addressed multiple needs and those that addressed a more narrow range.

Targeted clients. Most of the coordinated school-linked programs from the 44 quantitative studies were targeted for urban, low-achieving, and economically disadvantaged children, youth, and their families. In contrast, three programmatic areas, dropout prevention, teen pregnancy prevention and parenting, and the alcohol and drug abuse and prevention programs, focused on all children and youth. The six K-6 cross-programs also served low-income and low achieving students and families.

Program goals. The school-linked service programs from the 44 quantitative studies focused on different goals. Parent education and school readiness, teen pregnancy prevention and parenting, dropout prevention, and parent involvement programs all focused resources on improving students academic achievement. In addition, many of these programs aim to enhance parental competencies, family literacy, and child development and to refer students for mental health and health services. Selected programs such as teen pregnancy and chemical dependency had more narrowly defined goals associated with the programs special emphasis (for example, providing information about alcohol and drugs). The six cross-programs in the case studies addressed the multiple needs of students and families, including academic achievement, but their primary goal was to provide direct access to services to meet these needs, not just information about obtaining these services. This direct access characteristic was the reason for the inclusion of the six cross-programs in this study.

Curriculum vs. Service Orientation. In most of the coordinated school-linked programs in the 44 quantitative studies, both curriculum and services were offered as part of the programmatic intervention, but the emphasis was often on curriculum combined with a narrow range of services most often a referral for services rather than direct access to such services. The curriculum presented in most coordinated school-linked service programs provided knowledge and new skills in the programs area of emphasis. Services provided most typically were health care screenings, transportation to appointments, and counseling. Parent involvement programs relied primarily on curricular interventions. In the six cross-programs, curricular interventions were not emphasized. Direct access to a full range of services was emphasized instead.

Collaborators. In all the school-linked programs from the 44 quantitative studies, the most frequent collaborators were schools, families, and social and health care workers. A supportive but less central role was played by universities, private foundations, nonprofits, the media, religious institutions, law enforcement agencies, and the business community. Peers played a key collaborative role in modeling refusal and coping skills and in distributing current information in the alcohol and drug abuse and prevention programs. In comparison, the six cross-programs included a greater number of collaborators addressing a wider range of students and families needs.

Practices of Coordinated School-linked Service Programs

The cross-program case studies were analyzed to find widely implemented practices. Only the six cross-program studies were analyzed because many of the 44 quantitative studies did not provide sufficient detail about their day-to-day practices. Thirty-six day-to-day practices were mentioned in the six cross-program case studies. Of these 36, only 18 were mentioned in three or more cases and were considered to be widely implemented. Table 5 presents the 18 widely mentioned practices.

Many of the frequently mentioned practices were related to identifying and appropriately serving client needs. Identifying client needs requires distinguishing among the problems of community members, families, and students and devising site-specific approaches to meeting their needs. Generic goals for coordinated school-linked service programs which are insufficiently tailored to local community characteristics, culture, and needs are not likely to lead to effective planning or motivate reticent providers and clients.

A second cluster of frequently mentioned practices was focused on the collaborative process itself. Inclusive procedures planning, managing operations, interacting frequently about the program, identifying common outcomes are the building blocks of coordinated service programs. Without these shared procedures, the services may be co-located, but there is little integration or coordination of services and little sharing of service providers expertise.

Of the 18 frequently-mentioned practices, about one-third were family related issues such as including family as clients, identifying family goals, sensitivity to families cultural backgrounds, providing outreach, making services available at times when family members can attend (after school, weekends), and distinguishing among student, family, and community needs. The frequent mention of family related issues in the six cases is an endorsement for family participation in school-linked service programs. The case studies provide evidence that experienced practitioners regard family participation as a necessary condition for successful service delivery and treatment outcomes. While the six cross-program sites provided key information about day-to-day practices, the 44 quantitative studies provided systematically collected, objective evidence of program effectiveness.

Effectiveness of Coordinated School-linked Service Programs

The 44 quantitative studies provide evidence of the overall effectiveness of school-linked service programs, attainment of specific program goals, and the programs impact on student cognition, affect, and behavior.

Overall Effectiveness. Each of the 44 sources identified in the literature search was categorized into one of the six program areas, coded, and all of its outcomes were analyzed. Among the program outcomes examined were student cognition, affect, and behavior, number of services delivered, new knowledge and skills acquired by family members, reduced drug usage, and reduced teenage pregnancy rates. Results of the quantitative synthesis indicate that of the 176 outcomes reported in the 44 sources, 140, or 80%, indicated school-linked services programs produced positive results; 20, or 16%, reported no evidence of change; and 7, or 4%, reported negative effects (see Table 6). These results reflect the direction of the reported outcomes positive, neutral, or negative; they do not represent the magnitude of the effects. The percentage of positive outcomes ranged from 95% in Integrated Services to 68% in Parent Involvement programs. Even 68% is strong testimony to the efficacy of collaborative, school-linked programs. The overwhelmingly positive results suggest that coordinated services programs hold much promise as a means of enhancing the well-being of children and youth, especially those in adverse circumstances.

Attainment of Program Goals. In addition to each programs overall results, each program was designed to attain specific goals, such as delay of first usage of drugs and alcohol or reduction of pregnancy rates. The success of each of the program areas in attaining its particular goals is presented in Table 7. These results indicate that collaborative programs largely achieve the goals they set forth.

Effect of School-linked Services on Student Cognitive, Affective, and Behavioral Outcomes. The five most commonly measured student outcomes in studies and evaluations of school-linked programs were: attendance, academic performance (i.e., achievement test scores, grade point average, and academic grades), reduced behavior problems, self-esteem, and drop-out rates. As reported in Table 8, the influence of school-linked services on specific student outcomes was very positive. The number of positive results reported ranged from reduced behavior problems at 76% to reduced dropping out at 89%. These results suggest that school-linked service programs increase student engagement in the life of the school, improve academic performance, reduce behavior problems, and increase self-esteem.

Summary of Quantitative Synthesis Results. In the past five years, coordinated school-linked services have become a familiar feature of the educational reform landscape. Results from the quantitative synthesis revealed an overall pattern of positive outcomes for these programs, in terms of both specialized program goals and impact on students educational outcomes. Although positive outcomes dominate the results, they must be treated with guarded optimism since the research and evaluation designs employed in a number of the studies were insufficiently rigorous.

Evaluating Coordinated School-linked Service Programs

The state-of-practice in evaluating coordinated school-linked service programs must advance in order for stakeholders to obtain valid and reliable data about program effects. Current evaluation practices were identified from the 44 studies in the quantitative synthesis and the six case studies in the cross-program analysis. In general, the quality of the process and outcome evaluations was insufficiently rigorous. Although there were some exceptions, a number of flawed practices were identified. Many coordinated school-linked service programs made use of varied, and sometimes conflicting, goals, assumptions, definitions, procedures, and analytic tools. Frequently, the studies and evaluations of these programs were designed by teams of interdisciplinary researchers from social and health care agencies, as well as educational organizations. The influence of the social and health care professionals cast many of these studies in a service delivery perspective. In many cases, the program descriptions that provided potential consumers and stakeholders with information on school-linked services were insufficiently detailed. They glossed over the programs nature, function, and operation. Too little information was provided on program components and the relationships among them. Oftentimes, the delivery system was ill-defined, as was the support system or infrastructure that maintained the program. Descriptions of program infrastructure rarely provided information about the physical environment, system maintenance, professional development for service providers, or the monitoring system used by management to guarantee that program quality was maintained.

Providing detailed descriptions of program clients was also lacking. Some of the research and evaluation studies examined for this paper simply identified clients as low-income urban students and families. In fact, many of these programs served particular minorities, socioeconomic classes, and ethnic groups whose values, norms, and expressed expectations differed vastly. In addition, the experiential and educational histories of clients participating in, for example, integrated programs as compared to dropout prevention programs were likely to be radically different. Adequate descriptions of clients are paramount to determining the context in which these programs are effective. Differences in clients cultural, economic, linguistic, and experiential backgrounds affect the design and delivery of coordinated school-linked service programs. The outreach and delivery of services in a drug abuse and prevention program whose participants are middle-class, suburban youth differs from that of a program serving ghetto youth. Providing adequate detail describing the program and its clients is a critical first step toward contextualizing coordinated school-linked service effects.

A further step toward contextualizing program effects is describing what resources are available for use by the program staff. These resources may not have been used, but are available if needed. Among possible resources are money, prior experience with school-linked service programs, expertise, technology, transportation, equipment, print materials, quality of intake procedures, and number and quality of community organizations. None of the evaluations and studies examined provided detailed accounts of available resources. Providing information on the programs resource base identifies available sources of strength that can support program success. Not only should program resources be identified, but a needs assessment of the population to be served should be reported as well.

Only a few evaluations reported the use of needs assessments. In each of these instances, the needs assessment documented that changes were called for in the communities status quo and that school-linked services could fill the identified needs. The use of a needs assessment is important, because it signals that a priority for action exists. Scriven (1991) distinguishes between needs, wants, and ideals. Needs are necessary. In terms of school-linked services, availability of health services is a necessity, recreational programs are desirable, but not necessary. Ideals, such as providing clients in an emergency with access to free child care, food, shelter, transportation, and other services are utopian, but often impractical. The use of a needs assessment helps program planners in discerning facts about the functioning and availability of human services within the community. It identifies the functions and dysfunctions of existing systems. Program planners can build upon the empirical results of the needs assessment with some certainty that they are addressing essential and important concerns.

While needs assessments provide a starting point for program evaluation, it is also crucial to evaluate the process of change (process evaluation) during which the program is implemented. A process evaluation of school-linked services would investigate the implementation of the school-linked service program, including its legality, its morality, the truth of its claims, and the actual implementation process. Based upon the 44 quantitative studies and the six case studies, more and better implementation data are needed to generate evaluative conclusions about school-linked service programs. A key feature of the implementation process is the collaboration among participating individuals and agencies. The corpus of studies and evaluations examined in this paper used few direct measures of collaboration, such as: documentation of linkages among agencies, changing roles of administrators in schools and service agencies, changing role of staff, and the establishment of a management information system. Not documenting the collaborative process seriously threatens the conclusions drawn about program effectiveness.

Conducting a process evaluation only, without the use of an outcome evaluation, is not the method of choice. Only process evaluations were conducted in some of the cases used in the cross-program analysis and in some of the studies discarded from the quantitative synthesis because they failed to meet the criteria of containing outcomes. Sometimes, process evaluations alone are reported because the program is young and it would be premature to conduct an outcome evaluation. When school-linked programs have been implemented for a reasonable length of time, however, process evaluation must be supplemented with a rigorous outcome evaluation. The reason for the program is its outcomes, not its process. Thus, ideally, both types of evaluations should be conducted.

Many of the outcome evaluations and research studies examined in this study revealed positive results. These results, however, must be interpreted cautiously since many of the studies employed research designs that were not rigorous. Oftentimes, comparable control groups were not identified, only one wave of data was collected, sample sizes were small, and high rates of attrition reduced the external validity of the findings. Interestingly, some of the more rigorous designs were those developed for multi-site evaluations and reflected input and expertise from state and federal levels. However, the evaluation designs that were most sensitive to site-based concerns were locally developed. Outcome evaluations of coordinated school-linked service programs need to integrate both approaches and employ rigorous, site-sensitive designs.

Conclusions about the effectiveness of coordinated school-linked service programs must be based upon multiple outcomes collected from students, families, and agencies. The types of student outcomes utilized in these evaluations and research studies tend to be commonly used measures achievement test scores, grade point average, course grades, attendance, reduced behavior problems, self-esteem, and dropout rates. Family-based outcomes, such as improved access to basic education, achievement in functional literacy, obtaining a General Equivalency Diploma (GED), parent involvement, employment, and gains in family income were also widely used, especially among the six cases used in the cross-program analysis. Records of services delivered, participant satisfaction, and unobtrusive measures of engagement were also used; however, most programs relied on records of services delivered. Direct measures of the collaborative process were rarely employed. This is discouraging because early findings from Driscoll, Boyd, & Crowson (1994) suggest that many school-linked service programs manage to co-locate their services, but few really achieve vigorous collaborative activities. Presumably, direct measures of collaboration used throughout the implementation phase of these programs could jostle program planners out of complacency and incite midstream change in the programs operation and culture.

To date, most research studies and evaluations of coordinated school-linked services report the intended program outcomes, but few have reported unintended outcomes. For example, is there any evidence that the time teachers spend on referrals to other service providers, or engaged in the collaborative process, reduces their instructional planning time? Does the school principals engagement in service coordination and management negatively affect her role as an instructional leader?

The data collection tools used in studies of coordinated school-linked service effectiveness rely most heavily upon: school records; interviews and self-reports of students, family members, service providers, and program directors; student achievement tests; student and family paper-pencil attitude measures; and agency logs of services delivered. Observations, anecdotal data, medical records, and performance tests were used less frequently, possibly because of their high cost in time. In terms of the types of data reported, most program evaluations report numerical information (frequencies, means, and percentages) and a subset of those report statistical tests, as well. Most of the studies that are published in scholarly journals employ numerical data and statistical testing of results. Qualitative and anecdotal data were reported far less often. However, in the synthesis of 44 quantitative studies, one of the inclusion criteria was the use of outcome data that could be readily summarized, which eliminated studies that relied solely upon narrative, anecdotal, or qualitative approaches. Thus, the results reported here are likely to be biased in the direction of studies that report quantitative, statistical, and numerical data. When results are reported in studies of coordinated school-linked services, they frequently rely upon summary statistics, which implies that the population affected by the impact of the services is homogeneous with reference to the outcomes of interest. It may be desirable for result reports to be disaggregated by categories such as ethnicity, age of service recipients, English proficient vs. non-proficient recipients, special education students, low achievers, and other groups. Effectiveness with any subgroup should be reported, even if there is little evidence of success with the overall population.

Collecting cost data on these programs is desirable without cost data it is difficult to judge the practical significance of these programs. Ultimately, it is expected that coordinated school-linked service programs will provide better access to necessary services and reduce financial, psychological, personnel, and time costs to participants, service providers, communities, and society at large. Surprisingly, however, very few of the programs analyzed in the quantitative synthesis and the cross-program analysis reported any cost data. Of the 44 quantitative studies synthesized, only ten contained cost data; of these, nine were simple dollar amounts for operating the program for one year, and five of the nine attached an annual project budget. The only in-depth cost analysis was reported in the meta-analysis of parent involvement programs (White, Taylor, & Moss, 1992). White and his colleagues analyzed the costs of 20 of the studies they meta-analyzed and applied techniques of cost-effectiveness analysis. Other than White et al.s efforts, there were no other rigorous cost data collected in any of the studies examined.

Having highlighted the state-of-practice in coordinated school-linked services, it is clear that program designers and implementers need to engage in more strategic planning, responsible implementation, and practical wisdom. Likewise, program evaluators and researchers are called upon to design and conduct more rigorous, informative studies. As the coordinated school-linked service movement unfolds, high-quality, systematic process evaluations must be a priority in order to validate and improve program effectiveness. The traditional treatment/yield paradigm and classic pre- and post test control group experimental designs are suitable for outcome evaluations, where conclusions about program effectiveness are drawn. However, this paradigm does not reveal whether a program has been successfully implemented. To advance understanding about the implementation of school-linked services, the following questions must be addressed: What elements of the program must be implemented and at what levels to make the program work? What program features must be observed to validate that the program has been appropriately implemented? What barriers interfere with successful implementation? Designing rigorous studies of program implementation and collaborative processes is the focus of the next section of this paper.

Evaluating the implementation of collaborative, school-linked programs poses a major challenge. To date, the measurement of collaborative processes has been elusive. Should collaboration be measured by linkages among agencies, accessibility of services to clients, or by self-reports of participating service providers? Without advances in the measurement of program implementation, including collaborative processes, the design of these programs is largely based on theoretical speculation. As a first step toward improved evaluation of program implementation, the detailed information on practices and procedures identified in the cross-program analyses was combined with understandings from theoretical and research studies on school-linked services. From this information, four categories and supporting indicators to successful implementation were identified (see Table 9).

The four categories, (a) planning for implementation, (b) a client-focused approach, (c) conditions that promote interprofessional collaboration, and (d) resource allocation and deployment, are clusters of practices that were frequently mentioned in the cross program analysis (see Table 5). The categories and indicators can serve as a blueprint for an implementation assessment that is site-specific. The indicators can be tailored to local program conditions. Evaluators and researchers can use the degree to which indicators are present as evidence that specified actions and procedures are being implemented. The categories/indicators might be formatted as a checklist, rating scale, observation scale, or structured interview. The categories in Table 9 are arranged in order of implementation. Categories 1 and 2 would be implemented almost immediately in the life of the coordinated school-linked service program, whereas Categories 3 and 4 would be implemented after the planning and initial program set-up were completed. Thus, evaluators and researchers would collect data with such an instrument at several points during the life of the program.

Information gathered using such an assessment would reveal how fully the school-linked service program had been implemented. Results could be used to design new programs, guide maturing programs, and guarantee that a program has been fully implemented and is a candidate for an outcome evaluation. To use the assessment for such purposes, it would have to be psychometrically sound. The results would have to be reliable and valid given that decisions about childrens and families lives, and the future of existing programs would be based, in part, on results from the evaluation. Given the momentum of the school-linked service movement, the social science research and evaluation communities should make the development of such instrumentation a priority.

Although coordinated school-linked services, an innovative approach to addressing the multiple, co-occurring needs of children and youth in at-risk circumstances, have undergone numerous conceptual shifts, a basic premise remains: broader social policies are required to achieve this nations vision of schooling success for all children and youth.

Nearly all coordinated school-linked programs seek to develop connecting mechanisms for effective communication, coordinated service delivery, and mobilization of latent energies and resources of professionals from multiple fields, as well as families and communities. Whereas some focus on building school-linked delivery systems to achieve service integration, others adopt school-based approaches to bring children and family services into schools.

Despite intense interest and implementation nationwide, findings from quantitative and case studies on the design, implementation, and effectiveness of these programs indicate that several major barriers must be addressed if effective, coordinated school-linked services are to be provided to our nations children and families.

1. Researchers, practitioners, and policy makers concur that the application of knowledge and expertise from varied disciplines and professions is crucial to the success of coordinated school-linked services. Despite this widely held understanding, there is no system currently in place to communicate and share the expertise and knowledge generated by research and innovative development activities. There is an urgent need to provide forums for sharing solutions to thorny problems that are inherent in interprofessional collaborations, identifying promising practices and policies, and evolving new approaches to program implementation and evaluation. Successful practices that prove effective in one community can serve as a knowledge base to guide replication elsewhere. Researchers, program designers and implementers, service providers, clients, and policy makers would benefit from opportunities to share understandings and discuss research questions and methodological approaches yet another level of collaboration that ultimately contributes to the improvement of coordinated school-linked service programs.

2. The extant knowledge base on coordinated school-linked services is limited by both the paucity and quality of research and evaluation results available. Few rigorous, systemic studies have been conducted. Results from those studies that have been completed are often equivocal due to high attrition, lack of comparable control groups, little implementation and cost data, and limited outcome measures. There is a need for rigorous, systemic studies to determine the effectiveness, costs, and benefits of school-linked service programs. These studies must identify effective practices and policies that enhance organizations capacity to deliver high quality, collaborative services to children and families. Rigorous, systemic studies will provide research-based information on the technical aspects of initiating, maintaining, and evaluating these programs, as well as insights into exactly what constitutes program effectiveness and the conditions that facilitate effective service delivery.

3. The process and outcome measures used to document the implementation and effectiveness of school-linked service programs are not sufficiently tailored to the phenomenon they are intended to measure. Outcomes measures typically used to document effects on students include standardized achievement tests, g.p.a., attendance, dropout rates and other widely used measures. What documentation there is of "unique" client outcomes, such as decreased pregnancy rates, increased family literacy rates, or increased usage of child care facilities, tends to be scanty. Most programs record number of services delivered using agency logs and other record keeping devices, but few track client follow-up. In terms of program effectiveness, it is important to know how many children and families are referred for services, attend an initial session, but never return for the necessary follow-up. If school-linked service programs do not entice clients to remain in the therapeutic setting, then their effectiveness is limited finding that should be of keen interest to stakeholders, including taxpayers.

While some effort has been made to find measures of program outcomes, little has been done to measure program implementation. There are few direct measures of how adequately a school-linked service program has been implemented. Without such information, judging the programs effectiveness becomes speculative there is no clear linkage between the intervention and the programs outcomes. Measuring "collaborativeness" or "coordination" poses another major challenge. Developing better measures of program implementation, collaboration, and program outcomes would advance the methodology used to study school-linked services.

4. Schools are expected to play a key role in the delivery of services to children and families. This expectation is based on the premise that the catalytic role schools play in bringing needed services to children and youth is central to their well-being and educational success. However, little is known about the impact of coordinated school-linked service programs on the academic achievement of students in schools where such programs have been implemented. While providing access to health and other social services for these children is an important goal in itself, do students benefit academically from participation in such programs? Do the instructional programs in the schools in these communities need to be upgraded as well? What are realistic outcomes for school-linked service programs? Studies of coordinated school-linked services should examine the short- and long-term impact of these services on the instructional program and educational performance of children and youth.

Coordinated school-linked services are becoming a common feature of the school reform landscape. To develop effective programs, more definitive research and evaluation must be conducted and the results made accessible to practitioners, researchers, policy makers, and clients. The intense interest and support for these programs should be marshaled to establish a national research agenda to examine the effectiveness, costs, and benefits of such programs.

American Public Welfare Association. (1992). Status report on JOBS case management practices. Washington, DC: Author.

Arvey, H. H. & Tijerina, A. (1995). The school of the future: Implementation issues in a school- community connection. In L.C. Rigsby, M. C. Reynolds, & M. C. Wang (Eds.). School-community connections: Exploring issues for research and practice. (pp. 311-355). San Francisco: Jossey-Bass Publishers.

Behrman, R. E. (Ed.). (1992, Spring). School linked services. Future of children, 2(1).

Chang, H., Gardener, S. L., Watahara, A., Brown, C. G., & Robles, R. (1991). Fighting fragmentation: Collaborative efforts to serve children and families in Californias counties. San Francisco: California Tomorrow and the Children and Youth Policy Project, University of California, Berkeley.

Council of Chief State School Officers. (1989). Family support, education and involvement: A guide for state action. Washington, DC: Author.

Crowson, R. L., & Boyd W. L. (1993). Coordinated services for children: Designing arks for storms and seas unknown. American Journal of Education, 101(2), 140-179.

Driscoll, M. E., Boyd, W. L., & Crowson, R. L. (1994). Collaborative services initiatives: Preliminary report of a national survey of program directors. Interim report: Project 3.2/3.3: Exploratory studies of school-community connections. Philadelphia, PA: National Center on Education in the Inner Cities, Temple University Center for Research in Human Development and Education.

Family Impact Seminar. (1993). Data integration and evaluation: Essential components of family-centered systems-reform. Highlights of seminar meeting. Washington, DC: AAMFT Research and Education Foundation.

Gomby, D. (1992). School-linked services programs: Lessons from the past. Unpublished manuscript. Los Altos, CA: Center for the Future of Children: The David and Lucille Packard Foundation.

Gomby, D., & Larson, C. S. (1992, Spring). Evaluation of school-linked services. In R. E. Behrman (Ed.), The Future of Children, 2(1), 56-67. Los Altos, CA: Center for the Future of Children, The David and Lucille Packard Foundation.

Gray, S. T. (1993). A vision of evaluation A report of learnings from the Independent Sectors work on evaluation. Washington, DC: Independent Sector.

Hodgkinson, H. L. (1989). The same client: The demographics of education and service delivery systems. Washington, D.C.: Institute for Educational Leadership/Center for Demographic Policy.

Kagan, S. L., Rivera, A. M., & Lamb-Parker, F. (1990). Collaborations in action: Reshaping services for young children and their families. New Haven, CT: Bush Center for Child Development and Social Policy.

Kirst, M. W. (1991a). Integrating childrens services. Menlo Park, CA: EdSource.

Kirst, M. W. (1991b). Toward a focused research agenda. In William T. Grant Foundation Commission on Work, Family, and Citizenship and the Institute for Educational Leadership, Voices from the field: 30 expert opinions on AMERICA 2000 (p.38). Washington, D.C.: Editor.

Knapp, M. (1994, August). How shall we study comprehensive, collaborative services for children and families? Paper presented at the Working Conference on Comprehensive School-linked Services for Children and Families, Washington, D.C.

Levy, J. E., Kagan, S. L., & Copple, C. (1992). Are we ready? Collaboration to support young children and their families. Washington, D.C.: American Public Welfare Association and Council of Chief State School Officers.

Marzke, C. H., Chimerine, C. B., Morrill, W. A., & Marks, E. L. (1992). Service integration programs in community settings. Falls Church, VA: Mathtec, Inc.

Mickelson, R. A., Yon, M. G. , & Carlton-LaNey, I. (1995). Slipping through the cracks: The education of homeless children. In L. C. Rigsby, M. C. Reynolds, & M. C. Wang (Eds.), School-community connections: Exploring issues for research and practice (pp. 357-393). San Francisco: Jossey-Bass Publishers.

National Association of State Boards of Education. (1991). Caring communities: Supporting young children and families. Alexandria, VA: Author

National Commission to Prevent Infant Mortality. (1991). One-stop shopping: The road to healthy mothers and infants. Washington, DC: Author.

Public in poll backs change in education. (1992, August 28). New York Times, pp. A9, A12.

Rigsby, L. C., Reynolds, M. C., & Wang, M. C. (Eds.). (1995). School-community connections: Exploring issues for research and practice. San Francisco: Jossey-Bass Publishers.

Schorr, L. and Schorr, D. (1988). Within our reach: Breaking the cycle of disadvantage. New York: Anchor.

Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, CA: Sage Publications, Inc.

Stake, R. E. (1986). Quieting reform: Social science and social action in an urban youth program. Urbana: University of Illinois Press.

State Reorganization Commission. (1989). Services integration pilot projects: An evaluative report from Arizona, Florida, Maine, Oklahoma, South Carolina. Columbia, SC: Author.

Tyack, D. (1992). Health and social services in public schools: Historical perspectives. The Future of Children, 2(1), 19-31. Los Altos, CA: Center for the Future of Children, The David and Lucille Packard Foundation.

United States Department of Education & American Educational Research Association. (1995, April). School-linked comprehensive services for children and families: What we know and what we need to know. Washington, D.C.: United States Department of Education.

United States General Accounting Office. (1993, December). School-linked human services: A comprehensive strategy for aiding students at risk of school failure. (GAO/HRD-94-21). Washington, D.C.: Author.

Wagner, M., Golan, S., Shower, D., Newman, L., Wechsler, M., & Kelley, F. (1994). A Healthy Start for Californias children and families: Early findings from a statewide evaluation of school-linked services. Menlo Park, CA: SRI International.

Weiss, H., & Jacob, F. H. (1988). Evaluating family programs. Hawthorne, NY: Aldine de Gruyter.

White, K. R., Taylor, J. J., & Moss, V. D. (1992). Does research support claims about the benefits of involving parents in early intervention programs? Review of Educational Research, 62(1), 91-125.

Zetlin, A., Ramos, C., & Valdez, A. (in press). Integrating services in a school-based center: An example of a school-community collaboration. Journal of Community Psychology.