 |
 |
Temple University Center for Research
in Human Development and Education |
|
|
In addition to evidence of student learning, there is a growing need to provide evidence of accountability.
Policymakers and administrators want to know, "Does it work?" often using scientific evidence to validate claims.
But this is not a simple question. To systematically answer the question "Does it work?" requires the formulation
of specific indicators to assess successful implementation in the classroom. Is "it" the technology, pedagogical
approach, content, or other factors that may influence teaching and learning? What is the unit of analysis or
the target of investigation: the student, teacher, classroom processes, school as an organization, or something
else? What does "work" mean? What forms of evidence will convince decision makers that technology is working?
A Flexible Approach for Assessing Successful Implementation
There is no easy solution to how accountability can be addressed that satisfies policymakers,
funding sources, school personnel, and researchers. People involved in educational technology
need to change their conception of the criteria for acceptable proof of successful implementation.
The transitory nature of schools makes it difficult to follow a controlled scientific methodology
that would produce evidence of causal links between technology inputs and student achievement.
|
|
| |
School administrators who seek direct causality for the implementation of technology often look
to standardized test scores –measures that were not developed for that purposes and therefore are
not valid indicators. They also seek to link expenditures for technology to student performance,
assuming the more money spent, the greater the impact on outcomes. This is a necessary rationale,
given limited budgets and overwhelming demands and competing priorities for funds and resources.
However, despite the pressing need to justify technology expenditures, there are too many intervening
influences to be able to make such causal statements in educational settings. Lee Cronbach
(1963), one of the foremost experts in educational
evaluation, suggests that focused case studies triangulating
among measurement strategies, asking different questions,
targeting multiple levels of analysis, and concerning
various aspects of teaching and learning–both
process and product–may
provide the best sources of data. There also must be
sufficient exposure to technology-based applications to
obtain evidence of impact. Often instructional exposure is
limited, unrelated to curricular objectives, and the desired
outcomes are not linked to assessment procedures. Ideally,
there should be direct feedback loops between instruction
and assessment, regardless of the instructional medium.
Given the dearth of reliable and valid research results, critics have questioned
the practicality of assessing the merits of educational technology. After all, such attention has not
been given to the impact of other educational tools such as the pen or blackboard. Obviously, there are
different expectations for educational technology. While educational technology can be implemented in
diverse ways, we have some sense of what good implementation looks like. We know it when we see it.
The education community, however, lacks a common conception of what is acceptable proof of successful
implementation beyond teacher and administrator anecdotes. In fact, indicators of successful
implementation will differ across schools and even within schools, depending on specific objectives,
constraints, and environmental conditions. Generalization across sites may be difficult. What works for
one teacher may not work for others. Certain general principles may be possible, but it is essential to
remain flexible and receptive to new forms of evidence that may be only locally valid. By being
attentive to the specific factors that shape the implementation of technology in a given setting,
assessors will be able to formulate insightful indicators of successful implementation with which to
conduct a sensitive evaluation of technology's impact on their schools' educational processes and
outcomes. *
References
Cronbach, L. C. (1963). Course improvement through evaluation. Teachers College Record, 64, 672-683.
Lesgold, A. M. (2000). What are the tools for? Revolutionary change does not follow the usual norms. In S. P. Lajoie (Ed.),
Computers as cognitive tools (Vol. II) (pp. 399-408). Mahwah, NJ: Lawrence Erlbaum Associates.
No Child Left Behind Act of 2001, Pub.L. No. 107-110, 115 Stat 1425 (2002). Retrieved June 2002, from http://www.ed.gov/legislation/ESEA02/107-110.pdf
U. S. Department of Education. Office of Educational Technology. (2002). e-Learning: Putting a world-class education at the fingertips of all children. Retrieved December 2002, from http://www.ed.gov/Technology/elearning/
* Ellen Mandinach is the Associate Director for Research at the Education Development Center's Center for
Children and Technology in New York City.
Previous
Page
1 2
|
|
|
Copyright 2001 © MARTEC
|
|
