 |
 |
Temple University Center for Research
in Human Development and Education |
|
|
Guidelines for Selecting
Technology Assessment Instruments
121
by
Patricia Hendricks, Kelly Feighan, and Paul Droms
Temple University
Well designed surveys can help educators meet the No Child
Left Behind objectives of effective and continuous decision
making informed by data. Judging survey quality, however,
requires an understanding of several methodological
concepts. Surveys of quality are typically pretested,
revised, clearly written, and pose minimal time and memory
demands on respondents. In addition, leading questions that
might bias responses are omitted and reasonable steps are
taken to protect respondents' privacy. Survey data can shed
insight into many characteristics of a target population.
This techno-brief serves as a guide for educators interested
in reviewing surveys that measure teacher proficiencies and
educational technology integration. Although we have not
captured every aspect of survey design, we provide a
framework for understanding the basic components of a good
survey.
|
|
| |
The bullets
below organize ten characteristics of a quality assessment
instrument into topical areas, or domains. Each domain is
accompanied by a specific instrument example, reviewed only
once to provide maximum exposure to a wide array of
available instruments currently in use.
Clearly Stated Purpose. A good assessment
instrument has a clear purpose aligned with educational
technology goals. The instrument captures the depth and
breadth of technology integration by asking well-written,
succinct questions. An instrument designed to collect
information about the participants' attitudes and behaviors
should also elicit contextual information to control for
differences in resources and abilities. The goal is to
collect data that help explain (not merely describe)
participants' behavior. North Central Regional Educational
Laboratory's enGauge (2002) offers a systemic picture of an
educational agency. Included in this comprehensive tool are
nine surveys intended for nine different stakeholders;
however, this instrument is not a series of isolated
surveys. It is a systemic framework that allows each
stakeholder's self-reported behaviors and attitudes to be
considered in relationship to the other stakeholders.
Institutional Context. A technology assessment
instrument should capture an accurate description of the
institution and its culture. It should include measures of
the technology infrastructure, teacher's technology access
and use, institutional incentives and expectations,
teacher's pedagogical style, and teacher's technical skills.
An illustrative example is the Pennsylvania Department of
Education-eTechPlanner (2000). This survey allows districts
in PA to create a comprehensive picture of their technology
access and use.
Respondent Demographics. A quality instrument
collects respondent demographic information in order to
understand and explain the context of technology
integration. Demographic characteristics include a
respondent's age, race, gender, and level of education.
Information should also be collected on other
characteristics such as the respondent's current position or
grade level, primary and secondary teaching assignment,
years of teaching experience, years in current position, and
participation in activities such as professional development
programs. An illustrative example is Chadwick's School
District Staff Education Technology Needs Assessment (Slowinski,
2000). This free online tool includes a comprehensive
demographic section.
Confidentiality. A well-designed assessment
tool builds in reasonable safeguards for confidentiality. As
the American Statistical Association explains, a survey
organization should do everything it can to protect privacy
by ensuring that identifying information be removed. During
the data collection process, research staff may temporarily
know a participant's identity before information can be
coded. However, only aggregate data should be reported. An
example is Sun-Associates Educational Technology Integration
(2000, August 14). Sun assigns each participant a unique
identification number. The survey carries this
confidentiality clause: "Your answers on this survey are
confidential. We do not track the identity of individual
respondents and our external evaluators will not share raw
data with anyone in the district."
Length. Surveys should be long enough to
capture individual and contextual data but not so long as to
deter participation. A major usability issue of online
surveys is the ability to navigate a long survey. Surveys
must be divided into meaningful sections giving the user the
option to complete it in different settings. The Utah
Technology Awareness Project's technology concepts (2002)
are divided into seven categories: Basic Concepts/Skills,
Personal/Professional Productivity Skills,
Communication/Information Skills, Classroom Instruction
Skills, Educational Leadership Skills, Administrative
Leadership-Technology Implementation Skills, Technical
Troubleshooting Skills. Each category contains subcategories
with rubrics for assessing teachers' proficiency.
Page
1 2
Next |
