This site will look much better in a browser that supports current web standards, though its content is accessible to any browser or Internet device.

::Skip Top Navigation::

Home > Regional Networks > Tech. Coordinators Network > 12/18/02

Creating effective purchasing strategies for educational software

Wednesday, December 18, 2002

Discussion Questions:

Expert Panelists:

Dale Mann, Managing Director of Interactive, Inc and Professor, Teachers College, Department of Organization and Leadership, Columbia University will discuss educational software and research proven strategies. Dr. Mann's organization recently conducted a statewide analysis of the achievement gains associated with Pennsylvania's Students Achieving Standards program. This study considered the impact of instructional technology (integrated learning systems) on student academic achievement.

Karen Billings, Vice President Education Division, Software and Information Industry Association (SIIA) and Mark Schneiderman, Director, Federal Education Policy, SIIA Education Division, will discuss some of the best practice strategies districts use when purchasing educational software. The Software Information Industry Association, Education Division provides a forum for companies that produce and market code, content, and related technologies as well as provide services for use in educational settings.

Introductions and Background:

Patty welcomed each participant as he/she dialed in. For the first time, MAR*TEC experimented with utilizing an online chat in conjunction with the conference call. As participants logged on, they announced where they were from and provided a little bit of background. This summary contains information from both the online conversations and the phone conversations. (See online chat)

Lisa explained that she is a teacher on special assignment for technology at the Tredyffrin Easttown School District. T/E is located in Chester County, Pennsylvania. Wayne is the Director of Technology at the Penn Delco School District, located in Delaware County, southwest of Philadelphia. Penn Delco has 6 schools-4 elementary, 1 middle, and 1 high school-with approximately 3,300 students. Sandy is the Director of Technology for the Governor Mifflin School District; she has 5 schools and 4,100 students. Her job duties are broad and include the integration of technology into the curriculum, information technology, supervision of bids and purchases, data analyst, security officer for cybercrimes, and student information system administrator. Ken is the Director of the National Technology Institute for School Counselors. Mary works in the Technical Department of the New Hope-Solebury School District, which encompasses 3 schools and 1,300 students. Dale explained that his company (Interactive, Inc.) evaluates educational software including the Students Achieving Standards (SAS) program in Pennsylvania.

1. No Child Left Behind requires that districts use proven research strategies before implementing new educational programs. Who should conduct research on software?

Since the producers and vendors of educational software are often the same people who are documenting its effectiveness, a conflict of interest arises. Patty posed the questions, "what kind of research can educators trust? What strategies should educational leaders use when considering vendor produced software evaluations? What tools are available to help schools evaluate educational software?"

Dale explained that it takes money to do a decent evaluation; schools have other priorities and are hard pressed for money. Dale added that there are some companies that are ahead of the curve in respect to reliable research about their products.

Sandy suggested strategies currently used in the Governor Mifflin School District. She has committees at each building to consider software. Their district also looks to their Intermediate Unit for advice. She belongs to listservs of all the technical directors in the county and they communicate with one another. She also tries to conduct her own research. She found that if teachers were comfortable with a particular choice of software, they would implement it. In addition, the leaders in their district do a lot of reading and she contacts people who are getting advanced degrees in technology to evaluate and test different software.

Sandy remarked that it helps to read research reports that can be verified by more than one reviewer. Otherwise, questions arise about the validity and reliability of the reports. She is not certain that she can put 100% trust in any single research study. Mark then asked what it would take for her to be able to trust a product review. Sandy replied that she would like to know what went into the evaluation, such as the methodology employed, the types of students involved, the teachers' expertise and experience, etc.

Jon said that the West Shore School District does not use vendor research to make software decisions. Lisa remarked that their local software research is very limited; they allow grade level teachers to make decisions and will use a pilot program in a single class. In addition, they utilize previews of software and use 30-day trial periods, etc. Karen Billings, SIIA, added that vendors are interested in partnering with LEAs to research their products. The LEA usually needs to provide control groups of students. Mark said that there is a shared burden between vendors and LEAs to conduct the research; the vendors are willing to put up the cost but need willing participants. Dale added that the U.S. Department of Education has implemented a gold standard for evaluation. However, this standard will not be required of software evaluation. Mark said that, theoretically, a placebo has no effect but the person receiving the treatment does not know that the placebo will not be effective. Therefore, putting a student in front of the computer, knowing that the software will have no effect is a waste of class time and the student's learning. Dale remarked that he believes that more comparative research should be done on software effectiveness. For instance, research that looks at the effectiveness of one type of software versus another type of software. Patty noted that current educational research (research conducted by educational researchers and not associated with specific vendors) often lumps computer-based instruction (CBI), computer-assisted instruction (CAI), or integrated learning systems (ILS) into one group. The assumption with this type of research is that all products have equally good curriculum and instructional delivery. This assumption is not necessarily true.

2. How reliable is the information from software publishers and vendors?

Karen feels that reliability should be judged by the same factors, regardless of source. Dale agreed that it is the quality and relevance of the evaluation, not the source. Johann then questioned if we have a clear framework to judge information regardless of it source. He added that scientific evidence seems to be a common need from participants. Dale replied that it is a combination of understandable research procedures and practical considerations of the sort that happen in real classrooms. Both Wayne and Sandy expressed a desire to see web-based information from vendors. Johann said that some vendors do use the Web to disseminate their promotional materials but, most likely, they will release their most successful implementation only. It would be useful to include peer schools as a source of information with vendors and independent researchers. Karen agreed that web-based information on current uses and research would be great for the districts considering a purchase and added that publishers might be concerned about providing too much information for their competitors.

Wayne said that there are also some ISTE SIGs where information on products can be shared. Sandy added that control groups are tough to set up; her district tends to shy away from control groups. Wayne said that, in his district recently, people shared views on PowerSchool and how they implemented it.

3. Who has purchasing power for educational software? Does centralized or distributed purchasing power work best?

Patty asked participants, "who has purchasing power for educational software in your district?" Sandy answered that teachers recommend software to the school technology committee. In the Governor Mifflin School District, the Director of Technology reviews the requests, looks for the best price, and then determines the purchasing. Ultimately, she has the authority to make the purchasing decisions. In the Penn-Delco School District, the Director of Technology also has purchasing power, and all major purchases are supposed to go through that department. If principals or anyone else wants to purchase software, they are supposed to contact the Director of Technology, but that doesn't always happen. Consequently, they have had individual teachers who have purchased software that was incompatible with the school's software. Lisa agreed, saying that platform issues have been a problem in the Tredyffrin Easttown School District as well. However, teachers will often consult the IS team for final technical specifications. Lisa said that purchasing decisions in the Tredyffrin Easttown School District are made by several groups: 1) Director of Technology, 2) team facilitators, 3) principals, and 4) directors and supervisors of curriculum. Jon said that purchasing power in the West Shore School District is restricted to the technical coordinator and the technical specialist for final approval but added that anyone may suggest or request software.

4. How has legislation mandating scientifically based research affected district software purchasing decisions?

Many of the participants agreed that it is "business as usual." Sandy foresees a change in budget planning and sees state initiatives impacting the budgeting. A participant commented that if the burden of researching the validity of educational software falls on schools, that would be very unfortunate. Purchase orders may have to be canceled unless vendors could prove that their software is scientifically based. A participant added that no one is sure what constitutes scientifically based research; that term and what falls under that umbrella is a bit confusing. Karen commented that many districts are looking at products in alignment with current state standards and assessments. Wayne said that they do try to make decisions with the curriculum and state standards in mind.

Patty posed the question of whether, with the push to align curriculum standards to software evaluation, teachers naturally know how to align standards or need training.

Karen said that there are courses offered in how to align topics, curriculum, etc. with legislation. The courses are, conveniently, state specific; the information provided to the teachers in Texas, for example, is different from the information presented to teachers in New York City. Sandy commented that teachers have a natural love and instinct for teaching and she has found that it takes a mixture of natural instinct and professional development; there is not a single model that works. Online learning works, but online learning completion has only been about 50%. She said that they use technology for critical thinking and usually purchase more tool-based software. She added that staff development is essential, and training is involved. One teacher takes the lead and they use the IU curriculum folks as well. Their strategic plan works in K-12 with approved PA standards. They use summer curriculum funding to sponsor implementation. Wayne said that they start with their curriculum coordinators on the standards/curriculum software because they are responsible for some of the data entry. Then they train the teachers on where to find the information.

Johann posed the question of whether formal or informal types of staff development are more effective in helping teachers effectively integrate software. Sandy said she favors informal, and Wayne said they utilize more formal types and added that they have an IT coordinator (a new position this year) who gets into schools on a regular basis. Sandy said that they just expanded their Act 48 to include study groups and hired three technical assistants who have associate's degrees in educational technology. Teachers will receive Act 48 credits for each hour of work in a study group. Sandy said they support collaborative lesson planning and then do assessment working. Wayne added that his IT coordinator has been an invaluable addition, as teachers now feel they have support after the formal staff development. His teachers receive Act 48 credit as well. In addition, they have individual building tech. committees that meet monthly with the IT coordinator. The IT coordinator also works with teachers on projects.

5. How can school districts gaining leverage and strike effective deals with software companies?

Dale said that he is interested in districts combining their purchase orders to get more leverage from the vendors. Sandy added that her district does that; it enables them to afford certain software that they would not otherwise be able to purchase. Sandy said they were having trouble getting AutoCAD because of its expense. They used the purchasing power of the IU. By being the broker for the whole state, the software became affordable. She added that that model works, but it takes a lot of effort and time on the part of the IU. She is working to leverage customizations in PowerSchool to reduce the cost to the district. She feels that the licenses for pilots are perfect; she has used Jostens, CCC, and another vendor in schools.

Ted said that, in NJ, they have implemented group purchases of Microsoft licenses, but he has not seen anything as far as a consortium in regards to educational software. A participant added that there is a drive on part of the publishers to do more research, which enables districts to partner with these researchers and, in turn, can strike deals and receive significantly lower costs on software. In Delaware County, Wayne explained, they have a consortium for their countywide fiber network and Internet service. They also have bi-monthly meetings of technology coordinators at the IU where they discuss large purchases for all county schools through the IU.

Having worked at a number of companies that produce software products, Karen said that they will provide licenses so that schools will have the information they need to make a direct purchase. If they ended up purchasing from her company, that was great. If not, they still got great feedback.

If a school district wants to express interest in testing software, Karen recommends making a contact at the company and letting them know that you're interested; it is difficult to find districts who are willing to participate, so when districts express interest, it is taken very seriously. Karen added that they send out e-mail updates to their members and can contact interested parties one-on-one with vendor recommendations depending on what products a school or district is looking for.

Dale remarked that many publishers are more serious than ever about research; Interactive, Inc. has been working with publishers and districts to create "standing panels" of districts ready to participate. One benefit is often free access to next generation products.

Dale said he is ambivalent about the press to provide scientific research to educational software. While taking into account the empirical side of things, one cannot ignore the practical side. Karen said that schools should come forward with their frameworks and their needs."here is what we need; here is what we want.do you have something for us?" Her advice is to stay in control. In the end, that clarity will continue to give you the curriculum materials that you need.

Mark added that the Department of Education has been backing off a bit on mandates regarding scientifically based research. He feels that there are many opportunities to "step it up" and get more research done. He fears a derailment of efforts; there is a need to keep moving forward but it can get messy with No Child Left Behind and other legislation implemented.

Resources

Home > Regional Networks > Tech. Coordinators Network > 12/18/02