Wise Young
12-02-2006, 07:10 PM
If true and then confirmed by larger studies, the findings of this group have important implications for the education of boys and girls. While differences of language learning between boys and girls have long been believed by teachers, particularly those who teach in all-boys or all-girls schools, most of the beliefs have been based on conjectures that were not well-backed up by data. For example, many teachers believe that girls rely on verbal memory whereas boys tend to rely on rule-based processing of information. This study suggests that girls make similar mistakes as boys in "over-regularizing" grammatical rules but for different reasons. They make the mistakes by analogy whereas the boys tended to make the mistakes as a result of rules processing.
Like many studies of its kind, however, I cannot help but feel that the authors are over-generalizing based on relatively little data. The number of subjects were small and there was really no control. They had an unexpected finding (that girls actually over-regularized past tenses) and proposed an alternative explanation which is based on the better lexical memory of the girls and therefore their ability to use associative learning from previous examples. The study would have been far more rigorous if they had done the study on larger numbers of boys and girls that had been exposed to different training. The rigorous study would have been a large controlled study where boys and girls were randomized to different training methods and then tested for the degree to which they over-regularized their expression of verb tenses.
http://explore.georgetown.edu/news/?ID=20915
Study of Language Use in Children Suggests Sex Influences How Brain Processes Words
Washington, D.C.--Boys and girls tend to use different parts of their brains to process some basic aspects of grammar, according to the first study of its kind, suggesting that sex is an important factor in the acquisition and use of language.
Two neuroscientists from Georgetown University Medical Center discovered that boys and girls use different brain systems when they make mistakes like “Yesterday I holded the bunny”. Girls mainly use a system that is for memorizing words and associations between them, whereas boys rely primarily on a system that governs the rules of language.
“Sex has been virtually ignored in studies of the learning, representation, processing and neural bases of language. This study shows that differences between males and females may be an important factor in these cognitive processes,” said the lead author, Michael Ullman, PhD, professor of neuroscience, psychology, neurology and linguistics.
He added that since the brain systems tested in this study are responsible for more than just language use, the study supports the notion that “men and women may tend to process various skills differently from one another.” One potential underlying reason, suggested by other research, is that the hormone estrogen, found primarily in females, affects brain processing, Ullman said.
The study, whose co-author is Joshua Hartshorne, was published earlier this year in the journal Developmental Science.
Researchers know that women tend to be better than men at verbal memory tasks, such as remembering word lists, and that this ability depends on declarative memory. Included within declarative memory is a “mental lexicon” in which word forms are memorized and remembered. The grammatical rules that allow us to combine words in sentences depend on “procedural” memory. Researchers have found that both boys and girls may be equally adept at this process, which depends on a different part of the brain than declarative memory.
In this study, Ullman and Hartshorne hypothesized that girls would be better than boys at remembering irregular past-tenses of verbs, like “held”, since these words are memorized in declarative memory. And if girls remember “held” better than boys, they should make fewer errors like “holded”, since these “over-regularization” errors are made when children can’t remember irregular past-tenses, and so resort to combing the verb with an –ed ending, just as they do for regular verbs like “walked”.
So they studied how a group of 10 boys and 15 girls, age 2 to 5, used regular and irregular past-tense forms in their normal speech. To their surprise, and contrary to their predictions, the researchers discovered that the girls over-regularized far more than boys.
They then investigated which verbs the girls made the mistakes on, and found an association between the number of similar sounding regular past-tense verbs, and the particular verb that was over-regularized. For example, girls tended to say “holded” or “blowed” because many other rhyming verbs use the regular past-tense form (such as folded, molded, and flowed, rowed, stowed, respectively).
The researchers say this kind analogy-based processing suggests the girls were relying on their declarative memory to create the past tense. “This memory is not just a rote list of words, but underlies common patterns between words, and can be used to generalize these patterns,” Ullman said. “In this case, the girls had memorized the regular past tenses of rhyming words, and were generalizing these patterns to new words, resulting in over-regularization errors” such as “holded” and “blowed”.
In contrast, for the boys there was no association between the number of similar sounding regular past-tense verbs, and the particular verbs that were over-regularized. So the boys did not make more over-regularizations on verbs like “holded” or “blowed” that have many rhyming regular past-tenses. This suggests, Ullman said, that the boys were not forming these words in declarative memory, but were probably using the rule-governed system to combine verbs with –ed endings.
Other types of evidence also suggest that adult women tend use declarative memory more than adult men do in their use of language, Ullman said. “Although the two sexes seem to be doing the same thing, and doing it equally well, they are using two different neurocognitive brain processes to do it,’ Ullman said. “This is a novel and exciting finding.”
About Georgetown University Medical Center
Georgetown University Medical Center is an internationally recognized academic medical center with a three-part mission of research, teaching and patient care (through our partnership with MedStar Health). Our mission is carried out with a strong emphasis on public service and a dedication to the Catholic, Jesuit principle of cura personalis -- or "care of the whole person." The Medical Center includes the School of Medicine and the School of Nursing and Health Studies, both nationally ranked, the world-renowned Lombardi Comprehensive Cancer Center and the Biomedical Graduate Research Organization (BGRO).
Note that I reproduced the entire contents of the article (because it was a press release). Here is the abstract of the study.
Hartshorne JK and Ullman MT (2006). Why girls say 'holded' more than boys. Dev Sci 9: 21-32. Women are better than men at verbal memory tasks, such as remembering word lists. These tasks depend on declarative memory. The declarative/procedural model of language, which posits that the lexicon of stored words is part of declarative memory, while grammatical composition of complex forms depends on procedural memory, predicts a female superiority in aspects of lexical memory. Other neurocognitive models of language have not made this prediction. Here we examine the prediction in past-tense over-regularizations (e.g. holded) produced by children. We expected that girls would remember irregular past-tense forms (held) better than boys, and thus would over-regularize less. To our surprise, girls over-regularized far more than boys. We investigated potential explanations for this sex difference. Analyses showed that in girls but not boys, over-regularization rates correlated with measures of the number of similar-sounding regulars (folded, molded). This sex difference in phonological neighborhood effects is taken to suggest that girls tend to produce over-regularizations in associative lexical memory, generalizing over stored neighboring regulars, while boys are more likely to depend upon rule-governed affixation (hold+-ed). The finding is consistent with the hypothesis that, likely due to their superior lexical abilities, females tend to retrieve from memory complex forms (walked) that men generally compose with the grammatical system (walk+-ed). The results suggest that sex may be an important factor in the acquisition and computation of language. Brain and Language Laboratory, Departments of Neuroscience, Psychology, Linguistics and Neurology, Georgetown University, Washington, DC 20057, USA. http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=16445392
This study brings up an interesting and complicated issue of the rigor of evidence based standards applied in the field of education compared to biomedical science. Although clinical trials are often not accorded much respect by scientists in other fields, I think that the level of evidence required for proof of risk and benefit in clinical trials far exceed those in other fields of science. For example, the evidence that is presented in this study would be considered no better than phase 1 or, at best, phase 2 clinical trials, showing some promising results but not enough to generate approval by the FDA. It shows feasibility and some promising results. Yet, this kind of data may lead to implication of education policies that utilize billions of dollars every year.
The National Institute of Child Health and Development (NICHD), the premier NIH institute that is responsible for child health research in the country, was recently recruited into the "no-child left behind" program by the Bush Administration. The thought that was medical clinical trial designs be used to test educational theories that are being used or considered for use in the country. I must say that I personally thought that this did not belong in the province of NIH and should have been funded by the Department of Education, and that it took valuable resources from the biomedical field. On the other hand, it was very interesting how much impact biomedical research is beginning to have on education research where they don't have a long tradition of controlled trials. To tell you the truth, neither has medicine. For example, randomized clinical trials were not a requirement in medicine until the 1970's, when the FDA adopted the now widely used standard of phase 1, phase 2, and phase 3 trials for approval of treatments.
A strong movement for evidence-based practices has developed in the educational field (Source (http://www.wrightslaw.com/nclb/rbi.htm)), required by the No-Child-Left-Behind-Law:
http://www.wrightslaw.com/nclb/rbi.htm
A primary focus of this law is the requirement that school districts and individual schools use effective research-based reading remediation programs so all children are reading at grade level by the end of third grade. The law authorizes funds 'to provide assistance to State educational agencies and local educational agencies in establishing reading programs for students in kindergarten through grade 3 that are based on scientifically based reading research, to ensure that every student can read at grade level or above no later than the end of grade 3.' (20 U.S.C.§ 6361)" (page 73, Wrightslaw: No Child Left Behind)
Led by Reed Lyon at NICHD, the field of learning disabilities is beginning to adopt biomedical standards of evidence (Source (http://www.ldonline.org/indepth/reading/nichdbrochure.html)).
http://nhida.org/docs/ReadingDisabilities.pdf
Reading Disabilities: Why Do Some Children Have Difficulty Learning to Read?
What Can Be Done About It?
by G. Reid Lyon, Ph.D.
The National Institute of Child Health & Human Development (NICHD) considers that teaching
and learning in today’s schools reflect not only significant educational concerns, but public health
concerns as well. Our research has consistently shown that if children do not learn to understand
and use language, to read and write, to calculate and reason mathematically, to solve problems,
and to communicate their ideas and perspectives, their opportunities for a fulfilling and rewarding
life are seriously compromised. Specifically, in our NICHD-supported longitudinal studies, we
have learned that school failure has devastating consequences with respect to self-esteem, social
development, and opportunities for advanced education and meaningful employment. Nowhere
are these consequences more apparent than when children fail to learn to read. Why? Simply
stated, the development of reading skills serves as THE major foundational academic ability for
all school-based learning. Without the ability to read, the opportunities for academic and
occupational success are limited. Moreover, because of its importance, difficulty in learning to
read crushes the excitement and love for learning, which most children have when they enter
school.
<more>
A number of organizations have sprung up to promote evidence-based education policies in the United States (Source[/ur]). The term evidence-based education, however, has been controversial because there are many who believe that there are complexities and subtleties to teaching that will be very difficult, if not impossible, to capture in randomized trial designs (Source (http://www.cemcentre.org/RenderPage.asp?LinkID=30310000)). A new Institute of Education Science was created to replace the Office of Education Research and Improvement (Source (http://eduscapes.com/tap/evidence.html)). As part of the shift towards better accountability, the Department of Education has introduced the National Education Technology Plan (http://www.ed.gov/about/offices/list/os/technology/plan/2004/site/edlite-default.html)). The Department of Education has issued guidelines for "identifying and implementing educational practices supported by rigorous evidence" (Source (http://www.ed.gov/rschstat/research/pubs/rigorousevid/index.html)). The field of "instructional experiments" has begun (Source (http://www.readingonline.org/articles/art_index.asp?HREF=/articles/rt/9-03_column/)). It would be interesting to know what people think.
As Michael Priestly pointed out
http://www.readingonline.org/articles/art_index.asp?HREF=/articles/rt/9-03_column/
A Few Things Reading Educators Should Know About Instructional Experiments
Michael Pressley
I am confronted almost daily by the 2001 No Child Left Behind (NCLB) legislation and its demand that reading instruction be scientifically evidence based. Assertions are also being made that reading instruction provided to children in the United States should be evaluated in randomized experiments, especially reading instruction supported by public funds, such as those provided by the Reading First program.
As an experimental psychologist, I have contributed many "true" experiments (i.e., experiments with random assignment to condition) to the scientific literature. I am proud that I have mastered the ability to conduct research experiments. I am also proud that as part of my education I learned a great deal about the strengths and weaknesses of experimentation—when experiments are credible and useful and when they are not.
Many of the contemporary assertions about experimentation and reading instruction concern me. Many claims are debatable at best and sometimes just plain wrong. I spoke about my concerns at the 2002 International Reading Association Convention. At the conclusion of that address, the editors of The Reading Teacher requested that I prepare an article instructive to teachers about experimentation and evidence-based reading instruction.
I present in this article 12 points about experimentation that I think all teachers concerned with evidence-based reading instruction should keep in mind as they evaluate the flurry of assertions that are overflowing the marketplace of ideas in literacy education. After presenting these points, I reflect briefly on how responsible experimenters who wish to inform literacy instruction should communicate research results to classroom practitioners.
<more>
It would be interesting to know what people think of this development. If anything, this will be pointed to as one of the legacies of the Bush Administration in future years. It is important to note that the trend began before George Bush took office. As pointed out by Martha Rapp Ruddel, the Chair of the National Reading Conference Committee:
http://www.readingonline.org/articles/art_index.asp?HREF=/articles/purcell-gates/index.html
In spring 1998, the director of the National Institute of Child Health and Human Development appointed the National Reading Panel (NRP) to carry out a request by the United States Congress to study the effectiveness of various approaches to teaching children how to read. The panel subsequently defined the procedures by which it would conduct its business and the questions that it would use to guide its study. The panel has now published its results, available at [url]www.nationalreadingpanel.org/Documents/default.htm.
The National Reading Conference (NRC), a professional organization dedicated to literacy research, created a committee to respond to the NRP report by addressing the following questions:
1. Did the panel do what it said it would do to answer the questions it had posed?
2. What, if anything, did the panel leave out within the questions it created and in defining the questions themselves?
3. What research traditions, if any, have not been considered in the panel's work?
Following is the first in a planned series of commentaries prepared by members of the NRC committee. It presents one response to the third question.
Martha Rapp Ruddell
Chair, National Reading Conference Committee to Respond to the National Reading Panel
Past President, National Reading Conference
As Victoria Purcell-Gates pointed out in the above cited web site,
http://www.readingonline.org/articles/art_index.asp?HREF=/articles/purcell-gates/index.html
The Role of Qualitative and Ethnographic Research in Educational Policy
Victoria Purcell-Gates
The National Reading Panel, charged with identifying research that could inform policy and practice in reading, has now concluded its exhaustive review. The panel focused its review on experimental and quasiexperimental research which, by necessity, meant that it focused on a small set of questions. I say “by necessity” because the areas of instruction in reading that have been examined by experimental or quasiexperimental research are of themselves few and far between, although certainly of central importance to the learning-to-read process.
The point I wish to argue here is that by limiting the type of research used to address the issue of how to guide policy and practice in reading instruction to experimental and quasiexperimental studies, the panel missed critical areas that have been examined by qualitative and ethnographic research. Further, a clear danger resulting from this limitation to particular types of methodologies is the establishment in the minds of funders and users of research that only experimental and quasiexperimental research provide us with answers to educational problems. A second theme of this commentary, then, is my view that we do not stand a chance of solving problems of learning and teaching if we confine ourselves solely to these methodologies. Rather, it is only by the considered and judicious use of a range of methodologies, rigorously applied and intelligently synthesized, that we as educators, policy makers, and the informed and concerned public can hope to make real advances and reforms to our system and in our procedures for educating all our students to their full potentials.
Proving, Informing, or Both?
There is no doubt that experimental, and to some degree quasiexperimental, research is required to “prove” the effectiveness of an instructional approach, method, or intervention. This to some degree explains the panel's exclusion of other methodologies in its research review (see, e.g., Shanahan, 1999; online document). Identifying the effects of specified instructional techniques can only be accomplished with carefully designed studies that employ random sampling, random assignment to condition, control groups, large Ns, and analysis techniques that allow for the accounting of the myriad confounding variables that are inevitable in the study of socially situated activities such as teaching and learning. Generalization from such studies is bounded and constrained by descriptions of subjects, teachers, teacher training in the technique under study, fidelity of intervention implementation, and evidence of sustainability and scalability.
Clearly, this ideal, requisite though it may be, is difficult to attain, expensive to accomplish, and limited to answering only those questions that can actually be examined with such methodologies (Boruch, 1999, & Gueron, 1999, single online document). Gueron, drawing on her experience with research on training programs and welfare reform, lists eight criteria useful for judging when random assignment is the right tool:
1. When the key question is “Does the program or reform make a difference?” (questions of program impact)
2. When the program under study is sufficiently different from business as usual and you can maintain that distinction over time
3. When the research does not denying people access to an entitlement
4. When the researchers are addressing an important or unanswered question
5. When adequate procedures are in place to inform program participants about the study and to ensure confidentiality of data
6. When there is no easier way to get a good answer
7. When you can get cooperation
8. When you have the resources and ability to do a quality study
While experimental and quasiexperimental studies are the gold standard for examining program impact, there are many critical issues facing education and educators that raise other types of questions. For example, most reasonable people would probably agree that a central concern in the field of education is that of differential achievement rates. Throughout the history of public education, children and adults from nonmainstream groups or for low socioeconomic status (SES) have as a whole failed to achieve academic success to the same levels and at the same rate as learners from mainstream, sociopolitically empowered groups (Kaestle, Damon-Moore, Stedman, Tinsley, & Trollinger, 1991). Descriptive and correlational studies (both types of qualitative research) repeatedly have confirmed this disturbing pattern. Other studies using qualitative and quantitative research methods have examined possible causes as subcorrelates -- language differences, cognitive differences, differential access to books, inequalities in schooling, and so on. Clearly, without the ability, or right, to control probable confounding variables or randomly assign to condition (of socioeconomic status?), we can never establish true causation. However, we can gain real insight into the issue by employing nonexperimental methods. Further, with judicious and careful synthesis, we can come to understand the landscape of nonmainstream status and educational achievement to such a degree that we may be able to design experimental studies that examine the impact of educational programs on this SES-related achievement gap.
Many of our pressing and unanswered educational questions are of this type -- deeply embedded in issues of race, culture, class, gender, and family income. For most of these questions, we do not possess sufficient information or insights, gained through systematic, rigorous, nonexperimental research, to design reasonable experimental interventions and research studies that meet the criteria set forth by Gueron (1999), listed above.
LeCompte and Schensul (1999, p. 2) describe ethnography, one type of nonexperimental research that uses both qualitative and quantitative data, as a research method “designed for discovery.” Ethnography, they point out, is particularly suited to gaining insight into questions embedded in social and cultural communities and practices such as education. It is scientific, investigative, uses the researcher as the primary tool of data collection, employs rigorous research methods and data collection techniques to avoid bias and ensure accuracy of data, emphasizes and builds on the perspectives of the people in the research setting, and is inductive, building local theories for testing and adapting for use both locally and elsewhere.
The rigor represented in codified ethnographic research methods produces scientifically valid and reliable data. The same is true for other nonexperimental research designs using qualitative and quantitative methods -- descriptive and correlational methodologies, and different combinations of these. Each type of research has its own criteria for rigor, and each answers different types of questions. Thus, all types of research, from experimental to ethnographic, provide us with important lenses through which to study and examine critical educational questions and issues. Because they are different, each type of research provides us with unique answers and insights, and each type of research presents its own limitations affecting generalizability, types of questions it can address, and the nature of the insights it can offer.
Like many studies of its kind, however, I cannot help but feel that the authors are over-generalizing based on relatively little data. The number of subjects were small and there was really no control. They had an unexpected finding (that girls actually over-regularized past tenses) and proposed an alternative explanation which is based on the better lexical memory of the girls and therefore their ability to use associative learning from previous examples. The study would have been far more rigorous if they had done the study on larger numbers of boys and girls that had been exposed to different training. The rigorous study would have been a large controlled study where boys and girls were randomized to different training methods and then tested for the degree to which they over-regularized their expression of verb tenses.
http://explore.georgetown.edu/news/?ID=20915
Study of Language Use in Children Suggests Sex Influences How Brain Processes Words
Washington, D.C.--Boys and girls tend to use different parts of their brains to process some basic aspects of grammar, according to the first study of its kind, suggesting that sex is an important factor in the acquisition and use of language.
Two neuroscientists from Georgetown University Medical Center discovered that boys and girls use different brain systems when they make mistakes like “Yesterday I holded the bunny”. Girls mainly use a system that is for memorizing words and associations between them, whereas boys rely primarily on a system that governs the rules of language.
“Sex has been virtually ignored in studies of the learning, representation, processing and neural bases of language. This study shows that differences between males and females may be an important factor in these cognitive processes,” said the lead author, Michael Ullman, PhD, professor of neuroscience, psychology, neurology and linguistics.
He added that since the brain systems tested in this study are responsible for more than just language use, the study supports the notion that “men and women may tend to process various skills differently from one another.” One potential underlying reason, suggested by other research, is that the hormone estrogen, found primarily in females, affects brain processing, Ullman said.
The study, whose co-author is Joshua Hartshorne, was published earlier this year in the journal Developmental Science.
Researchers know that women tend to be better than men at verbal memory tasks, such as remembering word lists, and that this ability depends on declarative memory. Included within declarative memory is a “mental lexicon” in which word forms are memorized and remembered. The grammatical rules that allow us to combine words in sentences depend on “procedural” memory. Researchers have found that both boys and girls may be equally adept at this process, which depends on a different part of the brain than declarative memory.
In this study, Ullman and Hartshorne hypothesized that girls would be better than boys at remembering irregular past-tenses of verbs, like “held”, since these words are memorized in declarative memory. And if girls remember “held” better than boys, they should make fewer errors like “holded”, since these “over-regularization” errors are made when children can’t remember irregular past-tenses, and so resort to combing the verb with an –ed ending, just as they do for regular verbs like “walked”.
So they studied how a group of 10 boys and 15 girls, age 2 to 5, used regular and irregular past-tense forms in their normal speech. To their surprise, and contrary to their predictions, the researchers discovered that the girls over-regularized far more than boys.
They then investigated which verbs the girls made the mistakes on, and found an association between the number of similar sounding regular past-tense verbs, and the particular verb that was over-regularized. For example, girls tended to say “holded” or “blowed” because many other rhyming verbs use the regular past-tense form (such as folded, molded, and flowed, rowed, stowed, respectively).
The researchers say this kind analogy-based processing suggests the girls were relying on their declarative memory to create the past tense. “This memory is not just a rote list of words, but underlies common patterns between words, and can be used to generalize these patterns,” Ullman said. “In this case, the girls had memorized the regular past tenses of rhyming words, and were generalizing these patterns to new words, resulting in over-regularization errors” such as “holded” and “blowed”.
In contrast, for the boys there was no association between the number of similar sounding regular past-tense verbs, and the particular verbs that were over-regularized. So the boys did not make more over-regularizations on verbs like “holded” or “blowed” that have many rhyming regular past-tenses. This suggests, Ullman said, that the boys were not forming these words in declarative memory, but were probably using the rule-governed system to combine verbs with –ed endings.
Other types of evidence also suggest that adult women tend use declarative memory more than adult men do in their use of language, Ullman said. “Although the two sexes seem to be doing the same thing, and doing it equally well, they are using two different neurocognitive brain processes to do it,’ Ullman said. “This is a novel and exciting finding.”
About Georgetown University Medical Center
Georgetown University Medical Center is an internationally recognized academic medical center with a three-part mission of research, teaching and patient care (through our partnership with MedStar Health). Our mission is carried out with a strong emphasis on public service and a dedication to the Catholic, Jesuit principle of cura personalis -- or "care of the whole person." The Medical Center includes the School of Medicine and the School of Nursing and Health Studies, both nationally ranked, the world-renowned Lombardi Comprehensive Cancer Center and the Biomedical Graduate Research Organization (BGRO).
Note that I reproduced the entire contents of the article (because it was a press release). Here is the abstract of the study.
Hartshorne JK and Ullman MT (2006). Why girls say 'holded' more than boys. Dev Sci 9: 21-32. Women are better than men at verbal memory tasks, such as remembering word lists. These tasks depend on declarative memory. The declarative/procedural model of language, which posits that the lexicon of stored words is part of declarative memory, while grammatical composition of complex forms depends on procedural memory, predicts a female superiority in aspects of lexical memory. Other neurocognitive models of language have not made this prediction. Here we examine the prediction in past-tense over-regularizations (e.g. holded) produced by children. We expected that girls would remember irregular past-tense forms (held) better than boys, and thus would over-regularize less. To our surprise, girls over-regularized far more than boys. We investigated potential explanations for this sex difference. Analyses showed that in girls but not boys, over-regularization rates correlated with measures of the number of similar-sounding regulars (folded, molded). This sex difference in phonological neighborhood effects is taken to suggest that girls tend to produce over-regularizations in associative lexical memory, generalizing over stored neighboring regulars, while boys are more likely to depend upon rule-governed affixation (hold+-ed). The finding is consistent with the hypothesis that, likely due to their superior lexical abilities, females tend to retrieve from memory complex forms (walked) that men generally compose with the grammatical system (walk+-ed). The results suggest that sex may be an important factor in the acquisition and computation of language. Brain and Language Laboratory, Departments of Neuroscience, Psychology, Linguistics and Neurology, Georgetown University, Washington, DC 20057, USA. http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=16445392
This study brings up an interesting and complicated issue of the rigor of evidence based standards applied in the field of education compared to biomedical science. Although clinical trials are often not accorded much respect by scientists in other fields, I think that the level of evidence required for proof of risk and benefit in clinical trials far exceed those in other fields of science. For example, the evidence that is presented in this study would be considered no better than phase 1 or, at best, phase 2 clinical trials, showing some promising results but not enough to generate approval by the FDA. It shows feasibility and some promising results. Yet, this kind of data may lead to implication of education policies that utilize billions of dollars every year.
The National Institute of Child Health and Development (NICHD), the premier NIH institute that is responsible for child health research in the country, was recently recruited into the "no-child left behind" program by the Bush Administration. The thought that was medical clinical trial designs be used to test educational theories that are being used or considered for use in the country. I must say that I personally thought that this did not belong in the province of NIH and should have been funded by the Department of Education, and that it took valuable resources from the biomedical field. On the other hand, it was very interesting how much impact biomedical research is beginning to have on education research where they don't have a long tradition of controlled trials. To tell you the truth, neither has medicine. For example, randomized clinical trials were not a requirement in medicine until the 1970's, when the FDA adopted the now widely used standard of phase 1, phase 2, and phase 3 trials for approval of treatments.
A strong movement for evidence-based practices has developed in the educational field (Source (http://www.wrightslaw.com/nclb/rbi.htm)), required by the No-Child-Left-Behind-Law:
http://www.wrightslaw.com/nclb/rbi.htm
A primary focus of this law is the requirement that school districts and individual schools use effective research-based reading remediation programs so all children are reading at grade level by the end of third grade. The law authorizes funds 'to provide assistance to State educational agencies and local educational agencies in establishing reading programs for students in kindergarten through grade 3 that are based on scientifically based reading research, to ensure that every student can read at grade level or above no later than the end of grade 3.' (20 U.S.C.§ 6361)" (page 73, Wrightslaw: No Child Left Behind)
Led by Reed Lyon at NICHD, the field of learning disabilities is beginning to adopt biomedical standards of evidence (Source (http://www.ldonline.org/indepth/reading/nichdbrochure.html)).
http://nhida.org/docs/ReadingDisabilities.pdf
Reading Disabilities: Why Do Some Children Have Difficulty Learning to Read?
What Can Be Done About It?
by G. Reid Lyon, Ph.D.
The National Institute of Child Health & Human Development (NICHD) considers that teaching
and learning in today’s schools reflect not only significant educational concerns, but public health
concerns as well. Our research has consistently shown that if children do not learn to understand
and use language, to read and write, to calculate and reason mathematically, to solve problems,
and to communicate their ideas and perspectives, their opportunities for a fulfilling and rewarding
life are seriously compromised. Specifically, in our NICHD-supported longitudinal studies, we
have learned that school failure has devastating consequences with respect to self-esteem, social
development, and opportunities for advanced education and meaningful employment. Nowhere
are these consequences more apparent than when children fail to learn to read. Why? Simply
stated, the development of reading skills serves as THE major foundational academic ability for
all school-based learning. Without the ability to read, the opportunities for academic and
occupational success are limited. Moreover, because of its importance, difficulty in learning to
read crushes the excitement and love for learning, which most children have when they enter
school.
<more>
A number of organizations have sprung up to promote evidence-based education policies in the United States (Source[/ur]). The term evidence-based education, however, has been controversial because there are many who believe that there are complexities and subtleties to teaching that will be very difficult, if not impossible, to capture in randomized trial designs (Source (http://www.cemcentre.org/RenderPage.asp?LinkID=30310000)). A new Institute of Education Science was created to replace the Office of Education Research and Improvement (Source (http://eduscapes.com/tap/evidence.html)). As part of the shift towards better accountability, the Department of Education has introduced the National Education Technology Plan (http://www.ed.gov/about/offices/list/os/technology/plan/2004/site/edlite-default.html)). The Department of Education has issued guidelines for "identifying and implementing educational practices supported by rigorous evidence" (Source (http://www.ed.gov/rschstat/research/pubs/rigorousevid/index.html)). The field of "instructional experiments" has begun (Source (http://www.readingonline.org/articles/art_index.asp?HREF=/articles/rt/9-03_column/)). It would be interesting to know what people think.
As Michael Priestly pointed out
http://www.readingonline.org/articles/art_index.asp?HREF=/articles/rt/9-03_column/
A Few Things Reading Educators Should Know About Instructional Experiments
Michael Pressley
I am confronted almost daily by the 2001 No Child Left Behind (NCLB) legislation and its demand that reading instruction be scientifically evidence based. Assertions are also being made that reading instruction provided to children in the United States should be evaluated in randomized experiments, especially reading instruction supported by public funds, such as those provided by the Reading First program.
As an experimental psychologist, I have contributed many "true" experiments (i.e., experiments with random assignment to condition) to the scientific literature. I am proud that I have mastered the ability to conduct research experiments. I am also proud that as part of my education I learned a great deal about the strengths and weaknesses of experimentation—when experiments are credible and useful and when they are not.
Many of the contemporary assertions about experimentation and reading instruction concern me. Many claims are debatable at best and sometimes just plain wrong. I spoke about my concerns at the 2002 International Reading Association Convention. At the conclusion of that address, the editors of The Reading Teacher requested that I prepare an article instructive to teachers about experimentation and evidence-based reading instruction.
I present in this article 12 points about experimentation that I think all teachers concerned with evidence-based reading instruction should keep in mind as they evaluate the flurry of assertions that are overflowing the marketplace of ideas in literacy education. After presenting these points, I reflect briefly on how responsible experimenters who wish to inform literacy instruction should communicate research results to classroom practitioners.
<more>
It would be interesting to know what people think of this development. If anything, this will be pointed to as one of the legacies of the Bush Administration in future years. It is important to note that the trend began before George Bush took office. As pointed out by Martha Rapp Ruddel, the Chair of the National Reading Conference Committee:
http://www.readingonline.org/articles/art_index.asp?HREF=/articles/purcell-gates/index.html
In spring 1998, the director of the National Institute of Child Health and Human Development appointed the National Reading Panel (NRP) to carry out a request by the United States Congress to study the effectiveness of various approaches to teaching children how to read. The panel subsequently defined the procedures by which it would conduct its business and the questions that it would use to guide its study. The panel has now published its results, available at [url]www.nationalreadingpanel.org/Documents/default.htm.
The National Reading Conference (NRC), a professional organization dedicated to literacy research, created a committee to respond to the NRP report by addressing the following questions:
1. Did the panel do what it said it would do to answer the questions it had posed?
2. What, if anything, did the panel leave out within the questions it created and in defining the questions themselves?
3. What research traditions, if any, have not been considered in the panel's work?
Following is the first in a planned series of commentaries prepared by members of the NRC committee. It presents one response to the third question.
Martha Rapp Ruddell
Chair, National Reading Conference Committee to Respond to the National Reading Panel
Past President, National Reading Conference
As Victoria Purcell-Gates pointed out in the above cited web site,
http://www.readingonline.org/articles/art_index.asp?HREF=/articles/purcell-gates/index.html
The Role of Qualitative and Ethnographic Research in Educational Policy
Victoria Purcell-Gates
The National Reading Panel, charged with identifying research that could inform policy and practice in reading, has now concluded its exhaustive review. The panel focused its review on experimental and quasiexperimental research which, by necessity, meant that it focused on a small set of questions. I say “by necessity” because the areas of instruction in reading that have been examined by experimental or quasiexperimental research are of themselves few and far between, although certainly of central importance to the learning-to-read process.
The point I wish to argue here is that by limiting the type of research used to address the issue of how to guide policy and practice in reading instruction to experimental and quasiexperimental studies, the panel missed critical areas that have been examined by qualitative and ethnographic research. Further, a clear danger resulting from this limitation to particular types of methodologies is the establishment in the minds of funders and users of research that only experimental and quasiexperimental research provide us with answers to educational problems. A second theme of this commentary, then, is my view that we do not stand a chance of solving problems of learning and teaching if we confine ourselves solely to these methodologies. Rather, it is only by the considered and judicious use of a range of methodologies, rigorously applied and intelligently synthesized, that we as educators, policy makers, and the informed and concerned public can hope to make real advances and reforms to our system and in our procedures for educating all our students to their full potentials.
Proving, Informing, or Both?
There is no doubt that experimental, and to some degree quasiexperimental, research is required to “prove” the effectiveness of an instructional approach, method, or intervention. This to some degree explains the panel's exclusion of other methodologies in its research review (see, e.g., Shanahan, 1999; online document). Identifying the effects of specified instructional techniques can only be accomplished with carefully designed studies that employ random sampling, random assignment to condition, control groups, large Ns, and analysis techniques that allow for the accounting of the myriad confounding variables that are inevitable in the study of socially situated activities such as teaching and learning. Generalization from such studies is bounded and constrained by descriptions of subjects, teachers, teacher training in the technique under study, fidelity of intervention implementation, and evidence of sustainability and scalability.
Clearly, this ideal, requisite though it may be, is difficult to attain, expensive to accomplish, and limited to answering only those questions that can actually be examined with such methodologies (Boruch, 1999, & Gueron, 1999, single online document). Gueron, drawing on her experience with research on training programs and welfare reform, lists eight criteria useful for judging when random assignment is the right tool:
1. When the key question is “Does the program or reform make a difference?” (questions of program impact)
2. When the program under study is sufficiently different from business as usual and you can maintain that distinction over time
3. When the research does not denying people access to an entitlement
4. When the researchers are addressing an important or unanswered question
5. When adequate procedures are in place to inform program participants about the study and to ensure confidentiality of data
6. When there is no easier way to get a good answer
7. When you can get cooperation
8. When you have the resources and ability to do a quality study
While experimental and quasiexperimental studies are the gold standard for examining program impact, there are many critical issues facing education and educators that raise other types of questions. For example, most reasonable people would probably agree that a central concern in the field of education is that of differential achievement rates. Throughout the history of public education, children and adults from nonmainstream groups or for low socioeconomic status (SES) have as a whole failed to achieve academic success to the same levels and at the same rate as learners from mainstream, sociopolitically empowered groups (Kaestle, Damon-Moore, Stedman, Tinsley, & Trollinger, 1991). Descriptive and correlational studies (both types of qualitative research) repeatedly have confirmed this disturbing pattern. Other studies using qualitative and quantitative research methods have examined possible causes as subcorrelates -- language differences, cognitive differences, differential access to books, inequalities in schooling, and so on. Clearly, without the ability, or right, to control probable confounding variables or randomly assign to condition (of socioeconomic status?), we can never establish true causation. However, we can gain real insight into the issue by employing nonexperimental methods. Further, with judicious and careful synthesis, we can come to understand the landscape of nonmainstream status and educational achievement to such a degree that we may be able to design experimental studies that examine the impact of educational programs on this SES-related achievement gap.
Many of our pressing and unanswered educational questions are of this type -- deeply embedded in issues of race, culture, class, gender, and family income. For most of these questions, we do not possess sufficient information or insights, gained through systematic, rigorous, nonexperimental research, to design reasonable experimental interventions and research studies that meet the criteria set forth by Gueron (1999), listed above.
LeCompte and Schensul (1999, p. 2) describe ethnography, one type of nonexperimental research that uses both qualitative and quantitative data, as a research method “designed for discovery.” Ethnography, they point out, is particularly suited to gaining insight into questions embedded in social and cultural communities and practices such as education. It is scientific, investigative, uses the researcher as the primary tool of data collection, employs rigorous research methods and data collection techniques to avoid bias and ensure accuracy of data, emphasizes and builds on the perspectives of the people in the research setting, and is inductive, building local theories for testing and adapting for use both locally and elsewhere.
The rigor represented in codified ethnographic research methods produces scientifically valid and reliable data. The same is true for other nonexperimental research designs using qualitative and quantitative methods -- descriptive and correlational methodologies, and different combinations of these. Each type of research has its own criteria for rigor, and each answers different types of questions. Thus, all types of research, from experimental to ethnographic, provide us with important lenses through which to study and examine critical educational questions and issues. Because they are different, each type of research provides us with unique answers and insights, and each type of research presents its own limitations affecting generalizability, types of questions it can address, and the nature of the insights it can offer.