Attempting to find quantitative studies in L2/composition turned out to be particularly difficult this week, as I suspected it might be. I did a fairly thorough search of at least five of the journals I’ve been examining this semester. In journals that don’t specialize, such as College English it was difficult enough to find articles related to ESL issues, and that was before the quantitative work came into play. Likewise, the Journal of Basic Writing had very little overlap for the two, and in the Journal of Second Language Writing when using “quantitative” as either a keyword or word in the title, only 26 results total came up. It was about the same in the TESOL Quarterly in which looking at either keywords in the abstract or title about 30 results were returned. Clearly, we’re not a discipline that values our quantitative research very much, or I am somehow looking in the wrong places.

I did find one article that spoke to some of the likely deficiencies in quantitative research that shed some light on this general lack in the field of ESL/SLA. In “Statistical Literacy Among Applied Linguists and Second Language Acquisition Researchers” by Shawn Loewen et al., they note that statistical and quantitative analysis is very important in both applied linguistics and SLA, but also that it is “relatively rare” (361), noting that a 2000 study by Lazaraton found that over 90 percent of all journal publications were qualitative studies rather than quantitative in the field of SLA (363). Ironically, the authors note, this is despite the fact that many studies have shown that most teachers in the field, no matter whether they come from TESOL, applied linguistics, or other educational backgrounds, have statistics background, with Lazaraton’s study showing that on average, most participants in his study had taken two statistics or research methods courses (365).

Because such a quantitative study of statistics methods had not been done, really, since the comprehensive Lazaraton study, Loewen et al. wanted to follow up and find out more about how statistics were being used in these fields in 2014. They sent out 1000 surveys and received 331 replies (366). After analyzing the data, they discovered that 81 percent of the respondents had taken some sort of statistics class (369), although 40 percent of PhD students did not feel they had adequate training in statistics, along with 30 percent of professors (370). The study also reaffirmed that most of the respondents still felt more prepared for qualitative studies than quantitative (375). They conclude by suggesting that it is important to improve statistical literacy for quality research and encourage further quantitative work in these fields (377-378).

I thought this piece was interesting because although it did not have insight into composition, it did back up my struggle to find quantitative data when I searched among a wide variety of journals, including journals for composition, which would suggest that this is a “problem” across both disciplines. The fact that 90 percent of studies being done are qualitative shows that even if teachers feel they are at least somewhat trained in quantitative methods, they are certainly not comfortable using them. What this study shows me is that more quantitative data is needed to help to quantify in new and interesting ways the gaps between ESL and composition. Such data could provide further insights into how to best help these populations in new and meaningful ways.

In addition to the Loewen et al. article, in “Focus Article: Replication in Second Language Writing Research” by Graeme Porte and Keith Richards, the authors argue that research methods in the field of L2 writing are missing significant replication studies that would show further validity of pas results, both qualitatively and quantitatively. They argue that despite this ongoing lack, which they argue has been over the course of L2 language studies as a discipline, replication is “feasible, necessary, and publishable” (285).

First, they examine issues with quantitative studies, noting that often “novelty” is what drives research in the field, with no one wanting to replicate. Unfortunately this leaves large gaps in our knowledge and its validity, which, particularly in L2, which is a “young field.” They note that while there is much to be explored, discovering what we already know about our present knowledge is equally important (285). They also note that error happens in data collection and analysis and replication can help to fill in some of these details about what we do know. They suggest that one reason the field might shy away from replication is the idea that only exact or literal replication, which is nearly impossible, will work. However, Porte and Richards argue that the more flexible approximate replication is enough. In this research, getting matching results is not the goal or the ideal, but instead discovering that if there are nonmatching results, and analyzing what this might mean for the field or that particular data set, is still important. Such work will tell us more about our learners than attempting new research constantly, they believe (286). To begin to meet such a goal, instead of simply publishing and stating that “more work” can be done, Porte and Richards suggest that authors instead give enough background on the study so it can be replicated by other readers (287).

Likewise, in quantitative research, some researchers have shied away from replication because of the belief that “’human behavior is never static, [therefore] no study can be replicated exactly regardless of the methods and designs employed.’” (288). Porte and Richards dismiss this logic but suggest that it is important for researchers to include full and complete description for their “research design, data collection, and analytical procedures” so the work can be replicated, even with another set of people. They also argue that technology should make this easier, as authors can make further data available to others who want to replicate their study (289).

Finally, Porte and Richards argue that without replication in both qualitative and quantitative work, “we might expect at best considerable conflict among research outcomes, and at worst, confusion and stagnation.” It is only through replication, they argue, that we can better understand the field and how to help our learners (291).

Despite the fact that this was, again, not an actual quantitative study, it’s an interesting one to think about in the sense that again, there are big gaps in our knowledge as far as empirical methods because of the practices (or lack of certain practices) used in our field(s). Like Loewen et al. point out, it’s clear that there are large gaps in the way we obtain and analyze data about our students and our fields, which is doing a disservice us all. It seems that if there are gaps in each of these fields (as divergent as they clearly are), there are almost certainly not studies of the kind I am truly looking for – ones at the intersection of ESL and composition. Therefore, further work in both qualitative and quantitative methods seems really crucial for further understanding the overlaps and gaps I have noticed.

Despite mostly finding articles this week on the gaps in our research, I did manage to find one interesting study that did seem to address L2 and composition using some qualitative methods that is worth addressing. In John Hedgcock and Natalie Lefkowitz’s article “Feedback on Feedback: Assessing Learner Receptivity to Teaching Response in L2 Composing,” they want to discover more about the differences in feedback types preferred by foreign language and L2 students. They start by noting that such studies have been undertaken for L1 writers, but not for L2 students (142). They do note one such study by Leki (1991) that they attempt to partially replicate with some moderate changes, which had discovered that L2 students generally “display a strong concern for grammatical accuracy” while L1 learners usually do not prioritize grammar in their writing (143). Likewise, they discover that the most (ironically) consistent finding in their research background is how disparate the types of feedback given by L1 and L2 writing instructors is, with some focusing on “substance, organization, and writing style” and others on “spelling, capitalization, and punctuation (144). However, despite these differences, both L1 and L2 instructors often focus on giving learners time to reflect, seek clarity and coherence, and deemphasize correction of grammar until the later stages. However, Hedgcock and Lefkowitz note that given the “wide variation” in teacher response, it is important to examine what is most useful in the eyes of the students (144-145).

Because Hedgcock and Lefkowitz feel that some of the work looking at differences in L1 and L2 feedback assessment has been done, they instead decided to focus their quantitative survey study (sent to 247 writers) on ESL vs. foreign language (FL) writers (146). What they discover are fairly large discrepancies in the types of feedback these writers prefer. The most important findings are that while all writers in both groups most prefer written feedback combined with writing conferences, and both groups were worried about grammatical accuracy, the ESL writers cared much more about idea development, style, and sequencing, and FL writers cared more about formal accuracy (150-151). The authors note this difference is likely because ESL students care more about the work they will encounter in freshman composition classes, which care more about “the generation of substantive ideas” instead of “editorial concerns” (151). Therefore, ESL instructors need to know how to teach students to practice the rhetorical styles they will encounter in their various academies, as well as being more consistent with their teaching of a process over product approach (152).

This is a fascinating article in the way that it does use some quantitative data to compare overlaps not only in ESL and FL but also to composition in some important ways. What it shows is that regardless of the types of feedback ESL students are receiving in the academy, these students have a fairly good idea of the types of feedback and the writing opportunities they need to be successful in the academy. In fact, as writing is going to be an important part of their studies outside of English classes, being clear that rhetorical knowledge is more important than perfect grammatical accuracy is insightful and promising. What this shows is that if more overlap is being done between the fields of composition and ESL, not only do students perceive that they are gaining more from their coursework, but ultimately they probably are having better outcomes.

One limit on this study, I think, is that it is only a survey of students. Something that could make this more well-rounded would be additional information regarding the types of assignments these students were encountering in their classes and how this also fed into the types of feedback they were interested in. Likewise, knowing how their teachers assessed them would be equally meaningful as this might also affect the types of feedback they preferred. Additionally, though Hedgcock and Lefkowitz note that they undertook this study to look at ESL vs. FL learners because L1 and L2 learners had already been compared in the past, I am disappointed that the study itself did not focus further on L1 writers, as the findings here would have been very useful for my own work. Perhaps this is an avenue for replication in my own future research.

Though I don’t know if I ultimately found what I was looking for this week, what I did discover gave some good insight into the types of research and work our fields are doing and the types that are lacking. I believe that the lack of good, critical qualitative data that would help us look more clearly at the split between L1 and L2 writing courses is something that needs further work (and replication!). While Hedgcock and Lefkowitz got me a bit closer to seeing some of those overlaps, and in fact replicated part of Leki’s 1991 study, it seemed that they stopped a little bit short of what more they could have done with quantitative methods and making their study a bit more all-encompassing and useful. Finding further ways to introduce and assess L2 writers on the types of writing instruction they need to be successful beyond the ESL classroom appears to be a great place for future quantitative research across both disciplines. Likewise, nearly any quantitative research in the field(s) would help to open up new, promising avenues of discovery and further exploration.

Works Cited

Hedgcock, John, and Natalie Lefkowitz. “Feedback on Feedback: Assessing Learner Receptivity to Teaching Response in L2 Composing.” Journal of Second Language Writing, vol. 3, no. 2, 1994, pp. 141-163.

Loewen, Shawn et al. “Statistical Literacy Among Applied Linguists and Second Language Acquisition Researchers.” TESOL Quarterly, vol. 48, no. 2., 2014, pp. 360-388.

Porte, Graeme, and Keith Richards. “Focus Article: Replication in Second Language Writing Research.” Journal of Second Language Writing, vol. 21, 2012, pp. 284-293.