Mastering Academic Language: Organization and Stance in the Persuasive Writing of High School Students

Abstract


Introduction
In a world where classrooms are becoming increasingly diverse, where students with distinct languages, different socio-economic statuses, various ethnicities and ways of communicating are interacting ever more closely with one another, understanding the within-grade variability of writing performances is critical to better serve all students.As the writing demands of our current society continue to increase (National Commission on Writing, 2004), we need to search for innovative pedagogical tools and strategies to respond more effectively to the needs of all students 1 .We argue in this paper that while teachers are well aware of differences in writing performances across students in their classes, more precise research-based tools to identify adolescents' strengths and weaknesses in academic writing are sorely needed to inform more individualized instruction.Currently, students are assessed in schools mostly through wholistic rubrics that offer useful global judgments but no precise information to guide targeted teacher feedback (Alderson, 2007;Beck, Llosa, & Zhao, 2011).The present study is motivated by our conviction that educational linguistics can be instrumental in generating relevant findings for teachers and researchers to work together towards the design of pedagogically informative research-based tools.While acknowledging the multicomponential nature of writing development and the numerous social and cognitive factors that influence writing performances, this study focuses exclusively on the academic language of adolescent students' writing.
Undeniably, the language proficiency of a writer constitutes a critical dimension, even if only one among many others involved in mastering skilled writing.Beyond the mechanics of writing and the linguistic conventions of standard English, we conceive academic writing proficiency as 1 In the 2007 NAEP assessment, for instance, only 24% of 12 th grade students performed at the Proficient or Advanced level (Salahu-Din, Persky & Miller, 2008).Additionally, despite recent small increases in overall writing test scores, the gap in writing scores between historically privileged White students and historically ill-served students from Hispanic or African-American backgrounds has not displayed a significant improvement since 1998 (National Assessment of Educational Progress, 2008).
the flexible use of a repertoire of later-acquired lexico-grammatical and discourse forms to organize ideas and express a stance in a variety of school texts (Ravid & Tolchinsky, 2002;Snow & Uccelli, 2009).With the ultimate purpose of contributing to the design of pedagogically relevant tools, our study explores lexical, syntactic, and discourse-based features as predictors of overall academic writing quality in persuasive essays produced by an ethnically and linguistically diverse group of high school students.

Academic writing: A socio-cognitive pragmatics-based framework
Influenced by pragmatics-based theories of language acquisition (Ninio & Snow, 1996;Snow & Uccelli, 2009), functional linguistic approaches (Berman & Ravid, 2009;Halliday, 2004;Schleppegrell, 2004), and ethnographic research on language and literacy (Heath, 1983;Ochs, 1984), we understand oral and written uses of language as socioculturally situated cognitive practices (Blum-Kulka, 2008;Bazerman & Paradis, 2004).Thus, we see individuals' language performances as always influenced not only by the immediate context, but also by the history of opportunities they have had to use oral language and written texts in particular ways (Ochs, 1984;Heath, 1983).This framework implies that as learners grow and navigate an expanding set of language-mediated social contexts, they continue to learn new ways of speaking and writing.
This view also implies that some speakers/writers might be successful language users in some social contexts (e.g., sharing personal anecdotes with friends), yet much less skilled in other contexts (e.g., constructing effective arguments at school).We conceptualize adolescent writing as part of adolescents' development in "rhetorical flexibility", i.e., the ability to flexibly use an increasing repertoire of language forms and functions in an ever-expanding set of social contexts, orally and in writing (Ravid & Tolchinsky, 2002).Students come to school with abundant language resources learned through situated communicative experiences within their native communities which undeniably need to be valued and incorporated into instruction (Delpit, 1992;Gee, 2001).Yet, it is also important to attend to the new language demands of school learning, in particular in light of research documenting the substantial language challenges faced by many adolescent struggling readers and writers (Schleppegrell, 2001(Schleppegrell, , 2004;;Snow & Uccelli, 2009).

The linguistic demands of academic writing
At school, students are expected to master not only new genres or text types (e.g., explanation, argumentation), but also new school-relevant registers or language repertoires prevalent in the social context of school (e.g., the language of history, the language of science) (Bazerman & Paradis, 2004).The progress in mastering new genres or types of texts has been characterized by Martin (1989a) and Schleppgrell (2004) as moving progressively across three categories: (1) personal genres, such as narratives and recounts; (2) factual genres, such as procedures and reports; and (3) analytic genres, those focused on analysis and argumentation (explanations, persuasive or argumentative essays).Recent empirical data suggest, indeed, that while written narrative organization tends to be well-mastered by age 10, analytic genres constitute a later developmental accomplishment (Berman & Nir-Sagiv, 2007).The persuasive essay is a particularly prominent school genre which consists of a writer's position or thesis about a topic, followed by organized stepwise argumentation that includes precise claims, data, warrants, counterarguments, and rebuttals that lead to a well-justified conclusion (Toulmin, 2003).It is during the middle school years that persuasive essays start to be introduced consistently in writing instruction and assessment, with the expectation that students will become skillful writers of this genre by the end of high school (Hillocks, 2002).High-stake exams required for college admission, such as the SAT or ACT, routinely assess students' writing by eliciting a persuasive essay that needs to be produced within a limited timeframe (College Board, 2012).In reaction to these testing practices, researchers and teachers have called for more valid and authentic assessments (Hillocks, 2011).The need for future research on better alternatives for writing assessment cannot be sufficiently emphasized.Yet given the influential role of current high-stakes testing methods in determining students' future opportunities, in this study we explore written persuasive essays produced by students in the context of a timed SAT-like prompt.
As students gradually master the organization of new academic genres, they also need to learn the academic registers characteristic of school texts.Registers comprise the constellation of lexical, grammatical and discourse features that characterize and are prevalent in particular social contexts to accomplish specific purposes (Bar-Ilan & Berman, 2007;Biber, 1995;Halliday and Hassan, 1989).As many other school-relevant genres, academic persuasive essays are expected to fulfill expectations characteristic of more academic registers, such as: (1) lexical precision, (e.g., using diverse and precise vocabulary); (2) dense information packing (e.g., including nominalizations and complex syntax); (3) explicit discourse organization (e.g., using markers to signal text transitions); and, among others, (4) academic stance, (e.g. using markers that index the writer's attitude towards the claims advanced) (Schleppegrell, 2004;Snow & Uccelli, 2009) Even though academic discourses constitute pragmatic solutions generated to facilitate formal study of abstract ideas and precise scientific communication, many academic words, grammatical and discourse structures are so different from more colloquial ways of using 2 We acknowledge that different content areas in schools have unique language expectations, including specialized vocabulary, and langauge that reflects disciplinary forms of reasoning and methods for evaluating evidence (see Bazerman & Paradis, 2004or Toulmin, 2003 for details of these differences).However, in this study we focus on a few language forms and functions that tend to be prevalent across disciplines as they help to fulfill general expectations of a variety of academic registers, such as communicating in a precise and concise manner while expressing a cautious stance towards claims advanced.
language that many adolescents find them obscure and challenging (Schleppegrel, 2004;Snow & Uccelli, 2009).Schleppegrell (2001) argues that the linguistic expectations of school-based tasks are rarely made explicit to students.Teachers are so well-versed in the language of school that usually the complexity and challenges of the additional language forms students need to learn across content areas go unnoticed (Bailey, Burkett, & Freeman, 2008).This line of research implies that, unless the language needs of students are explicitly addressed, the educational system runs the risk of reproducing the socio-political structure from the outside world by rewarding privileged students for what they have mastered spontaneously outside of school, while complicating the path to academic success for students who have not mastered such language.Joining forces with other researchers currently working in this area (Bailey, 2007;Christie & Derewianka, 2008;Scarcella, 2003;Schleppegrell, 2004;Zwiers, 2007), our study seeks to shed initial light on relevant language elements present in adolescents' academic writing.Borrowing tools from Systemic Functional Linguistics (Halliday, 2004) and applying a pragmatics-based framework for analyzing academic language (Snow & Uccelli, 2009), our study explores well-established measures of lexical diversity (how many different words are used in an essay), syntactic complexity (how complex are the clauses used), and lexical density (how much information is packed into a single clause) in high school students' persuasive essays (Schleppegrell, 2004).In addition, informed by prior research on textual analysis (Hyland, 2005) and developmental linguistics studies on adolescents' stance -or propositional attitudes- (Berman, Ragnarsdóttir, & Strömqvist, 2002;Reilly, Baruch, Jisa, & Berman, 2002), we investigate two more innovative discourse dimensions: (1) discourse organization, and (2) writer's stance.Organization and stance constitute two dimensions that are present from the onset of discourse development.By age 9 or 10, informal narrative discourse and its correspondent organizational markers (e.g., temporal markers) and stance markers (e.g., adjectives that express sadness, or fear) are well-mastered (Peterson & McCabe, 1983).In contrast, for many adolescents writing persuasive essays poses new linguistic challenges as effective academic persuasive writing often involves organizing discourse, not around a sequence of events, but by imposing a stepwise argumentation structure to a series of ideas, often through the use of later acquired discourse markers (e.g., nevertheless, on the one hand…).In addition, academic persuasive essays go beyond expressing emotions or reactions towards events and require that writers mark their stance towards particular ideas, such as expressing degree of certainty about particular assertions (e.g., it might be that, it is certain that…) (Berman & Nir-Sagiv, 2007).
Informed by these different research traditions, this study seeks to identify analytic tools that capture variability in the academic language features of persuasive essays written by bilingual and monolingual high-school senior students.Two research questions guided this study: 1. Controlling for essays' length, are word-level and sentence-level measures of academic language associated with overall writing quality in the persuasive essays produced by a linguistically and socioeconomically diverse group of senior high school students?2. Beyond the contribution of length, word-level and sentence-level measures, are discourse measures of academic language -frequency or diversity of organizational markers or stance markers-predictive of persuasive essays' overall writing quality?Discourse organizational markers.In academic writing, organizational discourse markers contribute to the cohesion of a text, functioning as explicit guidelines for interpreting relations across sentences and discourse fragments (Givón, 1992;Vande Kopple, 1985).Within the field of adult academic writing, extensive research on textual analysis has identified a repertoire of organizational markers characteristic of academic discourse.Drawing from Hyland's (2005) prior work on experts' academic argumentative writing, we focus on what he calls interactive metadiscourse markers, i.e., words and phrases used to explicitly mark the coherent organization of the information in a text in order to guide their readers.Not surprisingly, only a subset of Hyland's (2005) markers proved relevant for our analysis of written essays produced by still novice learners of academic discourse.These frame markers included (e.g., first, second; one reason…another reason), code glosses (e.g., for example; in other words), transition markers (e.g., however, consequently) and conclusion markers (e.g., In conclusion, In sum).
Discourse stance markers.Stance refers to how writers' use language to express their attitudes towards the information conveyed (Berman & Ravid, 2009) 3 .In contrast to oral colloquial conversation, where stance is typically subjective and involved (e.g., let me tell you something!), and often expressed through non-linguistic means (e.g., gestures, intonation); academic discourse stance is encoded linguistically through a variety of later-acquired forms and functions used to express the characteristically assertive yet epistemically cautious attitude most typical of expository writing.Following Berman and colleagues (Berman, 2004;Berman et al., 2002) and focusing on stance markers that are salient in academic persuasive writing, in this study we focus on: (1) epistemic markers, those that signal the writer's belief about the degree of truth, relibility, or possibility of a given statement (e.g., it is possible that…; people might be…); and (2) deontic markers, those that signal an attitude that conveys a judgmental and categorical perspective (e.g., people should not resent others; it is wrong that…).One prior cross-linguistic and cross-sectional study has documented a shift from deontic to epistemic stance in the oral and written discourse of adolescents (Reilly et al., 2002).These authors claim that the deontic stance is related to the cultural worldviews and value systems into which children are socialized from early on and which children are likely to express through unexamined versions of right and wrong; as children grow older, they seem to move towards more cognitive/reflective attitudes relating their assertions to particular states of knowledge (Reilly et al., 2002).These intriguing findings come from developmental linguistics research conducted with students from middleclass families and aimed at documenting average developmental performances, without much attention to individual variability.In educational research we are well aware of the enormous variability in students' language and literacy skills within a classroom.Therefore, in this study, we focus on epistemic and deontic stance as lenses for capturing variability within a highly diverse cohort of students.
On the basis of prior writing research on traditional linguistic measures, we predicted that essays' lexical diversity, and syntactic complexity would be positively associated with essays' overall writing quality.One prior study on middle schoolers' writing has found syntactic complexity -measured as words per clauses-to be predictive of students' expository writing quality (Beers & Nagy, 2007).Similar results were found by McNamara, Crossley, and McCarthy (2010) who found syntactic complexity (as measured by number of words before the verb), lexical diversity and word frequency to be predictive of college essay writing quality.
Extrapolating from prior research on the textual analysis of experts' writing (Hyland, 2005), we hypothesized that essays with either a higher frequency and/or a higher variety of organizational markers would exhibit higher quality scores.Finally, building on the intriguing findings of the developmental move from deontic to epistemic advanced by Reilly and colleagues (Reilly et al., 2002), we hypothesized that the frequency of epistemic markers would be positively associated with writing quality.

Participants
As displayed in Table 1, 51 high school seniors participated in this study.By the end of their senior year, participants had an average age of eighteen years and four months.The sample included a slightly larger proportion of females (56.9%) than males and most students came from families with low socio-economic status, with 66% of students qualifying for free/reduced lunch.
Participants came from a variety of ethnic and/or racial backgrounds.The majority of students identified themselves as African American (60.8%); the next largest group was comprised of students who identified themselves as Hispanic/Latino (21.6%); and finally, only a minority of students identified themselves as White (7.8%) or Asian (5.9%).Two students did not report their race/ethnicity.This sample was also linguistically diverse.Almost half of the participants were monolingual English speakers (47%), whereas the rest reported a variety of home languages.The largest group of bilingual speakers reported Spanish as their home language (23%), followed by groups of students who reported Vietnamese (8%), Haitian Creole (6%), Cape Verdean Creole (4%), Portuguese (4%), Tigrinya (4%), Chinese (2%), and Somali (2%) as home languages.In general, students represented a range of academic performances.As measured by the statewide English Language Arts test of the Massachusetts Comprehensive Assessment System (MCAS-ELA), almost half of the students in our sample scored at the "proficient" performance level ( 22), half at the "needs improvement" performance level (25), and only a few students fell in lowest and highest performance levels: warning (2) and advanced (1).One student's MCAS-ELA score was not available.This group's academic performance was below the average MCAS score for the state, but it was representative of urban public high schools in the area.

Data collection
Participants produced a persuasive essay as part of their regular schoolwork during their senior year.The prompt resembled the types of prompts students encounter when taking the SAT.The school personnel chose the writing prompt, and students received a typed copy of the prompt during the data collection period (Appendix 1 displays the writing prompt).Following standard SAT conditions, students were given 25 minutes to complete their responses, so their essays reflect unedited, timed responses.This type of writing prompt and situational conditions are common in this school -and in many public schools-to provide practice for students in their high school preparation for SAT and college format writing.

Data Analysis
The original essays were handwritten by students.In order to homogenize the data and to avoid any subjective impressions due to calligraphy or spelling, all essays were transcribed as word documents.Subsequently, in order to use the automated language analysis program from CHILDES (The Child Language Data Exchange System), essays were transcribed by adapting the CHAT (Codes for the Human Analysis of Transcript) to written data (MacWhinney, 2011).
After essays were transcribed and verified by a second researcher, the following measures were generated to analyze the data: WRITING QUALITY: Writing quality was estimated using a wholistic writing rubric with essays ranging from a total score of 2 to 12. Essays were scored by a team of practitioners with high school teaching experience.Scorers were given instruction in using a 6-point SAT-like rubric, and each essay was assessed by two scorers whose points were added (when their scores were exact or adjacent).Percent agreement between scorers was 89.5%.Only 10.5% of the essays did not exhibit either exact of adjacent agreement.In those cases, an expert SAT scorer intervened to settle the disagreement and this score was doubled as the final score.
LEXICAL AND SYNTACTIC MEASURES: First, essays were divided into clauses.Following Berman and Slobin's (1994, p. 660), a clause was defined as a unified predicate describing a single situation (activity, event, state).Berman and colleagues have shown the clause to be a reliable unit of text segmentation across oral and written narrative and expository texts (Berman & Verhoeven, 2002).Subsequently, the data were analyzed using the CLAN (Computarized Language Analysis) tools to generate the following lexical and syntactic measures: -Length was calculated as the number of clauses per essay.
-Lexical diversity was measured through the widely used vocD measure, which reduces the impact of length in estimating the variety of words in a text (Durán, Malvern, Richards, Chipere, 2004).
-Lexical density was measured by estimating the frequency of content words (nouns, adjectives, verbs and some adverbs4 ) in a text as a ratio of total clauses.This is a widely used measure in systemic functional linguistics and developmental linguistics, and indicates the density of information packed in a text (Berman & Nir-Sagiv, 2004;Christie & Derewianka, 2008;Halliday, 2004).
-Syntactic complexity was calculated as the number of words per clause.This specific syntactic measure was selected because prior research has found it to be associated to the quality of persuasive essays produced by adolescent students (Beers & Nagy, 2007).

DISCOURSE MEASURES:
Informed by prior research on academic writing (Hyland, 2005) and adolescent writing development (Reilly et al., 2002), essays were coded at two discourse dimensions: Organizational markers: using Hyland's (2005)  Stance markers: using prior research on stance by Berman (2004) and Reilly et al. (2002), essays were coded for deontic stance, which involves a judgmental viewpoint, and epistemic stance, which entails, degree of possibility, certainty, or acknowledgment of the writer's beliefs about the truth of certain assertions or state of affairs: a. Deontic markers: express obligation, necessity, prohibition or (dis)approval displaying an absolute stance.For example, People should not lie…, It is wrong to lie… 2. Epistemic markers: include markers that express the writer's degree of certainty or beliefs about the truth of a given statement.a. Epistemic hedges: markers of degree of uncertainty that index a writer's cautious attitude toward the truth of an assertion.These markers include modal auxiliary verbs, adjectives, and adverbs (e.g., It might be true: it is possible that; probably…).b.Epistemic boosters: markers of emphatic certainty that index the writer's emphatic commitment to the truth of an assertion, e.g., It is true; It is absolutely true.c.Markers of beliefs: mental verbs through which the writer acknowledges that assertions are the result of self or others' beliefs, e.g., I believe, People think.These markers were analyzed as a separate category because, as has been reported by other researchers, it was not always possible to decide whether these forms were indexing an epistemic stance, or were only used as discourse sequencing devices (Reilly et al., 2002).
All essays were coded by researchers who were blind to the writing quality scores assigned by teachers.Inter-rater reliability was assessed using 20% of the data and yielded high levels of reliability with a Cohen's kappa statistic of .94 for stance coding and .87 for organizational markers.Most organizational coding disagreements were a matter of one coder overlooking one transition marker, and all disagreements were subsequently resolved.

Analytic Plan
Descriptive statistics were estimated for essays' writing measures (wholistic writing quality, length, lexical density, lexical diversity, use of organizational and stance markers).
Correlational analysis results informed the construction of the series of hierarchical regression models to explore the predictive power of demographic variables, lexical, grammatical, and discourse measures on overall writing quality.

Statistics
Table 2 displays descriptive statistics for overall writing quality, well-known lexicogrammatical measures of academic language, and the more innovative discourse measures.As can be observed, essays exhibited a wide range of writing quality scores with an average performance of 5.86, and a range of 2 to 12 points.Essays displayed an average length of 40 clauses with considerable variability across essays.Syntactic complexity ranged from 4.7 to 7.13, with a mean of 5.78 words per clause.Lexical density estimates indicated that essays exhibited, on average, two content words per clause.Finally vocD was on average 72.25.These average lexical diversity estimate is close to those reported for written narratives produced by English-speaking high school students from middle class backgrounds (vocD= 70 to 80), and lower than vocD values of 80 to 90 reported for those same students' expository texts (Berman & Verhoeven, 2002).However, as displayed in Table 2, essays displayed considerable variability in lexical diversity.
At the discourse level, Table 2 also captured substantial variability both for organizational markers and stance markers.In their essays, students used between 1 and 26 instances of organizational markers, with an average of 10.63 markers per essay.The diversity of such organizational markers ranged from 1 to 4 per essay, with an average of almost two distinct markers per essay (1.98).The frequency of stance markers, however, was much lower, with an average of 2.84 instances per essay and a range of 0 to 11. Essays displayed between 1 and 3 distinct types, with an average of 1.43 distinct stance markers per essay.
- -----------------------------------------INSERT TABLE 2 ABOUT HERE--------------------------------To take a closer look at how students used discourse markers in their essays, Table 3 displays organizational markers and stance markers disaggregated by subtypes.Not surprisingly, the most frequent type of organizational markers within and across essays were transition markers (e.g., however, consequently, therefore).In addition, more than half the students used code glosses in their essays (e.g., for example, for instance), but only less than a third used frame markers to explicitly mark the organization of their argument (e.g., first… second; some say…others say).Finally, the least often used type of organizational marker were conclusion markers (e.g., in conclusion, in sum).Not only was this marker the least used within a single essay, which would be expected as essays usually display only one conclusion marker, but only less than a fifth of all students used such markers in their writing.
Whereas all essays displayed at least one type of organizational marker (i.e., transition markers), stance markers were much less prominent.Only 21 essays displayed any one type of stance marker.Even the two most widely used types of stance markers -epistemic hedges (e.g., seem to, might, may) and markers of beliefs (e.g., I think, people believe)--displayed an average use of only less than one marker per essay.Approximately a third of essays used deontic markers (e.g., you cannot, people must).Finally, epistemic boosters, i.e., markers of emphatic certainty (e.g., it is true, the statement is true), were the least used stance marker type, both within and across essays.

Correlation of academic language features with overall writing quality
Interestingly, none of the demographic characteristics was associated with overall writing quality.Correlational analyses indicated, however, that several of our word, sentence and discourse level variables were significantly associated with writing quality.As expected, the significant correlation with essay length indicated the need to control for this variable in our subsequent regression analysis.At the word and sentence level, syntactic complexity and lexical diversity displayed significant positive associations with writing quality indicating that essays with higher levels of syntactic complexity and more diverse vocabulary received higher writing quality scores.However, lexical density (the number of content words per clause) was not significantly associated with writing quality, most likely due to the limited variability of this measure.We were encouraged by the fact that our discourse measures displayed significant positive relations with writing quality.As expected, these variables were also positively associated with length, but no significant associations with lexical or syntactic measures were detected.Both frequency and diversity of organizational markers showed significant associations with writing quality, with the former exhibiting a somewhat stronger relationship than the latter.
As for stance, frequency but not the diversity of stance markers was positively associated with overall writing quality.When we disaggregated stance markers into subtypes, frequency of one type of marker -epistemic hedges-was revealed as the only one significantly associated with writing quality.
Finally, it is worth noticing that lexical diversity and syntactic complexity were significantly associated (r=.73, p<.0001), supporting the exploration of a latent lexicogrammatical intricacy construct for the linear regression modeling.

Predicting writing quality
On the basis of the correlations results, length was used as a control variable in our exploration of regression models to predict writing quality.In addition, using principal component factor analysis, we created a lexico-grammatical intricacy latent variable from syntactic complexity and lexical diversity.A single factor with an eigenvalue greater than 1 was extracted, and the two observed variables loaded highly onto that factor (see Table 5).

------------------------------------------INSERT TABLE 5 ABOUT HERE--------------------------------
To explore whether discourse measures would explain any unique variance in writing quality even after controlling for the effect of more traditional measures, in our series of regression models, we first entered length as a control variable, followed by lexico-grammatical intricacy latent variable in the second model.As displayed in Table 6, length alone explained 35% of the variability in writing quality.Inserting the lexico-grammatical composite contributed 5% of additional variance.Once we established Model 2 as our baseline model, we explored the additional impact of the discourse variables.We found a significant main effect for epistemic hedges, which explained an additional 8% of the variation in writing quality over and above length and lexico-grammatical intricacy.By adding frequency of organizational markers, we were able to predict an additional 4% of the variance, but this variable only approached significance (p<.10).Nevertheless, inclusion of an interaction term between organizational markers and essays' length increased the R-square by 3%, with organizational markers becoming a significant predictor of writing quality, and the model explaining a total of 55% of the variance.
Therefore we selected this as our final model.As we graphed this interaction, it became clear that the impact of the frequency of organizational markers was less pronounced for shorter essays.Perhaps not surprisingly, for essays in the bottom quartile of the length distribution, the impact of organizational markers on writing quality was much weaker compared to essays in the higher quartiles.This interaction term only approached significance in this final model and, thus, it is only indicative of potential relationships for further research.

Illustrating domains of individual variability in academic writing: A few examples
In this section, we briefly discuss two persuasive essays that illustrate the higher and lower ends of the writing quality continuum for the sample of essays explored in this study.
Essay 1 received a score of 4 out of 12, and Essay 2 received the highest score possible on the wholistic scale (12/12).The wholistic rubric used in this study was calibrated to best capture the variability across essays within our sample, thus despite the identifiable opportunities for improvement in Essay 2, it is a good representation of the highest writing performance produced by this group of students.
- -----------------------------------------INSERT FIGURE 1 ABOUT HERE------------------------------In Essay 1 (see Figure 1), the student made minimal use of organizational markers and did not use any epistemic hedges.The small number of organizational markers results in a text that displays a concatenation of ideas without explicitly marking the unfolding of the argument or the overall text structure for the reader.In addition, the essay displays an abundant use of deontic stance markers to advance value judgments that inflexibly classify actions and beliefs in dichotomous categories of right or wrong, without ever entertaining the evaluation of knowledge sources or alternative viewpoints (e.g., this is not the right thing to do; what they've been doing wasn't the right thing).
In contrast, the writer in Essay 2 (see Figure 1) makes ample, strategic, and clear use of organizational markers.In addition to several transition markers that explicitly highlight conceptual relations across sentences (e.g., because, in order to), this writer uses frame markers, glosses, and conclusion markers that serve to orient the reader to the different parts of a persuasive essay and to the progression of her argument (i.e., The reason for this...; Another reason…; One last reason… For example…; Another example…; In conclusion...).Moreover, the epistemic hedges (e.g., could use, can possibly, might be presumed true) temper her argument, making her essay stronger.Instead of expressing an egocentric value judgment, this writer is evaluating alternative situations and cognitive/emotional traits that might lead to different consequences.In so doing the writing is closer in organization and stance to that of skilled academic writer in its explicitly organized argumentation in its assertive yet epistemically cautious stance.Interestingly, writers need to learn that in academic writing a more cautious, less categorical stance constitutes at the least one important component of constructing a stronger argument.

Discussion
In this study the essays produced by a diverse cohort of senior high school students were scored for writing quality by a group of experienced high school teachers and subsequently analyzed and coded for lexical diversity, syntactic complexity, organizational discourse markers and markers of epistemic and deontic stance by researchers who were blind to the essays' quality scores.Results revealed that beyond the contribution of length and lexico-grammatical intricacy, the frequency of organizational markers and one particular type of epistemic stance marker, i.e., epistemic hedges, significantly predicted persuasive essays' writing quality.Given that these features of academic language were associated with teachers' ratings, this research helps to reveal some of the implicit linguistic expectations that school teachers tend to have for student academic writing.Beyond the well-known areas of lexical diversity and syntactic complexity, this research documents discourse-level linguistic expectations that teachers value in students' persuasive essays, namely the use of a specific set of organizational markers that index explicit relationships between clauses and text fragments, and epistemic hedges or stance markers of degree of uncertainty, which allow the writer to hedge his or her argument and acknowledge that behaviors or statements may be evaluated from a point of view that moves beyond egocentric, categorically-determined judgments.Given that high school teachers are well-versed in academic language themselves, it is not surprising that the features they would value in their students' writing are in fact core markers of organization and stance in skilled academic writing.
While promising, our results must be viewed in light of some important limitations.Our small sample cannot be said to be representative of all high school students.Analysis of additional writing samples from a larger variety of schools and students would be necessary to confirm these findings and to further explore what additional academic language structures are predictive of writing quality scores.Longitudinal research that follows writers during the middle school and high school years is also needed to determine developmental trajectories and individual variability in the mastery of these linguistic structures -and others-over time.An additional note of caution is required.It is important to clarify that this study did not attempt to assess writers' skills, but had the more modest goal of assessing the features of a student's writing product.In other words, this analysis does not pretend to yield an assessment of writers, but the assessment of a piece of writing.In fact, the question of how best to assess students' writing skills to capture their optimal performance across a variety of genres and topics is still an important question that deserves serious attention in educational research.Our current findings offer just a modest but promising step forward in unraveling some pedagogically-relevant components of skilled high school academic writing.
Despite such limitations, this study contributes promising findings that might prove relevant for writing instruction and assessment.Our results on the impact of length, lexical diversity and syntactic complexity are aligned with prior research in this area.Beers and Nagy (2007) found syntactic complexity (measured by words per clause) to be correlated with expository texts produced by middle-school students.McNamara, Crossley, & McCarthy (2010) applied their computerized tool of Coh-Metrix analysis to a corpus of undergraduate college essays and found lexical diversity, syntactic complexity (measured as number of words before the main verb), and word frequency to be among the most important predictive indices of essay writing quality.
Our findings extend prior research by identifying additional discourse-level components that are predictive of writing quality and display considerable variability in writing performances across senior high school students within a single cohort.To our knowledge, this study is the first to look at this discourse dimension in educational research focused on individual variability among high school students.Interestingly, McNamara and colleagues explored cohesion -as measured by the Coh-Metrix tool-but they did not find it to be predictive of writing quality in their college sample.In fact, they report that essays at various quality levels were "relatively equal in terms of referential overlap and the use of connectives" (McNamara, Crossley, & McCarthy, 2010: 75).The present study had a different purpose than that of McNamara and colleagues as our motivation was not to apply a complex computerized tool, but to identify concrete components that teachers themselves could identify for assessment and instructional purposes.Whereas we did not score essays for cohesion more globally, our coding of organizational markers certainly overlaps with the causal cohesion assessed as part of the Coh-Metrix tool.Whereas it is difficult to compare the results of both studies due to the use of different measures, the findings seem to point to a possible discrepancy given that we found organizational markers (which comprise a broader set but include causal connectives) to be predictors of writing quality, whereas McNamara's and colleagues' found no association between connectives and writing quality.Several explanations can be hypothesized.First, maybe some of the functional human analysis of markers that allows for the identification of key form/function relationships might be missed through automated computerized analyses.
Alternatively, the findings might be the result of how the elements were grouped together as predictive indices in both studies.Yet another possibility is that in a more homogeneous sample of skilled college writers the variability in frequency of connectives -or maybe even organizational markers as defined in the present study-would be minimal and less predictive of writing quality.In any case, our findings strongly suggest that in the context of a diverse cohort of high school students, like the one studied here, organizational markers seem to be both a key dimension of variability across students and a positive predictor of writing quality.
Consequently, frame markers, code glosses, transition markers and conclusion markers are revealed as critical elements to consider in academic persuasive essays' assessment and instruction.
In addition to the contribution of organizational markers, the exploration of a second key domain, discourse stance, revealed epistemic markers --more specifically, the frequency of epistemic hedges (or markers of degree of uncertainty, e.g., it might be true that…; possibly)to be significantly predictive of writing quality.Whereas the frequency of deontic markers (those indexing a judgmental and prescriptive stance, e.g., it is wrong to…), epistemic boosters (e.g., it is absolutely true), or markers of beliefs (e.g., some people think, I believe) were not associated to writing quality, the frequency of epistemic hedges displayed considerable variation across essays and contributed positively to writing quality.From all stance markers coded, epistemic hedges were the most suitable to capture the epistemically cautious stance, defined as acknowledging degree of uncertainty or possibility in relation to the ideas presented.Whereas markers of beliefs could have been hypothesized as falling within the same category, it was in fact difficult to assess whether these markers were used with an epistemic value or simply as discourse sequencers in the essays studied.Our results can be interpreted in light of prior crosssectional studies in developmental linguistics that have identified a developmental shift from deontic to epistemic stance in adolescents' discourse (Reilly et al., 2002;Berman, 2004).This proposed shift has been interpreted as reflecting broader socio-cognitive developmental changes, with speakers/writers moving from points of view that are more egocentric and driven by social conventions to viewpoints that are more relativistic, move beyond self experience and consider multiple perspectives (Berman, 2004;Selman, 2003).Interestingly, instead of developmental trajectories, our study documents individual variability.Elucidating whether this individual variability is related to underlying socio-cognitive development or just to the understanding of the expectations of the persuasive essay as an academic genre goes beyond the scope of this study.Future research that explores the interaction between socio-cognitive and language development during the later years of school is necessary and has the potential to be highly relevant for instruction.Moreover, the effect of genre, topic and comprehension of the prompt needs to be further explored in relation to individual differences in writing performance, and in discourse stance, more specifically.Despite these still unanswered questions, our results are especially interesting as they fall at a core intersection between language and cognition.Thus, these results raise the possibility that through attention to epistemic markers, not only linguistic mastery might be supported and scaffolded but also socio-cognitive development, as has been documented for younger children in relationship to internal states or language-specific categorizations (Harris, Rosnay, & Pons, 2005;Slobin, 1991).It is worth clarifying, though, that these results do not imply that the exclusive use of epistemic hedges should be fostered.In fact, a skilled writer/speaker is one who can flexibly and competently select from within an extensive linguistic repertoire a combination of forms and functions to aptly present a stance -even a combination of stances--within a text in order to effectively convey meaning.For instance, one could imagine -or remember-a skillful presidential speech or op-ed where both epistemic and deontic stance markers would be competently used at different portions of the text to inform and persuade the audience.
Additionally, we could foresee a circumstance in which the over-use of epistemic hedges might detract from the overall effectiveness of the argument.Although that did not occur within our sample, future research could explore how including too many or too few epistemic markers impacts the strength or persuasiveness of the argument advanced.
The identification of critical features of academic writing associated to quality scores has the potential to inform the design of more specific and relevant assessment and instruction.
Whereas this study does not offer any evidence for the effectiveness of instructional practices, our results shed light on particularly relevant areas to consider in pedagogical approaches to academic writing.Recently, many states have adopted new common core standards that are designed to ensure greater levels of college readiness in students graduating from 12 th grade (CCSSI, 2011).The Common Core Standards put special emphasis on moving students at a faster pace towards text of higher linguistic complexity, both in reading and writing.In fact, the educational field seems to be in agreement about the need for providing explicit pedagogical attention to the academic language features of academic writing.However, in such climate, one important question becomes: what are the key features of academic language that would be relevant for writing instruction?On the basis of these results, future intervention studies may consider a focus on organizational markers and epistemic markers in an instructional design to promote academic writing in middle or high school classrooms.Far from advocating an instructional design focused on forms devoid of meaning, we consider the educational implications of these findings within a pragmatics-based view of language instruction (Snow & Uccelli, 2009).Within a pragmatics framework, students would be encouraged to understand language as a functional solution to the specific context of school and the particular communicative purposes of persuasive essay writing.Organizational markers and epistemic hedges could be presented as resources from which writers draw to sharpen their own meanings.
Rather than lists of forms to be memorized or used unreflectively in the classroom, future research could explore how to best promote students' reflective meaning-based choices, maybe through textual analysis of different models of skilled academic writing, abundant meaningful writing practices, and targeted teacher feedback (Sommers, 2012).More precise writing rubrics and assessments may also be designed with these features in mind.In contrast to wholistic writing rubrics, more precise tools can help reveal the often unnoticed language challenges of academic writing.In so doing, these tools could support teachers as they expand students' language resources in response to individual strengths and weaknesses.Our current findings offer a modest but promising step forward in unraveling some pedagogically-relevant language components expected of skilled high school academic writers.Future research is still necessary to explore the potential of these findings for instruction and assessment.My thoughts on the statement above is that it is very interesting.In today's world there are many people that do wrong things to people that they really know is wrong deep down.I believe that this is not the right thing to do, and the people who are doing these things should think about if it was done to them and how would they feel.I know this from experience that "bullies" will soon come to the light and realize that what they've been doing wasn't the right thing, and it wasn't the best way to handle the situation they were in.
I agree with the writer's assertion.The reason why I agree with this is because most people will do things that are wrong just to fit in with the crowd that is doing the wrong things.I believe that sometimes people don't think when they are doing things like that.People crave to have friends more than how they are degrading them.With that said I really don't think that this would ever change.
Essay 2: High writing quality (selected fragment) I do agree with the assertion that "we resent in others the very flaws that we ourselves possess." People turn a blind eye to their own flaws yet maximize another's.The reason for this, I believe, is because of the insecurity they feel.Another reason is because you are aware of the flaw and hate to see the same problem in other people.One last reason is that when two people have the same issue, and are alike in ways, they tend to argue more.This assertion does not always have to be a negative.A person could use their resent of others' flaws in order to make a positive impact.For example, … In this way, you can possibly make them research on metadiscourse markers, essays were coded for four types of organizational markers: a. Frame markers: markers that signal the sequence of claims or contrastive positions in the argument, e.g., First… Second…; One reason is… Another reason is… b.Code Glosses: markers used to introduce an example or to paraphrase, e.g., For instance; for example; in other words.c.Transition markers: markers that index inter-clauses or inter-paragraph relations of cause/consequence, contrast, and discourse transitions, e.g., because, even though, however, furthermore.Temporal markers, as well as the simple connective and were not included in this coding.Conclusion markers were coded separately.d.Conclusion markers: markers are used to express a summary or to explicitly introduce the author's conclusions, e.g., To summarize; In conclusion.

Figure
Figure 1: Students' Persuasive Essays: Two examples 5 Essay 1: Low writing quality change.Another example is … In conclusion, based on the examples I have provided this assertion can be presumed true.… 5 Misspelled words have been corrected.

Table 1 :
Frequency Distribution of Student Demographic Characteristics (n=51) LANGUAGE OF ACADEMIC WRITING 37

Table 3 :
Types of organizational and stance markers (n=51)

Table 4 :
Pairwise correlations between writing quality, discourse measures and stance markers

TABLE 5 :
Results from Principal Component Analysis for Lexico-Grammatical Intricacy

Table 6 :
Regression models testing the effect of discourse measures of organization and stance on writing quality, controlling for length and lexico-grammatical measures