Here goes. This is the first full draft of the proposal; it was used in the proposal defense (6/3/2008). My job is to address the questions and suggestions offered by the committee in a follow-up memo. I'll post notes on this (as well as the memo itself) on the dissertation blog. A final proposal draft will be posted here. In the meantime, any comments you have would be great.
There's no footnoting system in CommentPress, so I'll include footnotes as comments in the appropriate places.
Dissertation chair: Thomas Newkirk
Dissertation committee: Bill Condon, Elliott Gruner, Lisa Miller, Christina Ortmeier-Hooper
Overview
This proposal outlines a qualitative research project that examines student self-assessment in three college-level writing classrooms. The study seeks to analyze students’ and instructors’ responses to the task of structuring and implementing self-assessment writing projects (such as reflective portfolio cover essays) as an integrated component of their courses. Specifically, this study will examine the negotiation of teacher/student identity and authority that results from the shift away from traditional instructor-centered grading. Through an analysis of the written artifacts of self-assessment pedagogy, I’ll investigate the following questions in detail:
- How do teachers and students renegotiate their roles in a classroom where the traditional authority structure has been disrupted?
- How does this renegotiation manifest itself in the writing they produce in their construction of the classroom’s new assessment paradigm?
- How does the turn toward self-evaluation , specifically, highlight and intensify the role negotiation that takes place in these classrooms?
This collective case study will feature analyses of student-written self-assessment projects as well as the instructor handouts that situate and structure them within a specific classroom. In addition, selected class visits and instructor interviews will help to contextualize discussions of the self-assessment process, its perceived value and its aims in these classrooms. This study is meant to capture role conflicts (whether real or imaginary) that stem from self-assessment pedagogy as they play out in the classroom.
By outlining the negotiation process surrounding self-assessment, I hope to address concerns within the scholarly communities of composition and writing assessment regarding the ethical implications of self-assessment. Scholars have noted the lack of research into the consequences of asking students to construct honest “selves” within an unequal classroom power structure. These consequences have the potential to thwart the goal behind implementing self-assessment in the first place – that is, to balance institutional needs and expectations for assessment with teachers’ desires to develop self-aware, rhetorically savvy, independently motivated writers. My hope is that this study will outline ways in which the self-assessment process can become a conscientious examination into and negotiation of competing goals for education and needs for assessment.
Introduction: Where I’m coming from
This project has been in the making for years – since 2002, to be exact. It was in that spring that I taught my first college writing course and began to take the topic of student motivation seriously. In that first classroom, many of my students seemed to have minimal interest in their roles as writers, though they seemed to be well aware of their student roles. They would do the bare minimum of work required to earn the grade they wanted and not much more; the chance to communicate ideas to an audience didn’t seem as motivational as the grade. Of course, part of this problem was due to my inexperience as a teacher, but even in later semesters as I became a better teacher and had more interesting class sessions and conversations with students, I still felt that the classroom’s power imbalance, most obvious in the grading process, constrained what I could accomplish with many of them. They deferred to me too easily.
Scholars such as Marcy Bauman, a longtime composition teacher at the University of Michigan, have noted similar frustrations and attributed much of the blame to instructor-centered assessment:
In my experience, no matter how much I try to make the assignments "real," no matter how I try to encourage students to write for their own purposes and to make their own discoveries, no matter how easy I make it for students to take risks, as long as I'm the one grading their papers, students tend to understand the writing situation as one in which their task is to please me so that they can get a better grade. When I give grades, they tend to ask questions like "How can I make this paper better? Why didn't it get an A? What do you want me to revise?"—all questions that indicate to me that they haven't seen the writing of that paper to be a communicative act, but a demonstrative one.
(412)
Bauman’s description indicates a failure in motivating students not only to enjoy writing, but also to cultivate and trust their own writerly instincts – to become better writers through risk-taking and self-reflection. Bauman is convinced that many of her students will continue to dutifully write predictable, safe papers in their future courses for the transactional purpose of obtaining good grades. This is a potentially serious problem: if the right kind of motivation is lacking, students’ experiences with writing could be drastically limited. Their skewed perception of the purpose of writing could be reinforced.
As Lynn Bloom notes, grades carry all the weight after the completion of the course, with rare exceptions – to everyone but the student and instructor, they are the only visible artifacts of the course (215). Students know this. As a result, instructors’ genuine attempts to develop a good working relationship with students can be short-circuited as grading time draws closer, despite the best intentions of both: students can hardly be blamed if they stop paying close attention to their growth as writers and look ahead to the higher-stakes final grade. The authority of the instructor as the final arbiter of the grade is undeniable; because this role carries more weight in the final analysis than the instructor’s other roles (mentor, coach, expert writer, tutor, etc.), it can become dominant.
For this reason, many teachers have experimented with approaches to classroom assessment that attempt to shift the balance of authority, hoping to create an environment with more productive motivations. For example, Christopher Weaver’s “process grading” approach, based in part on Peter Elbow and Pat Belanoff’s A Community of Writers, removes grades entirely from the draft response process, shifting the emphasis to students’ “meta-awareness” of their writing processes. Weaver describes a system of grading reflective cover letters only: “[R]ather than base [students’] grade on whether or not their papers matched my expectations of what good writing should look like, I based their grade on how well their cover letters explained the different stages of their writing process and how well they persuaded me they were engaged in what I believed to be crucial issues of that process” (144). In short, Weaver rewards evidence of conscientious reflection. To Weaver, the effects of this reorientation toward the grading process are noticeable: he feels that he is no longer seen by students as “the [final] authority on their other writing – the papers to which the cover letters refer” (147) He believes that his students are motivated to “think seriously about the decisions involved in writing and revising their own work” and internalize criteria for good writing rather than simply having them externally imposed (148). Weaver argues that success in the course is no longer tied to the short-term goal of improving a few papers, but rather to the broader ability of becoming better self-assessors of their own writing – an ability that will continue to serve them well after the completion of the course (149).
These are the some of the main rationales underlying the work of writing teachers who have practiced some form of self-assessment, which covers the spectrum from “one-shot” portfolio cover letter writing to intensive, repeated class conversations and written projects that incorporate self-assessment. Reflective essay writing has its roots in the expressivist tradition , and those roots show: volumes such as Alternatives to Grading Student Writing rhetorically position themselves as counterarguments to the rigid, authoritarian philosophies of assessment that (presumably) underlay “current-traditional” composition. The narratives surrounding these experiments are liberatory in tone: they document a transition from an environment of oppression and drudgery with one of openness and greater motivation.
But classroom transformations are never truly that seamless, are they?
Although I too am a believer in self-assessment and incorporate it in some form into all of my courses, I recognize that the ideal version of self-assessment could only occur in a vacuum. It’s fascinating to see how complicated such an endeavor becomes, for both teachers and students, when it’s carried into the real-world composition classroom – a place where competing identities, roles and agendas intersect and compete for attention. It’s the complexity of the implementation of self-assessment that makes it so fascinating to me – and it’s the object of study for this dissertation. I believe there’s much to be learned from the teachers and students who take on the daunting task of “doing” self-assessment.
Ethical conflicts. My dissertation will respond primarily to the calls for ethical theorizing and research voiced in the work of Ellen Schendel and Peggy O’Neill (“Exploring the Theories and Consequences of Self-Assessment through Ethical Inquiry”); Susan Latta and Janice Lauer (“Student Self-Assessment: Some Issues and Concerns from Postmodern and Feminist Perspectives”); and Rebecca Moore Howard (“Applications and Assumptions of Student Self-Assessment”). While most early writers on self-assessment in composition courses discussed either the arguments in favor of self-assessment outlined above or the logistics of their own classroom practices, these scholars question the implications of asking students to inject “themselves” and “their” voice into a classroom assessment conversation in which teacherly voices have more power.
In a typical composition classroom using self-assessment, the instructor will create a series of prompts for student-written cover letters or essays. The student is asked to reflect on the process of conceiving, outlining, drafting and revising his/her major pieces of writing. A student might also be instructed to self-evaluate his/her work. For example, here are a few self-assessment prompts from my Fall 2005 First-Year Writing final portfolio handout:
• What do you believe good writing is? Define it and talk about your own strengths as a writer. What is good about your own writing? Look at your own writing for ideas. You'll be getting into your projects in detail elsewhere, but take some time to mention some general ideas about good writing here. …
• Please talk about what you have improved on this semester in order to become a better writer (and how you've worked on it), and talk about the things you know you need to keep working on in the future. Also talk about any technical or grammatical issues you've worked on. Please spend a couple of paragraphs talking about your writing strengths and weaknesses. …
• Talk about how well your [argumentative] essay and letter [to the editor] turned out (describe these separately). Include all of your drafts [in the portfolio] and highlight the parts that are best (with a highlighter), in your opinion. What can you show the portfolio reader as evidence of your hard work for this the essay and letter? Refer to the highlighted parts in your paper. Quote them. Talk about why they're examples of particularly good observation, analysis or research, or a well-supported point. …
• Talk about how you responded to feedback. What did you change, based on the comments given to you on each of your drafts? Why did you decide to make the changes in the way you did? What is better about your essay and letter now that you made the changes? Quote some of these changes…
These are just a fraction of the questions asked in the handout, and much of the contextual information is missing, but these questions give a good example of the type of narrative I asked students to construct at the end of this course. At face value, the questions are innocuous and not at all mean-spirited; as a matter of fact, I think they show I have a healthy interest in the progress of my students. But the problem I’m highlighting here is not in the questions themselves, but in the personas students must adopt in order to answer them – personas that are formed in response to the perceived expectations of the surrounding environment.
Power and the “academic self.” A self-assessment narrative is tied (either explicitly or implicitly) to a relatively high-stakes result: the grade. So “[w]hat happens if the ‘self,’ or the subject position, the student assesses is not the ‘self’ the teacher or institution wants them to inscribe?” Latta and Lauer ask (26). In their analysis of what could be called the academic self, Latta and Lauer find a tension between the individual student and the academic “community” s/he is trying to enter. Are students who fail to construct an acceptable self in response to the prompts above (that is, a self who can give an academically acceptable definition of good writing, a self who can trace a linear narrative of progress throughout each drafting process, and a self who values the teaching and learning philosophies underlying the course) penalized? Does their grade suffer? If their self-assessment factors into the grade, does it factor in less for these students than it does for a student with a conventional (and perhaps less honest) narrative? In other words, which “self” do I truly want when I ask students to self-assess? This is an important question to answer, because as the classroom representative of the academic community, I hold the institutional power to determine the acceptability and value of a student’s self-narrative.
Postmodern and feminist theories, which question the notion of the autonomous self, take into consideration the powerful demands and expectations that dictate the parameters of public versions of the self. As Howard notes, these theories not only question whether student “writers can exert the agency requisite to assessing themselves” but also “suggest that any student self-assessment would merely affirm and reproduce the student’s hierarchical place in the educational establishment” (37). Howard draws on the work of Bourdieu and Passon, who have asserted that “the function of education is the reproduction of established power relations” (40). Education asks students to accept their roles in the established power dynamic. Self-assessment within this framework is inevitably a political act, as is assessment of any type. As Schendel and O’Neill point out, teachers who incorporate self-assessment face an ethical dilemma: writers may improve, perhaps significantly, as they learn to self-evaluate, but on the other hand, “self-assessments may require that students participate in their own surveillance and domination” (200).
With that in mind, let’s consider the following introduction from one of my student’s self-assessment essays:
You can tell if a student takes class seriously by the good work s/he produces, if they show up to class on time, if they listen and participate in class, if they complete all their assignments on time and meet at least the basic requirements [. T]hey may improve their work or make changes to certain things … in order to improve their learning opportunity.
The language of this essay, which I’ve handed out as a model to accompany a self-assessment handout, is at times eerily similar to something I might write in a course handout. Although the student moves beyond this introductory language to assess her own work more specifically, nothing she says in her self-assessment in any way challenges the values she champions in this paragraph. So, should I be encouraged or disturbed at my student’s wholesale adoption of my stated values for writing and class work?
Schendel and O’Neill recommend that the “conflicted intents and consequences of self-assessments … be a site of attention for writing teachers” (200). Drawing from the work of Moss, who argues that “our conceptions of validity [should] include questions about why particular methods of inquiry are privileged” (qtd. in Schendel 202), Schendel and O’Neill pose the following broad questions for those who study the consequences of self-assessment:
• What are the implications for students’ selves, their writing, the classroom, the teachers, the profession?
• What discourse does the field use to situate and discuss self-assessment? What do we neglect or overlook?
• Why now, at this particular time, is self-assessment becoming more attractive?
• What do we privilege in self-assessment? Who is rewarded and who is penalized?
(202)
Defining the role of the academic self. Who/what is the “self”? How is it constructed? How is it presented? In an academic context, how do conflicting expectations and representations of the self complicate or limit the possibilities of educational environments? These questions lie at the heart of my study, making it important to understand contextual influences on an individual’s self-concept.
As I work on my own articulation of this phenomenon (which will be informed by my study findings), I’d like to start with Robert Brooke’s “identity negotiations theory,” which he uses in Writing and Sense of Self to describe the interaction of student writers in his workshop-based writing courses. Brooke posits that “learning is influenced more by the roles offered in school than by any particular content or material being taught, because it is in negotiating a response to these roles that individuals work out their future stances towards knowledge, towards authority, and towards academic learning” (11). The term identity negotiations draws from psychological studies and political and social theory to highlight “the development of the self [the identity part of the term] within a complex arena of competing social forces [the negotiations part]. … A person’s identity arises through negotiation with the many groups which provide [social definitions of self]” (12). Brooke highlights the tension between social or “implied” identity and internal or “felt” identity; his greatest interest is in how individuals balance these two, sometimes asserting an identity that will enable them to “belong” to a social group, and at other times engaging in behavior that will communicate their wish to be seen as outsiders to a group.
As Brooke freely admits, his analysis owes much to the work of Erving Goffman – specifically, Goffman’s concepts of “ego identity,” or projected identity (13). As Brooke notes, a “sense of ego identity through group affiliation” guides how individuals choose to “present their personal and social identities” (14). Identification is a motivation for public behavior. Depending on an individual’s standing within a social group (as well as the standing desired by the individual), behavior will consist of a mixture of compliance and resistance; “it is in their pattern that a unique identity emerges” (17). In short, individuals often consciously negotiate the expectations of competing social groups to construct a public identity.
Thomas Newkirk takes a slightly different angle in introducing Goffman’s performative theories. As he notes, Goffman’s discussion describes a type of presentation that relies on an internalized identity: “The sense we have of being a ‘self’ is rooted in a sense of competence primarily, but not exclusively, in social interaction. It is a sense of effectiveness, the robust feeling that we possess a repertoire of performances so natural that they cease to seem like performances at all” (5). In other words, ego identity, felt identity and social identity operate in feedback loops, with the results of social interactions influencing felt identity almost imperceptibly. “Negotiation” of identities often occurs subconsciously – at times before and after the individual’s interaction with others – and not just through the conscious acts of “performing” identity. To elaborate on Newkirk’s point that individuals suffer a crises of competence “when [their] performative routines fail [them]” (5), not only does this happen when their intentional acts of self-identification break down, but also when the simple act of behaving “normally” (i.e. seemingly in accordance with one’s felt identity) yields abnormal results. This could be when the ambiguities in one’s identity finally enjoy a moment of visibility: we don’t necessarily know what we subconsciously believe ourselves to be competent in until that competency becomes dysfunctional – that is, no longer “unmarked.”
So while Brooke’s assertion – that individuals select an identity projection in a given context based on known role expectations – is worth exploring, we could also argue that when individuals negotiate their response to these expectations, they might not know or be able to articulate the internalized values and tendencies that are causing them to take a particular negotiatory path. The “choice” to comply with some roles and not with others might not be entirely under the individual’s control. In addition, since expected roles in a given context are always multiple, individuals can rarely make choices with crystal clear intent, and certainly can’t do so with a full conception of the scope of the consequences.
For example, Newkirk cites an example of a novice student writer who writes as though she were a historian: “Slavery did exist in early America. There is no debate about that, but to what extent is the question” (82). Newkirk characterizes this role adoption as “learning by approximation.” Yet he cautions that young writers aren’t necessarily at a developmental stage where they can understand or articulate how they’re learning in this way, or how such writing choices position them as subjects within an academic hierarchy. As intentional as the act of approximating scholarly language might be at times, does the student understand where the intention comes from and where it’s going?
Contribution of my study. Given novice writers’ difficulties in understanding the nature of their developing self-concept within a network of influences, self-assessment classroom projects are problematic: they seem to require a level of competency that students haven’t yet developed. But a “pedagogy of self-assessment,” which would gradually move students closer to expertise in their own “selves” as they invent and are invented by the university, seems like a possible solution. Such a pedagogy would attach low stakes to the “accuracy” of a student’s articulated sense of academic self, but would nevertheless attempt to introduce students to the idea of negotiating among versions of the self – and documenting that negotiation – within a rhetorically demanding context such as a writing classroom.
My dissertation project would be a step toward defining the terms of an open discussion of student and teacher roles by outlining the moves each make as they continually frame, reframe and respond to the task of self-assessment. Within this complex negotiation, my study asks, where do students and teachers appear to speak with the same voice? When does that voice match the (perceived) institutional voice? When does it resist? And when do instructors and students speak with different voices? What is the effect of shifting among these stances? Furthermore, because this study will examine role and authority negotiation as a component of the grading process, it will frame these questions within a discussion of power imbalances in the classroom – how they affect the presentation of self. For example, a teacher might encourage a sort of “sanctioned rebellion” against institutional narratives, but usually this rebellion has limits – and the teacher’s final authority enforces these limits even in the most “student-centered” classrooms.
Many composition scholars have echoed the assertion of Egon G. Guba and Yvonna S. Lincoln that “all stakeholders put at risk by an evaluation have the right to place their claims, concerns, and issues on the table for consideration” (12). It’s only through an equitable consideration of teachers’ and students’ goals that we might understand the impact of assessment and learn how to use it as an effective learning tool (rather than merely as an accessory to learning) in writing instruction. This project will help teachers conscientiously construct self-assessment activities in ways that enable students to more fully understand the expectations that will be placed on their “selves” as they progress through their courses of study.
Defining the scope. The major research project of this dissertation will consist of an examination of three writing instructors who initially employ varying “levels” of student self-assessment in their courses ; I’ll document their approaches to incorporating student-centered approaches to classroom assessment into their courses and how their students respond. These courses will take place in Spring semester 2009.
In Kathleen Blake Yancey and Jane Bowman Smith’s conclusion to Self-Assessment and Development in Writing, they list a number of activities to which the term self-assessment can refer, from mental processes to written products. But I’m most interested in their fourth definition: a set of “heuristics that help students to establish a habit of critical inquiry that is active rather than passive, to integrate the learning into what is already known, and to project what more can or should be learned” (170-71). These heuristics are typically delivered in two ways: via classroom discussion and via self-assessment project handouts. The discussion of self-assessment, the handouts and students’ written responses to them will be the objects of my study. I won’t attempt to describe the workings of students’ minds as they self-assess, or the direct impact of self-assessment pedagogy on composing processes; definitive claim regarding the effectiveness of self-assessment is beyond the scope of this study. Instead, I’ll use instructors’ and students’ responses to the task of self-assessment to examine and describe that task as a culturally laden component of teaching, learning and evaluation within an institutional setting that presents conflicting definitions of all three. As mentioned earlier, this study will note patterns of identity/role/authority conflicts evident in the writing that comprises self-assessment projects. One might call this primarily a study of articulated and negotiated classroom relationships and power dynamics, rather than a study of writing or development (although my literature review will comment heavily on these latter topics, as will the student and instructor narratives I collect). Its implications and recommendations will be framed within the network of competing expectations and demands that surround college-level writing classrooms.
My research questions are stated in the introduction to this proposal, but there are others connected to my proposed data sources:
• How do department, institutional and professional expectations (explicit or implicit) appear to influence an instructor’s decisions in defining and structuring self-assessment, particularly in handouts?
• In written student self-assessments, where can we see evidence of a struggle among student roles: student/writer, evaluator/evaluated, etc.?
• How do intangibles such as personality and instructor-student rapport appear to play a part in class discussions that frame self-assessment activity?
• When do the efforts to coordinate these elements break down? When and how does the imbalance of classroom authority trigger an “ace condition” (Charles McKenzie’s term) in which the thoughtful negotiation process grinds to a halt and teachers and students “circle the wagons” in defensive postures? To what extent do these episodes damage the overall classroom orientation toward self-assessment?
Participants. This study will be situated within the classrooms of three different composition instructors, each of whom will be teaching at least one section of the same course in both Fall 2008 and Spring 2009. In a midsemester Fall 2008 staff meeting of undergraduate composition instructors, I’ll introduce my study, describing its scope and goals, the level of commitment required from participants and the benefits for participating; I’ll give out forms on which instructors can indicate their interest in participating. As I’ll note in the meeting, I’ll select three instructors who are interested in experimenting with their approaches to assessment and grading by incorporating self-assessment and, in particular, some element of student self-grading. I’m choosing to work with three different instructors because I hope to show how different instructor choices in structuring self-assessment affect the way authority negotiation plays out throughout the process. Each instructor will bring a very unique approach; however, if possible, I’d like to recruit different types of teachers (perhaps a new teaching assistant, an experienced TA and a full-time lecturer).
Setting. This study will take place within the University of New Hampshire undergraduate composition program, which is located within the English department. While the department currently employs around 40 tenure-track faculty, the majority of composition instructors are TAs, adjunct faculty and lecturers. Even among the tenure-track faculty who occasionally teach composition, there are few who would teach the same course in both semesters of 2008-09, so it’s likely that my participants won’t include a tenure-track professor. Nevertheless, thanks to the longevity of several lecturers’ careers and the curricular flexibility of the program, I’ll still be able to choose participants from a wide variety of experience and teaching styles.
The program’s ties to the “process movement” in composition shouldn’t be understated: portfolios, conferences and other process-oriented pedagogical devices are already deeply ingrained into the curriculum and culture of undergraduate composition at UNH. And though the program is standardized in some ways (a common syllabus and textbook for inexperienced TAs; a common attendance policy; expected genres of writing for each course), there are no enforced department-wide grading rubrics or grade distribution policies. So assessment practices among my participants prior to the study will vary significantly and will no doubt influence their approaches to self-assessment in different ways.
Mainstream undergraduate composition at UNH consists primarily of four courses: First-Year Writing (English 401); Introduction to Creative Nonfiction (English 501); Introduction to Technical Writing (English 502); and Persuasive Writing (English 503). Teaching assistants generally teach English 401 during the first two years of their respective programs, and experienced teachers often gravitate toward English 401, 501 or 502 (in other words, they indicate a preference in teaching assignments). For this reason, I’ll be able to find participants who will teach at least one section of the same course in both semesters of AY 2008-09. Because of the limited availability of English 503, this course will most likely not be involved in the study.
Study orientation. This study is best described as an collective instrumental case study, defined by John W. Cresswell as a study of more than one case that is “used instrumentally to illustrate [an] issue” (62). Such a study draws on multiple sources of information, most notably interviews, observations and physical artifacts (in this case, as mentioned, the instructor handouts and student self-assessments). These data are collected to enable a detailed description of each case. The interpretation of this data will consist of within-case analyses of each course narrative as well as a cross-case analysis that analyzes the broader themes (this distinction in analyses is drawn from Cresswell 63). As is common in case study design, I’ll construct an overview of each course that features closer descriptions of major classroom events related to self-assessment as well as patterns in self-assessment writing. As with other case study research, generalizability is not a goal here (the main reason for multiple cases is to demonstrate how a unique classroom population, a unique teacher’s classroom design and other settings that change substantially from classroom to classroom impact the implementation and negotiation of self-assessment exercises), so the total number of cases will be small.
This study will provide a case study narrative for one course per participant in spring semester 2009. This spring course will be based on plans the participant and I design prior to the semester, as indicated below. The bulk of my primary research will be conducted during spring 2009, though interpretation and supporting secondary research will obviously extend beyond this time frame (see Dissertation Structure, page 12). The following is a breakdown of my different data sources.
FALL 2008: end-of-semester interviews. In the middle of fall semester, I’ll recruit instructors for the study, as described above. At the end of the semester, I’ll conduct an initial interview to determine each instructor’s interests in changing his/her assessment approach. To help them define these interests, I’ll ask instructors to articulate their perceptions of their fall students’ self-assessment abilities.
Initial interview. Prior to this interview, I’ll send a list of questions to each participant, which I’ll ask them to answer in the interview. These questions will include the following:
• Generally speaking, how do you feel about your fall students’ abilities to assess their own work? Where do you see evidence that any of your students made successful independent judgments to improve their work from draft to draft, from assignment to assignment, or across the span of the course? Were any students noticeably unsuccessful at this?
• Based on what you’re seeing this semester, and given that we’re going to work on self-assessment next semester, what do you know you would like to change? What are some of your ideas for helping students to develop their skills of self-assessment? What would you like to see happen?
In our short, recorded end-of-semester interview, I’ll ask each instructor to respond to these questions as we talk about possibilities for the spring. Before the interview, I’ll suggest that instructors bring course projects and/or portfolio cover essays from the semester if doing so would help them answer the questions specifically. I won’t collect these essays, but we might walk through them while talking about goals for the spring semester. My hope is that this discussion will establish basic needs and expectations for our winter break work. It will also provide material for my overview of the course.
WINTER BREAK 2009. During the winter break, I’ll devote time to working with each instructor to develop assignments, heuristics and lesson plans that move their classrooms toward a “Level 3” or “Level 4” self-assessment environment (see the Levels descriptions in Appendix 1), depending on how far they’d like to go. The instructor will (re)design his or her own course; my job will be to assist the instructor in this design by providing sample handouts, templates and other resources on self-assessment, peer evaluation, group evaluation and so on. The specific goal of this planning is to build in enough assignment-based instruction in self-assessment that the instructor would ultimately be comfortable asking students to try their hand at self-grading as one of the activities of their self-assessment process. Each instructor will approach the task of building self-assessment into his/ her curriculum in a different way. Additions to the course may include short lectures and classroom discussions on values and expectations for academic writing; in-class practice response and grading exercises; in-class “inductive rubric” design and criteria mapping based on sample essays; peer and group response methods incorporating assessment; and, of course, midterm and final portfolio assessment projects .
It’ll be our job to figure out which instruments for self-assessment, and which implementations of those instruments, best fit these criteria and the existing structure of the course at hand. In short, we’ll be asking questions such as, “How can we create a systematic approach to self-assessment that ‘meshes’ well with the classroom goals and the teacher’s methods?” Of course, these plans must address the changes the instructor wishes to see in his/her students’ work as outlined in the Fall semester interview.
Most likely, there will be two meetings in the Winter 2009 planning process: the first to define the major areas of (re)design and the background material to read, and the second (to be conducted after a few days of course revision) to work out the details of each course activity within the syllabus and to plan researcher visits.
Required elements. Though the instructor will have the final say in the design process, there will be a few details stipulated beforehand:
• Though the number of self-assessment activities will vary according to the instructor’s preference, there will be, at a minimum, a midterm (approximately week 8) and final (end-of-semester) self-assessment essay assigned to each student in the spring course. These will be the student writings I collect for the study; other writings will be collected as regular coursework by the instructor only.
• Though all students will be turning in self-assessment projects as mentioned above, any student may choose to opt out of the study without penalty (as is required under IRB and FERPA guidelines). I will only be conducting close analysis of four self-assessment essays per class at each stage, so it isn’t necessary for all students to release their materials. Ideally, the self-assessments will be collected in a restricted electronic environment, such as a Blackboard group space, that is only accessible to the instructor and student; those who consent to join the study will simply add me to their space.
• As part of our agreement, each instructor will incorporate student self-grading into the self-assessment process for the course; that is, each student will at some point propose and rationalize a course grade as a component of their self-assessment process. It’ll be left to the instructor’s discretion whether this is done only in the final essay or throughout the course, as well as how much weight s/he will attach to these grades in the determination of the final course grade, as well as whether to reveal how influential the students grades are in the overall scheme. As I will argue, the mere presence of the letter grade, weighted or not, changes the stakes of the course work substantially enough to produce an observable effect on the teacher-student power dynamic.
The planning of initial classroom discussions on self-assessment will be especially important in this planning process. Hilgers et al. list topics for such classroom discussions, which include “a teacher’s description of an assignment and expectations” (which establish a baseline for discussing how these expectations vary across stakeholders); criteria generated in teacher-student conferences or peer-feedback groups; characterizations of “good writing” mentioned in discussion of readings; and so on (11). This series of discussions will provide necessary context for the self-assessment prompts and handouts.
SPRING 2009: handouts, observation data, interview data and student self-assessments. The spring study will record each course as it unfolds under the guidance of the instructor and students – there will be no researcher intervention in the classroom itself. I’ll collect classroom materials and will continue to interview the instructor on perceptions of the classroom.
Handouts. I’ll collect all handouts that describe the midterm and final self-assessment projects specifically, as well as any others that define the process of self-assessment, its value and its function in the course.
Field observations (classroom visits). In the spring, due to the frequency of assessment-related classroom activities, there will likely be a number of discussions related to the handouts that define the self-assessment projects. I’ll be present for the first two class discussions of the semester, which should provide a sufficient view into the way the instructor and students set the basic terms for class work in self-assessment. During my observations, my eye will be focused on the interaction of the teacher with his/her students during the framing of exercises. My job here is to act as a reporter – to note the language that is used while describing activities, the questions and negotiations that occur in the working out of these activities, and the end-of-activity comments given by the instructor. (see Data Analysis, page 11-12).
This examination will consider the non-verbal communication as well as the verbal. Specifically, I will be taking advantage of scholarship on outcropping (a term defined by Mary Sue MacNealy as “something visible … indicat[ing] something not visible,” 229). I’ll take notes on proxemics (social distance between teacher and student and arrangement of the classroom) and kinesics (body language and facial expressions used by teachers and students to accompany the spoken words) and analyze them under the guidance of established research on nonverbal behavior. Catherine Marshall and Gretchen B. Rossman, among others, describe ways to provide context for nonverbal cues in order to establish a reliable basis for interpretation. I’ll be looking to see how teacher and student identities and roles are defined and projected, as well as how authority is asserted, via this nonverbal communication.
Interviews (midterm and final) and selected student self-assessments. I’ll meet with each instructor twice to discuss the instructor’s perceptions of the class and their development in self-assessment. In recorded interviews, we will talk about the instructor’s satisfaction with their students’ self-assessment abilities (as evidenced in their written course projects and self-assessments), as well as any perceived needs for improvement in their students’ proficiency in self-assessment.
Midterm pre-interview questions, emailed to the instructor in advance, will include the following:
• How satisfied are you with the self-assessment work you’ve received in this course so far?
• Can you pick out two examples of students (from those who have consented to be part of the study) who appear to be effectively self-evaluating their work, their strengths as writers and their needs for revision? Can you pick out two who are not? What evidence suggests this to you? We’ll talk about these essays in our interview.
• Based on what you’ve received so far, is there anything you plan to add to the class discussion on self-assessment in preparation for the final self-assessment and self-evaluation?
I’ll ask the instructor to write informal notes in response to these questions. In the hour-long interviews, I’ll ask the instructor to elaborate on his/her answers the questions above based on his/her notes, and to walk me through the drafts s/he has chosen. I’ll analyze the chosen self-assessments as noted below (page 12).
Final pre-interview questions will include the following:
• How satisfied are you with the final self-assessment essays you’ve received in this course?
• Let’s follow up on the four students whose work you picked for discussion in our midterm interview. What is your opinion of their final essays in comparison to their midterms? Do they appear to have done similar work, improved, regressed, stagnated, etc.? In what ways? We’ll talk about their final essays in our interview.
• Based on the results of this experiment, what do you think about the self-assessment process we worked out for this semester? In what ways did it change your course? Would you do it again? Would you change your approach at all if you did – and if so, how?
• How has your role as an authority in the classroom and in assessment changed this semester? What do you think of the change? What aspects of it are positive, and which are negative? Which were unexpected?
In this study, handouts, interviews, observations and self-assessment essays will be analyzed together within a single interpretive framework. Because self-assessment essays have rarely been analyzed in print since the 1980s (and self-grading in writing courses has never been studied, to my knowledge) no established framework currently exists. Therefore, my coding schema will necessarily be emergent; however, I can now point to existing work that will contribute to the conceptual frame.
Role metaphors/subject positions. As mentioned earlier, my overarching theory of identity negotiations will draw heavily from Brooke’s (which in turn draws from Goffman’s). This work establishes a basic premise underlying my analysis: that both students and teachers perform identities (whether consciously or not) to negotiate perceived expectations within an educational environment. Brooke’s theory describes how teacher and student roles are embedded into “progress narratives” (“teacher as evaluator, student as performer”; teacher as “diagnostician,” student as “developing adolescent,” etc.) and that progress and development are discussed in terms of the socially established relationship between these roles (39-42). One of my duties will be to locate references to role metaphors, such as those mentioned above, in the classroom discussions I attend and in the handouts I collect. I’ll also note any shifts in the way the instructor characterizes his/her role or that of the students, and to what extent the handouts acknowledge and attempt to resolve the conflicts in different roles and expectations. I’ll then need to see how those metaphors are carried over to the students’ self-assessment essays and whether they change or are challenged in the transition. To define which roles I should be looking for, I’ll rely on the work of Roz Ivanič (Writing and Identity: The Discoursal Construction of Identity in Academic Writing), which utilizes an extensive coding scheme for subject positions in student writing.
Identity-based stances toward the task of self-assessment. In “The Arts of Complicity: Pragmatism and the Culture of Schooling,” Richard E. Miller uses the work of James Scott to distinguish between “public transcripts” and “hidden transcripts” within the culture of schooling. A public transcript appears in “open interaction between subordinates and those who dominate,” while a hidden transcript “describes the discourse ‘that takes place “offstage,” beyond direct observation by powerholders’” (quoting Scott, 661). Miller uses this distinction to show how analyses of power often fail to consider hidden transcripts and therefore “ceaselessly [produce] evidence that the disempowered willingly and thoughtlessly participate in the system that ensures their own subordination” (661). Miller argues that no student work should be taken for granted as an authentic reinscription of powerless student identity – much if not all of it contains elements of the hidden transcript. Some student work contains an intentional rupture in the public transcript – an open questioning of teacher authority or competence, for example, or a stated lack of faith in the goals of a classroom or an academic program. Given that a student knows the risks of such outward dissent, we can read these moments as purposeful rhetorical gambits. How will the teacher respond? How will this renegotiate authority in the classroom? These students are experimenting to find out the answers to these questions.
Student self-assessments are interesting pieces of writing within this framework because it’s always clear whether they’re meant to be “public” or “hidden.” If the teacher establishes self-assessment as a subversive activity and establishes him/herself as also rebelling against “powerholders,” s/he might momentarily be able to “sell” a self-assessment essay as a hidden transcript. Depending on the student, a self-assessment essay (or at least parts of it) might include open challenges to institutional authority, seemingly at the instructor’s behest. But because “[t]he classroom can tolerate all manner of nonconformity, but every classroom has its limit” (667), a self-assessment essay will undoubtedly also bear characteristics of a public transcript.
Based on these ideas (as well as my past experience with self-assessment essays), I expect to see textual evidence of the following stances:
• The “obedient response”: This stance is signaled by concepts and phrasings in instructor discussions and handouts that uncritically champion social, institutional and programmatic ideals for good writers and students, as well as student responses that appear to mimic the instructor’s language.
• Sanctioned rebellion: This stance is signaled by phrasings and concepts in student self-assessments that appear to counter institutional ideals but are championed by the instructor in his/her self-assessment discussions or handouts.
• Unsanctioned rebellion (subversive): This stance consists of rhetorical moves that place a student’s self-assessment beyond the limits of rebellion sanctioned by the instructor – for instance, this might include an open questioning of the course for which a self-assessment essay is being written.
• Unsanctioned rebellion (conservative): This stance indicates a student’s attempt to “pull back” from a teacher’s “progressive” goals in the assignment of self-assessment projects – for example, such a stance might include an unsolicited deferment to the teacher’s judgment (“I prefer to let you decide”).
My goal is to note the handout guidelines or questions that prompt a shift into one of these stances (as well as any others that might emerge); the phrases that signal them; and the ways in which students create an overarching narrative that allows space for different stances to coexist within a single self-assessment essay. The post-midterm and post-final instructor interviews will capture instructors’ reactions to these rhetorical moves, showing the responsive stances they choose to take.
I’ll be particularly interested in noting the shifts that take place in close proximity to the self-grading portion of these self-assessments. As mentioned at the beginning of this proposal, many students automatically shift into the mode of “obedient response” when faced with the immediate prospect of a grade – but in these cases, it might be the case that other stances simply become more veiled, rather than disappear. Since many instructors either assign self-grading only as an element of the final self-assessment (or only issue a highly tentative midterm grade) it’ll be interesting to see how the final self-assessment differs from the midterm. No matter what, students’ stances will be very interesting to chart, because they’ll provide a more realistic picture of the “meta-awareness,” independent learning, increased motivation and “internalized criteria” for good writing that are cited as advantages of self-assessment by its advocates.
• Introduction: Overview and rationale (context: pre-study reports). Sets up the general problem of the dissertation and contextualizes the main research questions.
• Chapter 1: Literature review.
• Chapter 2: Methodology and background of the 2008-09 study. Includes Fall 2008 initial interview data.
• Chapter 3: Study findings: Spring 2009 classroom description.
• Chapter 4: Study findings: Spring 2009 self-assessment review.
• Chapter 5: Discussion of 2008-09 study findings.
• Conclusion: Implications of the study.
• Spring 2008: Complete and defend proposal.
• Summer 2008: Complete IRB documents. Write the introduction and Chapter 1.
• Fall 2008: Recruit participants. Conduct fall pre-study (initial instructor interviews). Write Chapter 2.
• Winter break 2008-09. Work with each instructor to modify his/her classroom for Spring 2009.
• Spring 2009: Conduct the classroom study. Submit proposal for dissertation-year fellowship.
• Summer 2009: Write Chapters 3 and 4.
• Fall 2009: Write Chapter 5. Enter the job market.
• Spring 2010: Write the conclusion. Complete and defend the dissertation.
Posted by Mike G on September 12, 2007
Tags: Meta
Comments on specific paragraphs:
Click the
icon to the right of a paragraph
Comments on the page as a whole:
Click the
icon to the right of the page title (works the same as paragraphs)