Computers grading writing?

Journals all over the world, Korea included, have been delving into the question of whether or not computers can or should be used to assess student writing.  Teachers and students continue to be opposed to the idea, even if it would make their jobs quite a bit easier.  The NCTE just released its statement on the matter.  Unsurprisingly, teachers, teacher advocates, and those associated with NCTE feel that writing is too nuanced, too complex, in short, too human an activity to be graded by machines.  Here is a great little article that gives some practical examples of how difficult it is to grade writing.

I'm personally opposed to the idea of computers assessing writing for a number of reasons... but I don't think it is helpful to play the "we're essentially human and computers can't get that" argument.  Mostly because if that argument still holds water, it probably won't for very long.  It's an argument that's bound to end up on the losing side of technological history.  

Let's face it, if a human activity is based on any sort of pattern or set of conventions, sooner or later there will be an algorithm that can anticipate and reproduce it.   Those who discuss the nuances of writing and the beautiful complexity of human thought that writing encompasses are playing a waiting game.  Technology is only getting better, and that argument is not only doomed to historical failure, it completely misses the real issue at hand here.  

Humans (and especially students) are masters at adapting to their environments.  Language use is no different.  It doesn't take very long for most students to work out what a teacher wants from them, what they have to do to gain approval, and what they can get away with.  In short, school writing, like any kind of language use, is a social activity that is always in a process of adaptation.  Users of language are always "adapting to others' adaptations".  I can remember plenty of times that I wrote to teachers or professors in order to project a certain image of myself, by referring to a random comment they may have made in passing, trying to establish a common point of interest, political perspective, or world-view...  In short, writing is about making a connection with a reader.

I'm sure this sounds like I'm slipping into the "but we're so wonderfully human" argument... I'm not.  

As we human-folk are always in a process of adapting to our environments, I'd like to consider what it would mean to ask students to write their school essays to a computer.  What kind of writing can we expect from a student who is (very wisely) adapting to the expectations of a machine that is judging her work?  Gone is the opportunity to 'impress' your reader, to coax the reader into or out of a particular way of thinking, of projecting an image to the reader, of surprising the reader....  I think that we might expect this student to write more efficiently, in a more standard format, to take fewer risks, and to generally do no more or no less than complete the required task. 

In a fabulous short essay called Why I won't be Using Rubrics to Respond to Students Writing, Maja Wilson  demonstrates how teachers have the capacity to become machine-like in their grading and how there is constant pressure for us to do so.  

One might suggest that the more standardized and quantified our responses to our students become, the more feasible it becomes that a computer could do our work.  The task is, then, not a matter of exposing the shortcomings of current technology but in addressing the ways that our jobs as teachers have already been reduced to work that could be done by a machine.  The issue has nothing to do with computers taking the place of humans but of questioning the context in which such a thing could even be conceived.

[an aside]
I remember co-teaching in a high school English class.  The full-time teacher and I decided to put together a lesson on juxtaposition.  We asked students to bring in a photograph and to select a song.  During the class, students swapped songs and pictures.  While listening to the songs, we asked them to write their thoughts- "what does this mean?" / "how does the song relate to the picture?" and so on.  Before we started, we got the obligatory "how long should it be?" / "what exactly do you want us to write?" / "is this for a grade?"  

What struck me, both before and during the activity, was that we were envisioning the students writing for the purpose of catharsis or expression.  We were hoping for students to communicate a message through the song/picture juxtaposition and for those listening to not only 'get' the message, but to produce a creative expression of how the juxtapositions impacted them.

What we got were short sentences that were obviously written only in order to complete the assignment.  

My point is not that the lesson flopped (though it clearly did).  My point is that writing... school writing, is something that we have been in a process of reducing to its lowest common denominator for years.  As we make writing into a 'school' activity, we subject our students' ideas, thoughts, and expressions to a numerical grid.  Now that computer programs are being written which are just as efficient, consistent, and accurate (perhaps) at quantifying written thoughts, teachers are calling on the "all too human" card.  We (not everyone mind you, but public education as an institution) have been acting as imperfect computers for years.  It strikes me as ironic that we are now in an uproar over the very natural progression to computers acting as imperfect humans.  

I hope that the fact that computers can (or soon will) be able to quantify writing as effectively as humans might serve as a wake-up call.  My hope is that this exposes not the imperfections of computers but the long evolving process of reducing human experience, activity, and expression to quantified measurements.  The current debate strikes me as little more than humans acting as imperfect computers complaining about computers acting as imperfect humans.  All the while, we are leaving unquestioned the dehumanizing effects of an education system that has given birth to the idea that the work of teachers could be reduced to the calculations of automated machines.  

Ideology, Queer Theory, and Classroom Practice(?)

Last week's discussion seemed to emphasize the point that we can read a text in any way we choose. In other words, if we wish to find sexism in a text, then we will find sexism. If we want to find class issues, colonialist perspectives, or racism we will certainly find those as well. In other words, we see what we look for.

I think this clip offers a great example of this point:  

http://www.youtube.com/watch?v=2nxIi3MctdQ



Sexism?
One could look at the woman who is positioned as subservient to the male customer. He calls her by her first name and she has to accept this because she is this unequal position.  One could argue that this commercial reiterates traditional and somewhat oppressive roles that a woman must take up, and that it suggests that the successful male is in a much different (and more powerful) social role than the successful female.  Her role has been reduced to accepting his mildly sexual advances and this is the acceptable way to respond to such men.

Marxism?
What kind of man is this? What is the significance of the alpha-male persona and what does this say about the successful modern man? It seems reasonable to suggest that this advertisement produces an image of material success and seeks to convince viewers that in order to be successful in our globalized world, one must be stylish, well-groomed, international, and wealthy enough to travel in business class.

Racism or Colonial Concerns? 
One might consider the relationship between the white male and the submissive non-white woman whose duty is to make this man's life comfortable.  Further, this advertisement (from a company based out of Hong Kong, I believe) seems to be tacitly drawing on a historical relationship between Britain and Hong Kong and reinstating a position of subservience to a past colonial power (symbolized by the White Englishman) and associating the former colony with servitude and femininity.

...perhaps...

One of the primary goals of the course thus far has been to present a variety of lenses through which we can view various texts. We have been hoping to show that our 'common sense' or habitual way of 'reading' is only one of many, and that there are always other ways to understand and interpret the messages in a given text. But my concern is that in focusing on all these different 'lenses', we might be overlooking very simple and very important questions: “what do I think of this text?”  "How does this text speak to my experience?"  In other words, I worry that the structure of the syllabus might have led us to overlook the value of personal experience and reduced critique to little more than an intellectual exercise.

As I have said all along, the purpose of all this cultural theory we have worked in/through is NOT to simply understand theory.  The purpose has been to grasp new tools and to consider various ways of thinking and reading.  That said, is there a danger in viewing these political issues as mere 'tools'?  Would those impacted by sexism, racism, and other forms of institutional violence be comfortable with these issues being presented as 'lenses' or 'tools'?  In other words, is the approach to the syllabus one that could only have been constructed by a straight, White, middle-class, native English speaking, American, male?

I would like to stress that the capacity to view a text from multiple perspectives does not necessarily mean that these various readings (feminist, classist, postcolonial, racist) are not extremely important explanations.  Racism is real.  Sexism is real.  The effects of hundreds of years of colonialism are still with us.  I do not wish to make light of political and social struggle.  

Yet still.... I think it's always important to view these issues from a historical and context specific perspective.  Consider the following promotion picture from the group "The Bubble Sisters".  This is a dated photo and a long since forgotten band, and there has been a good degree of discussion on this already here, here, and here.  So I'll just say that given the historical significance of blackface, it seems more than fair to call out the racist nature of the photo.  Though given the fact that the history and the cultural references in Korea are much different, there is a question of whether or not cultural distance is an excuse.  Another example is in the Japanese commercial that depicts a talking monkey, who is dressed in a tie and speaking at a podium calling for "Change".  Is this a reference to Obama?  Are the advertisers racist?  Again, my immediate reaction is "YES, of course!"  Clearly the history and the cultural references are troubling.  But these are not the same in Japan... so again, the question that comes up is, "to what degree is this culture responsible for the possible meanings of its texts?"

I am not going to delve into a deeper reading of these problems.  But I will say that it seems clear that both history and context are equally important factors to consider.  For example, what are the implications of a White male calling Koreans 'racist' for producing these kinds of ads?  What kinds of power have enabled me to make these kinds of proclamations?  Is there a difference between me calling this image racist and a black woman who is living in Korea calling it racist?  It seems that the meaning(s) of a given text is never completely closed.  So while a text might project obvious and overt racism, there are always a variety of depths to which this can be read.  And if I dare say, there remains the potential for negotiation and exploration.

http://www.youtube.com/watch?v=eBBODzM2bNw

Ideology

So this leads to the question of ideology.  In the work we have done so far, ideology acts as a primary explanation for our interpretations. We interpret things in a certain way because that's the 'normal' way of seeing things. If a woman is represented as 'serving' and 'pleasing' a male in a dominant position, that's because we view this as a normal and acceptable practice. And it isn't until we recognize the ideology of sexism that we begin to question the 'naturalness' of these kinds of representations.

Uncovering such ideologies is essentially what we've been trying to do with our focus on Received & Oppositional readings of texts. Received readings are meant to allow us to recognize the 'normal' or 'naturalized' way of understanding something.  What assumptions are made, how is the audience supposed to react?  Oppositional readings are geared towards provoking readings that see beyond these damaging ideologies. Concepts like feminism, Marxism, postcolonialism, and institutional racism are meant to offer us more liberating (and more accurate) views of the world and how we are positioned with it.

In other words, by revealing the nature of ideology, we might get a glimpse of the 'true' nature of oppression and we are therefore empowered to begin envisioning a more just and equal world.   That means that ideological approaches to reading assume that a more just society and a freer self are lying dormant, waiting for us to lift the veil of ideology so we can begin creating a better world.  

In terms of feminism, these ideological shifts have often been described as follows:

> Liberal Feminism
(1st wave)
Opposed to men.  Women can do anything men can do.  We must remove traditional gender roles in order to benefit women and society as a whole.  Equal pay for equal work, equal opportunities, and legislation can lead to equality.


> Cultural Feminism
(2nd wave)
Opposed to masculinity.  Women have unique and essential qualities (namely, the capacity to give birth) that has produced a feminine sensibility and female ways of 'being in the world'.  Women are inherently more subjective, nurturing, and emotional.  Rather than seeking to join a masculine-dominated world, women would do well to create spaces free of masculinity and patriarchy.

> Poststructuralism
(3rd wave)
Opposed to essentialism.  There is no single quality that captures the essence of anyone.  All categories are social conventions and these social conventions are produced through relations of power.  The problem is not so much in the oppression of a specific group of people as much as it resides in the creation of categories.  Naming and categorizing are tendencies that must be challenged and the nature of the individual is always fragmented, partial, and unfinished.  "What is intolerable is no longer so much that which does not allow us to be what we are, as that which causes us to be what we are"    - Miguel Morey

Performativity & Queer Theory  

These are two concepts that grew out of the later work of Michel Foucault and are closely associated with Judith Butler.  Butler borrows from Austin's idea of Speech Acts, and applies it to subjectivity.  Put simply, Austin classifies two kinds of statements... constatives and performatives.  Constatives simply describe the nature of reality.  They are propositions that can be either true or false.  "The light is on" or "today is my birthday" would be examples.  Performatives are statements that themselves carry out an act.  "I do", "you pass" and so on.  In speaking, the speaker carries out some real world function.  These statements are not true or false, they simply carry out an act.
[incidentally, this is the means by which the right to 'free speech' does not give someone the right to yell "fire" in a public place.  In yelling "fire", one is not only 'speaking' but also performing an act.  Stanley Fish has also argued that this is the grounds for banning hate speech in places that guarantee free speech.  In spreading hate speech, one is actually performing an act of hatred and therefore not simply 'speaking', but also acting.  And of course, it would be absurd to make a law protecting one's right to free action because that would, by nature, make all other laws null-and-void.]

Anyway, Butler suggests that in saying "It's a girl" at the baby's birth is not a constative statement, but a performative one.  In saying "it's a girl", one ascribes a certain range of possible identities to this individual and creates the conditions into which this individual can be recognized.

Interestingly enough, modern medicine is now beginning to make similar claims that there are no objective grounds for determining sex in the ways that we conventionally do so.

These categories are actually on a continuum, meaning that individuals are not simply men or women in a simple sense, but actually fall within a combination of continua, the result of which is a somewhat arbitrary socially constructed category.

There is no objective foundation for an essential sexuality.  The issue for poststructuralists is in the 'naming', compartmentalizing, and categorizing of individuals.

This is a great documentary that goes into much more detail....

http://www.youtube.com/watch?v=GS6iMHER2G4



A quote by Ben Franklin shows the degree to which race is a historical and social construct:
"All Africa is black or tawney; Asia chiefly tawny; (native) America wholly so.  And in Europe, the Spaniards, Italians, French, Russians, and Swedes are generally of what we call a swarthy complexion; as are the Germans also, the Saxons only excepted, who with the English, make the principal body of white people on the fact of the earth."




> So what does this have to do with reading?  With classroom practices?


Interpellation
The 'hail' and the 'turn'

"Practical" instructor / "Theoretical" instructor / Native speaker / "Westerner"




Transgression








A classroom application

Nancy Chunn's New York Times art project:

http://thecriticalbeatdown.blogspot.kr/

And just for kicks, a new video just went viral:
http://youtu.be/bQ2VAV3HYdU

Some Resources and Readings

This is a short article by Scott Thornbury that includes an outline of basic tenets of critical pedagogy.

He lists the following principles:
Critical Pedagogy...

1. is transformative, and seeks social change 
2. foregrounds social inquiry and critique
3. challenges the status quo and problematizes ‘givens’
4. devolves agency to the learner
5. is participatory and collaborative 
6. is dialogic 
7. is locally-situated, and socially-mediated 
8. is non-essentialist, i.e. it doesn’t reduce learners to stereotypes, but rather legitimizes individual identities 
9. is self-reflexive


And here are two entries from his blog that touch on the idea of critical practice:
Critical Pedagogy
Linguistic Landscapes

There are also a number of blogs that offer resources, ideas, activities, and thoughts on critical practices in language learning.  One of the most well known is esletc.  Though a lot of the materials seem to be aimed at highly proficient learners, there is a lot of information there that might be useful to you.
The Freire Project is another site with information, resources, and links for critically minded educators.  And here is a blog aimed at sharing lessons and activities for English learners in Turkey (some of which seem to have a 'critical' edge to them).
I encourage you to have a look around the web for other resources, idea, and activities that you might be able to adapt to your own teaching and learning settings....


Here is a guide to reading images (adapted from resources at "The Critical Thinking Consortium") that you may be able to adapt to your practice:

Explaining Images

adapted from “The Critical Thinking Consortium”

                                                            Observations                                                               Inferences

Who is in this picture? (Gender, Social Class, Sexuality, Status)
What are these people doing? / What is happening?
When was the picture taken?
Where is this? (Region, Country, Culture...)
Why is the person doing this?

We can also combine this exercise with adaptions of the “critical questions” we discussed in class: 

1)  Who took this picture?  Whose perspective does this show?  What other perspectives can we imagine?

2) Who is the intended audience?  Who will see this picture?  What will they probably think about this picture? 

3) Who is represented?  Who or what do the figures, places, and objects represent?  What might be missing from this image?  

Freire

What is the story?

Who tells the story?

What names do we use?

The Dialectic

Oppressor  -- Oppressed 

Theory  --  Practice

Object  --  Subject

Banking Education
Problem-posing
Praxis
Dialog
Conscientization





Received Readings and Oppositional Reading

Who is the ideal audience?
What is the intended message for the ideal audience?

Some Guiding Thoughts/Questions on Freire

In response to a few people who mentioned they would like to focus more specifically on our readings, their meaning(s), and their relevance, I have attached a few thoughts/questions related to the first two chapters of "Pedagogy of the Oppressed".  You do not need to answer these for class, but they might bring some focus to your reading:

First, Freire is offering us a very clear example of dialectic thinking.  In dialectic thought, any idea or "thesis" is understood in relation to an opposing idea or "antithesis".  Knowledge is gained when one is able to move beyond what seem to be contradictions between the "thesis" and "antithesis" towards a "synthesis".  In Freire, we see repeated references to various dichotomies.  For example:

Oppressor - Oppressed
Object - Subject
Theory - Practice

In each of these examples, Freire seeks a synthesis that brings us to a new level of understanding.  When reading these two chapters, try to pay close attention to the ways Freire attempts to synthesize these various dichotomies.

Oppressor - Oppressed
What does Freire mean in saying that a dichotomy between Oppressor and Oppressed leads to the dehumanization of both?
What can be done to "synthesize" this dialectic?  Why is this "synthesis" so difficult? (hint, he discusses this for the first several pages of Chapter One)

Object - Subject
This dialectic is very closely related to the ways we name the world, and involves the relationship between our thoughts and our actions (see p. 56).  Freire denies the idea that we have access to a purely objective world, just as he denies the idea that our subjectivity can be separate from the world.  In short, the objective world and the self are in a dialectic relationship.
What does Freire's object-subject dialectic (p. 32) suggest about HOW we might transform the conditions of oppression?  What is "praxis" and how does it attempt to resolve a dichotomy between theory and practice?
Why/how does 'banking-education objectify the world?  How can problem-posing challenge this objectification?

Freire claims that "problem-posing education affirms men and women as beings in the process of becoming-as unfinished, uncompleted beings in and with a likewise unfinished reality"? (p. 65).  What do you think is the significance of understanding ourselves as incomplete or unfinished beings and how does this help us synthesize the oppressor - oppressed dialectic?


Some terms commonly associated with Freire:
Banking Education 53-57
Problem-posing 60,63
Dialog 47,49,
Praxis  33-35,47,60
Conscientization 48,49,60,63

"Any situation in which some individuals prevent others from engaging in the process of inquiry is one of violence....  to alienate humans beings from their own decision-making is to change them into objects (p. 66)."

Critical Theory & The Frankfurt School

Why think about pop culture?
What are some of the effects of pop culture?
(How) Can we 'read' pop culture in a critical manner?
Should pop culture become a relevant part of our work as critically engaged language teachers?

(+)                                                                                           ( - )
Improving the Korean 'brand'                                                      Unrealistic depictions of beauty
Generating capital                                                                    Distraction
                                                                                               Pressure to be beautiful




Tools for Thought:
"The philosopher, pacing his Oriental rug, was in a dark mood"
(example by Paul Fry)





The Frankfurt School:  Europe in the mid 1920's

Industrialization, modernization, and the invention of "modern times"

What is "The Culture Industry"?

‘진달래 꽃”(김소월)

나 보기가 역겨워

가실 때에는

말없이 고이 보내 드리오리다.

영변에 약산진달래꽃

아름 따다 가실 길에 뿌리오리다.

가시는 걸음 걸음놓인 그 꽃을

사뿐이 즈려밟고 가시옵소서.

나 보기가 역겨워

가실 때에는

죽어도 이니 눈물 흘리오리다.

“Azeleas” (역례 3)

If you go away

Through with me

I will quietly let you go.

I will gather azleas in armfuls

At Yaksan hill in Yongbyon

To scatter them on your path.

Tread with a tread,

Light and gentle,

On the flowers as you go.

If you go away

Through with me

Never will I weep though I perish.

Standardization, Passivity and Pseudo-individualism?






















"The Aura" and Ritual













The destruction of the aura through traditional artistic display












Aesthetics in Politics vs  Politicizing Aesthetics

Reading an image:

School Violence in Korea

I realize that I am, and in some respects will always be, an outsider in the Korean education system.  After all, I have yet to learn the language skills to communicate with teachers in Korean, to read the official policy, or to read reports and popular media in the original language.  That said, some things just strike me as very problematic and very very cold.  It's always a bit of a struggle for me to decide when to chime in with my opinion and when to keep my mouth shut.  In terms of electronic media, I usually just keep my mouth shut. This topic, however, really bothered me- as it is literally a matter of life and death.
A Korea Times piece published this week addressed the suicide of a 15 year old boy who left a note blaming his death on five boys who had been bullying him at school. The writer tied this to the bigger issue of school violence in Korea.

President Park was quoted saying "the essential change in school violence will come only when teachers show affection to students and interest in them."

A close and caring relationship between students and teachers is a noble goal.  Yet that last I heard, the average class size in Korea was still between 35 and 40.  Moreover, high stakes testing and further proposals for merit pay (based on students' test scores) position teachers in ways that make the warm and fuzzy emotional aspects of teaching and learning less of a priority than the more immediate need to dispense information and offer effective test taking strategies.  To my knowledge, there is nothing in standard teacher (or student) assessments that measure things like kindness, interest in students' lives, or an ability to mediate conflict.

I have worked with a lot of public school teachers throughout my career.  In that time, I have met teachers who may not involve themselves deeply in their students' lives.  I have even met a few who may not care all that much about their students.  But this is by far, NOT the norm.  Teachers in both the U.S. and in Korea (the two countries where I have worked in teacher education) care deeply about their students' well-being.  Many teachers I have met spend an incredible amount of time worrying about individual students.  These individuals not only go to extreme efforts to help students with their personal difficulties, but they are continually worried about how to address the needs of individual students without ignoring the needs of the other 35+ students.  My guess is that those who taught the 15 year old boy who killed himself are extremely traumatized over what happened.  To have government officials suggest that they are responsible for this seems downright cruel.

The only alternative offered in this article is to direct more funding towards the psychological well-being of students.  This would mean, presumably, the hiring of more guidance counselors and the implementation of more professional psychological assessments.  So the overall message in the article seems to be that the solution to the problem of teen suicides, bullying, and the larger culture of violence in Korean schools is in some debatable balance between greater teacher accountability and more funding.

I would like to suggest a different approach to the problem.  It seems to me that one of the principle reasons that public schooling breeds such violence among students is because our schools are not specifically designed NOT to do so.  The purpose of mass schooling in Korea and abroad is to differentiate between students- to place them on the educational and life path best suited to their individual ability.  The outcomes of our education system cannot be separated from this competitive function.  Students are placed in this system in order to compete with one another.  Those who do well in school go on to do well in life, and "doing well" is a matter of doing better than everyone else.

I remember working in a high school in Daejeon a number of years ago.  One day I noticed a piece of A4 paper taped to the wall, right at the front of the classroom.  The paper contained a list of every student in the class, ranked in order of their latest test scores.  The question that comes to mind is how can we expect our students to treat one another in a manner that is more respectful than the way we treat them?  We place our young people, both in the classroom and on the national level, in competition with one another.  It is a system in which students identify themselves according to how they compare, favorably or unfavorably, to their peers.  This is the institutional tool through which they come to know themselves.  Yet we remain shocked that students themselves invoke such hierarchies in their day to day school-lives.

Put simply, this is not an issue that will go away by pressuring teachers to "show affection" or insisting on widespread standardized psychological assessment.  I would suggest that the violence we see in schools will continue (and perhaps escalate) for as long as we implicitly teach that school is a place in which to compete and to prove yourself better than others.  This issue is rooted in the much larger function of schooling in general.  Asking teachers to fix this problem with their individual efforts is akin to asking the crew of the Titanic to start bailing water.  And throwing more psychologists at the problem is reminiscent of the practice of placing suicide nets outside the windows of sweatshops.  It may alleviate some immediate problems, but it is just a means of avoiding the root of the problem.  If we are serious about confronting the culture of violence in our schools, then we would do well to understand these problems as expressions of the system that we have created.