How the Writing Sample is Scored

Introduction Development Description Content Structure
Evaluation Readers Scoring Reported Scores Figure 1

Introduction

This document is designed to familiarize you with scoring procedures for the Medical College Admission Test (MCAT*) Writing Sample. Included is background information on the purpose of the Writing Sample, the types of skills assessed, and the scoring procedures, as well as examples of essays with explanations of their strengths and weaknesses. This guide is intended for use by examinees preparing to take the MCAT* and by medical school admissions committees, health professions advisors, and others who must interpret Writing Sample scores.

Development of the MCAT* Writing Sample

As early as 1973, the Medical College Admission Assessment Program task force recommended that a test of written communication skills be required for medical school applicants. Throughout the 1970s, the public and lay press emphasized the need for physicians to communicate with patients as active participants in the delivery of health care. Early in the 1980s, medical school deans joined in discussion of the need to stress writing and analytical skills in the admission process. They reported that communication and writing skills often were deficient among medical students.

Deans and faculty asked that an assessment of written communication skills be added to the information available about medical school applicants. Prior to adoption of the MCAT* Writing Sample, several medical schools independently instituted writing assessments in which applicants provided writing samples at the time of application or in conjunction with the interview. No universal instrument for assessing the writing and analytical skills of medical school applicants existed, however.

Research and development on an MCAT* essay section was conducted from 1985 through 1990, and the Writing Sample became an integral part of the revised MCAT* in 1991. This section, which requires candidates to develop and present ideas in a cohesive manner, offers medical school admission committees evidence of applicants' writing and analytical skills and provides unique information unavailable from other sections of the MCAT*.

Description of the MCAT* Writing Sample

The MCAT* Writing Sample consists of two 30-minute essays designed to assess skills in the following areas:

1. Developing a central idea,

2. Synthesizing concepts and ideas,

3. Presenting ideas cohesively and logically, and

4. Writing clearly, following accepted practices of grammar, syntax, and punctuation consistent with timed, first-draft composition.

Content of Writing Sample Items

Each Writing Sample item provides a specific topic and requires an expository response. Writing Sample topics are selected from areas of general interest such as business, politics, history, art, education, or ethics. Topics do not pertain to the content of biology, chemistry, or physics; to the medical school application process or reasons for the choice of medicine as a career; to social and cultural issues not in the general experience of college students; or to religious or other emotionally charged issues. Specific prior knowledge about the topic is not necessary to complete the Writing Sample.

Structure of Writing Sample Items

Each MCAT* Writing Sample item consists of a statement that expresses an opinion, discusses a philosophy, or describes a policy. The statement is followed by three writing tasks. The first is to explain or interpret the statement. Because the statement is not intended to be plainly factual or self-evident, it usually cannot be explained in a single sentence. In addressing this task, examinees should explain the meaning of the statement as thoroughly as possible.

The second task requires consideration of a circumstance in which the statement might be contradicted or judged not applicable. Examinees must present a specific example that illustrates a viewpoint opposite to the one presented in the statement and should further explore the statement's meaning.

The third task requires discussion of ways in which the conflict between the initial statement and its opposition (expressed in the second writing task) might be resolved. Here, examinees must reconcile the two viewpoints. In responding to this task, examinees should apply their understanding of the topic to more general problems of principle, choice, judgment, or evaluation raised by the conflict between the opposing viewpoints.

Evaluating Writing Sample Responses

Writing Samples are scored holistically. Whereas some methods for scoring essays assign several scores to a single piece of writing (e.g. separate scores for organization, development, grammar and mechanics, or fluency), holistic scoring regards an essay as a whole without separable aspects. Tbis type of scoring is based on the assumption that the various factors involved in writing are so closely interrelated that an essay should be assigned a single score based on the quality of writing as a whole.

Readers

Writing Sample papers are rated by a group of trained readers, many of whom are experienced teachers of writing. Using a scoring guide consisting of score point descriptions similar to those summarized below and sample papers selected to be illustrative of each score, readers are trained to score the essays reliably. Readers who are able to assign scores accurately and reliably are accepted to score actual essays. Readers' performance is closely monitored throughout the scoring process to ensure that scores are assigned fairly and accurately.

Since each Writing Sample consists of two topics, two groups of readers are used for the scorings. Each group is responsible for scoring papers for one topic only.

Scoring the Essays

Each essay is rated in terms of its overall effectiveness. The score is determined by the depth, cohesiveness, and clarity with which the writing tasks are addressed and by the extent to which ideas are developed. Readers are trained to score each essay on the basis of the writing skills displayed and not to judge the writer's personality, attitudes, or beliefs.

Readers are instructed to keep several guidelines in mind as they score. They are not to be unduly influenced by length (although an essay must be long enough that its ideas are developed) or by handwriting. Readers are not to be overly harsh on papers that contain minor errors in grammar, spelling, sentence structure, or punctuation. Because Writing Samples are written under timed conditions, it is assumed that some mistakes will be made. Each essay is judged on its individual merits; readers do not score with any particular score distribution in mind.

Responses that are blank, illegible, written in a language other than English, or that consist entirely of an obvious attempt to ignore the purpose of the Writing Sample (such as a drawing) cannot be scored. If either of the two responses cannot be scored, the reported Writing Sample score is an "X" for "Not Ratable."

All personally identifiable information is removed from essays prior to scoring; essays are identified only by their code numbers. Every essay is scored by two different readers. Each reader scores essays in sets or batches, and the scores are recorded on machine scannable sheets, not on the essays themselves. (Readers record only the score on these sheets; they do not make any notes of an essay's strengths or weaknesses.) When a batch of essays is ready to be circulated to a second reader, a new scannable sheet is provided. Thus, the second reader does not know the score given to an essay by the first reader.

Readers score essays using the six-point scale described below. The final score for each essay is a function of the scores assigned by two readers. If an essay receives scores that are more than one point apart (i.e., a 2 and a 4), the essay is evaluated by a supervisory third reader who determines the final score.

Score Point Descriptions

Summarized below are the typical characteristics of papers receiving each score.

6	These papers show clarity, depth, and complexity of thought. The treatment of the writing assignment is focused and coherent. Major ideas are substantially developed. A facility with language is evident.
5	These essays show clarity of thought, with some depth or complexity. The treatment of the rhetorical assignment is generally focused and coherent. Major ideas are well developed. A strong control of language is evident.
4	These essays show clarity of thought and may show evidence of depth or complexity. The treatment of the writing assignment is coherent, with some focus. Major ideas are adequately developed. An adequate control of language is evident.
3	These essays may show some problems with clarity or complexity of thought. The treatment of the writing assignment may show problems with integration or coherence. Major ideas may be underdeveloped. There may be numerous errors in mechanics, usage, or sentence structure.
2	These essays may show some problems with clarity or complexity of thought. The treatment of the writing assignment may show problems with integration or coherence. Major ideas may be underdeveloped. There may be numerous errors in mechanics, usage, or sentence structure.
1	These essays may demonstrate a lack of understanding of the writing assignment. There may be serious problems with organization. Ideas may not be developed. Or, there may be so many errors in mechanics, usage, or sentence structure that the writer's ideas are difficult to follow.

Reported Writing Sample Scores

Each examinee's Writing Sample is scored by a minimum of four readers. As stated above, two readers score the first essay, and two different readers score the second essay. The final scores given to each of the two responses are summed. This numeric score, which may range from 4 to 24, is then converted to an alphabetic score. Alphabetic scores range from a low of J to a high of T. Only the alphabetic score appears on the MCAT* score reports.

Because a 21-point numeric scale (the 4 to 24 scale) is converted to an 11-point letter scale, there are multiple means by which examinees may obtain a given alphabetic score. They may receive similar numeric scores on both essays or may score slightly higher on one essay than on the other. For example, a score of 20 may be obtained by receiving two fives on the first essay and two fives on the second essay, two fours on the first essay and two sixes on the second essay, etc. Although it is not uncommon for examinees to write two essays of dissimilar quality, the majority attain a given letter score by performing equivalently on both essays. It is very rare for an examinee to score exceptionally well on one essay and very poorly on the other.

Because examinees may attain a given letter score in various ways (i.e., by performing equivalently on both essays or by performing better on one essay than on the other), making inferences about the writing skills associated with a given letter scores is not straightforward. Figure 1 provides some general guidance in interpreting letter scores. This figure shows the continuum of writing skills represented by the J to T score scale and describes the typical features of essays at three places on the scale.

Figure 1

Description of Writing Skills Associated with Writing Sample Alphabetic Scores

J K L M N O P Q R S T

	Below Average		Average		Above Average
	These essays demonstrate a degree of difficulty in discussing the topic and/or responding to the three writing tasks. They may show problems with the mechanics of writing or in addressing the topic at a complex level. The response my not be organized, and ideas may not be integrated. Although the three writing tasks may be addressed, the ideas may be underdeveloped.		These essays demonstrate a degree of proficiency in discussing the topic and/or responding to the three writing tasks. Few problems in the mechanics of writing are evident, and there is demonstration of control of language. The writing tasks are addressed in a clear, organized, and coherent manner. Ideas are developed to some extent and may show evidence of depth and complexity of thought.		These essays respond to the topic and the three writing tasks in a superior manner. The writing demonstrates a strong control of language. The response is presented in a clear, organized, and coherent fashion. Ideas are well-developed, and the topic is dealt with at a complex level.

*This figure portrays the continuum of writing skills represented by Writing Sample alphabetic scores. At the "Below Average" level, Writing Samples assigned scores of J or K consist of two essays which both exhibit nearly all of the characteristics included in the description. As scores move up the scale, essays possess fewer of the "Below Average" characteristics and more characteristics described under the "Average" level. For example, a Writing Sample that receives an M typically consists of two essays which exhibit some "Below Average" characteristics and some "Average" characteristics; at a score of N, even fewer "Below Average" characteristics and more "Average" characteristics would be present in the essays. Likewise, as scores increase beyond P, essays possess more of the "Above Average" characteristics and fewer "Average" characteristics. For Writing Sample scores of S and T, nearly all of the characteristics described in the "Above Average" description are present in both essays.