A Survey of Three Major International Exams and What Can We Learn From Them?

Assessment Center, BeiwaiOnline Education Institute Wang Xuedan

Abstract

“Language testing almost never takes place in isolation, it is done for a particular purpose and in a specific context. Its development goes along with advances in our understanding of language acquisition and language teaching”(Bachman, 1990). Two major concepts in modern ELT methodology of particular relevance to online education and testing are the communicative approach and learner-centred approach. How to develop suitable tests to measure the effect of online language teaching with these two concepts in mind is of central importance. This article aims at working out a desirable way of developing online language tests by learning from three major international exams.

This article starts with a brief overview of developments in ELT methodology over the past 2 decades. It then shows how these advances have been reflected in changes to three major international examinations: IELTS, TOEFL, and BEC. The examination formats are examined and compared in detail. In the third part, the article discusses features of online education. Then the author proceeds to talk about why and what online education should learn from these exams in section four. In addition, the paper pinpoints that online education can work out a suitable testing system only by integrating the features of online language teaching with best practice in international testing system. In the conclusion part, the author forecasts the future direction and responsibilities of language tests in online education.

Key words: communicative methodology learner-centredness approach

online language teaching best practice machine-marking test format

Preface

During the past two decades, great changes have taken place in language teaching. Many new approaches and concepts have emerged and developed during this time. Due to the reciprocal relationship between language teaching and testing, changes in language teaching methodology will inevitably affect the theory and practice of language testing.

As well as these changes in the academic field, technical advances have also brought people into thinking of new ways of language teaching and testing. These technical innovations have come about through the explosion that we call information technology, and in particular the Internet.

The explosive Internet development brings innovations everywhere in human society, especially in the way of presenting a brand-new medium in front of people. The attraction of the Internet lies in its integration of all kinds of media: text, picture and sound. Apart from this, it realizes people's dream of obtaining information anywhere, anytime and free of charge most of the time. Interactive communication, which was impossible in other types of media, has also been realized by this medium. Undoubtedly, all of these features enable the Internet to become the most powerful educational tool in this information era.

In response to this emergence, educational systems have also made great changes almost overnight. Online education has suddenly become a hot social and governmental issue. Many earnest educators have begun to explore new ways of teaching in this climate of innovation.

At this milestone, what is the importance to language testing of online education? How can the language testing system balance well under the double driving force of new language teaching methodologies and technology innovation relevant to testing? This paper aims at finding out the answers to all these questions.

Section One

Review of Language Teaching

When talking about language testing in whatever situation, online or offline, we need to bring into the discussion on the related issue of language learning and teaching. As J.B. Heaton said at the very beginning of Writing English Language Tests “Both testing and teaching are so closely interrelated that it is virtually impossible to work in either field without being consistently concerned with the other.” Lyle F. Bachman also wrote in his Fundamental Considerations in Language Testing “… advances in language testing do not take place in a vacuum; they are stimulated by advances in our understanding of the processes of language acquisition and language teaching. And developments in language testing can provide both practical tools and theoretical insights for further research and development in language acquisition and language teaching.” So equally, it is safe to say that developments in people’s understanding of language acquisition and language teaching will offer them corresponding theories and practice in language testing.

Figure 1 Developments in language teaching

As we can see from Figure 1, in the left column are the methodologies and strategies which dominated certain stages in language teaching history and are well known to the profession. While in the left column are the methodologies and strategies which have been responsible for major changes in language testing system quite recently and still are the trends of today’s language teaching. The contents of the left column will not be discussed here, since they are dealt with exhaustively in the literature. Instead issues in the right column will be illustrated in detail as they relate to recent development of English language tests as well as the focus of this paper --- language teaching and testing in online situation.

q Communicative Language Teaching (CLT)

This approach emphasizes the learner’s ability to use the language appropriately in specific situations and puts communicative competence at the centre of all its teaching.

The main principles are as follows:

· Learners learn a language through using it to communicate.

· Authentic and meaningful communication should be the goal of classroom activities.

· Fluency is an important dimension of communication.

· Communication involves the integration of different language skills.

· Learning is a process of creative construction and involves trial and error.

q Learner-centredness

This means as its name indicates that the language learners should be the centre of the language learning, that is to say the learners should take the initiative and main responsibility for their own learning.

“…Another trend in recent years which has stemmed from CLT has been the development of learner-centred approaches to language teaching, in which information by and from learners is used in planning, implementing and evaluating language programmes. While the learner-centred curriculum will contain similar elements and processes to traditional curricula, a key difference will be that information by and from learners will be built into every phase of the curriculum process. Curriculum development becomes a collaborative effort between teachers and learners, since learners will be involved in decisions on content selection, methodology and evaluation” (Nunan, 1989:19).

With the emergence of this new methodology, learners were no longer considered as passive recipients of knowledge; they were viewed as active participants who are involved in various stages of teaching and learning. Teachers, on the other hand, started to adopt the role of "communication facilitator" instead of being the source of information, and materials were designed in such a way that they would promote the real-life uses of the language, which the learners are likely to experience in an authentic communication situation.

This concept implies the following points:

· learners are involved in creative language use incorporating tasks which relate to learners’ real-life communicative needs

· learners choose what, how and when to learn

· learners and teachers are required to adopt a range of roles, and use language in a variety of settings in and out of the classroom

· learners are exposed to the language as system

· encourage learners to develop skills in learning how to learn

· controlled practice is provided in enabling micro-skills (reading, listening, speaking and writing)

· involve learners in creative language use

“The knowledge and personal qualities that learner involvement requires cannot be taken for granted, and need to be developed over time. A learner-centred approach needs therefore to contain an element of awareness development, which is designed to help learners deepen their understanding of language learning and develop their ability to play an active and self-directive role in their language study.” (Tudor, 1996: 34)

q English for Specific Purposes (ESP)

This term refers to teaching a specific genre of English for students with specific goals. Examples include English for Academic Purposes (students will enter an English-speaking university), business English (for business people), medical English (for nurses, doctors, other health care professionals). ESP can be (as the name implies) very specific -- a growing market is English language computer support personnel. In sum, ESP is English for vocational purposes, where the word vocation is used loosely to include education and all kinds of employment.

From the early 1960’s, English for Specific Purposes has grown to become one of the most prominent areas of EFL teaching today. Recently, ESP is defined in terms of two groups of characteristics which are universally known as Absolute Characteristics and Variable Characteristics.

· Absolute Characteristics

1 ESP is defined to meet specific needs of the learners.

2 ESP makes use of underlying methodology and activities of the discipline it serves.

3 ESP is centred on the language appropriate to these activities in terms of grammar, lexis, register, study skills, discourse and genre.

· Variable Characteristics

1 ESP may be related to or designed for specific disciplines.

2 ESP may use, in specific teaching situations, a different methodology from that of General English.

3 ESP is likely to be designed for adult learners, either at a tertiary level institution or in a professional work situation. It could, however, be for learners at secondary school level.

4 ESP is generally designed for intermediate or advanced student.

5 Most ESP courses assume some basic knowledge of the language systems.

According to the definition, ESP can but is not necessarily concerned with a specific discipline, nor does it have to be aimed at a certain age group or ability range. ESP should be seen simply as an “approach” to teaching, or what Dudley—Evans describes as an “attitude of mind”. Such a view echoes that of Hutchinson et al. (1987:19) who state,

“ESP is an approach to language teaching in which all decisions as to content and method are based on the learner’s reason for learning.”

These newly developed teaching methodologies lead English language learning and teaching into a new prospect. At the same time, insights gained from the progress of language learning and teaching also provide valuable information for designing and developing more useful tests.

Many international exams such as IELTS, TOEFL, and BEC, have just grown more and more mature in terms of academic during this process. In terms of technology, they are also the first responders to the Internet influence. Consequently, they are considered as best practice in international testing system and set as models for people from testing field to study and follow.

The author believes that in seeking better ways for language testing systems in online education, people in the profession can also gain certain enlightenments and instructions from studying these exams.

Section Two

A Survey of Three Major International Exams

Since we have already discussed the most influential language teaching and learning methodologies in today’s language teaching field, and the most advanced tool technology brings to us – the Internet, how these methodologies and the technology are reflected in the international testing system? We will be able to get the answer to this question by analyzing the best practice in international testing system. The author chooses three major international exams here for studying --- IELTS, TOEFL and BEC. Now we will have a closer look at them.

q Reasons for this choice

In order to clear up the doubts before a detailed discussion and pave a way for further analysis, we should understand why these particular three exams are suitable for our study.

1 All of the three exams have a relatively long history of research and development, which distinguishes them as reliable and authoritative exams in today's testing field. IELTS was lunched worldwide in 1989, TOEFL was developed in 1963-64 and BEC was introduced between 1993 and 1996. (refer to Appendix 1 Background Information)

2 All of them are advanced level examinations and represent best practice in testing all over the world. These three exams are recognized as preferred entry qualification among academic institutions, scholarship programs, immigration, government agencies or a particular course study as far as BEC test is concerned. (refer to Appendix 1 Background Information)

3 They are the exams developed by the most authoritative academic bodies and widely used all over the world. Both IELTS and BEC are produced by the University of Cambridge Local Examination Syndicate (UCLES), and jointly managed by the British Council and IDP Australia, while Educational Testing Service (ETS) in America is the designer of TOEFL. The popularity of these three exams never stops growing. For example, TOEFL is administered to approximately 800,000 candidates in more than 210 countries each year. (Refer to Appendix 1 Background Information)

4 With the changes of ELT methodology, the communicative needs have been addressed in these three exams. All of the three exams are the integration of assessing four language skills. They have always updated their test items in order to achieve the objective of checking learners' communicative competence. The materials they use in the test are authentic so that they provide a real-life situation for language learners to operate in.

5 The use of the exam and their special exam operation system encourage a learner-centred approach. Because the subscribers of these exams are those people who want to pursue a specific goal, they have the aspiration of taking the exam. For this reason, they have to learn for their own purposes. In addition, these exams are operated world wide by an independent authoritative body, which is different from courses in traditional school study. That is to say, the learners' learning is not supposed to be instructed by other people such as teachers. Instead, they take the initiative in learning and then sit the exams when they choose.

6 They are proficiency tests which have had a large effect on achievement tests. "Established proficiency test such as the TOEFL or the Cambridge examinations tend to generate a washback effect on instruction (preparatory courses and textbooks which are oriented towards the test) and hence come more and more to be used as achievement tests. This achievement-proficiency dynamic rightly leads to new proficiency tests being designed."(Alan Davis, 1999:154)

7 As far as BEC is concerned, the BEC test represents the rising use of English internationally, particularly in the fields of commerce, science, and technology which led to English for Specific Purposes (ESP) as an important component of English language teaching.

8 Due to the high recognition status of the three exams, they all have to organize large-scale examinations as well as large-scale test paper marking which will be the very model for other large-scale testing system to follow. Their experience can be considered as a valuable asset for other new types of language testing, such as the language tests in online education.

9 With the introduction of TOEFL Computer-based test (CBT), TOEFL has once again become a pioneer in technology application. Its experience in CBT design, score-marking and test format will surely be of direct relevance to language testing in the online situation.

q Test Format Analysis

Test format refers to the overall design of a test. This includes characteristics such as the test length, the number and type of sections, the skills or sub-skills tested, the item and task types used and their number, the ways in which stimulus material is presented and the kinds of responses required of test takers. Therefore, a test format serves as a blue print which releases all sorts of useful information about a test. Such as test sections tell what kind of language skills will be assessed; test length and item numbers suggest the weight of different sections or skills; task type and material used specify the purpose of the test, reflect the methodology a test advocates and imply the assessment concept a testing body holds.

Long-time testing practice develops a comprehensive test format. Vice verse, a well-designed test format can be instructional in testing practice. For this reason, the conclusion from analysing or comparing the best practice in international testing will be very instructive to develop new types of language test, such as, language testing system of the online situation we are pursuing now at BeiwaiOnline.

The author has done lots of research on the test format of these three exams. Based on the information collected, the author produced a test format sheet for each test. (See Appendix 2 Test format of IELTS, Appendix 3 Test format of TOEFL-CBT, Appendix 4 Test format of TASK and Appendix 5 Test format of BEC-higher) Due to varied information released by different test authorities, the test formats of these three exams may differ from each other in some minor ways. Nevertheless, the author has tried best to keep a consistent description of the three tests.

Based on the information collected, all of the features which may be of relevance to language test in online education are illustrated here:

1 Language Skills Tested All three tests include measures of four basic language skills: listening, reading, writing and speaking, which is clearly due to the influence of the communicative approach. The most remarkable evidence of this benign influence is the evolution of TOEFL. Since TOEFL was developed in 1963-64, it has experienced three periods of evolutionary modifications. In terms of the skills it assesses, TOEFL has grown from testing only one element of a skill, such as grammar or vocabulary, to adopting a much more integrated skills approach, and most recently, it has developed a self-assessing access to spoken tests. All of these changes must be attributed to the increasing awareness of the importance of communicative competence derived from the field of applied linguistics. As ETS reviewed in their TOEFL 2000 report:

“Base on a review of the various approaches, it was decided that the most practical way to organize language tasks for the TOEFL 2000 framework is by modality. Thus, the test will include measures of speaking, writing, listening, and reading. Within these four areas, the new test will include a variety of language features, including not only grammar and vocabulary but also discourse, pragmatics, and sociolinguistics, as well as setting and task.” (TOEFL Monograph Series TOEFL 2000 Framework: A Working Paper)

2 Source of Test Materials In the listening section, most materials are carefully scripted or chosen from real life activities, while the materials used in the reading section come from authentic source, such as newspaper items, magazines articles, books reviews etc. In particular, those practical texts, such as notices, advertisement, leaflets, curriculum vitae etc. are all "authentic" or "original" materials. Apart from verbal materials, these tests also employ different kinds of non-verbal materials such as photos, graphics and pictures to enrich the context and support the content. This act of employing real life material for testing is more evidence of the influence of the communicative approach. By this means, the examinees are presented with real life language events which may consist of some colloquial language usage, slang or substandard language phenomenon. Therefore, the language learners are highly stimulated to set communicative competence as their language learning objectives. As t has been stressed in Section one, after all, many language learners learn the language for the sake of communication rather than academic study.

3 Test Item Type Test item types simply means forms of response a test require from the test taker. A test item type can be the most influential element in assessing language ability and knowledge. Test item types used in a test will reflect which particular area a test designer want to measure of a test taker. In addition, numerous studies have demonstrated that test method has a sizeable influence on performance on language tests.

Let’s first look at TOEFL test (refer to Appendix 3), from which we can easily see that TOEFL-CBT is a continuation of traditional paper and pencil TOEFL test, which was a totally objective item. Apart from simplex multiple-choice questions, it has added matching, reordering in listening section; and identifying errors in structure section. However, it is obvious to us that actually all the item types a TOEFL-CBT uses are objective items or we can say “transfigurations” of traditional multiple-choice item. The statistics shows that many international students who are admitted with a high TOEFL test scores arrive on campus with insufficient writing and oral communication skills to participate fully in academic programs. This is partially because of the exclusive use of traditional, multiple-choice items to assess receptive skills. Nevertheless, we have to accept that for all of us, the most important advantage of this test item type is the fact that it is cost-effective and efficient in marking.

One of the differences between TOEFL and Cambridge exams (such as IELTS and BEC) is due to the choice of using test items. It seems that IELTS uses whatever item type they believe which can assess the language ability especially communicative competence, to the fullest extent. A brief look at the item types IELTS uses:

· multiple-choice;

· short answer questions;

· sentence completion;

· notes/summary/ diagram/flow chart/ table completion;

· labeling a diagram which has numbered parts;

· classification;

· matching;

· choosing from a “heading bank” for identified paragraphs/sections of the text; identification (yes, no or not given).

The completeness of varieties of employed item types is amazing. We can see more than half of them need the test takers’ “writing effort”, “analyzing effort”, “manipulation effort” and perhaps “creative thinking”, all these “efforts” they make will signal their ability in using more than one language skills' at the same time. Therefore, IELTS best reflects the communicative approach by test language skills interactively. For example, they test reading a text by writing a summary, or listening to a dialogue by completing a note. These integrated tasks will provide information about examinees’ ability in more than one skill area. All in all, a principle should be born in mind in testing which is that “Communication involves the integration of different language skills.”

4 ESP test Another sample is BEC which is a typical ESP test --- “a test designed for language learners in specific academic, professional or vocational fields, such as tertiary students, language teachers, doctors and tour guides. Such tests rest on the assumption that language performance varies with both context and test task.” (Alan Davies, et al 1999:113) BEC mostly follows IELTS’ style in terms of aiming at assessing test takers’ communicative competence. Due to its emphasis on testing language in a specific area, BEC has to choose those materials with special topics, content, forms and languages. From the format (Appendix 5), we can see that the materials BEC test uses are all business-related. All of them must be related in terms of content, so they should be from “authentic business-related source), some of them must be related in terms of both content and form, such as “advertisements, notices, books reviews, short newspapers items on related topics” or “source of original texts may be the general and business press, company literature and books on topics such as business.” The test item types also need to be for a “specific purpose” in order to facilitate the inferences about test-takers’ ability to cope in the target context. For instance, in writing test, BEC requires test takers to describe or compare figures from graphic input or write a short report, proposal, business correspondence. Although “ESP test typically claim to be testing language alone rather than content or non-language skills, in practice the boundary between language and these other skills is somewhat blurred.” (Alan Davis, et al 1999:113), BEC also contains a task of filling lexical gaps, which can be viewed as a content checking.

Since ESP is designed to serve the learners who have specific needs, ESP test has to shoulder more responsibilities than a general language test.

5 Speaking Test Format A prominent feature of communicative approach is to provide language learners with interactive activities. Accordingly, the corresponding test should also provide this to facilitate the assessment. Both IELTS and BEC use 3-part speaking test which both involves interactive communication and one-way or prompt communication. The topics for the oral test derive entirely from normal features of people's daily life. Learners who will be checked by this system have to be driven to practice communication in natural conversational situations.

TOEFL-TASK is a new self-assessment access provided for language learners who are aware of their speaking skills. This hasn’t been included as compulsory part of a TOEFL. However, the simple act of adding this component to TOEFL is a stride towards responding to the logic of the communicative approach. A distinguishing and advantageous feature of TASK (Appendix 4) is the combination of listening skill and oral skill, which was discussed before as a typical integrated skills checking in response to the communicative approach. Nevertheless, the lack of interactive communication is a continuing fault of this design.

6 Proficiency test From analyzing the test paper formats of these three tests, it’s easy to understand what a proficiency test should comprise. All the analysis shows that a proficiency test measures what the test taker has learned relative to a specific real world purpose, for example, does he/she know enough of the target language to follow a lecture, train as an engineer or work as a ski instructor in that medium, or to translate to the requisite standard out of that language. Therefore, a proficiency test is always taken by those learners who will be assessed more for checking their actually ability of mastering a language than for differentiation on passing or failing. So the takers are not “required” to be assessed but “require” to be assessed. Usually there is no textbook, course or instructor driving them to study, they study for their own objectives and developments. Therefore, they have to be self-directive and take initiative in studying. In one phrase, a proficiency test corresponds to learner-centeredness approach, which is a testing principle more useful for learners than the test designers.

Section Three

Features of Language Test in Online Education

Since the Internet becomes a teaching tool for education, it donates unique features to both teaching and testing. In order to work out a way for language testing of online education, the first task in front of us is to understand the features in online language teaching and testing. The author has summarized the following five features of language test in online education:

First and the most remarkable feature is use of the Internet. This is what online education all about. The Internet is the medium of online teaching, which explains how the courses are transmitted to the learners. The Internet provides an interface with all kinds of available teaching facilities and patterns by which the distant learners can study through text, video, audio, visual materials as well as an interactive platform. In the same way, the Internet provides more varieties in language testing types and means than traditional education. For example, because of the Internet, online self-assessment test and self-placement test become possible. The language learners can do the test and get a prompt feedback on their own. They can assess their reading, writing and listening abilities online. Quite possibly, they can even record their voice online and be assessed. Apart from checking these micro-skills, the Internet allows more item types within one skill area. For example, play a video clip and ask questions about it or give a voice prompt and ask for reply. The more people know about the Internet, the more people can explore from it. Proper use of Internet in online education will certainly promote language tests to a new prospect.

Second, online language teaching emphasizes the learning process and demands self-centredness approach to encourage the study. Learners not only need to master a second language, but also need to learn how to learn. Under the distant learning situation, learners should understand they must manage their own study. They should be aware of that the only people they can rely on in terms of studying and monitoring should be their own. As a result of this learner-centredness or autonomous learning approach, the test designed to respond this methodology should be able to orient self-study and offer access to self-assessment. One feature of this totally “independent test-taking system” should be developing self-explicit tests. Everything in a test must be absolutely clear and free from errors or ambiguity.

Third, test used for online language teaching is an integration of proficiency test and achievement test. Online language teaching is usually done for learners to master a particular course or skill of a language. Therefore, the corresponding test to this kind of study is kind of achievement test which refers to measure the mastery of what has been learnt, what has been taught or what is in the syllabus, textbook, materials, etc. Nevertheless, due to the very nature of language learning and teaching which emphasize the knowledge and ability of a language rather than the limited knowledge in a particular book. The test used for online language learning is usually a combination of achievement test and proficiency test. That is to say, in a test paper, part of it should measure what has been in the syllabus of a particular course and part of it should reflect the real world purpose or the course objectives so that a proportioned achievement and proficiency elements need to be combined in the test.

Forth, language test in online education measures the communicative competence of the language learners. With the emergence of communicative language teaching, the materials used in teaching and testing should be designed in such a way that they must promote the real-life uses of the language, which the learners are likely to experience in an authentic communication situation. The test for online language learning also needs to apply to this principle so that it should pay special attention to its test materials used, test objectives, test item types and the whole process of test development.

Fifth, the trend of English for Specific Purposes (ESP) is growing fast in language teaching online. At present, tertiary education is the main body of most online education institutions. Most distant learners joining online language education are adults who specialize in all walks of life wanting master a second language. The learning purpose of these learners is to develop their language ability in a particular field and for a specific use, such as medicine, business, engineering etc. Therefore, online language teaching has a tendency to develop more ESP teaching, which will result in more research and study on ESP testing.

Lastly, online language testing has to deal with large-scale exam administration work, such as exam operation and test paper marking. One of the charms and also one responsibilities of online education is to offer education for everyone who has access to the Internet. As a result, online education inevitably has to face a large number of learners. Apart from language teaching, special attention also needs to be paid to large-scale exam administration work such as test paper marking. One of the effects of this feature is the profound influence on test design. For example, the instructions used in the test have to be very clear, because there is no way to explain to the test-takers face to face. With the consideration of large-scale marking, machine-marking will be an unavoidable tendency which has a huge effect on test item construction. The test designers need to consider how to write “machine-markable” items which should also be valid enough to measure a particular language skill.

All these features of language test in online situation propose a question in front of us, what shall we do in order to develop a suitable te