Waddell (1978)

Notes on the text

Part I

Preliminary pages Contents, Membership
Chapter 1 Introduction
Chapter 2 Educational matters
Chapter 3 Structure of the examining system
Chapter 4 Cost
Chapter 5 Conclusions
Appendices

Part II

Preliminary pages Contents, Membership

Report of the Education Study Group (ESG)

Glossary, Introduction
Chapter 1 Feasibility of common exam system
Chapter 2 English
Chapter 3 Mathematics
Chapter 4 Science
Chapter 5 History
Chapter 6 Geography
Chapter 7 Modern languages
Chapter 8 Classics
Chapter 9 Commerce
Chapter 10 Social science
Chapter 11 Religious studies
Chapter 12 Craft design and technology
Chapter 13 Technical drawing
Chapter 14 Home economics
Chapter 15 Needlecraft and dress
Chapter 16 Art
Chapter 17 Music
Chapter 18 Further work
Appendix A List of witnesses
Appendix B Questions
Appendix C Statistics
Appendix D Joint examinations

Report of the Cost Study Group (CSG)

Chapter 1 Introduction
Chapter 2 Costs in 1976
Chapter 3 Costing a common system
Chapter 4 Changeover costs
Annexes

Report of the Steering Committee established to consider proposals for replacing the General Certificate of Education Ordinary Level and Certificate of Secondary Education examinations by a common system of examining

Chairman: Sir James Waddell CB

Presented to Parliament by the Secretary of State for Education and Science and the Secretary of State for Wales by Command of Her Majesty July 1978

London: Her Majesty's Stationery Office 1978
© Crown copyright material is reproduced with the permission of the Controller of HMSO and the Queen's Printer for Scotland.

Part I: Cmnd 7281-I
Part II: Cmnd 7281-II

Part II - ESG Report

Chapter 2 English (language and literature)
[pages 16 - 23]

Introduction

29. English as a subject involves the practice of complex skills in listening, speaking, reading and writing in many different and highly specific contexts. Teachers and users differ about the relative importance to be attached to the various aspects of the subject. All pupils, whether examination candidates or not, will be called upon to use English in a great variety of ways ranging from the functional levels of the four activities noted above, through the more sophisticated and subtle levels of those activities to using and appreciating the spoken and written words in their expressive and imaginative forms.

30. The assessment of these activities and functions is always in part subjective. In a common system of examining it must include evidence of the candidate's capacity on the one hand to understand and to use language for a wide range of utilitarian purposes, and on the other to appreciate and use it as a personal response to experience. This chapter considers, on the basis of the available evidence, whether a common system of examining can be satisfactorily applied to an appropriate selection of language uses which reflect good curriculum practice and take account of a variety of subsequent targets and a wide range of performance.

The evidence

31. The joint examinations listed below were studied and discussed with subject interests and two of the consortia concerned (ALSEB/JMB and TWYLREB/JMB). Further reference to the examinations is made by number.

Table 1

The ability range

32. For most of the joint examinations the entry appears to have been 'skewed' towards the lower end of the ability range. Since 1974, partly perhaps as a result of the raising of the school leaving age, candidates well below the 40th percentile of the range have been entered. But despite the lack of balance there have been enough candidates attaining grades A and B to justify conclusions about the whole of the intended ability range.

33. Different considerations apply to examinations in English language on the one hand, and English literature on the other. Because it is so widely specified as a qualification for entry to educational courses, or to employment, more candidates enter for the examinations in English language than any other subject, and the ability range is exceptionally wide. English literature, when examined separately, is not in such demand and the entry is more restricted both numerically and in ability range. For those reasons this chapter deals separately with English language and English literature, and devotes more space to the former. However, this treatment does not imply a judgement whether, as some teachers strongly believe, language and literature should be taught and examined together in a 'unitary approach' or not.

ENGLISH LANGUAGE

The syllabus and objectives

34. In contrast to some other subjects, examinations in English are not normally based on a syllabus setting out content in detail. The schemes of all the joint examinations state broad aims and then go on to outline what is required of candidates, so that it is to the form of the examination that one must look to ascertain whether the schemes lead to clear and worthwhile objectives for the whole target group; to accommodate, for example, within the four activities noted above, the understanding and handling of language used in conveying information and ideas, in marshalling argument, in persuading and in expressing a personal response, all the while observing at an appropriate level the conventions of spelling, punctuation and grammar.

35. The joint examination schemes contain aims and objectives which vary in their degree of precision. Three in particular (1, 2, 7) are helpful to teachers in indicating course requirements. Scheme 2 is clearly aimed at ensuring realistically that a variety of language skills is covered and at providing for teachers an appropriately broad and stimulating guide in terms of language activities for the ability range. Scheme 1 offers the most inclusive approach attempted, in that literature is to be examined as one component in a single examination. Scheme 6 is brief, generalised and deliberately unprescriptive: it is designed less to determine or to prompt and more to monitor what each school offers. There is detailed machinery for agreement trials (three per year) and moderation but the process contains more of the implicit than the explicit. This has made it difficult for outside observers to analyse the objectives of the scheme and we have therefore been able to make less use of this scheme than we might otherwise have done in coming to our conclusions about objectives.

36. The objectives presented by some consortia provide a good basis for encouraging balanced and stimulating work, though some schemes (eg 1, 7) may need to broaden their approach to accommodate more easily the pupils in the lower bands of ability. Taken all together the schemes provide satisfactory evidence that clear aims and objectives for a common system of examining in English language can be devised.

The syllabus and the examination

37. All schemes but one offer examinations whose form and content make plain what is expected, test a broad sample of the objectives and thus provide a clear framework within which teachers and candidates can work. Schemes 1, 2 and 7 test writing and understanding in a variety of functions and assume a range of essential skills, although with surprisingly differing emphasis on punctuation, spelling and grammar. Scheme 6 employs 100 per cent teacher-based assessment. In this scheme there is no explicit indication that pupils are led to write for a variety of purposes and there appears to be a tendency for subjective and personal writing to predominate, with much less evidence of a range of writing for different purposes than, for example, in 1, 2 and 7 which, taken together, offer a more satisfactory range of language tasks.

38. The range of activities tested includes listening comprehension in 1, 3 and 7, and speaking in all except scheme 6 which, alone of the current examinations, provides for no assessment of oral/aural work, although it is under consideration. The devising of such tests is in its early stages and there have been great difficulties in mounting tests of spoken English in particular. The difficulties are administrative and educational, are shared by the dual system and are therefore not peculiar to a common system of examining. Apart from these problems, which extension of this aspect of examining would increase, many teachers are not convinced that the right method of examination of spoken English lies in formal tests of the 'viva' type often modelled on written assessment procedures, or that the most valid means of assessment have yet been found. Since it is particularly important that an examination in English should include assessment of the spoken language, the best prospect lies in improving the school's own assessment of a variety of the pupil's spoken work. This practice has been adopted, at least in part, by all the consortia offering oral/aural components.

39. Assessment of the work a pupil has done over a period of time, set by the teacher who knows his interests and potential, offers an additional opportunity not constrained by formal examination conditions, to set tasks which match his capabilities and engage his interests. This applies to all levels of ability. A measure of internal assessment of written work is to be found in all the schemes, varying in amount from 100 per cent (6) to 20 per cent (5). It is well known that there is considerable enthusiasm for this form of assessment in the region covered by 6. Despite the administrative burden which it entails, it is this scheme which has achieved the most rapid growth. An important side effect has been to encourage cooperation among teachers and among schools in developing programmes of work and in assessing pupils. Discussions with representatives of this consortium stressed measures taken to improve standardisation and moderation procedures. From 1978 scheme 2 will make available a course work alternative to paper I (expression) which will, however, be subject to the same safeguards as to the range of work required as the external paper itself. This development is in line with the alternative syllabus offered by the JMB O Level examination.

40. There were several interesting and successful attempts to minimise any inconsistency due to subjective marking. Schemes 1, 2 and 7 used two or more examiners to mark each script - a practice already found in the dual system. All the papers marked in this way discriminated successfully. Analytical marking, adopted for open-ended comprehension questions, also discriminated well. Multiple-choice objective tests, which further obviate inconsistent marking, occupy a place in 1, 3, 5 and 7; from 1978 it will be the only form of assessment of comprehension in 1.

41. The most successful of the schemes (2) tested a broad and balanced range of written and oral skills and candidates were able to achieve the grades commensurate with what was expected of them. This conclusion is supported by evidence from 7 which shares some of the features of 2 in the written expression papers. The variety of assessment techniques used in all the examinations taken together offers a large body of useful experience which, although not always successful, will serve to point the way forward in a development period. The most effective approaches for a wide range of ability appear to be those of 2 and 6; those of 1 and 7 seem to be too strongly oriented towards the abler candidates without offering sufficient opportunity for less able candidates to perform well.

The examination and the ability range

42. The content and format of the question papers, their weightings and the mark schemes applied to them, are all of crucial importance for the wider range of ability envisaged by the common system. All the examinations attempt to cater for the broader range of candidates by means of a single examination in which all sections are compulsory, although there is some choice of subjects within the sections devoted to expression. It was unfortunate that the two attempts (in 3 by additional options and in 7 by comprehension questions of varying difficulty) to offer tests of graded difficulty were not pursued by the consortia; this approach could be explored in a development phase.

43. Of the forms of composition the discussion-type of essay in 1 and 4, and the summary (a traditional component of GCE examinations and found in 3 and 4), were not found by the consortia to be suited to an examination consisting of common papers since they discriminated only among the abler candidates. Other forms of composition, although they vary considerably as to the amount of writing required and as to the type and extent of the stimuli, are common to all examinations except 6. Little consistency of practice is discernible, however, although all the boards have claimed success for their approach. The variety and the capacity to discriminate of these forms of composition suggest that candidates have adequate opportunity to score appropriately. Nevertheless, on the whole the type of subject required tends to favour the abler would-be GCE candidate. It may be that a format is required which allows for choice either within common papers or by provision of differentiated papers, which would draw out more readily varying levels of ability.

44. One of the more intractable problems lies in the testing of comprehension, which is also subject to a number of approaches as to length and number of passages to be read, types of questions to be answered and their weighting. Longer passages have a valuable place, even though length makes for difficulties for the less able candidates. Subtlety and remoteness of style also create difficulties: 1 and 7 show general evidence of these characteristics; 2 did so in one: year. Some of the questions set were also likely to cause difficulty: reliance on factual information (1) did not give the abler candidates a chance to show their capabilities; loose questions (1, 2, 7) or unclear instructions (1, 7) were unhelpful, above all to the less able. Even in the most successful attempt to pitch passages and questions appropriately (2) the difficulty of catering for the lower CSE candidates is evident. The quality of passages selected and of questions set on them in this scheme has been high. There might be difficulties, as for 2 in the one year mentioned above, in maintaining this high level from year to year, and it is important that the skills needed for setting such questions should be developed and shared.

45. The evidence of speaking and listening comprehension tests is much less extensive since much is in the form of course work assessment. These tests are at an early stage of development and are subject to inherent difficulties as suggested in paragraph 38. The examples of scheme 1 betray unrealistic demands in speaking and an emphasis on factual recall in the listening test.

Marking and grading

46. The mark schemes and subsequent processes of adjustment translate objectives and criteria into hard practice. Scheme 6 offers helpful criteria specifying qualities to be looked for in the award of grades. Other schemes offer through their examiners' reports a cumulative experience of what examiners are seeking. However, in several schemes criteria are revealed in the mark schemes of which teachers and candidates may not be aware. The mark schemes also reveal a great variety of practice even on matters of common concern (eg mechanical errors); this may give some cause for disquiet in an examination intended to certify the performance of candidates on a nationally valid common grade scale. Nevertheless, 2 does show that it is possible to have formal expectations in respect of grammar, spelling and punctuation according to the ability of the candidate. Where the common system is particularly affected is in the narrowing of the band of marks awarded which can be discerned in the comprehension papers of 1 and 2; in 2 insufficient marks are allotted for the less able, and also for the abler if their potentially greater perception is to be rewarded; in 1 marks tend to be limited to factual items with little allowance for reasonable alternatives. In both 1 and 2 the procedures adopted for the adjustment of marks to the specified weighting resulted in further 'bunching'. However, there seems every possibility that these technical problems can be solved.

47. The main problem has been to allow sufficient scope for the less able to perform and be adequately rewarded at their level. In both the composition and comprehension sections schemes have tended for various reasons to favour candidates of GCE calibre but on occasion have failed to reward them adequately in terms of the greater subtlety or perception of which they should be capable. However, from the variety of experience some examples (2 and 7 in composition, and 2 in comprehension) suggest that a form of examination can be devised in English which caters for the ability range envisaged.

Extension

48. Although since the end of the experimental studies proper the joint examinations in English have gained momentum in two or three consortia, the analysis of their development has been much less detailed, so that it has been difficult to discover the effects of extension. Overall there will be an increased requirement for agreement trials and moderation; this will occur irrespective of the introduction of a common system. As more teachers become involved the difficulties of maintaining comparability and reliability of results may be increased.

Conclusion

49. The joint examinations have shown that in English language aims and objectives can be devised to cater for the target range of ability. The more successful schemes have effectively tested a broad range of written and oral skills. The various examination techniques tried indicate between them that techniques can be devised to cater for candidates of diverse levels of attainment.

50. In short, a common system of examining in English language is educationally feasible and could, though it need not necessarily, take the form of a common examination, provided that this is carefully and skilfully designed and makes provision for suitable choice for candidates to show their respective capabilities. Some of the joint examinations were notably more successful than others in solving the problems associated with a common examination. Further work should involve the sharing of experience and should build on the more promising lines of development.

ENGLISH LITERATURE

51. The response to literature is difficult to examine, whether assessment is based on set texts or on a free choice of reading. It requires a maturity and breadth of experience that many 16 year olds have not achieved; this makes it difficult to design an examination for a very wide range of ability. The problems are related to those mentioned in paragraph 44 in connection with the testing of comprehension. The plays of Shakespeare are the proper material for some pupils but beyond the understanding of others. If a reading of Shakespeare is equated with one of, say, Paul Gallico, what many people would regard as the higher order of achievement of the candidate who has read and responded to Shakespeare is not rewarded. Moreover, the pupil not only has to know what to look for in a literary work; he needs a specialised vocabulary if he is adequately to express his response. In short, the books and examining approaches which are appropriate for some candidates are inappropriate for others.

52. The general practice of schools is to enter their abler candidates for English literature as a separate subject in GCE O Level. Most CSE examinations have adopted the unitary approach, although a proportion of CSE candidates are now taking English literature as a separate subject. In 1976 the number of such candidates amounted to 10 per cent of the total CSE entries in English. Literature is occasionally available as 'further literature', that is, additional to a literature component in the unitary examination.

53. The evidence concerning literature from the joint examinations is slender. Only one consortium (8) offered an entirely separate examination in literature. One other (1), in addition to its unitary approach, offered extra GCE O Level certification for a further paper in literature. Scheme 3 (non-operational) also offered a further endorsement for literature but so few candidates took this option, which was designed for the more able candidates, that its evidence is of itself negligible. Consortia 2, 4 and 6 did not include literature in their schemes; 2 indeed has excluded it until a decision on a common system of examining is arrived at.

54. Up to and including 1977, schools using scheme 8 could offer any one of three approaches:

a set book examination comprising an external paper and course work;
a literary themes examination comprising a paper and course work;
a course work approach entailing 100 per cent teacher assessment.

55. The question papers of 8 (set books) again illustrate the problem. For instance, Dickens, Charlotte Bronte and Stan Barstow are alternatives in the prose section but are hardly equivalent in either length or difficulty. The drama and poetry sections are much more closely geared to the abler candidates in their level of material and questions. In 1 also, several of the set books would seem difficult for CSE candidates and its questions are generally limited to recall. The poetry question, however, is test of a comprehension which is similar to that frequently set in O Level. The set books approach also tends to offer essay questions occupying 35-40 minutes each.

56. Mark schemes in 8 have a positive emphasis. Lengthy and detailed schemes are offered in the set books paper. In the course work approach the syllabus has been amplified to include some guidance on the creative writing option and on the monitoring of the different tasks set by the teacher. However, a wide variety of practice is still emerging, making difficulties for standardisation.

57. The tests in literature offer little encouragement for extension. It is significant that the thematic approach which offered opportunities to the CSE range of pupils has been dropped because the consortium finds it unsatisfactory. The set books papers appear to offer scope only for the abler candidate. As to the course work approach consortium 8 has noted its concern with the problem of assessment. Scheme 1 also stresses the problem raised by the very wide range of books which might be offered by schools and which the examiners had to study. It mentions also the lack of confidence of teachers in carrying out their role in assessment. In spite of these doubts and those of teachers it is the course work approach, entailing expensive visiting moderation, which is to be the only form of assessment of literature for 1977.

Conclusion

58. The intrinsic difficulties of setting a common examination for the full ability range of the target group are greater for English literature than for English language. Although the evidence from the joint examinations is more restricted for literature it illustrates these difficulties, and it must remain doubtful whether a satisfactory common examination can be designed. Only one of the operational joint examinations explored the most likely approach to assessing the response to literature in a common system - that of differentiated papers. Insofar as the other joint examinations included the assessment of literature they confirm the validity of this approach and offer some useful indications which might be followed up in adopting it. This fact, supported by analogy with other comparable subjects, makes it reasonable to conclude that techniques of examining English literature within a common system can be devised. Among the matters deserving further consideration is the possibility of offering two literature options, one explicitly specialist for those intending to pursue studies in a literary subject in the sixth form, and another of a more general nature. Alternatively, a further and more difficult paper could be offered for literature additional to an examination in English based on a unitary approach.