Hadow (1924)

Notes on the text
Preliminary pages Preface, Contents, Membership, Analysis, Introduction
Chapter 1 History of development of psychological tests
Chapter 2 Summary of evidence
Chapter 3 Possible applications, conclusions and recommendations
Appendix I List of witnesses
Appendix II Recent experiments
Appendix III Use of tests in foreign countries
Appendix IV Standardisation and norms (Cyril Burt)
Appendix V Correlation in mental testing (Cyril Burt)
Appendix VI Grades in US schools (AE Twentyman)
Appendix VII Recent publications
Appendix VIII Examples of tests
Appendix IX General, special and group abilities
Index

Appendix II

SHORT ACCOUNTS OF SOME EXPERIMENTS RECENTLY CONDUCTED IN ENGLAND IN THE USE OF GROUP TESTS AND INDIVIDUAL TESTS IN FREE PLACE EXAMINATIONS AND IN SCHOOLS OF DIFFERENT TYPES
[pages 149 - 164]

(1) Experiments in the use of (a) group tests of intelligence, and (b) standardised scholastic tests at Blackpool (summarised from data supplied by the Director of Education).

(a) The selection of candidates for scholarships to central and secondary schools in the County Borough of Blackpool in 1922 was made by the following stages: in the first instance, an examination or review of the work of all pupils in elementary schools between the ages of 11 and 12 was made. About 900 children sat for the preliminary examination, and about half this number were selected to sit for a further competitive examination in English and arithmetic. The marks gained at the written examination and at the oral examination, together with a report from the head teacher of the elementary school which the candidate had attended, were taken into consideration in the award of scholarships. As a result 30 candidates were sent to secondary schools and 240 to central schools. As an experiment one of the published series of group intelligence tests was given to all the candidates selected for the second examination. The papers containing these tests were carefully marked, and the intelligence coefficient of each child was calculated from them, and a list made in order of merit. It was found that the list was not identical with the scholarship list, and it was noted that 16 of the first 30 children on the intelligence tests list were included in the first 30 on the scholarship list. This result, however, did not afford material for deciding whether the intelligence test or the ordinary examination afforded the more trustworthy result. Accordingly, as a further experiment, 92 of the first 100 pupils on the scholarship list were set four other group intelligence tests. The papers thus compared were three American tests and two English tests. The following table gives the position of each list for certain of the pupils:

Dr Cyril Burt worked out the correlation of these results with the scholarship results, arriving at the following coefficients respectively: .35, .55, .51, .46, .50, with a probable error from .05 to .06. Dr Ikin believed that some guidance could be obtained from the use of group tests which might be of great value in deciding borderline cases, but the chief result of the experiment in his opinion, was that though intelligence tests of this kind were being developed on right lines, they had not yet achieved such a state of perfection as would justify the award of scholarships on group intelligence tests alone. Dr Ikin had found that a comparison of the work of the pupils subsequent to the application of the tests was showing that these tests did assist in the discovery of educable capacity, and he believed that if a group test were given in addition to the ordinary attainment test, there would be a greater probability that the more suitable candidates would be selected for higher education, than if either type of test were employed alone.

Dr Ikin states that he is continuing his experiments with group tests of intelligence; and that it is probable that this year he will be able to combine a group intelligence test with the ordinary scholastic examination for awarding scholarships.

(b) Dr Ikin has recently applied standardised tests of arithmetic to all the children aged 7 to 14 in the ordinary elementary schools of Blackpool. The tests were applied after the scholarship examination had been set, and immediately after the children so selected had been transferred to secondary or central schools. The object was in part to discover whether any children of supernormal ability in arithmetic had been missed by the scholarship examination.

Those used were the tests of mechanical and problem arithmetic taken from Dr Cyril Burt's Mental and Scholastic Tests. They were set and marked by the teachers; the tabulated results were submitted to Dr Burt, with a request for comments; and his reply was then circulated to all the teachers concerned.

The following are some of the conclusions drawn from the experiment:

(i) The use of such standardised scholastic tests in the hands of the ordinary teacher presents no difficulties, and yields suggestive information in regard to the level of attainments of the children in the schools within a district.

(ii) Tests standardised in one urban area, such as London, seem directly applicable, with little or no change, to another urban area, such as Blackpool.

(iii) Dr Burt states that, as compared with London children, the younger children at Blackpool, particularly those aged 7, did unusually well in the tests; and that, in general, the Blackpool children were better at problem work than they were at mechanical work. It is interesting to note that the level of attainments in two such widely separated districts is all but identical.

(iv) The relative inferiority of the pupils over the age of 11 appears, in Dr Ikin's view, to confirm the anticipation that the best pupils had already been successfully selected by the annual scholarship examination.

(2) Results obtained by the use of group tests of intelligence in the examination for free places and for entrance to secondary and central schools at Bradford in 1922 (summarised from data supplied by the Director of Education)

In the examination for free places and for admission to secondary schools and central schools held at Bradford in March 1922, the maximum mark for group intelligence tests was 100, while that for arithmetic and English combined was 200. The results obtained by the application of the intelligence tests were not to be taken into account, except in borderline cases. As an instance of the effect of the inclusion of the marks for intelligence tests, it was pointed out that 448 pupils would thereby be raised from between 33 per cent and 50 per cent to over 50 per cent. It was found that pupils admitted on these tests justified their choice by profiting from a secondary education. It was stated that a number of cases had already been investigated, and, although the total number of candidates so examined had been small, nevertheless the enquiry, so far as it had gone, had confirmed the value of group intelligence tests as an indication of ability to profit by a secondary school education. The number of cases investigated in the secondary schools was 48 boys and 44 girls, and it was found that in general the results obtained from the intelligence tests corresponded with the progress made in the secondary school, as indicated by the terminal examinations, to a rather greater extent than obtained when a comparison was instituted between the marks awarded in the ordinary scholarship examination and the marks given in the secondary schools.

(3) The use of intelligence tests in examinations for free places offered by the Leicestershire Education Committee (summarised from a memorandum sent by the Director of Education)

All pupils in the elementary schools in Leicestershire (with the exception of those not recommended by their head teachers) between the ages of 11 and 12 on the 1 June in any year are examined in English and arithmetic, and also undergo an oral examination. For the marking of papers in the written examination, and for the conduct of the oral examination, district boards consisting entirely of secondary and elementary school teachers are appointed.

The term 'oral examination' is used to cover a consultation between the two visiting teachers (one secondary and one elementary) and the head teacher of the elementary school as to the capacity and promise shown by the pupils. For purposes of standardisation the oral examiners are supplied privately with a 'provisional qualifying mark' based on the results of the written examination in English and arithmetic. A preliminary order of qualification, drawn up by the head teacher, enables the examiners to make ready reference to any pupils who may have failed to do themselves justice in the written papers.

The visiting examiners may question the pupils on errors occurring in their written papers and may test their knowledge in other subjects, especially history and geography. In addition, the visiting teachers may impose any reasonable mental tests of general capacity.

For the purpose of estimating the actual intelligence, as distinct from the acquired knowledge of the children, the examiners, generally speaking, had not relied solely on mental tests according to the textbook models, but they had taken into consideration the results of the written papers, and especially the opinion of the head teacher and the record of schoolwork.

The total impression left upon the minds of the examiners was the correctness of the head teachers' estimates, subject to the invaluable check (which a general examination over a large area provided) upon possible faults of judgement and false standards of knowledge.

Experience did not show whether mental tests would have resulted in the same order of placing the pupils, had the examiners approached their task with an open mind and with no previous knowledge of the capacity and attainment of the pupil. Some of the most experienced teachers who had conducted oral examinations in Leicestershire were of opinion that 'intelligence' tests, whether oral or written, should not be the sole means of estimating the quality of candidates for admission to secondary schools. Such tests indeed revealed presence of mind and quickness in 'seeing the point', but ignored application and staying power. Nevertheless, they were invaluable in the oral examination.

(4) The use of group tests of intelligence in examinations for free places offered by the Northumberland Education Committee (summarised from memoranda sent by (a) Professor Godfrey H Thomson, and (b) the Director of Education)

(a) In a memorandum sent to the Committee by Professor Godfrey Thomson, he stated he had tested about 3,000 children in Northumberland in 1921 and about 14,000 in 1922 by means of group tests devised by him for the purpose, as well as some 2,000 children in other parts of Great Britain during the preliminary process of standardising the tests. He was convinced of their value in giving due weight to inborn ability in the exceedingly important and difficult task of selecting children for different types of schools at the age of 11. He was well aware, from his own work as well as from his study of the publications of Burt, Gordon and others, that intelligence tests, especially group tests, were by no means uninfluenced by schooling, so that a child from a poor school or from an uncultured home was still heavily handicapped. But he believed that this handicap was far less than the handicap felt by such a child in attempting, say, a county minor scholarship examination. He had retested a small number of children for several years by individual tests, and had found constant intelligence quotients. Further, he had retested 11,000 children at a year's interval by his own group tests, finding practically the same intelligence quotient in 85 per cent of the cases, but changes ranging up to, in one case, 30 points in the minority. On the whole, he believed in the constancy of the quotient. He stated that the practice in Northumberland was now to admit children to secondary schools who were very good either in intelligence tests or in acquired knowledge, provided they were at least average in the other respects. At the free place examinations in 1921, however, 14 children were awarded free places and maintenance on the result of the intelligence test alone. Of those one child had since died and one was doing badly, but the remainder were doing very good work in secondary schools. His experience in Northumberland had led him to reject for general educational use group tests which required accurate timing for each page, partly because the bustling nature of such tests was not in his opinion desirable, and in part because considerable skill and coolness were required in the supervisory, so that testing on a large scale (in which it was necessary to employ untrained supervisors) was impossible with such minute timing. (1)

(b) In a memorandum sent by the Director of Education for Northumberland it is stated that, in addition to the tests applied in 1921 and 1922, the Education Committee has sanctioned the experimental adoption of intelligence tests in examinations for junior scholarships, and as a supplement to attainment tests for the admission of fee paying pupils to secondary schools. For this purpose the following group tests have been used:

As a general rule, candidates who obtained satisfactory marks in the subjects of English and arithmetic proved also to have shown, as the result of the intelligence test, a mental ratio above the average. To this rule, however, there were many instructive exceptions.

The Director of Education has submitted a number of tables showing the result of an inquiry into the subsequent progress of pupils admitted to secondary schools during the past two or three years. The following are the more significant facts emerging:

(i) Among pupils credited by the intelligence tests with a mental ratio of 120 or more, all except one are reported as making satisfactory progress. In the ordinary examination in English and arithmetic many of these brighter candidates obtained low marks, and would not have received a scholarship had not the results of the psychological test justified special consideration. They are all well under the average age of the forms in which they are now working; and their position in their form is high - some taking first, second or third place in spite of low marks in the ordinary examination. One boy who obtained ratio of nearly 120, but only 15 marks out of 100 in the ordinary examination, is now second in his form, although he is a year younger than the rest of the class: the head master reports that 'though he failed in paper work on entrance, he has since made good'.

(ii) Three pupils who showed somewhat low mental ratios (between 83 and 90) were awarded scholarships on the ground of comparatively high marks in arithmetic and English. One is reported to be 'a steady worker'; and the work of the other two is described as 'fair' or 'very fair'. They have, however, obtained only low places in their forms; and two of them are much above the average age of their form. A fourth boy, with an equally low mental ratio, whose examination marks did not justify a scholarship, is reported as 'satisfactory'.

There are thus exceptions to the rule that only children with high mental ratios make satisfactory progress in a secondary school, but these exceptions appear to be exceedingly rare.

The Director concludes that 'on a broad view, the introduction of a psychological test, as a supplement to the examination in school subjects, has been of great value. It has at least tempered the application of a particular method of determining admission to secondary schools with entirely satisfactory results'.

(5) An experiment in the use of group tests of 'intelligence' in 16 elementary schools in the West Riding of Yorkshire (summarised from a memorandum sent by Mr TP Tomlinson, MEd, MA, Head Master of Love Lane Council School, Pontefract

Group tests of intelligence were applied to 1,600 children, aged 9 to 14, attending 16 schools in the West Riding of Yorkshire. The area thus surveyed included ten small villages, a small river port, and slum and residential districts of a neighbouring city.

The scheme of tests used was one termed by Mr Tomlinson the West Riding Scale. It consisted of eight tests of familiar types: (1) instructions, (2) analogies, (3) absurdities, (4) mixed sentences, (5) classification, (6) word meaning, (7) logical selection, and (8) arithmetical reasoning. Two parallel sets of similar tests were used.

The results of the experiment are briefly as follows:

(i) It was found that all the tests could be administered satisfactorily with a minimum of explanation by the head teachers of the various schools. The application of the tests took rather less than one hour; and the marking was practically mechanical.

(ii) This is one of the few experiments in which similar tests have been twice applied, so that some measure of the correspondence of the results is obtainable. The results of the second application agreed almost exactly with the results of the first, the reliability coefficient being throughout approximately .95.

(iii) For a few small age groups independent estimates were obtained, either from the teachers of the classes, or by the application of the individual tests of the Binet-Simon scale. The correspondence of these estimates with the group tests was again quite close, the correlations ranging from .71 to .84.

(iv) A point of special interest in the survey is the well-marked difference found between children from areas of different types. The results obtained from the rural schools are of peculiar interest. In the rural schools the most noticeable feature is the greater range in the distribution of intelligence. Further inquiry showed that the villages where a disproportionate amount of dull and backward children were found, were for the most part villages near a port or city - villages, that is to say, from which the brighter families had already migrated. Other villages, however, in a similar situation showed a disproportionate number of bright and average children: in such cases it was found that the less intelligent families had moved to the larger industrial centres owing to the demand in the latter for unskilled labourers; while, on the other hand, the village itself offered a desirable place of residence for better families, such as skilled artisans and professional people.

(v) Mr Tomlinson points out that such tests, in the hand of teachers working in relatively isolated schools, afford serviceable aid in rapidly sorting out the dull, the backward, and deficient, from the bright and supernormal. Without such standardised tests teachers in these remoter districts have little notion of the general level of the normal, backward, or supernormal child. He urges that such tests are needed more particularly for the brighter children. The subnormal are readily detected owing to their backwardness in school progress, but the brighter children may easily be overlooked. (vi) The following are typical instances of children whose intelligence and fitness for higher education have been wrongly rated by the ordinary examination.

(a) AB, a girl of 11½, had a mental ratio of 147 (Binet) and 148 (West Riding Tests). Her head mistress described her as a brilliant girl. Yet she had failed in the junior scholarship examination. She was subsequently transferred to a secondary school, and is now working there very successfully.

(b) CD had a mental ratio of 125, but is a highly nervous child. At the age of 6 he was able to read the daily newspaper with intelligence. At 8 his critical ability was so far developed that he was able to suggest improvements in the designs exhibited in his 'Meccano' book, and to embody these improvements in his constructions. At 11 he was doing very good work in Standard VI, both in arithmetic and English. Owing to his nervousness, however, he failed hopelessly in arithmetic at the junior scholarship examination.

(c) EF has gained a junior scholarship, owing, probably, to special preparation; his low mental ratio should have eliminated him from the list of awards. His most recent report shows that at the secondary school he is bottom but five in the lowest form, and his want of success, even in subjects taught in the elementary schools, is almost as evident as it is in the new subjects taught at the secondary school.

(6) Results obtained by the use of individual and group 'intelligence' tests at the Ben Jonson Council School, Harford Street, Stepney, London (summarised from information supplied by the Head Mistress of the Girls' Department, Miss G, M. Anderson)

Miss Anderson explained that this school was the largest provided elementary school in London. Eighty per cent of the pupils were the children of casual workers, and about 12 per cent were of foreign parentage.

For the purpose of classifying scholars on admission, individual tests were generally used - Terman's revision of the Binet-Simon scale. The test was applied during first term to pupils admitted from the infants' department, and immediately on admission to those admitted direct from outside.

Group tests were also used such as Dr Ballard's 'picture', 'absurdity' and 'vocabulary' tests, but the individual tests were considered the best. Of these individual tests, the 'comprehension' tests were the most reliable.

The school was organised so as to allow three currents of promotion, viz, the 'supernormal', with a mental ratio of 110, the 'normal' with a mental ratio of 90 to 100, and the 'subnormal' with a mental ratio below 90. Mental tests formed the sole basis of classification for the supernormal grade. No test was required in order to discover the subnormal children.

Promotions in the school were determined as a result of examinations held twice a year. The examination was chiefly in acquired knowledge, but mental tests in the form of 'comprehension' tests were always included. The order of merit in the two tests was on the whole parallel, especially in the supernormal class. The following table was given as a typical result of an examination in the upper division of the 'supernormal' class:

*This child was rather inaccurate in arithmetic.

The time taken to work the tests was regarded as indicative of intelligence, but the slow, sure, deliberate and accurate children were also discovered, and one such child had proved to be brilliant.

(7) Results obtained by the use of group tests of intelligence at the Mile End Central School, Myrdle Street, London (summarised from data supplied by Mr JG Tibbey MA, Head Master of the School and Chairman of the Research Committee of the London Head Teachers' Association)

Mr Tibbey had used psychological tests in two boys' schools and one central mixed school, sometimes to assist in organisation, and sometimes to test it. For this purpose he had mainly relied upon opposites, analogies, arithmetical series, and absurdities, and, allowing for errors in application, had found such tests materially helpful. Much depended upon care in the selection of tests and attention to procedure.

For the purpose of school grading at the present stage, it was advisable to consider the results obtained by intelligence tests in conjunction with those of the examinations in attainments and the class record.

Time tests were generally valuable, but needed to be applied with discretion. They appeared to be undesirable for pupils below 9 years of age where writing was involved.

He suggested that copies of intelligence tests of proved reliability should be made available to teachers by local education authorities.

(8) Account of an experiment in the use of individual non-linguistic tests of intelligence and quasi-vocational tests at the Acton and Chiswick Junior Technical School (summarised from information supplied by the Head Master, Mr Max Tagg BSc)

During the past four years, Mr Tagg has regularly applied psychological tests to all the boys in his school. The boys are about 150 in number, and range in age between 13 and 16 years. The tests used were designed to test partly general intelligence and partly special aptitude for engineering trades. They consisted of individual tests of a non-linguistic or so-called 'performance' type, e.g. tests of perception of form and of space, of gauging curves, diameters, lengths of lines and sizes of angles, of constructing geometrical figures, and of practical reasoning.

After considerable experience of these tests, he concluded that they afforded better estimates than the ordinary written examination. In a written examination boys of the type attending his school did not always do themselves justice, and sometimes failed to reveal their special aptitude for technical work. Often there was little or no correlation between the results of the written examination and the opinion which masters formed of individual boys during their attendance at the school. Mr Tagg himself now bases his final judgement of his pupils on the combined result of written, oral, and practical examinations, in all of which psychological tests play a prominent part.

(9) An experiment in the use of tests at Pate's Grammar School for Boys at Cheltenham (summarised from information supplied by the Head Master, Mr RR Dobson MA)

(i) Oral tests of intelligence. Mr RR Dobson carried out in 1922 an experiment in supplementing the ordinary annual examination for free places by an oral examination with 'intelligence' tests. There were about 150 candidates for 26 places. The tests used for this purpose were Dr Burt's graded reasoning tests.

From his personal experience in using these tests he concluded that they were undoubtedly of great value as a check upon the scholarship examination and as a useful subsidiary criterion for borderline cases.

In the main, the agreement between the psychological tests and the ordinary examination was so close that, in many elementary schools, the boys could have been selected by the results of the former alone. Since, however, a child must have some degree of educational attainments, it would clearly be undesirable to abolish the ordinary examination.

Where the results of the psychological tests differed from those of the ordinary examination he had inquired into the causes, and found cases of two types:

(a) The teaching in the elementary schools in a mixed area varied far more than was generally supposed, some being very efficient and some inefficient. Thus there could be little doubt that many children who did better in the psychological tests would have done equally well in the ordinary examination had they been as efficiently taught as the rest. In certain country schools, for example, the boys were, in natural intelligence, quite as well endowed as the town children; but, owing to the difference in staffing, often attained a far lower standard in the ordinary examination. Mr Dobson had accepted some of these boys solely on the ground of their success in the 'intelligence' tests, and their subsequent progress showed that the selection had been fully justified.

(b) Where a boy from a good home or school had failed to do the work of the class to which, owing to his more attractive bearing, or his greater success in the scholarship examination, he had been sent, such tests had been of great service in indicating that his apparent merits were simply a superficial and transitory effect of fortunate conditions, and in distinguishing, in such instances, lack of native ability from lack of industry and zeal.

(ii) Written tests of intelligence. Mr Dobson has since carried out extensive experiments with written or group tests of intelligence. The tests employed consisted of five separate tests, constructed by Dr Burt, practically identical with those used later by Mr Vaughan at Rugby School. With the written tests of intelligence he has examined nearly 600 persons, including boys and girls from Pate's Grammar Schools at Cheltenham, students in the training colleges and University at Bristol, and a small group of university lecturers and professors.

The written tests gave a correlation with attainments much closer than that given by the individual tests, the correlation being somewhat higher with the younger grammar school pupils than with the older university students. Mr Dobson believed that the results of the oral and individual tests were the more reliable as indications of native intelligence; but the written tests, which took far less time, might for practical purposes be more serviceable.

The following is an extract from the tables submitted by Mr Dobson to show the agreement between the results of the tests and the pupils' progress in the ordinary work at the school.

Among the group aged 15 to 16, the three top boys in the test were all matriculated students, doing well in an advanced course. KL, aged 11, who obtained 132 marks (50 marks above the average for his year, and by far the highest obtained among boys of his own age), is known as an unusually bright boy, working well with grammar school boys whose age is 14. MAP, aged 17, who gained the high mark of 166, almost the highest in the whole school, had obtained a State Scholarship at the age of 15, with distinction in mathematics in the Higher Schools Certificate, and has since obtained an open mathematical scholarship at Clare College, Cambridge.

Mr Dobson found that inborn intelligence, as thus tested, appeared to increase regularly from the lowest ages tested up to about the age of 16; after that age there seemed little or no increase in intelligence as distinct from experience and attainment, except among persons of exceptionally high ability; the university lecturers and professors, for example, obtained unusually high marks, well above those obtained by fourth year students in training who had already obtained their degree with honours.

Mr Dobson stated that the boys admitted into the school in 1922 graded by the oral reasoning tests are all doing good work in the second year of the school course.

At the re-examination of these boys at the end of one year's interval, all kept their places well. There appeared to be a slight increase in intelligence quotient, possibly due to scholastic training and environment.

(10) Memorandum on an experiment in the use of group tests of 'intelligence' at Rugby School by the Head Master, Mr WW Vaughan MVO, MA.

'On the last day of the Christmas term 1923, a group test was set to the whole of Rugby School (600 boys). The test done was one constructed by Dr Burt, for the National Institute of Industrial Psychology, and consisted of series 33 of their group tests. The series consisted of a printed booklet of five different tests - namely, opposites, analogies, mixed sentences, completion and reasoning - similar to those described in Appendix VII. To make the marking as uniform and as mechanical as possible, alternative answers were printed, so that the candidate had nothing to write, but simply to pick out and underline the correct reply. Such a test necessarily affords no opportunity for constructive imagination or literary expression, but was selected as being the type of test most suitable for testing a large group of boys differing widely in age and ability. In a previous year a similar test had been employed on a smaller group of boys at Rugby School, and the results were found to correlate with the masters' independent estimates of intelligence to the high extent of .83. The work was done form by form, under very strict conditions, and I think the instructions were most loyally obeyed. The scripts were sent to be corrected by a clerical assistant nominated by Dr Burt. The results are interesting. From the tables given it will be seen that the upper forms of the school did better than the lower forms; the older boys better than the younger boys; those who hold scholarships better than those who are normally considered backward in their work. Of those who came out in the first 20 all except two were in the sixth form, or the adjoining form. The school position of one boy who was in the last four of those examined was about 150th from the bottom by school work, but he has gained this place by very hard work. He finds all his subjects difficult except Greek grammar, in which he does very well.

'From my knowledge of all the boys in the school I conclude that the test is a very good test of teachable ability. This seems to me a better word than 'intelligence'. In a few, but very few, cases, the boys were upset by the time limit.

'The test, it is true, gave no reliable indication of a boy's power of literary expression, or of his imagination. It did, however, give a better judgement than I had expected of his industry. As a rule a boy who does not waste time in school or in preparation was more successful than the dawdler. The results seem to disprove the contention that there is very little change in a boy's intelligence between 16 and 18.

'It may be worth while to add that amongst the members of the school who took the tests were a scholar of Balliol, a scholar of Trinity College, Cambridge, and a scholar of King's College, Cambridge'.

Footnote

(1) cf. Professor Thomson's description of this Northumberland experiment in the British Journal of Psychology for December 1921 and July 1922.