A Critical Analysis of Sullivan's 'Cultural Capital and Educational Attainment'
Published in the journal 'Sociology' (2001, vol. 35, no. 4 pp 893-912, herein known as '2001 35-4'), the article 'Cultural Capital and Educational Attainment' is based on work Alice Sullivan carried out and completed in 2000 for part of her doctoral dissertation, 'Cultural Capital, Rational Choice and Educational Inequalities'. The article in the journal is not a direct copy of the original paper with the same name, although the vast majority of the text and tables are the same. Sullivan's paper aims to assess Bourdieu's theory of cultural reproduction "that children from middle-class families are advantaged in gaining educational credentials due to their possession of cultural capital" (2001 35-4: 893).
This essay seeks to go beyond merely understanding the article so as to analyse it from many different angles,providing critique of findings and suggesting possible errors and false presumptions. It looks to analyse the way in which the article has been written, outlining ways in which the article excels as well as fails in terms of readability and accuracy. This will lead to a conclusion as to the overall significance and acceptance of the article, with suggestions made as to improvements and extensions to it.
In order to critically analyse the article, it is certainly necessary to understand the context in which it was written. People can never be fully objective - they have consciousness which makes them subjective beings. Further it must be remembered that, whilst it is presumed that Sullivan sought objectivity and fairness in her paper, this should not be taken for granted and must be included in the analysis. In this respect, it is important to consider the funding for the paper and possible motives to search for certain findings. To take something at face value without any analysis is naive.
Sullivan's doctoral work was funded by an ESRC (Economic and Social Research Council) studentship. The ESRC is an independent organisation under Royal Charter which receives the majority of its funding through the Government's Office of Science and Technology. With her doctorate taken at Oxford University, interestingly Sullivan is now holds a post there. There are no obvious reasons for questionable motives here. But it should be remembered that writing this paper and having it published as an article in a prestigiuos journal would have advanced her career.
The article starts with an abstract which is commendable for its bevity and conciseness. It sets forward the articles aims and conclusions made in easy to understand English. Following this is an introduction which is equally concise, outlining four questions to be addressed through the article in bullet points. However, the bevity does lead Sullivan to make a minor factual error in mentioning "GCSE attainment at age 16" (2001 35-4: 893) even though no all GCSE candidates in the survey were 16. But this minor, if somewhat anal omittance is remedied later in the article in the 'Methodology' section where the target sample are explained as being in their final year of compulsory education, of about 16 years of age.
The next section is 'Cultural Capital' in which Sullivan discusses at length Bourdieu's theory and arguments in an analytical fashion. She makes good use of direct quotations from Bourdieu's works, addressing the presumptions made by Bourdieu and showing ways in which Bourdieu appears to contradict himself, for example in his denial of theorising despite evidence otherwise. Throughout, Sullivan's choice of language and scentence structure enables the reader to maintain focused on the thread of discussion, quite in contrast to Bourdieu's own vocabulary which at best reads like a riddle. Sullivan's vocabulary is more widely accessible and for this it is applauded: it does not appear to be that an elitest academic intent on reducing the number of people capable to reading her article to a significant minority. In doing so, Sullivan avoids a trap into which many academics appear to fall in their pursuit of proving their superiority and intellectual capacities.
Sullivan's concluding comments of Bourdieu evidence are quite harsh but seem fair, denying Bourdieu of his entitlement to make certain presumptions and associations of causal relationships, claiming that "Bourdieu assumes much of what he sets out to prove" (2001 35-4: 896).
Sullivan continues with 'Operationalisation' in which she explains how she classes the concept of 'cultural capital'. She does this with heavy but appropriate reference to fellow researchers and their own takes on the concept. But there is a point on which I will pick up on in this section which is of crucial importance to Sullivan's study. She states that "[t]he return on cultural capital takes the form of educational credentials and, ultimately, occupational success" (2001 35-4: 897). However, I argue that educational credentials are not necessarily the best or only way of measuring educational attainment, but simply the commonly accepted way in Western society. Credentials show how well someone did in an exam, and is presumed to be linked intrinsically to educational attainment. However, exams are by no means perfect in their deductions and it could be argued that an individual with a high exam score has not attained as great an education as a second individual with a lower score, perhaps because the second individual became more educated in topics not relevant to examination, or simply malperformed during the exam. Further, to be educated literally means to learn and to improve, not just to know. Hence a true measure of educational attainment should refer to how much is learnt over a period of time and not just how much is known during a set of exams.
In 'Methodology' Sullivan explains how she carried out her research, explaining her decisions. Her choice method was surveying pupils in year 11 in four comprehensive schools in England in 1998. A pilot questionnaire was used first, suggesting that it was well thought out, however the fact that the survey itself is not available to us means that we must trust that the survey was well written and was not ambiguous or misleading in any way. Ideally, it would be good to at least have the option of being able to view the questionnaire, for example in an appendix or online. Whilst it is not common practice for the original questionnaire to be available to readers, I see no reason why it should not be - a high level of transparency should be vital to the integrity of research.
Sullivan explains that her findings are not relevant to the population, as her sample design is for four comprehensive schools and does not cover the full educational system. There is also no mention of geographical location, except for 'England', which means her data may only be relevant to certain parts of the country. However, she rationalises this by stating that "[c]ultural reproduction theory is concerned with general processes, which are not contingent on any particular school context" (2001 35-4: 898). But this could be taken as a simple excuse not to extend the study - just because the theory does not consider their to be differences between schools, does not mean that the reality mirrors this theory. It is good to see Sullivan explaining her rationale though, rather than allowing less learned readers to make false presumptions about the survey's scope.
Naively, Sullivan appears to presume that all pupils surveyed live in a nuclear family with one set of parents. To be sure of this, it would be crucial to have access to the original questionnaire, but in the article Sullivan talks of "parent's social class" and "[m]other's or father's qualifications" (2001 35-4: 898). If the questionnaire asked similar questions then this could have confused some children, for example those with only one parent, foster parents, more than one set of parents etc. Also, think for example a child who grew up for most of their life with one parent but recently moved to live with another. The survey should surely be concerned with the household in which the child has spent the majority of their youth, as it is from their that the most cultural capital should have been passed down. Similarly, what of parents' job changes over time - no account appears to be made for this either. Again, it would be extremely useful to see the original questionnaire to see how Sullivan worded her questions. Cynically it may be argued that Sullivan is covering up obvious methodological hiccups.
|Fig. 1: Taken from Sullivan's original paper, not included in the journal article.|
Sullivan's assessment of pupils' cultural capital is undoubtedly subjective, but I can think of no way in which it can possibly not be. However, some aspects seem a little too subjective, for example in awarding points to reading books "of the sort that receive reviews in the quality press" (2001 35-4: 899, italics mine). Indeed, I believe this particular judgement to have an air of elitism surrounding it. To assess cultural knowledge she merely asked pupils to categorise twenty-five famous cultural figures according to their field of excellence (fig. 1). But what of pupils who didn't care much for names, and were perhaps more interested in geography or cultural history instead? Apart from small criticisms such as these, Sullivan's overall scope appears to be broad and well thought out, and she admits that her test is "not intended to reflect all aspects of a pupil's cultural knowledge. However it at least provides us with some indication of cultural knowledge" (2001 35-4: 899-900).
It is interesting that Sullivan notes that "many parents would no doubt have been reluctant to participate" (2001 35-4: 900) in the survey themselves, and that pupils' responses regarding their parents are unreliable. It also triggers an important question which Sullivan does not answer: were the parents told that their children were participating in the survey, and if so, were they asked for permission? It is quite possible that parents may not have wanted their children to participate, for example perhaps they did not agree with the ethics behind and the possible findings of the survey. Schools must be trusted as they have tremendous power in our society, so if permission was not asked then this is grossly unethical. I would assume that Sullivan made every attempt to follow a reasonable code of ethics, perhaps enforced in relation to her ESRC studentship, but I do not know this for sure.
Up to the 'Analysis', Sullivan's work should be consistently praised for its English and accessibility, as has already been mentioned. However, and quite regrettably, her analysis (and in particular her tables of data) is shocking for its lack of accessibility. Her first table, on 'Pupils' Cultural Activities', takes up a whole page and includes no less than 36 rows and four columns of data. Accompanying the table are one or two paragraphs of text which explain the meaning of 'Model 1' and 'Model 2' but little more. Analysing the table is a painstaking chore aggrevated by a lack of explanatory notes - it works something like solving a puzzle or mystery. We are told that linear regression is used (even though Sullivan fails to explain why the relationship should be a linear one - although if it is not linear then her findings will simply underestimate, not overestimate, the strength of relationships) and then presented with a statistic aptly named 'B' in the table, which includes two figures, one of which is placed in brackets. What this magical statistic represents is by no means obvious - but of course a select few in the world who conduct social surveys on a daily basis may be able to hazard a guess. Having an A at A-Level maths myself, one would imagine that I may have some advantage in deducing its meaning. Or two degree-level maths undergraduates who studied the table surely should immediately be able to understand it. Finally, after half an hour of careful scrutiny, web research and graph doodling I think I may be able to explain what 'B' means: the top number, not in brackets, is a mean average, whilst the lower number is probably the standard deviation, or possibly some other confidence interval.
The table is certainly not aided by the lack of boxes grouping categories together. Neither by lack of explaining 'corrected model' or 'intercept' either. To its credit (and yes this is said with more than a hint of sarcasm), eta2 is explained in a footnote as describing "the proportion of total variability in the dependent variable attributable to the variation in the independent variable. It is the ratio of the between group sum of squares to the total sum of squares" - I'm sure that will clear up any confusion. A more useful explanation of eta2 would be to say that the bigger it is, the more significant the determinant concerned must be, adding that it is a correlation ratio perhaps.
|Fig. 2: The table from Sullivan's original paper - a little less confusing|
A web search returned what appears to be an original copy of Sullivan's paper in electronic form. Whilst the article in the journal is a near word-for-word duplication of this, there are some notable changes. In particular, fig. 2 shows the table on 'Pupils' Cultural Activities' in its original form. There are considerable differences between this and the journal version: the original is condensed, showing only significant paramenters, and does not include the eta2 statistic. Because of this, the original is easier to read, however I presume that the journal wanted, for completion, to show all variables. However, in doing so they have complicated an already complicated table, and they have not remedied this through the use of boxed categories and explanatory text, as one might hope that they would. Better would be a graphical representation of the data, similar to that used to show parents' qualifications against cultural capital in the original paper (see fig. 3).
The 'Analysis' continues, accompanied by three more similar tables, one of which can't even fit on a single page. Easier reading is a smaller table showing 'Cultural Capital and Gender'. Regardless of the tables, the wording of the analysis is generally sound, explaining her main findings. At one point Sullivan says that "[t]here is no space to show overall significance and eta2 values for categorical variables in the tables" (2001 35-4: 902) yet they sound from her analysis to be the most important values to show - a change in priorities over what to show in figures and tables would have been much appreciated. Similarly, she states that "...giving 1 point for a G grade, 2 for an F etc. This point score is approximately normally distributed - see Table 5" (2001 35-4: 906). The table itself does not have the capacity to show normal (or other) distribution: that is not its function.
|Fig. 3: Graph from Sullivan's original paper - easier to read than longwinded tables|
Regrettably, the shoddy use of figures and tables in the article undermine its integrity. It leaves one with a burning question: did Sullivan really understand the data she was collecting, and the statistical analysis performed with it? People are not infallible, and it is quite possible that Sullivan rushed the research, without properly understanding it, to get it finished by a deadline, or because it was mind-numbingly boring after three years of constant study on it. Sociology students are know for being generally lousy with anything in the slightest mathematical or computational - these are not their strong points. If this is the case, it is hoped that Sullivan acquired the help of a statistician to mull over data, validating it and suggesting appropriate statistical methods. But, as I have already criticised Sullivan for earlier in relation to the questionnaire, there is no mention of this.
At this point, the critical analysis can shift momentarily from focusing on Sullivan to the journal itself. A journal should be concerned with ensuring the utmost quality in articles published within it. This should include checking over many of the points outlined in this critique. Because of this reason, the findings put forward by Sullivan are given extra validity, as they can be said to have been rigourously examined by an independent adjudicator. But the fact that the journal article was produced with such confusing tables is startling: surely someone who works for the journal could have realised that a large percentage of the population would find these totally useless?
The conclusions made by Sullivan are particularly interesting but ultimately worthless in the framework of the article due to the article's failings. Sullivan's study's aim was to use emprical data to analyse Bourdieu's theory, but she has not proved to me that she fully understands her data, and is not transparent enough with her findings and research methods to allow for deeper investigation. This is all of great shame as Sullivan has conducted an extremely interesting and worthwhile research project, and I would like to believe that what she says is true. But doing so would be naive: if all printed works were taken at face value, believed to be 'true' without and questioning, then all manners of false findings - influenced by the researchers' subjectivity and motives - could make their way into our common knowledge.
[2,915 words, written in 2003]
Bibliography & References
Sullivan, A (2001) 'Cultural Capital and Educational Attainment', Sociology, vol. 35, no. 4, pp. 893-912
Sullivan, A (?) Cultural Capital and Educational Attainment, Online: www.u-bourgogne.fr/IREDU/sem2433s.pdf (accessed: 6/5/03)