DISCOURSE ANALYSIS MEANS DOING ANALYSIS: A CRITIQUE OF SIX ANALYTIC SHORTCOMINGS

A number of ways of treating talk and textual data are identified which fall short of discourse analysis. They are: (1) under-analysis through summary; (2) under-analysis through taking sides; (3) under-analysis through over-quotation or through isolated quotation; (4) the circular identification of discourses and mental constructs; (5) false survey; and (6) analysis that consists in simply spotting features. We show, by applying each of these to an extract from a recorded interview, that none of them actually analyse the data. We hope that illustrating shortcomings in this way will encourage further development of rigorous discourse analysis in social psychology.


Introduction
In the past fifteen years, discourse analysis has had a major impact on social psychology, especially in Britain. It has introduced new methods of research, new ways of conceptualising research questions and new ways understanding the nature of psychology itself. In this time it has gone from a marginal perspective developed by a handful of scholars to an approach that is represented in wide range of different empirical and theoretical journals, seen in different conference presentations, and developed in a growing body of PhDs. For an increasing number of academics discourse analysis is the prime way of doing social psychological research.
At the same time, there has been a proliferation of forms of discourse analysis. The geography of the discourse terrain is complex, with widely disparate assumptions being made about fundamental topics such as method, theory, the nature of discourse, the nature of cognition, and the nature of social structure. We will not be mapping this terrain here (but see, for example, Jaworski & Coupland, 1999;van Dijk, 1996;Wetherell et al., 2001). To give a sense of the variety, we note that in social psychology some discourse work is close to conversation analysis (for accounts of which, see Hutchby & Wooffitt, 1998;Sacks, 1992), while some has been influenced by critical discourse analysis (Fairclough, 1995) and post-structural and Foucauldian thinking (Hodge & Kress 1993) among other schools of thought. In social psychology, analysts have focussed on the actual conduct of conversational interaction in institutional or mundane settings (for reviews and examples, see, for instance, Antaki, 1994;Edwards & Potter, 1993;Edwards, 1997, Widdicombe & Wooffitt, 1995, and on talk and written text in the study of ideology and social critique (again, for reviews and examples, see Billig, 1992;Burman & Parker, 1993;Hollway, 1989;Parker, 1992;Wetherell & Potter, 1992).
There are sometimes tensions between these different aims, and the styles of work associated with them (see, for example, Nightingale & Cromby, 1999, Parker & Burman (1993 and the extended debate in the pages of Discourse and Society [Billig, 1999a;Schegloff, 1997Schegloff, , 1998Schegloff, , 1999Wetherell, 1998;Stokoe & Smithson, 2001]). Our aim here is not further to rehearse these debates and issues, but to highlight some methodological troubles that are visible from whatever discourse perspective one adopts.
We are aware that some of what we will be arguing is already familiar in the broader social science literature on qualitative methods in general (e.g. Coffey & Atkinson, 1996;Gilbert, 1993;Silverman, 1997;2001). We are concerned with the variable quality of discourse work specifically in our own discipline, and hope to contribute to the literature that has already grown up within it. General overviews can be found in Coyle (1995), Gill (1996), Potter & Wetherell (1987), Potter (1996cPotter ( , 1997, Wood & Kroger (2000) and Wooffitt (1993). Billig (1997a) and Potter & Wetherell (1994) work through the process of analysis with a specific example. Potter & Wetherell (1995) discuss the analysis of broad themes and interpretative repertoires drawn on in interview talk. Potter (1998) compares grounded theory, ethnography and discourse analysis in the analysis of clinical materials. Edwards & Potter (2001) discuss discursive psychological analysis of the role of psychological talk in institutions. Yates, et al., (2001) introduce and compare a range of different approaches to analysing discourse. All of these have positive things to say about doing analysis. But they leave implicit what is not analysis. That is what we want to make explicit in this paper.

Well-and poorly-founded criticism of analysis
It may be questioned why we feel the need to state what might seem obvious. There are basically two reasons. The first is that discourse analysis still can be misunderstood by those who have been schooled in quantitative analysis. It might appear to quantitative researchers that 'anything goes' in qualitative work in general, and discourse analysis in particular. However, that certainly is not the case, though we believe that the quality of discourse work has been variable -as variable, of course, as any other kind of work. It is not surprising that this is so. Although an increasing number of researchers are producing discursive theses, reports and articles, they sometimes have to do this through self-education, possibly in institutional settings characterised by incomprehension of, or even direct hostility to, discourse analysis.
The second reason is that work continues to be produced, submitted to journals and sometimes published that embodies basic problems. When we compared notes from our experience of refereeing journal submissions across a wide range of discourse and social psychology journals we noticed that a particular range of shortcomings appeared with great regularity.
Under these circumstances, it is important to make a statement that reiterates and emphasises the analytic basis to discursive studies. Such a statement might have value for those who are learning the trade. In addition, it might help prevent researchers from producing work that might lend credence to the quantitative researcher's dismissal that, in discourse analysis, 'anything goes'. This basic position is not out of line with those who comment on the study of discourse in other disciplines. David Silverman, for example, makes similar critical points in the conclusion to his recent book on analysing qualitative data in social sciences in general (Silverman, 2001). In the domain of journal publishing, Teun van Dijk, in the first editorial of Discourse and Society, the journal founded to study discourse and its relations to social processes, goes out of his way to emphasise the need for 'explicit and systematic analysis' based on 'serious methods and theories ' (van Dijk, 1990, p. 14).
What we shall do in this paper, then, is to identify things that might superficially give the appearance of conducting discourse analysis in social psychology, but do not in fact do so. We have collected together six such non-analyses: (1) under-analysis through summary; (2) under-analysis through taking sides; (3) under-analysis through overquotation or through isolated quotation; (4) the circular identification of discourses and mental constructs; (5) false survey; and (6) analysis that consists in simply spotting features. It would be invidious to single out one or even a small number of studies as representing these problems (although it is not hard to find such studies). Instead we will sketch out the problems in a more general way, and illustrate them in relation to a single piece of data.

An extract to work through with examples of non-analysis
Discourse analysis can be performed on a wide variety of talk and text. For convenience we reproduce an extract from an interview, but we do not mean to imply that interviews are specially preferred sources of data. We will reproduce the extract (on the nature of marriage) here in its entirety, as it will be drawn on repeatedly in the course of the paper.
The data have been transcribed using conventions, now common in much discourse analysis, developed by the conversation analyst Gail Jefferson (see Hutchby & Wooffitt, 1998, or ten Have, 1999 for details; a brief summary is provided in an appendix). (

1) Under-Analysis Through Summary
Qualitative analyses share something important with quantitative analyses in that they both want to do something with the data. Neither is content merely to lay the data out flat.
A quantitative researcher who merely presents the raw data from subjects in an experiment without putting it to some sort of statistical testing would hardly be said to have analysed it. So it is with qualitative data.
An interview, doctor's consultation or television talk show might be transcribed. Even complex transcription notation might be employed, to indicate the rise and fall of intonation or pauses and hesitations, as in the data extract presented above, taken from an interview with a young woman. We recognise that what to put in a transcript, and how to notate it, are far from easy questions, and that in that sense 'theory' cannot be kept out of transcription. The point is a venerable one in discourse analysis (see, for example, Edwards & Lampert, 1993;Jefferson, 1985;Psathas & Anderson, 1990). For our purposes here, however, we mean to warn against the notion that transcription can be a replacement of, or substitute for, analysis. Transcription prepares the data for analysis.
However, it is not analysis in itself.
Analysis must mean doing something with the data, but not just anything. A quantitative analyst who presents a selection of their raw data in some graphical form, hoping that the reader might see a trend or a pattern, would not have done anything statistical on their data. A qualitative analyst will be doing the equivalent if they present their data as a prose summary. However, summarising the themes of what participants might say in an interaction typically does not involve any analysis of the discourse that they are using. A summary is likely to lose the detail and discursive subtlety of the original. The summary will be shorter and tidier. It will be phrased in the analyst's words, not those of the original speakers (or writers). It will lose information and add none. Under-Analysis through Summary, then, is the first of our list of things that are not discourse analysis.
The failures of summarising can be seen in relation to our interview extract. It would be possible to offer a summary of the main themes that the Respondent seems to be saying.
One might say: 'the Respondent is expressing a belief in the desirability of marriage and the necessity to work hard to maintain marriage relationships; he stresses that in his view the demonstration of commitment is important and that divorce has become too easy'.
Such a summary does not provide anything extra. In fact, it provides less: much of the complexity of the speaker's comments is lost. For example, at a relatively gross level, such a summary does not draw attention to his apparent switch around in lines 64 and 74 and following, when he appears to concede that marriage doesn't necessarily mean that one will be together in forty years. A summary of the switch does not analyse what effects the switch might have and precisely how it was presented. It misses, for example, the rhetorical and discursive effects of saying "in sort of (0.7) forty years time" and not just "forty years' time". At a rather more fine grain level, such a summary does not draw attention to the laughter that accompanies the interviewer's question (line 6) and the trouble shown in the understanding check (line 8) and the various aspects of 'dispreference' shown in the start of the participant's response (lines 11-12). Such examples can be multiplied by as many utterances as there are in the text.
In general, summarising does not offer an analysis of the discourse that the speaker was using. The analyst in the summary might be drawing attention to certain themes, pointing to some things that the participant(s) said, and not to other things. However, this pointing out is not discourse analysis. It might prepare the way for analysis, but it does not provide it. It can impede analysis, if it distorts the original by presenting the speaker as being more consistent, smoother and briefer than they might have been. And it will distort if it is freighted with heavy implication: if the summary attributes beliefs, policies and so on to the speaker as a short-hand, then it risks changing the object of analysis even before the analysis starts in earnest.

(2) Under-Analysis Through Taking Sides
If data analysis requires that the analyst offers something additional beyond presenting or summarising the data, then this does not mean that every additional offering is analysis.
It certainly does not mean that every added element of analysis is discourse analysis. In some writing one sees the additional offering of the analyst's own moral, political or personal stance towards what the quoted speaker or text is saying. This on its own is not discourse analysis.
There is a debate amongst discourse analysts whether analysts should take positions with respect to the material that they study. It is not our intention to enter into that debate.
Nor, indeed, do we agree amongst ourselves on this issue. What we do insist upon, however, is that position-taking -whether analysts align themselves with, or critically distance themselves from, the speakers whom they are studying -is not analysis in itself.
Sympathy and scolding (either explicit or implicit) are not a substitute for analysis.
When the analyst is primarily engaging in positioning themselves vis-à-vis their data, The data presented above do not show an example of a powerless, victimised speaker.
Nevertheless, an analyst might wish to take a stance vis-à-vis the issues being discussed.
For instance, the analyst might wish to align with the sort of position that the speaker is outlining. The analyst's summarising might contain pointed references. It might be said that the speaker 'realises' or 'appreciates' how relationships need hard work. Or the analyst might add that the respondent 'takes seriously' the idea of marital commitment and 'sees the problems' of divorce. Such language might subtly, or not so subtly, indicate that the analyst is aligning himself or herself with the position taken by the respondent. The crucial point is that such alignment of support on its own does not constitute analysis of the discourse used by the speaker.
By the same token, a critical dis-alignment by the analyst does not constitute analysis.
For instance, an analyst from a radical feminist perspective might be critical of the institution of marriage, claiming it to be a patriarchal institution. The analyst might quote or summarise the respondent in order to distance themselves from the position he seems to be taking. The analyst might summarise the respondent's comments and add that the

(3) Under-Analysis Through Over-Quotation or Isolated Quotation
There is a particular form of under-analysis that seems, at first sight, to avoid the dangers of Under-Analysis by Summary. Under-Analysis by Summary fails to get to grips with the text. As it were, it leaves the text behind. There is a reverse fault when the analyst fails to get beyond the text or texts. This can happen if the analyst is doing little more than compiling a list of quotations snipped from the data. Quotation, like summarising, is not discourse analysis in itself.
Under-Analysis through Over-Quotation is often revealed by a low ratio of analyst's comments to data extracts. If extract after extract is quoted with only the occasional sentence or paragraph of analyst's comment, then one might suspect this type of underanalysis is happening. In the example of the interview about marriage, an analyst might think of chopping up the whole extract into quotable extracts, omitting the interviewer's questions. After presenting the quotations, the analyst might summarise the collection of quotes with a comment such as 'so we can see that the respondent had strong views about the importance of marriage and commitment'. This would not be analysis. The list of quotes divorces the utterances from their discursive context, with the result that it would not be possible to analyze them as responses to questions.
More typically, Under-Analysis through Over-Quotation is liable to occur when the analyst is piecing together responses from different speakers. For instance, the analyst might wish to show that a number of interviewees had responses rather like the one in our extract. Selective quotation from such respondents might be given. There can be analytic and theoretical reasons for presenting profiles based on piecing together such quotations. However, this profiling is not normally of itself discourse analysis, for again it does not of itself get down to the business of actually analysing in detail the discourse that is used. Indeed, as has been mentioned, the over-quotation may impede certain forms of discourse analysis by removing utterances from their discursive context. Two tell-tale signs of Under-Analysis through Over-Quotation would be the small amount of analyst's writing in proportion to the large amount of quotation, and the tendency of the writing to refer to the quotations rather than analyse them.
In addition to Under-Analysis by Over-Quotation is the related error of snipping out a single quote and allowing it to 'stand for itself' as if it required no further comment. This is Under-Analysis through Isolated Quotation. An author might feel that their argument can be illuminated by a quote from their respondent or from the textual source they are working on. The quote is not actually analysed, but set up as self-evidently consistent with, or even proof of, the author's argument. For example one might extract lines 86-90 from the material in the interview extract and simply place it in the text as a self-evident specimen (say, a specimen of the discourse of 'modern times'). At best, this may be a rhetorically powerful embellishment of an analysis done elsewhere; but Under-Analysis through Isolated Quotation is not itself analysis.

(4) The Circular Discovery of (a) Discourses and (b) Mental Constructs
Compiling quotations into a profile can be part of a discourse analysis. For instance, an analyst might be seeking to investigate whether speakers, in framing their individual utterances, are using commonly shared discursive resources. Some analysts examine how particular rhetorical and conversational devices are used in specific contexts. Some researchers examine how speakers may be using shared patterns of understanding or interpretation. There are a variety of terms to describe the sort of discursive resources that speakers may share. For instance, Potter and Wetherell (1987) refer to shared 'interpretative repertoires', Billig et al (1988) and Billig (1991) to 'ideologies' and Parker (1991) to 'discourses'. Each signals a different set of theoretical and analytic assumptions. Accordingly, some discourse analysts will consider it a matter of theoretical and methodological importance to show how particular utterances are themselves formed out of wider, socially shared 'repertoires', 'ideologies', 'discourses' etc. The analyst might present a profile of quotes in order to show how different speakers might be drawing upon common repertoires etc.
In theory, such profiling would seem to fit the requirement of discourse analysis. An analytic extra is being added. The reader is not merely being informed that the speakers made these utterances, but the additional claim is made that all these utterances have something in common, being manifestations of a shared pattern of talking. The problem comes when care is not taken to substantiate the claim. Again, the data cannot be left to 'speak for itself', as if a series of quotes is sufficient in itself to show the existence of the repertoire, ideology or discourse. Moreover, the analyst runs the risk of circularity if the socially shared entities are cited in explanation for the utterances. This is just the concern expressed by Widdicombe when she writes: the analytic rush to identify discourses in order to get on with the more serious business of accounting for their political significance may be partly responsible for the tendency… to impute the presence of a discourse to a piece of text without explaining the basis for specific claims (Widdicombe, 1995, p 108).
Widdicombe then goes on to make a strong case for her observation by re-analysing another writer's data, and, in being more explicit in her analysis, coming to very different conclusions about it.
To return to our interview example, quotations could be selected from the speaker's comments about marriage and relationships, requiring commitment. Indeed, other speakers might be quoted, if the analyst is suggesting that they are all talking along the same lines. On the basis of such quotations, the analyst might then claim that the speakers are using the repertoire, ideology or discourse of 'marital commitment'. The analyst may even claim to have 'discovered' the repertoire / ideology / discourse on the basis of the interview material.
If that is all the analyst is doing, then these terms function merely as summaries. They add little if anything to the analysis of the utterances, for they are only handy ways of describing the common features that the analyst is claiming to summarise. However, if the analyst then moves towards an explanation of the quoted discourse in terms of these entities, then a step towards circularity is taken, and we have Under-analysis through Circular Discovery. The quotes, which provide the justification for claiming the existence of a 'marital commitment discourse' (or repertoire, or ideology) are then explained in terms of this entity. Such circularity would occur if the analyst, having quoted extracts to claim the existence of a 'marital commitment repertoire / ideology / discourse', then goes on to imply that the speakers made those particular utterances because they shared this discourse, repertoire or ideology.
This is not to deny that there can be discursive analyses of repertoires, ideologies or discourses. Such analyses must provide some extra elements. The analyst might, for example, want to show how particular repertoires, ideologies or discourses are drawn upon to deal with specific features of the conversational interaction, such as particular moves from the interlocutor; or that when speakers use this repertoire in a general way, they will tend to qualify it by introducing counter-themes (as the speaker does in lines 64 and following). Such an analysis would draw attention back to the details of the talk, as the analyst seeks to relate specific use of themes to specific conversational junctures.
Much more will be required than quotation and assertions of commonality to sustain such an analysis. The analyst would need to demonstrate the commonalities in detail.
Alternatively, the analyst might seek evidence that is beyond the specific conversational extract, to substantiate the claim for the existence of such repertoires, ideologies or discourses. The analyst would need to state something about the nature of these entities.
For instance, historical evidence might be cited to show the origins and development of various cultural patterns of talk. The particular analysis would aim to show how these wider patterns of talk are mobilized by the speaker in the particular context of the interview or conversation that is being studied. This wider historical perspective, then, would lead back to questions of why particular conversational manoeuvres are being made and what speakers are doing by using these common patterns of talk at these conversational junctures. Again, the perspective would lead back to examining the details of interaction. Indeed, it must do so, if the dangers of circularity and mere summarising are to be avoided.
In addition to the circularity of identifying discourses there is a parallel danger of circularly identifying mental constructs. In orthodox social psychology, analysts tend not to explain people's views in terms of socially shared constructs, such as repertoires, ideologies or discourses. Instead, the focus has been to explain external actions, including discursive actions, in terms of inner mental processes, such as 'attitudes' or 'schemata'. Discursive psychology has tried to combat this move, insisting that such inner processes are only hypothetical constructs whose existence the analyst is inferring from the outward use of language. In this respect, discursive psychology rests upon a particular philosophy of mentality. Some discursive psychologists stress the philosophical heritage of Wittgenstein and Austin (Billig, 1999b, Harré & Gillett, 1994Edwards, 1996;Potter, 2001) and some Volosinov (Billig, 1997b) and even classical rhetoric (Billig, 1996). Whatever the philosophical origins of the stance, the implication is clear: the discursive psychologist should resist positing mental entities and should concentrate upon examining the use of psychological language in discourse.
A failure to take seriously this philosophy of mind can lead to an analogous sort of circularity that was discussed in the previous section. For instance, an analyst might concentrate upon quotes in data where speakers use psychological phrases such as 'I think' or 'I feel' and then claim that these statements give direct access to the person's inner thoughts or feelings. The circularity comes into play when the analyst cites these inner thoughts or feelings as reasons why the speaker speaks as they do.
The processes. Instead, such phrases would need to be analysed discursively. One might say that the interview situation is one in which the respondent knows that they are expected to engage in the discursive business of 'giving views'. In order to avoid appearing dogmatic and to demonstrate recognition that others have opposing opinions, speakers will use such phrases as 'I believe', 'I think'. Such an analysis of the rhetoric of giving views, then would look to see how the speaker manages the dilemmas of presenting opinions forcefully but without seeming to be dogmatic. to state that the speaker is expressing their beliefs is either to risk under-analysis through summarising or making the circular discovery of an inner belief.

(5) False Survey
There is a danger of extrapolating from one's data to the world at large. This error is not unknown in quantitative research, of course. It may be avoided by explicitly surveyoriented studies, but is not uncommon in experimental social psychology when findings are subtly generalised from the sample of the experiment (say, a set of North American undergraduates) to the universal categories they are supposed to represent (women, high achievers, people with a certain attributional style). Discussion sections of experimental papers sometimes use such unqualified terms, with the logical implication that they encompass all members of that category.
The same danger of False Survey lurks for qualitative work that discovers that certain respondents use certain discourses or ways of speaking. It is fatally easy to slip into treating one's findings as if they were true of all members of the category in which one has cast one's respondents. For example, an analyst reading our interview extract might see, in the respondent's way of talking, a 'traditionalist discourse of marriage'. They might then be tempted to attribute that discourse to all people in his position ('non-University-educated young women', if that was the demographic information supplied along with the extract). This attribution might be done explicitly, but is still more likely to happen unconsciously, in the way the writer uses demographic categories to refer to the people in their data.
Probably few discourse analysts want or intend explicitly to be reporting surveys; but without care, their reports may give that impression. Such a fault makes the work an easy target for the quantitatively-minded, who will properly see it as failing to supply appropriate evidence for its claims. If a survey is wanted, survey tools must be used.

(6) Under-Analysis through Spotting
If discourse analysis demands an attention to the details of utterances, this does not mean that all such attention qualifies as satisfactory discourse analysis. Analyses provided by discursive, conversation and critical discourse analysts have, over the past twenty-five years, noticed and labelled a wide variety of conversational and rhetorical procedures.
Anyone engaging in these sorts of analyses should properly acquaint himself or herself with such work. They should be able to recognize these conversational features in data extracts. The same is true of rhetorical tropes in printed persuasive materials and so on.
However, the recognition of features does not constitute analysis, at least at a research level. It may be appropriate in training exercises as one seeks to acquire the skills of analysis. But research does not, and should not, consist principally of feature-spotting, just as analysing the history and functions of the railway system cannot be accomplished by train-spotting. Thus there can be Under-Analysis through Spotting.
The interview extract contains many features known to conversation analysts. As the interviewer says mm or yeh they provide 'continuers' which acknowledge the respondent's turns. Thus by saying mm a speaker can concede their turn at transitionrelevant points. Similarly when the interviewer asks a question, they are making the first move in an adjacency pair that expects an answer. These and other well-known structural features of the talk can be spotted in this extract. Indeed, such spotting is possible in virtually any such extract of interactional talk, just as the rhetorician will be able to spot familiar tropes in a piece of formal speech-making.
An analysis that consisted primarily of such spotting would not count as original research. It would be like a training exercise in running a well-known illusion such as the it's doing in this particular set of materials. Good analysis always moves convincingly back and forth between the general and the specific.

Concluding Comments
It is worth revisiting the two reasons we had for writing this paper. One is to help those who approach DA enthusiastically, but in an environment where there is less support than there would be for more traditional methods of analysis, and so less opportunity to test and refine methods among sympathetic colleagues. The other is to scotch the sort of errors that give comfort to the traditionally-minded who accuse DA of 'anything goes'.
We hope we have shown the difference between something that is discourse analysis -of whatever sort -and something that is not. Writers are not doing analysis if they summarise, if they take sides, if they parade quotes, or if they simply spot in their data features of talk or text that are already well-known. Nor are they doing analysis if their discovery of discourses, or mental constructs, is circular, or if they unconsciously treat their findings as surveys.
We should be at pains to say that we do not think that identifying these inadequacies tends positively toward any one particular level or style of discourse analysis. What it does is show up how some ways of writing have the sheen of analysis without its substance. We have deliberately stopped short of saying what does count as analysis, because of the variety of directions in which analysis can go, and because much more has been written on this elsewhere. Perhaps it is safe to say that analysis means a close engagement with one's text or transcripts, and the illumination of their meaning and significance through insightful and technically sophisticated work. In a word, Discourse Analysis means Doing Analysis.