The Social Life of Methods

The Social Life of Methods is an interesting programme of research undertaken by Evelyn Ruppert, Mike Savage and John Law. This is orientated towards what the authors term the ‘methodological complex’: a dominant way of understanding method, tangled up in a particular division of labour, which precludes the investigation of methods as objects in their own right. In contrast to the ensuing view of methods as neutral tools for producing knowledge about the social world, they want to study the ‘social life’ of these methods and consider how this has shaped their emergence. In this paper they talk about the dual sense in which methods can be said to have a social life:

Methods are shaped by the social world

Methods don’t come into being without a purpose and methods don’t come to prominence without advocates. In this sense, we can link techniques of map-making and surveying to the needs of nascent nation states to map their territory. The census came about as a result of the need to constitute a governable national population, measurable and therefore susceptible to intervention. For instance, Servius Tullius, the sixth king of Rome prior to the republic, instituted a census and divided the common people (plebs) into social classes: these were defined hierarchically, with rights and responsibilities ensuing from what they could contribute to the city. All the sixth class were seen to be able to contribute was their children (source). The UK census began in 1801 and, with the exception of 1939 and 1966, has been undertaken every ten years since. The present government has discussed replacing the census: “There are, I believe, ways of doing this which will provide better, quicker information, more frequently and cheaper” opined Cabinet Office Minister Francis Maude in 2010. The technologyused to conduct the census has changed radically, with a move towards handheld computers and online completion perhaps giving rise to an entirely online census supplemented by existing data or even its complete abandonment. The classifications used in the census change, existing classifications change, for instance ‘pensionable age’ increases, and new classifications are introduced, such as ‘ethnic group’. In fact this didn’t appear in the census until 1991, with Margaret Thatcher having dismissed its proposed introduction in the 1981 census, seeing it as a left-wing dogma incompatible with her planned direction for the UK (weirdly placing her in alignment with some on the radical left who were suspicious of the government collecting such data at a time of racial tensions and with a rising far-right). The purposes underlying the census change, both in terms of the intentions of the government but also the uses to which groups like academics and charities make of the census data, leaving different groups with vested interests who help shape its future direction. The vast majority of the population of England and Wales completed the census form and, while there are legal sanctions for a refusal to complete it, this degree of compliance necessitates an extensive public engagement operation and makes winning public supportcrucial. In this sense, the logistics of the census itself are of interest, given how all this activity is compressed into the space of a few weeks.

What Ruppert, Savage and Law are arguing is that we miss all the complexity if we simply treat the census as a neutral tool to study an object. It also helps constitute that object, ‘chopping up’ a population in various ways according to political needs and social contingencies, though the ensuing data cannot be dismissed as simply a product of the exercise. The census helps constitute a ‘population’ susceptible to intervention. We confront similar questions when considering the UK government social surveys, which began in 1941 during the second world war.

while the census made people inhabiting a territory into a national population, subsequently that population reality could be further calibrated through the technique of sample statistics in the mid-twentieth century. So there’s a history of method to be told here. In the US, for instance, sample surveys grew up with the Gallup polls of the late  1930s, and then with agencies such as the Department of Agriculture during the Second World War. Mike Savage traces their analogous though later rise in the UK. He shows, for instance, how during the Second World War, the Government Social Survey became a key instrument in generating knowledge of the circumstances and concerns of the British population. It proved popular in part because its methods of anonymous sampling avoided relying on known informants, in a way that had attracted popular opprobrium. The government proved a key player in promoting survey research into the 1960s, seeing it as part of a modernising form of government that no longer needed to rely on the views of the ‘good and the great’. It proved important in shaping educational reform in the 1960s, with surveys being used to explain the social selectivity of grammar schools. As Hilton and Savage have shown, these methods were thus embraced by a technocratic middle class, seeking to distinguish themselves from older gentlemanly intellectuals

http://research.gold.ac.uk/7987/1/The%20Double%20Social%20Life%20of%20Methods%20CRESC%20Working%20Paper%2095.pdf

To talk of the social life of methods entails inquiry into the purposes underlying methods and advocates for particular methods, as well as how these change over time. These purposes might not be lofty or consequential ones. The focus group was largely absent from the academy until the 1980s but had been developed and refined as a tool for market research. Likewise corporations have led the way in the analysis of consumer transactional data obtained through mechanisms such as store cards. Considering the social life of such method entails looking at the origins of these methods, their purposes and advocates, as well as how they have changed as they moved out of the commercial sphere and into the academic one. What assumptions are loaded into their use? How are their operations understood by those who us them? What status is ascribed to the data that is produced using them?

Method contribute to shaping the social world

The notion that methods are shaped by their context is relatively straight-forward. It might be under-theorised and understudied but it’s unlikely to be a contentious claim. However the authors argue that methods also contribute to shaping that world. This notion of the performativity of method can be slightly harder to grasp. They offer the sample survey as an example:

Take an example: the sample survey. This is not any example, because the sample survey is one of the most legitimated methods in use today. In the UK University teaching is assessed by the scores academic departments achieve in the National Student Survey. Our measures of crime are dependent on the British Crime Survey, which ‘corrects’ for the under-reporting of much crime to the police. The inflation rate is determined by responses to the Family Expenditure Survey, and so on.

The survey works by first sampling people. And then it works by asking them questions about matters of fact (like age, gender, income bracket, religious affiliation, lifestyle choices) and matters of opinion (such as the performance of the government, attitudes to abortion, or meat- eating). We might think of a survey as a bit like a methodological package deal. Like all package deals it has great virtues. It tells those who advocate it something about how people are planning to vote, or, say about their attitudes to global climate change. It’s also exceedingly useful because the basic methodological thinking doesn’t have to be done again. Surveying, after all, is a bog-standard method that has been industrialised and routinised. Its standards of quality control have been set, are widely agreed, and ethical guidelines are in place to police them.

At the same time, like all package deals, it is indeed standardised. You get to see parts of social reality in particular ways, while you don’t see things that escape the package. Or more strongly (and now we’re getting to the point we want to make about constituting), it may be that you get to perform certain kinds of social realities whilst not performing others. You’re actually bringing realities into being while you’re shutting down others

http://research.gold.ac.uk/7987/1/The%20Double%20Social%20Life%20of%20Methods%20CRESC%20Working%20Paper%2095.pdf

I think the framing here is confusing. But the underlying point is an interesting one. On the most basic level, people (sometimes) act on the basis of research and so what the methods open up and what they close down shapes the ensuing action. In so far as research methods helps shape how people see the social world then methods have consequences, including the oversights and blind spots latent to any particular method. As Einstein (allegedly) put it, “Everything that can becounted does not necessarily count; everything that counts cannot necessarily becounted”. Things that matter may not show up and, conversely, things that don’t matter may show up’. However the authors are making a much deeper point than this. Research methods construct individuals as having capacities and properties. To ask someone about their choices presupposes their capacity to choose. To ask someone about their attitudes presupposes that their reported attitudes at one moment can tell us something about their real attitudes at another i.e. it assumes their relative stability. Methods also construct collectives as having certain properties:

Individuals are abstracted from a place and then taken as representatives of that spatially delimited place. A particularly important example is the modern nation. There is no independent nation which does not appropriate to itself the ability to conduct censuses and surveys on its national population delimited by its sovereign boundaries. It is not incidental that new and developing states see the capacity to conduct national censuses and surveys as central parts of their ‘statehood’

http://research.gold.ac.uk/7987/1/The%20Double%20Social%20Life%20of%20Methods%20CRESC%20Working%20Paper%2095.pdf

This process of abstraction constructs a ‘population’ composed of individuals. In measuring this (newly constituted) population, they become susceptible to intervention. Who needs what? Who can contribute what? The answers to these questions may have become far more complex but the questions themselves are still rather similar to the ones addressed by Servius in pre-Republic Rome. These abstractions enter into the lives of individuals in terms of the expectations placed upon them, the facilities offered to them and the interventions they are subject to. Consider for instance, the vast auditing apparatus involved in the tax system. Methods address subjects as beings with particular sorts of competencies and encourage them to respond as such: for example a life history interview makes a whole range of assumptions about the coherency of an individual’s trajectory and their capacity to recall it. These ontological assumptions may be explicit but they’re often implicit. The latter case makes it even more imperative that we analyse methods and extract these latent claims.

The point the authors are making is that research methods have to “pick and choose between different individual and collective realities”. Methods assume the stability of their object but they also contribute to the reproduction or transformation of that object by foreground certain aspects and backgrounding others. They offer this as an argument about all methods, as opposed to quantitative or qualitative ones. We discover things about the world through methods but we also act in and on that world in way that contributes to changes in the objects we study.

First we’re saying that they make discoveries about the world, and that those discoveries may surprise us. That’s why we conduct interviews and surveys and all the rest. But also, and counterintuitively, we’re saying that they also make more or less self- fulfilling assumptions about the character of the social world. And that in so doing they tend to constitute it, so to speak, below the radar in ways that we scarcely notice. In short, that they tend to produce what John Law calls collateral realities: that is, realities that we don’t think about very much but that we’re all busy reproducing as we go about the daily methodological work of gathering and analysing data ‘about’ the social

http://research.gold.ac.uk/7987/1/The%20Double%20Social%20Life%20of%20Methods%20CRESC%20Working%20Paper%2095.pdf

This suggests that studying methods as objects in their own right encompasses theory: what theoretical assumptions are loaded into methods, how do they condition the enactment of the method and the status ascribed to the results? It encompasses history, in so far as that studying the emergence of the methods, its advocates and purposes, helps us understand how that method came to be constituted in the way we find it. It involves technique, in the sense that we must attend to the operation of the method but without bracketing it off in the manner implicit in the ‘methodological complex’.

The suggestion that research methods have a double social life seems uncontentious to me. The claims being made are that (1) methods are shaped by the social contexts in which they emerge and (2) methods in turn help shape those contexts. So research methods should not be understood as neutral tools developed in isolation from the social world they are orientated towards. Instead, we need to recognise the manner in which methods are shaped by that world and in turn contribute to its shaping. This involves rejecting what Law, Savage and Ruppert describe as the ‘methodological complex’:

It assumes that methods are tools for learning about the social world. That this is what they are. End of story. We see this in methods courses. Juxtaposed and differentiated both from theory, and from substantive courses, these tell us about techniques for knowing the world. Which to choose. How to use them. How to analyse data. And how to present it.

There’s nothing wrong with this in certain senses: in social research indeed we need methods, and it’s not a bad idea to use those methods properly. But to think of methods in this way – simply as appropriate tools – involves consequences, some of them unanticipated, which create a baggage which can be heavy, even burdensome. We can distil this as ‘the methodological complex.’

http://research.gold.ac.uk/7987/1/The%20Double%20Social%20Life%20of%20Methods%20CRESC%20Working%20Paper%2095.pdf

This ‘methodological complex’ entails a particular division of labour for empirical research and a particular conception of how research can be undertaken. Theory, methods and substance are construed as distinct spheres of activity. Research questions are derived from theory, inviting the use of methods to address them in relation to distinct areas of substance. They also argue that this involves the ontological presupposition of a stable world, with definitive features that can be reported and turned into data:

We’re distinguishing between the world on the one hand, and representations of that world on the other. In this way of thinking it’s methods that bridge the gap. If we get those methods right then our representations will match the realities of the world. Tools have a better or worse capacity to do the job at hand. They  will, as the philosophers of science say correspond to it; or at least (this is what the  pragmatists say) they will describe it sufficiently well to be treated as accurate. This means that they are tools for handling the world. If we get them wrong then our accounts of reality, our data, will be flawed.

http://research.gold.ac.uk/7987/1/The%20Double%20Social%20Life%20of%20Methods%20CRESC%20Working%20Paper%2095.pdf

I’m hostile to any attempt to refute naturalism on this basis, arising from the obviousness with which these points can be reconciled to a critical naturalism (see Roy Bhaskar’s Possibility of Naturalism). But I think it’s important to explore them because the analysis seems entirely plausible to me, even if I’m sceptical about the prescriptions many would draw on the basis of them. I also agree that, as the authors put it, “oscillates between an objectivist concern with ‘bias’ and a humanist response which seeks refuge in an ‘ineffable’ human moment which somehow lies outside this purview of representational methods”. Roy Bhaskar makes a similar point when he argues that positivism and hermeneutics share a view of natural science, framing reality in terms of a schism between matter andmeanings with the former being the domain of the natural sciences and the latter the domain of the (hermeneutical) social sciences. In fact I find their analysis congruent with Bhaskar’s, complementing it productively as a result of a sightly different focus:

By reducing issues to questions of technique, it allows different parties to come together around some kind of shared project, whatever their goals,values, orientations and identities. If we need to create random samples, then this is because it is important to avoid undistorted samples. If it is dangerous to avoid recruiting so-called professional participants to our focus groups, then this is because we’re looking for people who are naïve and untutored in appropriate ways. If the ethnographer needs to avoid the outsiders who flock to talk with her when she first arrives in the field, then this is because she’s on the lookout for gatekeepers or people at the core of the community rather than people with grudges on the periphery. We learn all these things in a million different versions in the hope of reducing bias; in the hope of knowing and describing the world accurately. This search to avoid bias and to use our ‘tools’ more effectively is pervasive, indeed ubiquitous. We share it. But it then also leads to an automatic response, from even the most positivistic researcher, about ‘what is left out’ by any specific method.

http://research.gold.ac.uk/7987/1/The%20Double%20Social%20Life%20of%20Methods%20CRESC%20Working%20Paper%2095.pdf

Their point is not that a concern to use tools effectively is wrong but rather that an exhaustive treatment of methods in these terms serves to preclude consideration of others aspects of methods that are salient to the practice of social research. Their project seeks to recognise that “methods are fully of the social world that they research; that they are fully imbued with theoretical renderings of the social world” and to think through the implications of this for how we understand them. These are the questions that we lose sight of if we focus on using tools effectively. As I understand their point, they accept that tools are used in the production of knowledge but argue that to understand these ‘tools’ we need to stop and consider them as objects in their own right. Their point is not a trivial constructionist one, such as to assert that ‘methods are socially constructed’ (well of course they are, would anyone argue that methods are natural kinds?) because to do so would enact precisely the oscillation between objectivism (there is a world out there with fixed properties which we use neutral methods to investigate with a greater or lesser degree of efficacy) and subjectivism (there are first-person human realities which are intrinsically beyond the purview of objective representation using neutral methods) – in critical realist jargon, I think they’re proposing a systematic framework for investigating the transitive dimension of social science.


Categories: Outflanking Platitudes

Tags: , , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *