how we know: epistemological foundations of information systems research

I just finished reading a pretty fabulous paper: Orlikowski, W.J. and Baroudi, J.J., 1991, Studying Information Technology in Organizations: Research Approaches and Assumptions, Information Systems Research, 2(1): 1-28. basically, they do a survey of 155 papers from 1983 – 1988, arguing that the large majority adhere to a positivist approach, a small minority follow an interpretive approach, and none adopt a critical approach. the reason this paper rocks so much is the cogent, clear, concise, summaries it provides of these three epistemological traditions, citing both strengths and weaknesses, complete with case examples of each. quick overview:

positivist – reality exists external to the researcher, and phenomena of interest accessible to through observation via sensory organs or measuring instruments; “the phenomenon of interest is single, tangible and fragmentable;” “he researcher and the object of inquiry are independent;” “nomothetic statements, i.e., law-like generalizations independent of time or context, are possible” (Orlikowski and Baroudi, p. 9); the goal is the discovery of general laws or universal principles.

interpretive – “reality, as well as our knowledge thereof, are social products and hence incapable of being understood independent of the social actors (including the researchers) that
construct and make sense of that reality;” assumptions and potential limitations clearly stated up front; “the aim of all interpretive research is to understand how members of a social group ... enact their particular realities and endow them with meaning” (ibid., p. 13); “the researcher in part creates the reality she is studying” (ibid., p. 15).

critical – “social reality is historically constituted;” rather than studying or explaining reality, “critical researcher attempts to critically evaluate and transform the social reality under investigation” (ibid., p. 19); a major component is exposing, and likely attempting to overturn, currently unseen or unvoiced assumptions and hidden contradictions; “interpretation of the social world is not enough. The material conditions of domination need also to be understood and critiqued” (ibid., p. 20); major focus on socio-economic class, labor relations, and the conflict therein.

another summary and comparison with other work can be found here. the article is quite insightful and chock full of demonstrative examples. I highly recommend you go read it.

furthermore, it was particularly useful for me, as I’ve recently been thinking through a lot of related issues, often via questions of legitimation of knowledge construction and enactment of disciplinary rigor. the paper did, as most good papers do, also help raise some further questions.

much of the paper describes how these epistemological stances apply to information systems research, arguing for their various applicability or inapplicability to social phenomena. for example, the positivist approach asserts that “organizations... have a structure and reality beyond the actions of their members” (ibid., p. 9), whereas an interpretive approach would assert that the organization’s structure exists because of its enactment by its members. I was left wondering, how do these various epistemologies apply to physical or natural sciences? there is some allusion to the growing idea in such sciences that positivism is something of a myth, but I think such treatment is beyond the scope here (potentially belonging in an STS paper). I wonder, how would interpretive particle physics or critical evolutionary biology look and feel? how would natural/physical sciences change if, rather than believing that they were discovering the true nature of reality, scientists instead believed that they were actively creating reality? would it simply be an acknowledgment of the social process involved in scientific investigation, or would there be deeper, more substantial changes?

in the discussion of the interpretive perspective, there is a distinction between the “weak” and the “strong” view. in the weak view, positivist and interpretive approaches are suited to understanding different phenomena in different ways and producing different types of knowledge. the two can complement one another. the strong view, on the other hand, rejects the very grounds on which positivist research is predicated and does not believe that it can produce valid knowledge. the notion that the two are different lenses through which to perceive different aspects of related phenomena I find appealing, but I can also see why, due to the rather different philosophical commitments involved in each, it might be difficult to reconcile findings from positivist and interpretive studies.

the critical approach emphasizes “totality, which implies that things can never be treated as isolated elements” (ibid., p. 19). phenomena being studied must be understood in terms of the social, historical, cultural, political, and other contexts that cause them to come to be. however, it seems that, ultimately, everything is related to everything else. how, then, does one practically scope a study? arguably, the point of totality is that context is not just context (i.e., with the text), its text, the thing being studied itself. taken seriously, totality seems to pose a major problem in terms of tractability.

I also noticed, as Orlikowski and Baroudi point out, that the critical approach focuses very heavily on socio-economic class, labor relations, capital, and markets, often at the expense of “other factors such as race and gender” (ibid., p. 23). this is clearly related to the problem of totality; not everything can be taken into account, your analytic focus must be somewhere. furthermore, it makes me wonder if “critical” is in some ways a politically correct, though thinly veiled, guise for “Marxist.”

Orlikowski and Baroudi’s major point is that none of these epistemological stances is perfect—each has its strengths and weaknesses, philosophical commitments and founding assumptions—and the job of the researcher is both to be aware of the impacts of choosing a particular stance, and to acknowledge the validity of alternative stances. I couldn’t concur more. especially coming from a highly interdisciplinary field, some of the major tensions I see arise from epistemologically-oriented issues, though they sometimes come in the form of methodological critiques.

however, I was also left wondering, must a researcher commit entirely to a given stance, with both its strengths and weaknesses? or is there the potential to hybridize these approaches, doing so with a conscious and reflective eye to knowledge production? for example, could you conduct a quantitative survey, informed by positivist methods, that takes into account an interpretive perspective and seeks to achieve critical destabilization of the current social order? similarly, could you build a computation system, using tools and techniques developed under a positivist paradigm, but deploy (and potentially evaluate) that system in a way that foregrounds users’ interpretation of the system and of their reality as experienced with and through the system, specifically focusing on using those experiences to critique the status quo for both users and designers of the system? yes, this is a bit of a silly hyperbole, but it’s meant to make an important point: why do we have to accept all the baggage of a single approach? can we not learn from analyses such as that of Orlikowski and Baroudi in order to advance our reflective knowledge production? perhaps obviously, I think the answer (in my part of the information science/studies world) is a tentative yes, but I’ll leave explication of why and how for another post.

technological extravention

technology can solve any problem. or at least, that’s what plenty of folks seem to believe. lots of work (I’m thinking here largely about research on computational technology, but this line of thinking can also be applied more broadly) describes a problem and then presents some sort of technological intervention intended to solve the problem. every so often, I’ll see a paper describing how, in the process of attempting to solve one problem, the technological intervention actually led to more problems than it solved, and once in a rare while someone will argue that perhaps the best approach would be not to introduce the technology in the first place, but such critical reflection is, in my experience unfortunately rare. I’ve ranted about related notions in the past, particularly with respect to the ways in which companies and NPOs foist technology in places where it might not be wanted or needed, but never with a specific alternative course of action.

I want to suggest such an alternative approach here. rather than studying the impacts of a technological intervention, what if we to conduct a technological extravention? that is, how might our understand technology use, particular the ways in which that technology is interwoven among larger social and cultural constructs, by removing the technology? being a bit of an etymological enthusiast, I’ll admit that the etymology here is a bit off; something like “technological extraction” or perhaps even “technological extradition” might be a bit more accurate, but I think the neologism I’ve used helps emphasize the nature of the critique.

envision conducting a study of preventing a group of people from, for example, using text messaging, or sending email, or reading blogs, or tweeting (I suspect there might be a difference between forcibly preventing people from using some technology and people willfully avoiding its use, but I’d hesitate to speculate what the exact differences might be without further consideration of the specific technology and specific individuals involved). how would people adapt to such situations? how would the renegotiate their various social interactions that are currently mediated via these technologies? at the conclusion of the study, how might people’s long-term patterns of use change? obviously, there would be plenty of logistical challenges to overcome (how would you find people willing to participate in this sort of a study? how would you ensure that participants were complying with your requested non-use? what if the study disrupted the conduct of their work, connection with their families, or some other basic aspect of their lives?), but I suspect the results could be highly informative and worth the difficulties. while an admittedly small step in the direction I’ve suggested, such a study might be a concrete way of suggesting that, in some cases concerning technology, perhaps less is better.

(thanks to various members of the Social Code Group, conversations with whom started my thinking along these lines)

research instigating critical thinking

I just saw an idea come up again that I've noticed in a couple places: the notion that research (generally academic) is not only beneficial, informative, and potentially transformative for the researcher, but also for the participant.

I don't recollect all the other places in which this idea has arisen. in my brief look at action research, it seemed that research as an agent of social change was a central concern in that approach. that is, research is not just about the researcher learning something about the subjects, but the subjects actively participating in, and contributing to, the research, somewhat a la participatory design. the experience also came up in my own research on blog readers, wherein participants became more aware of, and reflective about, their own blog reading practices and habits as a result of participation in the research. most recently, while reflecting (further) on questions about knowledge attribution in blogging and ethnographic research more broadly, Lilia Efimova points out that "often it’s not only the researcher who learns new things, but also people who participate in the research, when their thinking on a subject is triggered as a result of an interaction" [emphasis added].

it seems that doing research is particularly good at getting people thinking, not just the researchers, but also the participants. rather than seeing this as a side benefit, what if we were to engage in research where the sole purpose is to get participants thinking critically? how might research look differently if the primary goal was not making a "novel and significant contribution to knowledge" but rather fostering critical thinking and reflection on the part of those involved in the research? I'm not saying that forsaking knowledge making in favor of flipping bits in people's heads is necessarily desirable. rather, I'm wondering aloud how research might look and feel different if creating a particular type of experience for participants' was made a concern of greater importance. how would this different approach manifest itself, and what might it be able to tell us about the purpose and place of doing research?

local knowledge and knowing in the doing

about a year ago, I saw Lost in Translation. it's really an excellent film, but what I found most striking was the depiction of being an American visiting Japan. I was there for a conference back in 2006 (and incidentally took my phd qualifying exam while staying in Hakodate, but that's another story), and I found myself watching various scenes say, "yeah, that's exactly how it feels." watching the movie felt like being a lone American traveling in Japan.

however, the other people with whom I watched the film weren't quite as taken with it. only afterward did I consider that none of them had been to Japan or any non-western country (here, I'm counting most of western Europe, Australia, and New Zealand as "western"). I suspect that part of the reason they didn't care for the movie is that they couldn't particularly identify with it. the plights of some of the characters certainly transcend the particulars of the situations that brought them to Japan, but the particular aesthetic, the feel, the experience of watching the film resonated with my experiences in Japan so strongly that I saw that as one of the film's greatest strengths, its ability to so perfectly capture and express that experience.

I've recently been doing a bit of research about blog readers. one particularly striking thing I've noticed during this process is the difference between reading about blogs (and virtual communities and online identity); no matter how many times I'd read Nardi et al. or boyd or Miller and Slater or any of these other folks, I never could have gained the understanding I got from actually doing the research myself. rather, I could never have gained the same kind of understanding from only reading. I draw at least two important conclusions.

first, there is no substitute for actually "getting one's hands dirty" doing research. I heard it suggested recently that, often, it doesn't matter what you set about to research, as long as you research something, because even if you think you know what you're studying, you're going to wind up working on something different. to that I'd might add that even if a large majority of your results are reiterations of those from previous studies, you as the researcher come to know those same results in a different way (perhaps even more fully) than the way you would know them from reading alone. this sort of knowing, the "knowing in the doing" or what Schon calls knowing-in-action, seems impossible without the doing.

second, it makes me wonder about the place and purpose of scholarly writing as a means of sharing ideas. it's not that the type of knowing that comes from reading isn't valuable. rather, the author/researcher, because of her or his unique experiences, understands what s/he knows in a way that the reader cannot, unless of course the reader has had similar experiences. granted, there is a sort of deeper philosophical issue about how we can ever come share meaning, but I'm going to side step that for the moment in favor of something slightly more pragmatic. if the major method of disseminating knowledge in academia is writing papers, and the kind of knowing/understanding that comes from reading is different from the kind of knowing/understanding that comes from doing, why not have more of an emphasis on doing, especially doing together? I think this sort of emphasis on doing is at work in various places, but it doesn't seem to be fore-fronted as much as might be beneficial (this is probably also my bias towards constructivist/constructionist learning coming through). rather than having conferences consist of presenting papers, lets get together and do studies together, analyze data together, theorize together, design together, and generally engage in our practices and concomitant (collaborative) knowledge construction together.