It Appears That Sociological Ethnography Is Not, and Can Never Be, Science………

Alice Goffman released what many consider to be a seminal work of sociological ethnography.

It has now been revealed that many of the details of this study are false.

This is actually not an issue of a violation of professional ethics. In fact ethics in this field requires the obfuscation of data:

Late last month, a Northwestern University law professor published an article calling into question the veracity of a widely lauded book by Alice Goffman, one of sociology’s brightest young stars. The book, On the Run: Fugitive Life in an American City, is an ethnographic study of a black neighborhood in Philadelphia where, according to Goffman’s research, residents live in a mini–police state, constantly in fear of being arrested and sent to jail or prison, often for minor offenses. Goffman conducted her fieldwork, first as an undergraduate at the University of Pennsylvania and later as a graduate student at Princeton University, by embedding herself with a group of men from the neighborhood—they are all given pseudonyms in the book—and carefully tracking their lives over the course of about six years. The result is an extraordinarily detailed portrait of a community—nicknamed “6th Street” by Goffman and never identified by its actual location—in which the criminal justice system dominates people’s lives and systematically cuts them off from opportunity.

On the Run received nearly universal acclaim upon its publication. But according to Steven Lubet, the Northwestern law professor, the book is seriously flawed. Lubet points to two anecdotes that he believes could not have happened as described and a third that seems to implicate Goffman in a felony. The article in which Lubet laid out his concerns touched off a debate both within the academic community and outside of it, one that spilled from sociology message boards onto the pages of the New York Times and caused some observers to wonder whether they were witnessing the opening scenes of an all-too-familiar story of intellectual deception, exposure, and professional disgrace.

Lubet’s article, originally published in the New Rambler Review and adapted by the New Republic, may have touched off the public backlash against Goffman, but he was not the first critic to attack her credibility. A few weeks earlier, many in the world of sociology had become transfixed by a strange unsigned document that had been sent to scores of influential scholars in the field, along with Goffman’s department chairwoman at the University of Wisconsin–Madison. In the document, which quickly made its way online, an unnamed critic made the case against On the Run over the course of more than 30,000 words and 45 numbered “problems,” highlighting what appeared to be inconsistencies in the book’s chronology and casting doubt on the integrity of the research behind it. The document appeared to be the work of an obsessive with an agenda—its tone was at times angry and bitter, and its reasoning could be muddled and hard to follow—but some of the author’s observations seemed damning, or at least in need of explanation. Goffman felt compelled to write a line-by-line rebuttal and submit it to her department.

I spoke to Goffman on the phone on June 5, not long after the University of Wisconsin–Madison released a brief statement saying that it had found the accusations of academic misconduct that had been leveled against Goffman in the anonymous letter to be “without merit.” With that statement in mind, I asked Goffman about what I considered the most problematic points in the letter and asked her to respond to them. From there we talked more generally about her book and how it compares to other works of nonfiction aimed at exposing the strife and degradation suffered by underprivileged populations.

Ethnography can look like an uncomfortable hybrid of impressionistic data gathering, soft-focus journalism, and even a dash of creative writing.

I came away from the conversation with a sense that there are indeed factual inaccuracies throughout On the Run. However, they are not the product of the kind of fraud we’re used to seeing in publishing scandals, and it would be unfair to say they place Goffman in the company of fabulists like Stephen Glass or data-cookers like Michael LaCour. That’s because the majority of what I’m calling “inaccuracies” were introduced into On the Run because the conventions of sociological ethnography required them. In keeping with the methodological protocols of her chosen discipline, which typically demands that researchers grant their subjects total anonymity, Goffman changed details and scrambled facts in order to prevent readers from deducing the identities of the people she was writing about. In the process, she made her book all but impossible to fact-check.

The imperative to anonymize subjects is, in most cases, specifically mandated by the institutional review boards that approve social science research in American academia. And while it does seem possible to me that Goffman embellished some aspects of her narrative in order to tell a more compelling story, it was the steps she took to protect her sources—many of whom commit serious crimes in the book—that have made On the Run all but defenseless against skeptics like Lubet and the author of the anonymous letter. In other words, there is a good bit more than just the credibility of On the Run at stake here. At the heart of this controversy are the fundamental limitations of ethnography as a mode of inquiry. As practiced by many scholars, what is supposed to be a scientific undertaking aimed at systematically revealing truths about the world looks more like an uncomfortable hybrid of impressionistic data gathering, soft-focus journalism, and even a dash of creative writing.

Though it may be Goffman in the hot seat right now, any number of her colleagues could find themselves in a similar position, for the simple reason that producing research that is detached from reality to the point of being unverifiable is a central tenet of their discipline. As a result, some of the most vital and nationally relevant findings that come out of their field, including research on topics of urgent importance, like the conditions of inner city life, are vulnerable to questions about how much truth—and what kind of truth—they actually contain.

………

Elijah Anderson, Goffman’s undergraduate adviser

To that question, Goffman doesn’t have a good answer. As Lubet points out, she so disguised the people and locations that appear in On the Run as to make her accounts effectively unverifiable: She changed the name of every person in her book and altered what she refers to in the preface as people’s “identifying characteristics.” According to Goffman, that meant changing the names of the places her characters inhabit and visit, changing people’s ages and jobs, adjusting the number of people who were present at certain events, moving some of those events around in time. What’s more, Goffman revealed in an interview with the Philadelphia Inquirer that she shredded all of her notebooks and disposed of the hard drive that contained all of her files out of fear that she could be subpoenaed and thereby forced to incriminate her subjects.

………

The third discrepancy was of a different type, however. Goffman said that Chuck was indeed alive at the time of the court hearing. But the field note in which he is described giving his brother a ride was not written down in 2009—Goffman just labeled it that way in the book as part of the anonymization process. When it came to court hearings, she explained, she felt it was especially important to scramble dates because public records can be used with relative ease to identify cases and thus people. In this instance, Goffman said, her failure was in neglecting to make sure that the timeline as presented in the book was internally consistent.

It’s this last example that is most illuminating: unlike the other two, which seem to be innocent, if careless, mistakes, it illustrates the lengths to which Goffman went to change facts in order to protect her subjects. More to the point, that’s exactly what she was supposed to do, according to the rules of ethnography.

………

To find out, I called several sociologists and anthropologists who had either done ethnographic research of their own or had thought about the methodology from an outside perspective. Ethnography, they explained, is a way of doing research on groups of people that typically involves an extended immersion in their world. If you’re an ethnographer, they said, standard operating procedure requires you to take whatever steps you need to in order to conceal the identities of everyone in your sample population. Unless you formally agree to fulfill this obligation, I was told, your research proposal will likely be blocked by the institutional review board at your university.

“Your honor —your word—is the only thing you have to make your stuff believable.”

If all there is to verify your studies is your word, it's not science.  Period, full stop.

If sociological ethnography requires its adherents to produce studies that are by definition irreproducible results, then it is not a science. It is an exercise in historical fiction.

Science requires reproducibility, no saving throw.