CHI 2015 Conference Report

One nice(ish) thing about attending a conference on the other side of the world from where you live is that it gives you lots of time on the plane. In my case, it's something like 13 hours. To help the time pass, I'm writing some thoughts about things I saw at CHI this year.

I've grouped these under three loose themes: non-use and reflection; algorithms and computational analysis; critical considerations; and social computing.

One quick caveat: These are just a subset of the sessions I saw, so they probably reflect more my particular interests than any sort of representational cross section of the conference.

Non-use and Reflection

Alexis Hiniker, Kiley Sobel, Hyewon Suh, Yi-Chen Sung, Charlotte P. Lee, and Julie A. Kientz. Texting While Parenting: How Adults Use Mobile Phones while Caring for Children at the Playground.

Mixed-methods study including observation of caregivers (not just parents) at playgrounds in Seattle and a larger online survey. Results show roughly three different types of approaches to phone use while caring for children. First, confident absolutely never use their phone at the playground and feel great about it. Second, confident users use their phone regularly while caring for children and have no qualms about doing so. Third, the largest group (44%) felt that phone use should be kept to a minimum, but in practice they found it hard to do so. Furthermore, this conflict group also showed higher evidence of guilt about their own parenting/care-giving. However, when offered the possibility, the majority (74%) did not want a technology to help them limit their technology use while caring for children, including 60% in the conflicted category. One participant said it would be like having an app to cut down on apps.

It's great to see work that explores situated practices technology non-use, especially that highlights the tensions and the way that most people don't fall clearly into the category of user or of non-user. Furthermore, the points about morality are fascinating. One quote from a participant suggested that s/he shouldn't be using the smartphone so much anyway, let alone while caring for a child. These kinds of comments help elucidate some of the moral order around technology (non)use.

Eva Ganglbauer, Geraldine Fitzpatrick, and Florian Güldenpfennig. Why and what did we throw out? Probing on Reflection through the Food Waste Diary.

Describes an app developed to help users document food they waste in the home. Deployed simply by releasing on the app store. Coded the notes that users left to themselves by employing Fleck and Fitzpatrick's levels of reflection framework, plus an emotional dimension. The analysis found evidence for significant levels of reflection.

One of the really nice things here is the way the authors acknowledge the affective dimension(s) of reflection. This is a point I haven't seen discussed extensively in prior work on designing around reflection, and it's something that didn't come up in my synthesis of theoretical and conceptual approaches to reflection. However, it seems a potentially highly important point that future work in this area should almost certainly address. Also, the approach of deploying only via the app store rather than explicitly recruiting for a university study provides a nice means of studying "real world" us.

Helena M. Mentis, Jarmo Laaksolahti, and Kristina Höök. My Self and You: Tension in Bodily Sharing of Experience.

The paper focuses on a computational object/artifact that uses various sensors to record how people react as they move through an art gallery. These reactions are then replayed to others in the form of the object lighting up and vibrating in different ways. The paper focuses specifically on reflection, drawing on concepts from Satre to explore how reflection, both on the gallery art pieces and on the experience of the art gallery, were mediated by this embodied interaction.

The main thing I found nice here was the focus on the role of embodiment in reflection. So much work on (designing for) reflection, mine included, focuses primarily on the cognitive dimensions. It was great to see an example where participants' physical bodies played an important role. I'm interested to read the paper more closely to see how exactly they're incorporating Satre.

Algorithms and Computational Analysis

Motahhare Eslami, Aimee N. Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie G. Karahalios, Kevin Hamilton, and Christian Sandvig. "I always assumed that I wasn't really that close to [her]": Reasoning about invisible algorithms in the news feed.

The authors used the Facebook Graph API to set up two situations. In the first, participants saw the news feed content they would normally see. In the second, participants saw all posts their friends had made, highlighting the posts the newsfeed curation algorithm had removed. The authors called this system FeedVis. Reactions were pretty much what you'd expect. My favorite was from a participant who said it felt "like waking up from the Matrix." Also, the authors followed up about six months after the study and found most participants (83%) had changed their Facebook usage in some way. These changes were a mix of testing the system and gaming the system (e.g., if I look at Courtney's page more often, do I see more of her posts in my news feed?).

Some of these authors actually presented a prequel paper at alt.chi last year in which they described the FeedVis experimental set up. This strikes me as very clever and ties into Sandvig's broader interest in algorithm auditing. The other really interesting piece was the six-month follow up and how it evidences some of the ways that lay persons reason about computational/algorithmic systems.

Emilee Rader and Rebecca Gray. Understanding User Beliefs about Algorithmic Curation in the Facebook News Feed.

Grounded theory style analysis of survey responses in which the authors asked respondents if they believed that Facebook manipulated the content of their news feed (these data were collected before the emotional contagion experiment; so things may be different now, but Eslami et al.'s results suggest not terribly so). Lots of participants thought it was possible but weren't sure. For example, some described noticing that they only saw content from a subset of friends; when they went to other friends' pages, they noticed lots of posts they hadn't seen. There were also offline clues of the form "Did you see Diego's post yesterday?" and the participant hadn't seen anything from Diego. In lots of cases, they, the participants just assumed they had missed the post, i.e., that they were at fault. The authors introduce the term "sociotechnical bugs" to describe situations where a system is essentially functioning properly, humans are making reasonable inferences based on the system, but together they result in unexpected and/or unintended consequences.

I found two interesting things here. First, the detailed accounts from participants about why they thought Facebook was or wasn't manipulating their newsfeed exposes the kinds of reasoning that laypersons do about algorithmic systems (cf. Eslami et al.). Second, the notion of societechnical bugs offers a potentially compelling means of articulating what's happening in situations where an algorithm is wrong.

N. Sadat Shami, Michael Muller, Aditya Pal, Mikhil Masli, and Werner Geyer. Inferring Employee Engagement from Social Media.

Companies want to know how engaged their employees are, where engagement means applying discretionary effort to accomplish organizational goals. However, employee engagement surveys are time consuming and often conducted infrequently; in the case analyzed here only once per year. Instead, the authors considered if they could employ data about social media use to infer employee engagement. Demographic factors did a fairly poor job of predicting engagement, explaining only about 10% of the variance in engagement. However, the text that employees used on company social media (in terms of several dictionaries, including LIWC) captured 48% of the variance. Furthermore, by looking at changes over time, the authors suggest that engagement is not a fixed trait, as it has often been treated in the organizational literature, but rather a dynamic state.

My main interest here is the use of linguistic analysis to predict other variables (i.e., engagement). In particular, I find it fascinating that simple dictionary-based features enable the model to explain so much of the variance. I know lots of people argue that LIWC is, analytically speaking, incredibly powerful. It makes me wonder how well LIWC (and/or other dictionaries) might stack up against different approaches.

Jeffrey Warshaw, Tara Matthews, Steve Whittaker, Chris Kau, Mateo Bengualid, and Barton A. Smith. Can an Algorithm Know the "Real You"?: Understanding People's Reactions to Hyper-personal Analytics Systems.

The authors leveraged existing techniques that infer attributes of an individual from bits and pieces of data about them. For instance, (apparently) if you Like Hello Kitty on Facebook, you are more likely to be Christian and more likely to be a Democrat. The authors incorporated this technique into a system that inferred big five personality traits (among other things) and presented the results to users in a narrative form. Users could alter the description if they wanted, but in most cases they chose not to. That is, participants deferred to this "expert" algorithmic analysis.

This last point I find most interesting. I mentioned above questions that have been raised about when an algorithm might be considered "wrong." This paper raises a point that's come up in some of my work, as well, which is: when are we willing to let an algorithmically based system tell us that we're wrong?

Critical Considerations

Alan Blackwell. HCI as an Inter-Discipline.

The author suggests that HCI should not be thought of as a discipline in the traditional sense but rather as an interdiscipline, one that sits between and translates among other disciplines. This paper responds to one in the CHI 2014 proceedings that uses bibliometric analysis to suggest that CHI has no disciplinary core. Essentially, Blackwell is saying that HCI is not and *should* not be a discipline. Disciplines, he suggests, might best be thought about not in terms of bodies of knowledge but rather as creating theaters of thinking. Interdisciplinary approaches, then, become about challenging epistemological standpoints through sustained resistance to host disciplines.

This year, I served on the alt.chi jury. In doing so, I both juried and wrote a commentary for this paper. Overall, I find the work very compelling. HCI itself represents a somewhat novel disciplinary configuration, and so these kinds of papers are crucial to helping orient the field. My commentary on the paper includes some of my thoughts (e.g., how do we actually constitute a discipline? why does it matter, e.g., for organization of university departments, for distribution of funding?), so here I want to focus on the talk. First, Blackwell used a strategy I rather appreciated of soliciting comments from the audience throughout the presentation. Basically, he would make some some-outlandish claim, e.g., "HCI is not a discipline," and then ask, "Does anyone disagree?" While a bit unsettling, the format was actually better for discussion than the tradition approach of a talk followed by Q&A. The only downside was that there didn't end up being any extra time at the end for additional questions, but that's not a big issue. Some of the best questions, I think, came from Paul Dourish around what actually constitutes a discipline (cf. my commentary). For instance, in the 19th century, he said, disciplines were largely organized around phenomena of interest (e.g., biology). It's only in the 20th century that we started to see disciplines organized around methods. What we're starting to see now, he suggests, is a swing back toward disciplines as being constituted by the phenomena of interest. A good example might be something like communication, which uses a variety of methods and methodological orientations. HCI could potentially represent a similarly phenomenally-organized discipline.

Mark Blythe, Jamie Steane, Jenny Roe, and Caroline Oliver. Solutionism, the Game: Design Fictions for Positive Aging.

The authors describe the use of a card game that riffs off of Eugene Morozov's book of the same title to help designers come up with speculative design fictions. The paper here used the case of older adults living in homes -- not assisted living facilities (these folks were in many cases fairly active) but rather group living arrangements. For instance, one home consistent entirely of retired actors, all of whom were essentially stars in their own day. One of the important points Blythe made dealt with the fantastical nature of design fictions, that part of the reason they work is that we know that they are fictions.

During the Q&A, I asked about this last point, specifically the ways that the fantastical nature of design fiction(s) might contrast with the utilitarian idea of "solutions." Blythe suggested that part of the power of design fictions comes from their irony, that the "solutions" they suggest are really tongue-in-cheek. On the one hand, this makes sense. On the other, I'm not sure about reinforcing the problem/solution framing of design, i.e., that design is about solving problems.

Morgan G. Ames, Silvia Lindtner, Barry Brown, Daniela K. Rosner, Sidney S. Fels, and Roel Vertegaal. 10 Years of alt.chi: Reflections and Outlook.

In this panel, current and past alt.chi chairs reflected on the genesis, development, and future trajectory of the alt.chi track. Initially, authors could not submit to alt.chi as a separate track. Instead, it was a means of rescuing compelling but divisive papers; to appear in alt.chi, a paper had to be rejected from the main conference. alt.chi chairs would look for papers with disparate reviews (e.g., a 5 and a 1) and select some of them to be presented at alt.chi. Only later did it develop into a separate track to which authors could submit, incorporate open reviewing, add a jury selection process, etc. Brown suggested that, in some ways, alt.chi has become a ghetto; many of the important critiques raised there belong not in a separate track but in the main conference. Rosner asked if there might be ways that publication forms could evolves, e.g., what if publications were not static items in an archive but dynamically evolve? In a tweet, Florian Echtler suggest such a configuration might look a bit like this proposal.

The discussion following the panel was fascinating. To me, some of the most important questions raised here have to do with peer review -- what we think peer review does for us, what we want it to do for us, and how those functions might differ between various tracks and venues. Vertegaal made the point that, even though we pretend the peer review process is fairly scientific, it's really not (citing studies about multiple reviews of different papers that show inconsistent reviewer decisions). However, I wonder: is "scientific" the most important property towards which we should aspire in our peer review process? What about other attributes, e.g., could we design peer review to be as egalitarian as possible, to make sure that all voices are heard and represented equally? Any discussions about how different review and publications formats evolve should, I believe, address these types of questions about what (we think) they accomplish, both for individual submissions and for the community as a whole.

James Pierce, Phoebe J. Sengers, Tad Hirsch, Tom Jenkins, William W. Gaver, and Carl DiSalvo. Expanding and Refining Desing and Criticality in HCI.

HCI work has demonstrated lots of interest recently in critical design. However, a multiplicity of ways exist in which that term might be interpreted. In particular, the role that critical design as a practice plays in the discipline of design differs in important ways from the way that critical design has been broadly conceptualized and deployed within HCI. The paper argues for embracing different approaches to design and criticality, in particular, those that foster more dialog among design, design studies, and HCI.

This paper does some important work in terms of advancing the discourse around critical design. One of its greatest strengths is acknowledging that there may be a multitude of different ways in which a particular research project may be critical or may engage with design and valuing the unique contributions that each of those different approaches can make.

Jeffrey Bardzell, Shaowen Bardzell, and Lone Koefoed Hansen. Immodest Proposals: Research Through Design and Knowledge.

Considers the unique types of knowledge created by critical design artifacts. Applies approaches from literary criticism to provide reading(s) of one particular design artifact, the Menstruation Machine, which uses electrical muscle stimulators and even dispenses blood to simulate menstruation. During the talk, Bardzell argued that, unlike with art criticism, there will never be a class of critics focused on critical design work (especially) within HCI. The paper illustrates how multiple readings of the design expose the particular kinds of knowledge embodied in the design artifact.

The best moment of this talk came during the Q&A when someone got up and asked, essentially, "Is this design or is this art?" Bardzell was, I think, a bit visibly distressed (as well he should have been!). The response, which was spot on, basically said that that's not a productive question. Instead, the question should be about the kinds of interpretive approaches we can/should apply in making sense of these objects along with the texts surrounding them.

James Pierce and Eric Paulos. Making Multiple uses of the Obscura 1C Digital Camera: Reflection on the Design, Production, Packing and Distribution of a Counterfunctional Device.

Analyzes a deployment of a counterfunctional camera where one can take photographs but, in order to view the photos, must destroy the camera to get the photos out, thus eliminating the ability to take more photos. This buildings on the authors work around counterfunctional design presented last year at DIS. This paper focuses on deployment, but it doesn't provide a user study in any traditional sense of the term. It talks about packaging and deploying the cameras for users, but it doesn't analyze experiences at all. Instead, it talks a bit about how the design was simultaneously packaged and deployed in a particular way for the audience of HCI researchers, just as its packaged and deployed in a particular way for users. Instead, the potential user findings are suspended as a potentiality, just like images in the camera. Part of the point of design research, Pierce argues, is that it often has many [points]. Here, it's simultaneously about exploring whether counterfunctional designs is a category of objects that people could ever come to demand while simultaneously asking/raising questions about how we do design research and design knowledge.

Sometimes Pierce's work is almost (not quite, but almost) just a bit too clever its his own good. The multiple levels of interpretation (e.g., academics as an audience just like users are an audience; user study findings suspended potential just like the images in the camera) are really compelling in the way that they nest the argument self referentially within itself. I also think there are some interesting connections to be made between counterfunctional design (i.e., design that eliminates some normal aspect of use) and non-use (i.e., individual engagement with a technology that eliminates some aspect of use) but James and I are still trying to talk through exactly what these resonances are in a way that's productively interesting.

Conor Linehan, Sabine Harrer, Ben J. Kirman, Shaun W. Lawson, and Marcus Carter. Games Against Health: A Player-Centered Design Philosophy.

This paper reacts to the ubiquitous interest in "Games for Health." The authors suggest that much of what makes games compelling and interesting as a medium and as an experience is essentially antithetical to health, and vice versa. Thus, they argue, rather than making games for health, we should try making games against health, games that appeal because, e.g., they require as little physical movement as possible. One of the question askers pointed out that, sometimes, you don't expect entertainment to be entertaining, as it were; some movies are scary, some make us cry, and games can be similarly diverse. To this point, the authors agreed, but they also pointed out that you don't ever really see "movies for health."

In many ways, this paper continues the tradition of work critical/provocative coming out of this group. The talk was similarly insightful -- they actually had three "contestants" get up on stage during the talk to play a game against health. The contestants had to eat a cup of noodles and drink a soda as quickly as possible while moving as little as possible. The authors also handed out cigarettes to anyone who asked a question -- or at least they tried to; I turned them down. The presentations by this group, particularly at alt.chi, have at many times taken a form more closely resembling performance art. I find myself ambivalent about this development. On the one hand, it's fantastic to see alternative presentation formats, and I suspect alt.chi provides an apt venue. On the other hand, sometimes it's hard to tell if such presentations are overly theatrical bids for the Best Talk award or creative means of advancing the scholarly discourse.

Social Computing

Niloufar Salehi, Lilly C. Irani, Michael S. Bernstein, Ali Alkhatib, Eva Ogbe, Kristy Milland, Clickhappier. We Are Dynamo: Overcoming Stalling and Friction in Collection Action for Crowd Workers.

Describes the creation and development of We Are Dynamo, a labor organization for Mechanical Turk Workers. Essentially, Turkers want some kind of organized labor, but the traditional organization of a union with members provides an all fit, partially because there's contract between employer and work, and partially because many Turkers are transient and only work for short periods of time. We are dynamo represents an effort provide a voice. The paper focuses on barriers to collection action more generally. It describes stalling, when an effort loses momentum/interest, and friction, when tensions flare up. One of the major approaches to address this they termed "act and undo." When collective action stalls, make sure you can do something, but also make sure that it's done tentatively so that those actions can be repealed if necessary.

This work provides an awesome example of activist engagement in academia. I've heard multiple academics [e.g., Comp Ling paper] point out the major ethical concerns with Mechanical Turk and then proceed to suggest that we shouldn't use Turkers as study participants or as workers in the systems we build. To me, this paper represents an alternative; rather than seeing a system you don't like and opting out, you could instead attempt to intervene and change the system. Furthermore, the paper is not solely about activism but instead bumps it up a conceptual level to consider how their work could help inform collective action online in general. In talking with one of the authors (Bernstein) later in the week, his plans are fairly ambitious: use crowd workers to help design an alternative crowd work platform. Can't wait to see what they come up with.

Amy X. Zhang and Scott Counts. Modeling Ideology and Predicting Policy Change with Social Media: Case of Same-Sex Marriage.

Uses data from Twitter to predict the outcome of policy events (court decisions, state legislation, public votes, etc.) about same-sex marriage. Their model uses a variety of features, based largely on LIWC categories. The best models including data only from a single state achieve about 80% accuracy. Including data from geographically proximate states improves performance to around 87%.

One of the interesting questions this talk raises deals with people who don't use Twitter. During the Q&A, I asked how the author would interpret the fact that they can achieve better accuracy than public opinion polls even though they're missing systematic groups (e.g., certain socioeconomic status are under-represented in many social media). The author suggested that, with some prediction tasks, determining relative differences may matter more than establishing absolute truth. This kind of work makes me both excited and nervous.

Jeremy Birnholtz, Nicholas A. Merola, and Arindam Paul. "Is it Weird to Still Be a Virgin?": Anonymous Locally Targeted Questions on Facebook Confession Boards.

A phenomenon exists on Facebook where people will anonymously submit (usually via Survey Monkey or something similar) stories, comments, and questions, and a moderator will then post those in a locally-specific group, often associated with a university. The authors use this to look at how anonymity (vs. requiring users be identified by their real name) influences prevalence of taboo or stigmatized topics. A content analysis revealed that, indeed, taboo topics occur with some degree of frequency (32% of the questions analyzed). Furthermore, ofte the questions that got responses, only 5% were negative or harmful. This finding contrasts with popular media coverage that conflate online anonymity with flame wars, cyber bulling, hate speech, etc.

One of the really important points this paper makes deals with the nuanced role that anonymity can play. Furthermore, it acknowledges that anonymity is not a binary opposition. In the case studied here, posts were made anonymously, but replies were named. The authors suggest (and I agree) that it would be beneficial to explore the design space around different configurations of anonymity that are neither fully anonymous nor fully named.