A 5by5 conversation with Ruth Schmidt, Visiting Professor and Director of Strategic Initiatives at IIT, Institute of Design about the interplay of design and ethics
“As designers our responsibility is persistent, enormous, and non-negotiable. There’s no longer any question—if there ever was one—that as individuals we are not capable of being completely unbiased; our best bet is to recognize that fact, and find ways to push against the inherent biases we naturally have. In some ways it takes the pressure off feeling like we’re bad people if we recognize and accept our prejudices.” —Ruth Schmidt
Question 1
How might organizations and designers balance their desire to increase competitive advantage with the needs of users themselves, who don’t always have a say in the terms of how they engage with services and products?
Right now, the onus is typically on the user to read and adhere to terms and conditions — it’s a one-way conversation (if you can even call it a conversation… in some ways it’s more like a hostage situation!). With the increasing recognition that data is valuable, it’s completely understandable that organizations want to capture what they can. What’s not cool is that users don’t have a voice, which puts them at a grave disadvantage.
Add the fact that it’s difficult for many people even to conceptualize the abstract notion of “data” and what it can do, and you’ve got a perfect storm where end users have no choice, no leverage, and no real way of knowing what they are giving up.
This situation has partly arisen because of the slippery slope of adding just one more term or nuance to user agreements, along with the increasing sophistication of data analytics and data science. What might have seemed merely annoying at one point has now become an overwhelming morass of little things, which have accumulated into a big deal; one step organizations can take is to pause and recognize what they are doing by playing along.
Is it possible to have agreeable terms that don’t read like a deposition? Are there ways to be more transparent and honest with users, or establishing trust through beliefs and actions that people can get behind? Eventually, I have to believe that there’s a limit to the complexity of T&Cs and that someone will invent a new normal for how we agree to interact. This will require a radical acceptance that we can’t just keep adding one more toothpick on the camel’s back; let’s rethink what we’re asking that camel to carry in the first place.
“Encouraging diversity is perhaps the single more important thing we can do to address this — that can mean diversity of opinions in the room, diversity of inputs, diversity in who analyzes research data, diversity of leadership… across the board, having a more diverse set of minds and voices is pretty much required to counter bias.” —Ruth Schmidt
Question 2
As designers, what is our responsibility in checking inherent biases that have existed in organizations as we evolve products and services, and what can we do to change them?
Our responsibility is persistent, enormous, and non-negotiable. There’s no longer any question—if there ever was one—that as individuals we are not capable of being completely unbiased; our best bet is to recognize that fact, and find ways to push against the inherent biases we naturally have. In some ways, it takes the pressure off feeling like we’re bad people if we recognize and accept our prejudices.
Dolly Chugh just published a book called The Person You Mean to Be: How Good People Fight Bias on precisely this topic. Her point is basically that we should learn to be ok with being “good-ish” rather than striving to be perfect. Frankly, perfection is an unrealistic goal, and can actively keep people from probing more deeply into their own bias and recognizing when and how to address it.
Wrestling with organizational bias is equally essential. This often takes the form of “hidden in plain sight” tendencies. An assumption that the way we do things around here is somehow normal, or even natural. Chances are it is neither. When we get used to cultural and procedural norms and surround ourselves with familiar opinions that reinforce what we “know,” it’s not surprising that we internalize these things as apparent.
It’s this pattern of behavior that’s what leads to soap dispensers that can’t recognize non-Caucasian skin. If you haven’t fully considered the range of normal skin tones, it’s probably because you’re surrounded by people who all cluster around a similar lighter skin type and the lack of diversity is the root of the problem.
Encouraging diversity is perhaps the single more important thing we can do to address this — that can mean diversity of opinions in the room, diversity of inputs, diversity in who analyzes research data, diversity of leadership… across the board, having a more diverse set of minds and voices is pretty much required to counter bias.
Question 3
What other fields should design be looking to learn from as we continue to create solutions and interactions that are ethically sound for all stakeholders?
Design should probably start by looking at itself! Sometimes being honest about past missteps can be as, if not more, instructive than looking for the right way in. There’s no shortage of solutions that meant well but missed the mark, which might not have been ethically challenged, per se, but which made assumptions that led to failure or even damage.
One Laptop Per Child is the poster child for a well-intentioned solution that didn’t get at the heart of people’s real needs, but there are also plenty of examples of things that solved one problem only to create others.
Recent studies indicate that ride-hailing companies like Uber and Lyft, while wonderfully convenient for riders, increase system congestion and emissions, and research suggests that they also have a negative impact on the use of public transportation. Not to pick on Uber even more, but there’s been skepticism about their practices and tactics in incentivizing drivers to drive more than they might typically by employing behavioral economics tactics.
So we can’t think of ethics as a singular effort; it requires identifying where we, as a field, fall short. For example, we also need to recognize when populations are getting over-studied.
In an old role, I worked for a healthcare payer, who genuinely wanted to learn more about their members who belonged to “vulnerable populations”: rural, financially challenged, with low education. In isolation, this was the right impulse: Know your users as real people, not line items, etc. The only problem was that the user research they wanted to conduct tapped into the same swath of people who had been the subject of numerous other research efforts. These people had patiently told their stories to researchers, only to be left behind and more or less forgotten when the project ended. Once would be bad enough, but this happened a few times with the same population of individuals. While the insights gleaned from those interviews were useful for the organization, the individuals couldn’t help but feel discarded. You can only imagine that they did not exactly welcome subsequent research efforts with open arms. So it’s worth remembering that ethics is not just about the outcomes of the project or the nature of the inquiry, but must take into account the larger cycle of people and context.
Looking outside, I suspect that fields that either make diversity a cornerstone of their work or have learned the hard way how important it is would be interesting to look into. Disciplines that study culture and cultural norms might be an excellent place to start. Their bread and butter is understanding why cultures are the way they are, and what makes them distinct, but also what cultural norms and “orthodoxies” exist. Experts in sociology and anthropology would likely have interesting things to say with the caveat that, like anyone else, they might be blinded by their own, “but this is how we do things” lens!
“Real diversity means looking beyond the apparent classifications of things like ethnicity and gender which are necessary, and there’s no denying the representation matters, these are not sufficient to achieve true diversity.”
Question 4
What role should diversity and inclusion play in design, especially as we increasingly rely on data and algorithms to contribute to design inputs and activities?
First of all, we need to be better at diversity in our teams and cultures—from top to bottom. Silicon Valley is a case where a minimal perspective dominates; this means that “what good looks like” is narrowly defined, which impacts who gets hired, who gets funded, and what gets built on a fairly massive scale. #MeToo is another example where it’s not just products and services but whole modes of behaving and livelihoods that suffer from a narrow perception of what’s normal.
Real diversity means looking beyond the apparent classifications of things like ethnicity and gender which are necessary, and there’s no denying the representation matters, these are not sufficient to achieve true diversity. Presumptions about what it takes to be successful can limit who gets promotions or plum gigs or can mistake where someone went to school for talent.
It’s also interesting, of course, to explore the cautionary tales, especially when it comes of data and algorithms that may initially seem rational and exempt from bias. Nothing could further from the truth! Someone designed those algorithms, and someone interprets that data.
We need to question these issues even in daily activities: for example, more men than women contribute information to OpenStreetMap (OSM)—the crowdsourced set of landmarks that populate many of the nooks and crannies of online maps to supplement essential thoroughfares—by an enormous margin, like 95% to 5%. This may seem inconsequential, but it also means that landmarks that are meaningful to men are over-represented, and in the inverse is true for markers that are more likely to be predominantly on women’s radar.
So sports arenas, strip clubs, and bars have a higher likelihood of being represented on ostensibly “objective” maps than women’s health clinics, domestic violence resources, or childcare services. Does this mean these places don’t exist in real life? Of course not… but since we increasingly assume that virtual maps depict reality, they are, mainly, in danger of being conceptually erased.
Question 5
How might the future of ethics in design as a practice evolve? What’s missing today that will be important as we design for tomorrow?
It’s encouraging that there are more conversations about ethics and bias than in the past and that these conversations have expanded beyond whether it’s ok to not do work for industries that one finds problematic, like cigarette manufacturers or Big Pharma.
In behavioral science, for example, there’s a lot of discussion about what it means to “nudge”—who decides? It’s easy to assume that losing weight or quitting smoking is desirable, but there are a lot blurrier areas; what wins out when choices are made about things that might benefit me, but not you, or both of us but not the system at large, or when two systems have genuinely conflicting “best interests”.
How do we know that an intervention is really in someone’s best interest? Doing the right thing is not always incentivized, and taking a paternalistic approach can be questionable even if you’re informed about the context in which people make decisions.
Designers are in the business of designing something; it’s not an option to throw your hands up. I often look to that famous quote from Spiderman: with great power comes great responsibility. Designers need to be informed, embrace diverse opinions, and do the best they can.
I’m currently a Visiting Professor at ID, and the good news is there are a higher appetite and interest in having these discussions amongst students than even a few years ago. I’m hoping that this desire to continue the conversation and reach real answers also reaches more established ears and minds…when you have benefitted from a system, it can be a lot harder to question norms than if you’re starting to knock on the door.
If the implications of being ethical are seen to run the risk of putting your company or livelihood at a disadvantage, that’s going to be a hard sell.
I wonder if seeing some of the behaviors in our current government—where people in power seem to feel like they have the right to do whatever they possibly can even if that means embracing demonstrably manipulative actions—is going to gross people out. Will it cause people to push back on unethical or disingenuous behavior or will contribute to a sense that everyone does it and this is the only way to play the game? I dearly hope it’s the former.
5by5 