AI and Machine Learning Will Change UX Research & Design– During our Q and A session at the June Virtual UX Conference, an attendee asked Jakob Nielsen if and how artificial intelligence and machine learning might affect UX research and design. Here’s what Jakob had to say.
Well, I think that there are two angles to that and the first angle is that the user interface will employ more artificial intelligence than it did in the past and that will also therefore change what UX people have to do. And that’s all the way from a little bit more, maybe primitive use of AI for things like face recognition.
We did a study recently in China of how you pay by smile. “So I got my croissant paid for” it was one of our test users and they didn’t quite understand how it was doing it. A lot of mental model confusion. So even though face recognition is a great technology, right? And it’s certainly much better, particularly if there’s any virus going around that you don’t have to touch terminals and mess with things, because it’s just like you get recognized and then that’s how they know that you’re willing to pay.
But the way it was done had a lot of usability problems leading back to mainly mental model, which is one of our key concepts. And another more kinda maybe important advanced use of artificial intelligence is in feature interpretation so that basic computers ought to work on a “do what I mean” principle, not “do what I say.”
So computers have always been in some sense ideal in one way of thinking, because they do as they’re told and that’s great, right? But they do exactly what they are told. And so if you tell them the wrong thing, then they’re also gonna do the wrong thing. And that’s one of the areas where AI can come into place and have better interpretation of what you really want.
But again, that has a huge human factors component to it and big task analysis components to it of understanding the circumstances, but that’s one way usability can be vastly improved if computers can change from not having that literal interpretation of commands to having more of understanding interpretation of commands.
Also for more complex fields or complex areas, it’s often not possible to have the computers just like do it, which in some sense, productivity wise is the optimal, because then you don’t have to have a person involved, but there’s a lot of things that involve two.
Well, you have to have more human judgment that the computer can just do the thing. But what the computer can do is present likely options to the user and then the user can choose from them. That can be much more efficient than just having like, “Oh, here’s all possible options” and you have to do a lot of manual collation manually to get anything to work, which is kinda how current computers work. Instead, the computer can kind of shortcut a lot of that.
Not by making a decision, but rather saying, “Here’s the five most likely things that should be done, which one is actually the one?” And the person the human can decide on that. And that’s one of them should be one of the options should probably be none of the above. And then you’ve gotta go back to the manual involved complicated process. But I think AI can do a lot to really make deeper decisions, better decisions, but again we have to understand the task, the domain, and do a lot of those kind of deeper human factors type of the methods for that to work.
And so that’s the entire area of the user interface. And so it’s just, I guess the most kind of bottom line conclusion there is. Don’t assume just because it’s AI, then we don’t need any user interface, design any usability work any of the UX stuff at all. And in contrast, we actually needed more because we need to have a deeper understanding of both the person and the problem before we can apply AI methods to either fully or partially solve the problem.
I think partial solve is more likely to be often the case. And then we may need a little bit of variant methods but honestly, the methods where we have apply to this technology, does this apply to any other technology. But that said, I think there’s still gonna be some changes for UX work as well, because we can use AI in our own work.
And some of these are some of the things I kinda mentioned like, let’s say face recognition, not just face recognition, like who is this person? Because if we take somebody in for usability study, we know where they are but you can do some kind of like emotion recognition or other ways of tracking what goes on during the study in a more interesting and detailed way.
I mean, let’s say you can, actually you can not right now, but that’s one of the things I think that AI will hopefully be able to get us is a better use of lot of biometrics that currently we don’t really use because it is too confused and messy and misleading, but that I think is a really future important thing for us to get better use of research. I think also more simplistic things like automatic transcription of a user interview or usability study session, or maybe even ways of highlighting or recognizing key things to look for, I don’t necessarily think we can have a computer just do the user interview.
Maybe it could do some of the questions automatically, there’s sometimes by the way, people answer more truthfully, when they are doing an interview with a computer than when they’re doing it with a person, that’s been something like experiments in medicine.
So the classic example is like, how many drinks, how much alcohol do you drink per week, right? And when the computer asked the question, people will say they drank more alcohol than when a human doctor asks the question ’cause they’re embarrassed to tell the person I drink like 20 shots of whiskey a week, but they’ll tell a computer, even though the doctor’s gonna look at the computer’s response in the report later, but people I’m honest sometimes with the computer.
So maybe some types of user interviews could be automated but obviously many cannot because a lot of this is like react in the moment and understand the circumstances, but some could be done and then you could mass produce it over more people and you can automatic transcribe it, you can automatically identify areas of interest. And again, I don’t think perfect AI anytime soon anyway.
So this is gonna be a little bit like what I mentioned with the user interface that the artificial intelligence will identify possible things, and then the human which is the user researcher has to go and look at those and say, “Yeah, this is not really important, that one is,” and like interpret what it means which the computer cannot interpret what it really means, that’s so I think that will definitely still be a need for UX people, but we can do things more efficiently.
Similarly in design, you could have a lot of things where designers now sit and mess with things at a very tiny basis can be done more or less automatically by the computer or the computer can come up with like, here’s like five possible layouts that comply with all the known graphic design principles and then the designer can say, “Yeah, but you know, the principles are right.”
But sometimes best design comes from mainly following the principles but then like violating it in one particular spot and then you add at your human creativity, but you have saved a lot of work because you already can stop working from something that AI came up with. I mean, I think there’s gonna be a very large number of examples like that.
Also, for example, analytics, like just continuing analytics that just is in real time analyzed by an AI, which again, you can not have a human sit there and look at millions of data points every hour, but a computer can do that. And then again, it pops up to one of us like a human UX person that the AI thinks there’s something otherwise going on and then you gotta ponder that and analyze it and say “yes, it’s right or wrong, what should be done about it?”
Maybe we can also have automated A/B test variations generated by an AI as well, maybe as it was triggered by this real time monitoring by an AI. So there’s a variety and this keeps going many, many, many possible other things, right? And of course the basic, the real main conclusion is that nobody really knows what will be the truly genius use of AI in 10 years when it’s better.
But now cycling all back, the point is that it’s gonna make us much better, more efficient, more productive at our work. Now this backs in the follow-up question, which says, “Okay, higher productivity that means that we don’t need as many employees so we can fire half of them, right?” Actually I don’t think so in this case because I think that the need for UX is so enormously great in the world that if we can do more work, that doesn’t mean that there needs to be fewer people doing the same work, it means that the same people will do more work.
And it also means so actually what normally happens also is if productivity goes up, that’s the only way that you can increase people’s salaries, right? Because basic rule of economics is that a company cannot pay an employee more than they produce. “Cause if you pay people more than they produce, you’re gonna go out of business. So if you wanna make more money, they’re gonna produce more. Now, if we think that AI is gonna make you UX people let’s say 10 times more productive, I don’t think this means we’re gonna get 10 times higher salary, I think it means that we will do it too bad, right?
But I think I there will be some more higher salaries because we will be better, we will have more impact on the world, but much of that productivity gain is gonna be spent or used on more UX, right? So just gonna be more and more, more things, “cause that’s another basic rule of economics. If something is cheaper people don’t buy more of it. And so what I’m predicting is that UX work will be cheaper in terms of unit of insight if you wanna call it that, a unit of great design, which is very hard to measure in units like that.
But conceptually anyway, we can definitely talk about it that way. And so there will be more good UX done because the AI can take some of that manual work off our plate and so we can deliver more and that is needed because the world needs so enormously, much more UX work than is being done right now. And so one of the things that’s holding smaller companies back in particular I think is that the current price of a unit of great design or a unit of great user insight.
And so if we can drive that down by, a big, big factor, much more UX work will be done. And so that’s the way I think that all that we resolve that we’ll do more, it will be cheaper, we may get a little higher salaries, but not necessarily super higher, much higher salaries, but definitely no unemployment just because our productivity will go up because we are in a field where there’s endless, endless more need than what’s being fulfilled right now.
Web enthusiast. Thinker. Evil coffeeaholic. Food specialist. Reader. Twitter fanatic. Music maven. AI and Machine Learning!