BackStories/02: Electrifying the Amazon Fleet

Tirthankar Sengupta:
How can you give them a tool, because the van ultimately is a tool. How can you give them a tool which is the right tool for that job? And so, our quest is to build that right tool and design is all about building the right tools, right?

Justin James Lopez:
Hey everyone. Welcome to another episode of Amazon’s Backstories, a behind-the-scenes podcast that lets us get a sneak peek into the people and design decisions behind various services and products across all of Amazon. Now, this episode is hosted by Khalen Morehead and features Tirthankar Sengupta and Jackie Chang, as we explore the Amazon and Rivian partnership in electrifying our new vehicle fleet. Let’s hear their story.

Khalen Morehead:
All right. Welcome, everybody. My name is Khalen Morehead. I am a user experience designer for DDG, working on the Fire TV OBE device setup, and we’re going to be interviewing some very special guests here today. And so, I’d like to start by passing it over to Tirthankar.

Tirthankar Sengupta:
Hi, everybody. my name is Tirthankar Sengupta. I am a senior manager of user research and design in the last mile Experience Design group. Our group works on delivery software that powers all the delivery across the last mile network throughout the world. I have been at Amazon for over 10 years now. I started working on Echo and Alexa operating system, and then moved into last mile in 2016, where I started working on last mile Experiences, especially the delivery execution software for drivers. And I’m happy to talk to you today about our electric vehicle delivery application software.

Khalen Morehead:
Awesome. Jackie, why don’t you tell us a little bit about yourself?

Jackie Chang:
Hi, everyone. My name is Jackie Chang, and I’m a principal designer for the last mile Experience Design team. My expertise comes from automotive background. I had about 20 years of working experience, mixed with automotive OEMs research lab for autonomous driving. And now I’m working with the last mile team on everything about on the road experience. Thank you for having me.

Khalen Morehead:
Yeah, absolutely. Pleasure to have you both here today. And it’s really great just to be able to talk to you with all the shared experience that you guys have. And so kind of just getting into it, I’m really curious, can you tell me a little bit about how Amazon design specifically was involved in the rollout of these electric vehicles? What was your involvement, and how did you guys partner with Rivian to divide and conquer all of this?

Tirthankar Sengupta:
We started around end of 2019, and kind of looking at all the electric delivery vehicles that were coming into the last mile fleet. And our initial engagement was with Rivian. Now, one of the things around that electric vehicle was, the user experience inside the van had to be built for delivery drivers. So our design team engaged with Rivian design team to understand how we can create a custom application for delivery drivers that would actually enhance their experience throughout the day. At that time they were just using the mobile application software.

So we wanted to leverage all the technology that the electric vehicles provided, so that we can enhance their experience. When we started working on that app... So, our team was only involved in developing that part of the experience, which would make drivers deliver packages throughout the day. When Jackie joined in 2020 the team, then he started working on the relationship with the Rivian design team, and we had regular working meetings with them on a regular basis to understand how we can gradually move forward into the design, understanding their design systems, understanding our design systems, and combining them.

Jackie Chang:
Yeah. When I joined Amazon in 2020, I think that was in September, basically Rivian design team has the very first last mile solution from their own interpretation. So we took a look at it, and then the first thing we shared with Rivian was a deep understanding about how driver goes through their day in the life. Also, walking through with Rivian about the pain points they are having with drivers, so that we can collaborate knowing what are the pain points we are working together to solve. And from there we kind of have a really working as one team, on a weekly basis. And as we go through every week we just improve our design in alignment in the meeting.

And we are really able to make sure the vehicle experience, and the last mile delivery experience in the vehicle are cohesive and seamless. And beyond that, we also were involved working with Rivian UX team on some of the physical experiences, like the keyless experience, bulkhead door, climate control, seat comfort, more so importantly also to address the on road safety. What is the Amazon design tenant for safe to drive the vehicle on the road?

Khalen Morehead:
I’d really like to hear about that marriage of all the aspects of both the function of the vehicle, as well as the performance of delivery driver’s tasks, as well as just their overall comfort in trying to merge all those things together. What would you say are some of the favorite features that you all worked on? And how were you able to ideate some of these things into existence to help them have a much more safe and more efficient day-to-day?

Tirthankar Sengupta:
It started with the customer. So, at Amazon we always work backwards from the customer. And working back from the customer means understanding the customer, and then developing all the ideas and inspirations for design. We had done a lot of ride alongs with drivers on a day-to-day basis to understand what they go through throughout the day, to develop a very detailed understanding of their day in the life. And when we went through that, so our research team went out in the field on a weekly basis with drivers, and understood like, “Hey, at this point, what are you doing? At this point, what is the task? When you are parking, what kind of information do you need? When you are delivering, what kind of delivery information do you need?”

So going deep into their information requirement, their mindset, their needs at each and every step of their delivery journey. And what that gave us is a good understanding of where we are asking the drivers to do too much with technology, and also where technology can help the drivers. Using geofence to understand where the driver is located, or using proximity to understand whether the driver is close to the vehicle or not. Or using movements inside the vehicle to understand where the driver is located at, or at what stage of the delivery journey the driver was going to be. So those gave us some of the indications around what those unique features are going to be for this vehicle.

So for example, one of the features was park to drive actually initiated the delivery workflow. So when the driver went into the vehicle, they would start the van, and they would put the parking gear into the drive gear. That would automatically initiate the route to the next delivery location or stop. What that does is basically, it removes some of the steps that the driver has to do on the phone, he or she doesn’t need to use the phone. It’s completely hands free, the driver is just driving at that point. So the driver can focus on the road, focus on the driving, enjoy the experience of driving. And the vehicle actually takes care of the delivery workflow at that point of time.

Khalen Morehead:
Oh, that’s intuitive.

Tirthankar Sengupta:
Yeah. That’s one of the unique features. The other ones, we kind of not only looked at the delivery workflow, but also the driver’s movement inside the van. And there are interesting things inside the van that we did.

Jackie Chang:
Yeah. Let me add a bit about the more specific now that Tirthankar has shared about all the user value we able to bring in. So, to highlight some of the features, one thing is the maps and wayfinding experience. Because Rivian has two displays, like some of the most premium automotive out there in the market right now has dual display experiences. So taking this advantage of the hardware architecture, we were able to redefine was the maps experience, putting the most important maneuver guidance experience right in front of the driver, behind the steering wheel so their eyes are on the road. And putting things that are not important on the secondary screen within the center display.

So, I would say the map wayfinding experience, it’s probably first of its kind, a revolutionary in the logistic delivery industry, a good understanding of how user would use the content. And therefore we were able to redefine what’s a proper UI layout that can provide a safety use on road. The other important feature are advanced driver assistance systems which the automotive industries called ADAS. So we also collaborated with Rivian on all these active safety features. Working with Rivian, we were able to tackle the physical UX and digital UX synchronization. So both experiences are harmoniously synchronized based on driver most natural workflow of their day.

Khalen Morehead:
I would be curious to know, were there any sort of design solutions, where perhaps you were aiming for something more technologically savvy? But perhaps you just kind of realized it in the work and the normal flow of their duties, that the drivers didn’t necessarily need something? Didn’t maybe need that feature, or need something that elaborate, or something like that, and you just kind of had to take a much more human approach to it?

Tirthankar Sengupta:
We actually worked on... One of the things that we thought that would be a really great experience is voice. We were trying to use audio and voice a lot. And in the initial stages we realized that, voice as a complimentary experience. In some cases it was a little bit distracting for them. In some cases they were not using it for doing the delivery workflow, or they were just using the visual information more effectively than the audio information. Sometimes they are listening to music while they’re driving, or they’re enjoying their drive, and they don’t want to be distracted with all other modalities.

So it was an interesting revelation to us, that every driver is basically different. They enjoy their driving, and they enjoy their throughout the day. The way they carry out their deliveries is completely up to them in terms of how they plan, and how they are going to experience that throughout the day. And we had to work on the baseline of making sure that we are addressing only the features that makes it effective for them to focus on the driving, to focus on the safety of driving, to focus on making sure that they’re getting to the stops quickly and accurately. There was one more where we had a lot of information on the UI that we are trying to put in, and thinking that the drivers would really like to have most of the information in front of them.

And when we went through the user studies, we realized that they need just the next couple of stops. They don’t need everything. Because it is the next couple of stops that matter. So having that information, not having the information overload, but still having the right level of information so that they feel that they have control over their delivery workflow and delivery route, was really a good balance that we had to achieve in our design.

Khalen Morehead:
That’s so cool.

Tirthankar Sengupta:
Yeah.

Khalen Morehead:
I’m sure you’ve obviously seen the vans in person having worked on them, but how would you really describe the experience being able to see just these ideas, and all the things that your team has worked so hard on really coming to life, to existence like that?

Jackie Chang:
One of the rewarding moment is when you, at the close of the launch of the van, we were having this van in the field for driver to drive and test, and the driver sentiment was very positive. And that’s very encouraging. Because we hear a lot of them saying, “We cannot go back to normal van.” And to add that was a... It’s just as simple as it is, but it really enlighten the team, thinking all this effort, it really worth. The team was very excited. And in fact, even after we launched the product, we continuing improving the experiences from infield research. And we were able to continue improving that driver satisfaction. So it’s a very rewarding experience, the team on this project.

Tirthankar Sengupta:
Yeah. I would echo that. And the reason why that happened was because we started with that research in the initial stages. So, I would give kudos to the engineering team as well. The engineering and the product team was, the very first thing, the very first step they did was come to the design team and said like, “Hey, we need to understand the customer. Because this is a new product, a new experience, a new area for Amazon to go in.” It’s like there were no delivery vehicles that we had designed previously. We were just accruing vehicles in the fleet. So this was the first time that the design team, even as a researcher leading a design team, I was not in the fully comfort zone of having this. But when I looked at the potential and the opportunity, I said like, “Okay, let’s do it.”

How can you give them a tool? Because the van ultimately is a tool. How can you give them a tool which is the right tool for that job? And so, our quest is to build that right tool, and design is all about building the right tools, right? So we looked at alpha views of the van where they could view the pedestrians, how we were not blocking the view because of the UI, how we are not distracting them, how we are making their UI glanceable, even the text sizes, the font sizes were studied in order to understand how they can view it from when they’re at the driver’s seat. That distance as well as when they go back in the cargo area, the font changes on the UI.

Because they can see from the cargo area, which is around 10 feet. And so all of these were taken care of. And when the drivers actually drove it, the really interesting part was, the drivers were seeing all of this. The drivers who were driving after we launched versus the drivers who we tested with were different. And we could see that the drivers who were driving after the launch, they were actually recognizing how we have built in the whole driver experience into that design.

Khalen Morehead:
One thing I am particularly curious about is really there, how do you try to marry the design systems between Amazon and Rivian’s here?

Tirthankar Sengupta:
I would let Jackie take this one. Yes.

Jackie Chang:
Is that a really good question? Of course we all want to try to keep our brand identity in a product place. But I think it comes to understanding the user first. We know that today they are using a mobile phone tomorrow they may be driving a Rivian electric van. That mean it’s very important from user-centric point of view that their driver journey, they go through phone, different devices, now they’re driving the van. That experience shouldn’t be broken, it should be consistent throughout. So therefore there was a negotiation point with Rivian on trying to discuss, and understand, and we even have a meeting with leadership from their side and our side.

Both design team have put together proposals about using two different design system on one product. And we even have a research to support the outcome, if we use only Rivian or we using Amazon or both combined. So it was quickly comes to conclusion, keeping driver experience consistent within the last mile experience important, because that impact how they work, how efficient. If they are looking at the location pin, why on mobile phone is green and let’s say in a car is red, it should be consistent. We even quickly understood that. And after that design stress test on design system, user testing data, we quickly come to very convinced that yeah, we can do two design system in one product with no problem. And we have product engineering aligned from both sides and we quickly agree and align with that approach.

Tirthankar Sengupta:
Yeah. One of the things I would say that really gave us a way to make sure that we are not focusing on the design system itself, was to understand the driver experience as a whole. The holistic driver experience is not just the design system. There is one thing about the design system obviously. There is also the drive itself. The drive is not around the design system. That’s what the main experience is. So when we were discussing different features with Rivian, that would be... Like, there was one example was a floor mat. It’s completely nothing to do with digital experience, but we wanted to make sure that the floor was smooth, as well as had a little bit of friction so that the drivers wouldn’t slip, but it would be easy to move packages on the floor. So this surface design or the material design, that feedback had to come from drivers. And that feedback was given to the Rivian team, and they addressed it beautifully.

But the point I’m trying to make is, we were not talking about only, “Hey, this is the software, this is the design system, this is all about it.” We were just focusing it on the central was the driver. And then whatever happens to the driver in that van and around the van was the main focus of understanding the experience. There was another one which was very interesting, and it’s a digital experience, but there’s no design system involved in it. This was known as the rear view camera experience. So the rear view camera is basically, you have the camera at the back, and you take the camera and the feed and put it as a rear view mirror. Because since it’s a big van, you cannot use a rear view meter effectively. How do you present that camera in front on the central information display? Because what you want to do is, when you are reversing, you want to have the full... That’s a safety guideline according to the National Safety and Health Administration, or NHTSA, right?

Jackie Chang:
Mm-hmm. National Highway-

Tirthankar Sengupta:
National Highway Safety-

Jackie Chang:
... Safety Traffic Agency.

Khalen Morehead:
Safety traffic Administration

Tirthankar Sengupta:
Yes. So, we have to the full rear view camera on the central information display. How do you present it so that the drivers can see the best possible view to make decisions around that? So that was one, while reversing. While going on highway, you also want to use the rear view camera for making lane changes. How do you change that? So there were different decisions around that rear view camera compressing, and resizing, and how much view to show the driver so that they can make those decisions effectively. That didn’t need any design system, but it was a digital experience. And we were focused on that with Rivian. Like, okay, “Hey, can you build us different things?” So having multiple areas, and not just focusing on one also played in our role in terms of, we are working for the driver. So, what makes the most effective experience? We will take that path.

Khalen Morehead:
That’s great, yeah. I think that’s honestly what shines a lot about this project and what’s come out of it. And so even on top of that, it’s like, this wasn’t just a team that you took over to lead, but this was a team that was built from scratch. And so, what was it like trying to build a team to address this new area for last mile during the pandemic? And how do you account for that gap that we’d had in vehicle UX and other human factors during the course of all this?

Tirthankar Sengupta:
You’re asking me the question right now and I’m thinking, well, we did that. But when I was doing it I had no idea I was doing it. So one of the things that drove me to build the team, based on like, we need the best experience of best delivery experience in the van, was to make sure that the drivers actually loved these vehicles. Because if the drivers loved these vehicles, then they will adopt it. If they adopt it, then we will be very well on our way to the climate pledge. Because the more drivers do, the more they will drive the electric vans, the more they will be excited about it, and we will be able to invest in it. So that was one. The second thing was we kind of like, since it was a new area, we always wanted to innovate on the driver experience a lot. And the mobile application was there for two or three years at that point of time, and it had been built on a very basic prime now delivery scenario.

It had become extremely complicated in terms of the type of deliveries we made, the countries we were delivering into. So there was a refresh needed in terms of, what is the best possible delivery experience?

So this was really a good chance for us to create a new experience. And that also drove our understanding of, who should be a part of this team? Research was definitely already involved, so we had researchers like Paul, but I wanted to get a co-design technologist who would prototype all of these experiences very quickly and test with drivers. So I brought in Cassandra because of that. And then from our point of view of the strategic design, or having a long-term strategy, we wanted someone from the vehicle space.

Yes, there is obviously like we can get from the UX space or the tablet design space, but I considered it as the driver experience as a holistic experience of the van environment. How they interacted with the overall space, how they interacted with the outside as well. So that’s where Jackie came in, in terms of, “Hey, Jackie had 20 years of experience in Nissan.” Understanding not only just how to design vehicles or how to design the interior of vehicles, but more about the UX inside the vehicles and how the physical and the digital interaction played, and how they were actually informing the designs of different interior systems. Those were creating all those needs and requirements of the design team. Like, okay, what does this design team look like in order to create the right product? And especially this happened during the 2020 time period, so it was full on COVID going on the side.

Khalen Morehead:
Right. There were a few things happening around that time.

Tirthankar Sengupta:
Yeah.

Khalen Morehead:
Tell me a little bit more about, what was it like, and how the team was able to work together to bring this all to life. And particularly in the context of collaborating during the pandemic.

Tirthankar Sengupta:
Oh, wow. Yeah, I mean-

Khalen Morehead:
I don’t mean for it to be too loaded of a question, but...

Tirthankar Sengupta:
Yeah. I mean, it was very difficult. I won’t say it was difficult, it was very challenging to figure out how we will get this. Because when we started the project, there was no pandemic, and we had all the plans in place and everything. At the moment we wanted to execute the plan, COVID hit. So, we were like, “All right, how do we do the research now? And how do we do the design now? And how do we work with designers across the team?” And that was before Jackie joined. So essentially what we did, we had to start with the research. So, we implemented special COVID protocols inside the station. We had the vehicles had to be in secret, so there were black curtains all around the vehicles, and we would do the research inside the vehicle. My researcher Paul Barclay, he created the best COVID protocol to do the research in the station.

Used a lot of different types of tools, and monitors, and display connection software to create a makeshift usability lab inside the van, so that we can see and chime in inside the van cockpit, and understand what the driver is doing. So, there was a lot of technology being done. And then we invested in Protobi which is design tech software to create effective prototypes for drivers. So our design technologist worked with Jackie on all those prototypes to create the best possible close fidelity experience of those driver experiences. And obviously, getting the drivers at the right time, making sure that the drivers feel comfortable doing the studies, this was also a part of that, yeah.

Jackie Chang:
Yeah, yeah. Even from program team point of view, this is a very complex project. In order to keep the team in sync, we also have meeting person a day, a week in the office so that we can align and sync with the safety COVID protocol. Because which is a very high complexity with vehicle side, Rivian side, and that really helped a bit, a day a week, just with that. And also even for testing the vehicle on the road, we can only do a person at a time, and not two people in the van when we’re out there delivering and testing our design.

Tirthankar Sengupta:
Mm-hmm, yeah.

Khalen Morehead:
Yeah. I mean, it’s really great to know that you guys were able to try to just work up so many new solutions on the fly. I think that’s really the great thing about Amazon, and here is some of the innovative spirit. For me, I found the involvement with Rivian to be particularly exciting as someone who considers myself an automotive guy. And so, I am curious about, sort of from in that more consumer lens, do you think there are any of the features or learnings from this project that may ultimately cascade down into production vehicles? Whether for Rivian or other companies as far as, do you think there’s anything that was learned from this that would be applicable in those kinds of spaces that you’d like to see going forward?

I think there’s other questions for me around like, are these going to be vehicles that are proprietary for Amazon? Or are we looking to maybe take this technology, or some of the designs and things like that, and see if we can license them to other companies? See if there’s other ways we can try and help to electrify other fleets out there, and try to take this technology further than just where we are right now?

Tirthankar Sengupta:
So, right now what we are looking at, like you can parse this into two parts. One is obviously around the electrification of the fleet. The other one is around the new way of doing deliveries with the software that we have developed. Now, what our roadmap looks like is obviously we are going to electrify the fleet. And we will be using the software and all the capabilities of those vehicles in order to improve upon all the features that we have currently. But we are also looking at creating software for the ICE vehicles, or the internal combustion engine vehicles, as we say, the regular vehicles of the fleet. As we won’t be able to completely let go of all the vehicles immediately.

So how do we make sure all the drivers get the same experience? So that’s one of the next steps is like, “Hey, can we create experiences for drivers that are more modular in nature?” Or they can just take a tablet and do it. Or they can go through these experience. The drivers who are not driving the Rivian van, they are not bereft of these experiences. They are also able to leverage that experience, because that just creates better experiences for drivers. So that’s one of line of thought for us. The other line is, as we get into these vans and as we get into different types of technology and other sensors and cameras, et cetera, how do we leverage that to create more efficiencies across those workflows? Anything you want to add Jackie, on this one?

Jackie Chang:
Yeah. So for this question, when I think about it, we know that this project has been a revolutionary experience for drivers, really change how the way they work. And with that advantage like a Tirthankar has mentioned, we would like to make this experience and expand it. Make it available across all other combustion engine vans that can also offer these experiences. And therefore, I think with all these connected devices, think about a car and how it’s connected, and become smart and responsive to anything we want it to be. And that just bring in new way of thinking how delivery services can be offered internally.

And the whole idea is, we want to make these on-road last mile innovation platform available to our internal product team. And then, who know, maybe potentially this could be something that is a service that we can provide to other. And definitely working with Rivian, I think the best thing is that both sides, we both have learned a lot for instance to learn how to collaborate sometime on different agenda. But yes, we can collaborate, we can make it to work. And learning things like, what are the customer looking for? Sometimes we do have to learn how to constraint to certain things that are already in place from Rivian. It’s a continual learning process because every company has a different process.

Tirthankar Sengupta:
Now that you are talking about it, Jackie, one thing that triggered in my memory was, it’s not just only about working with 3B or like outside vehicle manufacturer companies, it’s also working internally. So we cover last mile. So there are other areas of the operations team which use vehicles for example, middle mile users, trucks and first mile users, some sort of vehicles. There are additional multimodal vehicles in last mile. So for example, we have bikes in some countries, we have walkers and Segways, and God knows how many types of vehicles that we use across different countries to make sure that we deliver to our customers as we promise them. How do we leverage our delivery software that we have built for Rivian into all those different modes?

Jackie Chang:
Yeah. And to do that, our team currently are creating more so on automotive UX design guide, so that we can enable more innovation with our internal partners. So we can have anyone who wants to bring in product to automotive, they can use our guide and then design and develop and integrate.

Khalen Morehead:
That’s amazing, y’all. I mean, it’s really great to just hear what we’re trying to do with last mile, trying to electrify the fleet, trying to really find other ways to leverage what the learnings were from here into the next steps. And the next phases, and the other areas that may not necessarily be exactly the same, but that we can still try to just enhance. And I think that’s really what it always comes back to. Gentlemen, I really appreciate the time that you’ve given us today, all the insight, and just hearing about everything that you’ve been working on, and obviously the work that you’ve done itself has been great. Tirthankar Sengupta, Jackie Chang, it has been absolutely a pleasure getting to talk to y’all today.

Tirthankar Sengupta:
Thank you so much Khalen. I’m just really proud of my team who actually did all the work, and delivered.

Jackie Chang:
Yeah. Thank you for having me.