In this episode of React Native Radio, Jamon, Robin, and Mazen are joined by special guest Dima Zaytsev from mobile.dev. Today we're talking about Maestro - a mobile UI testing framework that helps to simplify the testing process. Tune in as they explore the world of mobile UI testing and share how Maestro can improve your workflow.
In this episode of React Native Radio, Jamon, Robin, and Mazen are joined by special guest Dima Zaytsev from mobile.dev. Today we're talking about Maestro - a mobile UI testing framework that helps to simplify the testing process. Tune in as they explore the world of mobile UI testing and share how Maestro can improve your workflow.
This episode is brought to you by Infinite Red! Infinite Red is a premier React Native design and development agency located in the USA. With five years of React Native experience and deep roots in the React Native community (hosts of Chain React and the React Native Newsletter), Infinite Red is the best choice for your next React Native app.
Helpful Links:
Connect With Us!
Todd Werth:
Welcome back to React Native Radio podcast. Brought to you by the NSA. We know you have some concerns and we want you to know, we hear you. Episode 262 Maestro with Dima Zaytsev.
Robin Heinze:
Hey Mazen, do you have your kids' song stuck in your head right now?
Mazen Chami:
Yes. I'm actually singing Mickey Mouse Clubhouse right now instead of prepping for this podcast.
Robin Heinze:
(Singing) Yeah, that one.
Mazen Chami:
That's a very good, catchy song. Yeah. My Spotify is all out of whack.
Robin Heinze:
Yeah.
Mazen Chami:
My Discover Weekly, that recommends me songs?
Robin Heinze:
Oh no, it's trash. It's all-
Mazen Chami:
It's impossible.
Robin Heinze:
It's all kids songs. Yeah, don't-
Mazen Chami:
I used to love it.
Robin Heinze:
Don't...
Mazen Chami:
Yeah. No…
Robin Heinze:
Your Spotify recommendations are totally trash once you have a kid.
Mazen Chami:
Yeah, I can actually hear it right now. My wife is probably feeding him lunch, and has the music going on in the background, so that's probably my Spotify right now. She messing up my algorithm, so.
Robin Heinze:
I think Jamon's long past this stage of life. Your kids probably have their own Spotify by now.
Jamon Holmgren:
I don't even know what song you're talking about. What song are you talking about?
Robin Heinze:
Are you familiar with-
Jamon Holmgren:
Mickey Mouse, what?
Robin Heinze:
Mouse Mickey Mouse Clubhouse that maybe...
Jamon Holmgren:
Sort of, vaguely. Yeah, my youngest is nine, so.
Mazen Chami:
Well, there's also Baby Shark, which-
Robin Heinze:
Oh, yeah.
Mazen Chami:
He's into.
Jamon Holmgren:
Oh yeah, yeah.
Mazen Chami:
There's actually a version of Baby Shark that's about monkeys that he loves, also. So, yeah, we could have a whole episode about this.
Jamon Holmgren:
My girls are more into Taylor Swift, but then also, they weirdly are into a lot of songs that came out when I was 20.
Robin Heinze:
What?
Jamon Holmgren:
Backstreet Boys.
Robin Heinze:
Is it coming back?
Jamon Holmgren:
And stuff like that.
Robin Heinze:
I hear the '90s are coming back around. Fashion, music, it's coming back.
Mazen Chami:
It's NSYNC or Backstreet Boys, one of them go on tour or something recently?
Robin Heinze:
Did they?
Jamon Holmgren:
Maybe that's why.
Mazen Chami:
Yeah.
Jamon Holmgren:
I think so.
Mazen Chami:
I remember vaguely hearing something that one of them is back on tour.
Robin Heinze:
That was my first CD when I was 10 years old.
Jamon Holmgren:
Oh, NSYNC, okay. Yeah, it's funny because they'll just be rocking out to songs that I listened to. Yeah, it's fun. But hopefully I can avoid Mickey Mouse Clubhouse for as long as possible.
Mazen Chami:
It's catchy jingle. Be careful.
Robin Heinze:
It's a bop.
Mazen Chami:
I don't know.
Robin Heinze:
I think you wouldn't mind.
Mazen Chami:
Don't play it on YouTube because-
Jamon Holmgren:
I don't know. It seems-
Mazen Chami:
Yeah. It's a downward... Or on Spotify.
Robin Heinze:
It will get stuck in your head though, so maybe don't. Yeah.
Jamon Holmgren:
Yeah, it just gets in the algorithm, and then you're done. Okay. Well, should I do intros? We got to try to keep the earworms from spreading here. So, I'm Jamon Holmgren, your host, and friendly CTO of Infinite Red. I live in the beautiful Pacific Northwest with my wife and four kids. I play recreational hockey. I actually played super late last night. It was my third night in a row. I started the game at 9:45 PM, and didn't get out of there till after 11:00 PM. Yeah, 11:00.
Robin Heinze:
I can't believe you play hockey while I'm asleep. How do you even have energy at...
Jamon Holmgren:
I didn't.
Mazen Chami:
My team was supposed to play at 10:00 last night and I told them I can't make it because one, yes, I was asleep by then and two, I had to be up at 5:00.
Jamon Holmgren:
Why'd you have to be up at 5:00?
Mazen Chami:
Because our new tenant loves to wake up at 5:00 screaming.
Jamon Holmgren:
Your new tenant, your son?
Mazen Chami:
Yes, that tenant.
Jamon Holmgren:
I'm joined by my amicable co-hosts, Robin and Mazen, and our guest Dima, who I will introduce in just a second. Robin is our lead software engineer at Infinite Red. Robin recently got promoted. Congrats Robin. She's located west of Portland, Oregon with her husband and two kids, and has specialized in React Native for the past six years. Mazen lives in Durham, North Carolina with his wife and new tenant. He's a former pro-soccer player and coach and is a Senior React Native engineer also here at Infinite Red. And we also have Dima Zaytsev, he's a lead Engineer At mobile.dev. He lives in Amsterdam, and he comes from a background of Android development. He, for example, created photo APPRIGHT, which is a camera library for Android, a popular one. But says he does all kinds of things these days, so we're going to be diving into some of that stuff as we get into it. Welcome to the podcast, Dima.
Dima Zaytsev:
Hello. Thank you for having me here.
Jamon Holmgren:
Awesome. Can't wait to get into it, but I do have to mention that this episode is sponsored by Chain React, America's best React Native conference is back. It's happening May 17th through 19th in Portland, Oregon. We have workshops, we have lightning talks, we have some great speakers, and some great talks. We're going to be going through all the CFP stuff. I think it'll be closed by the time this comes out. Sorry if you didn't get your talks submitted. I did tweet about it like 45 times. But it's going to be an awesome conference, and I've seen some of the talks that have been submitted and they are absolutely fantastic. You're not going to get this content at other conferences. A lot of conferences are like react conferences that bolt on React Native. This is all React Native. 100% React Native.
Robin Heinze:
First class.
Jamon Holmgren:
All right, let's get into our topic. We're going to be talking about Maestro, which our clever title is The Absolute Solution for Mobile UI Testing.
Robin Heinze:
Can you guess who came up with that title?
Mazen Chami:
Not the person that delivers the mom jokes-
Jamon Holmgren:
Yes, I can guess.
Mazen Chami:
Or the person that delivers the mom jokes.
Jamon Holmgren:
Exactly. The person who delivers the mom jokes.
Robin Heinze:
Oh, I didn't come up with it. ChatGPT came up with it.
Mazen Chami:
Oh.
Jamon Holmgren:
Oh, okay.
Mazen Chami:
You couldn't easily said you did, Robin.
Jamon Holmgren:
And you even admit it.
Robin Heinze:
I could not take credit for pun that punny.
Jamon Holmgren:
That's pretty good, actually. All right, Dima, let's start. Just tell us a bit about yourself. What's your original background?
Dima Zaytsev:
Right. Well, I am coming from a Android development background. Something I've been doing for... Gosh, who knows how many years now, 10-ish. Until eventually, after hopping between companies, startups, mostly. Uber at some point. Yeah. Decided to pick up a role of developer platform engineer, if that makes sense, right? Combination of everything. Android, backend, frontend, iOS, you name it.
Jamon Holmgren:
Yeah. That's fantastic. Who else is behind Maestro? I want to know more about the team.
Dima Zaytsev:
Oh yeah, the team is unique. I would say in a sense that we're fully distributed across the globe. So, we have seven people and we have people in Brazil, India, EU, Dubai, United States from... Yeah, I would say stellar teams.
Robin Heinze:
That's quite an international team. Is it hard to work across that many time zones?
Dima Zaytsev:
We figured it out. I would say it is a bit of a challenge. We need to get used to it, but we do have a decent overlap in time zones, right? So about two hours a day, which also-
Robin Heinze:
That's your go time.
Mazen Chami:
Yeah.
Dima Zaytsev:
Yeah. Also limits the number of meetings we have, right? Because you cannot have meetings outside of this time window, so that's convenient in its own way.
Robin Heinze:
It really forces you to hone your async communication and written communication. Which is a good thing.
Dima Zaytsev:
Absolutely.
Mazen Chami:
It's also good because based on the countries you just mentioned, I feel like someone is always working on the product. So, when Rob and I, when we were working on Mercari, it was a team out in Japan. And when I signed off or when you signed off, it was starting their day.
Robin Heinze:
It was a hand off to Japan, and they work all night.
Mazen Chami:
Yeah, like, "Hey, can you guys... I need an API call, or I need this fixed," they'll fix it. You come in the morning, there's a Slack message, it's ready to go, you implement it and you just keep rolling. No one was really blocked. You were able to just keep moving forward.
Robin Heinze:
Well, let's talk about Maestro. Can you give us a TL;DR, what is it? Why were you inspired to make it? For our listeners who may not know.
Dima Zaytsev:
Well, now that you mention ChatGPT pun, I wonder if I can outdo it. My story is a simplest UI testing framework. There is really, I doubt we can find anything simpler in terms of usability.
Robin Heinze:
Having used it, and I agree.
Dima Zaytsev:
Yeah.
Mazen Chami:
That's a fact. It's not an outlandish claim.
Robin Heinze:
Yeah, that's not an exaggeration.
Dima Zaytsev:
It takes literally one line to install it, and I think, what is it? And three lines to write the simplest end-to-end task, which is useful.
Mazen Chami:
What was the inspiration behind it?
Dima Zaytsev:
Inspiration comes from a use case of our product. When mobile.dev started originally, we had a completely different product in the very beginning, and it was around measuring performance of applications, and we still do that, to be fair. But when measuring performance, memory leaks, you name it, you need to navigate through your application somehow, right? And he first thing which came to mind, "Well, we'll just ask our customers to do it themselves, just write the test, send it to us, we'll run it." And soon enough we'll realize two things. Either one, their tests are flaky, and they just don't work in 10% of the cases, which is decent, but when you run it on every poor request that creates problems. And two, that is other potential customers are simply afraid of writing your test because they burnt in the past trying it out. They realize it was, yeah, it didn't work for them and now they don't want to do it again.
Robin Heinze:
Well, it's a lot of time investment for it to not work, or be brittle, or not effective, and then you have to pivot and you've just wasted a bunch of time.
Jamon Holmgren:
Brittle tests just feel worse than no tests at all, pretty much. It's just so annoying. If it's going to fail two out of three times, which sometimes you get to that point, then it just feels like it's not worth it at all.
Dima Zaytsev:
Yeah, exactly. It doesn't have to be that way, right? If I will give you an application, say YouTube, right? Say we are working on YouTube app, and we are working on a feature, it's a search feature. And if I ask you to test it, that really is open the app, tap on the search, enter the text, verify something is visible. Now, try it with any testing framework out there and you realize that okay well, so many things you need to keep in mind, so much setup you need to go through. While the gap between explaining it to human being is just so big, right? So, that's something we try to replicate with Maestro's feeling of you just goes through the app like a user would.
Robin Heinze:
Mm-hmm (affirmative).
Mazen Chami:
So, we've used Maestro, and this next piece here that I'm going to talk about. We've also been able to use it, but there's been a lot of buzz lately, and I think Leland, who I believe is your partner at Mobile.dev, he posted a blog article about Maestro Studio. Can you explain a little bit what Maestro Studio is, and does it work in Tandem with Maestro, or is it something separate? How do the two interact together?
Dima Zaytsev:
You can think of Maestro Studio as lightweight ID for Maestro itself that allows you to write tests by simply clicking on the device screen. Or it runs in your browser and comes repackaged with a Maestro itself. So, it's a part of a C-Line tool which then opens on your browser.
Jamon Holmgren:
You can just run Maestro Studio?
Robin Heinze:
You don't have to install anything new. Yeah.
Jamon Holmgren:
Yeah, that's cool.
Robin Heinze:
Yeah, I tried it out on the project we're working on, and it's so cool. You could click around and it'll show you what you're clicking on, or you can write tests live just to see what they do and what works and how to get things to pass. And then, just recently, I know you guys released not that long ago, a new version, which lets you then export what you've just created so it's super slick.
Mazen Chami:
Yeah.
Dima Zaytsev:
Let's say motivation behind it, right? Can we get this tool into hands of everyone doing... Do you really need to be an engineer to write those tests? Can my product manager write it instead, right?
Mazen Chami:
Right. Yeah, that's exactly what I was just going to say. I think Maestro Studio unlocks the potential where a non-developer can come in and build it, export it, and then pretty much send it to the developers, and it can get committed in the code that way. I think also one thing to outline is the output file is in a .YAML. So, I think that's something some people might be scared about specifically, but I think it's very simple. Your documentation pretty much outlines how to write each one, and makes it easy and then Maestro Studio just does it for you. That's just pretty cool. Those that don't like YAML, use Maestro Studio.
Robin Heinze:
YAML has pros and cons.
Mazen Chami:
I like it.
Robin Heinze:
It's nice, it's clean.
Mazen Chami:
Yeah.
Jamon Holmgren:
That's spoken like a true lead engineer.
Robin Heinze:
Pros and cons.
Jamon Holmgren:
Yeah, Robin. Pros and cons.
Robin Heinze:
It depends.
Jamon Holmgren:
Yeah, trade-offs. You can have a very confident, wrong answer, or you can get, it depends.
Dima Zaytsev:
Yeah, we got quite a lot of feedback about YAML, specifically, because that's not something people are associating with testing generally. Right?
Robin Heinze:
Yeah.
Dima Zaytsev:
I need to write my test. It is a program which I'm writing and that's how we've been doing it for 10 plus years now. So, YAML seems like a toy almost, right?
Robin Heinze:
Mm-hmm (affirmative).
Dima Zaytsev:
But that's a point almost, right? When you are using the app, the same YouTube app, let's say you are not writing a program, you're just tapping on the screen, do this, do that. That's all right, right? Making it complicated is, of course, possible still even with Maestro, but that's something we're trying to dissuade people from doing in a way. As well as having it in YAML allows us to do quite a lot of optimizations, which would not have been possible with a regular programming language like JavaScript because we know in advance how your test will look like and you give us a whole plan, we can look at it, "Okay, after tapping on this, you're going to something else, right?" We can look ahead, we can look back, we can see you. We understand your test.
Jamon Holmgren:
Yeah, which is a lot harder to do when you have something that's more like in JavaScript or something, that's very difficult to do. So, one thing I want to do is cast this for our audience because React Native radio, we have React Native developers. Maestro can be used for React Native or Non-React Native, it's just any mobile app. Including built-in ones on your simulator, you can just test if you wanted to test the setting gap or whatever, you can do it. So, that's pretty cool, it's across. Now I want you to talk a little bit, Dima, about the difference between Maestro and the other end-to-end testing frameworks that are popular among React Native developers. So, Appium is the granddaddy, I don't know, maybe there's some others that were before, but it's the one that's been around for a long time. Probably one of the ones that was causing you grief, Dima, back in the day.
Robin Heinze:
Probably one of the most annoying. I think I spent a week once just setting it up.
Jamon Holmgren:
Just setting it up. That's the problem. It's very powerful, but it just has... I don't know, getting the developer experience to the right spot was just very difficult. Then you have Detox, which came out of Wix, which really tried to attempt to fix the problem of flaky tests by knowing what was going on under the hood. So, gray box testing rather than black box testing, black box being like, "We don't know what's going on, we're just going to tap around and observe what we see." Gray box Detox is like, "Okay, we know that there's a network request happening right now, so we're going to wait until that network request returns." And then we'll say, "Okay, now we're ready to move on." Great idea in theory and we have had the Detox team on the podcast and you should listen to that because there's some really great stuff in there.
Done some very cool work. But it never quite hit, I think, for a lot of people the promise of what we really wanted, which was just do your test, forget about it and move on. For some apps it worked that way, but it didn't seem like it worked all the way that way. And the bigger problems a lot of times came up like, "Okay, we want to test this in a device farm or something like that." And it wouldn't work because it wasn't really designed for that. So, hopefully I didn't spoil too much of your answer, Dima.
Robin Heinze:
Did you just answer your own question?
Jamon Holmgren:
I'd love to hear your perspective on what makes Maestro different, maybe better in some ways than Appium and Detox.
Dima Zaytsev:
Yes. Well, first of all wanted to give kudos to developers of Appium and Detox because we are building on the shoulders of giants in a way, right?
Jamon Holmgren:
Mm-hmm (affirmative).
Robin Heinze:
Mm-hmm (affirmative).
Dima Zaytsev:
They did have quite a lot of good ideas, some of which we inherited, some of which we redid our way. And I guess I'll start by talking about Appium first and what is different. So, you can think of Appium as a translator between your JavaScript codes and actual testing framework under the hood, right? It takes your logic, connects to the device and then delegates everything to a testing framework. So, it does make your test cross-platform, it does make them faster compile. And that's great. And I think that part works. However, fundamentally doesn't change a game. It still depends on say, Espresso under the hood or UI Automator or XCUITest or whatever the underlying mechanism. As plus, yeah, setting it up a bit of a challenge. And then Detox I think tried to fix some of those issues and whereas their gray box testing framework Maestro still is a black box in a way, but they do understand more about the app, right? That's a crucial component.
They already venture it into the area of, okay, something needs to change fundamentally about the testing behavior itself, right? It's not enough to just trust, say Google and Apple to handle that. Unfortunately it seems that testing has been a second thought for Google and Apple. Yes, framework works. Yes, you can test your applications, you can use it. But when it comes to answering hard questions, it doesn't handle it quite as well. Like network interactions being one, right? So, where Maestro is different is that all the brains of test execution actually live in Maestro itself. So, on your desktop, not on the device. Maestro tries to trust external frameworks as little as possible, right? Just bare minimum to get operations like tap on the button, right? Sorry, not even on the button. Tap on screen coordinates or input text into a field or give me a view hierarchy and that's it.
The rest happens within Maestro itself. Understanding what view to find, how to find it, then how to recover from failure. Say if tapping fails and what to do? If a limit is not found, what to do? Maybe scroll. And that's what makes it fundamentally different.
Robin Heinze:
That makes a lot of sense. It feels like this is going to be the way of the future for integrated or end-to-end integration tests on mobile specifically. This is the first time I've actually wanted to use end test. It's faced with the prospect of writing Detox or Appium tests. It's like a chore that you're like, "I should do that." But you don't want to. This is the first time I've ever actually had fun writing tests and I'm like, "Okay, this is really cool. I want to add these to my app." So, I think this is pretty revolutionary for the space for sure.
Jamon Holmgren:
I think the end-to-end tests in general, for me, they've always felt like the most bang for your buck if you can get them to work really well. It feels like the amount of effort you put into writing an end-to-end test, you test so many things. Everything has to work together in order for it to function how it's supposed to. And it feels like the most pure test in a way because at the end of the day, what matters is what the user is tapping on, seeing come up, interacting with-
Robin Heinze:
It doesn't matter what code is running to make that happen, it matters whether it works.
Jamon Holmgren:
Exactly. You literally could have a function that returns the wrong result but somehow gets corrected for later in the system and the app works.
Robin Heinze:
But that's what your unit tests are supposed to catch. That's why you have unit tests.
Jamon Holmgren:
Yeah, unit tests catch that. But at the end of the day, does the user care? They don't. They don't care that something under the hood is being patched together. They really just care about the end result. And so it feels to me like the most pure type of test. Some developers would probably disagree with me. They feel like a unit test would be the most pure, but to me, no way. It's all about the user. It's what the user is experiencing.
Robin Heinze:
I think you can have an entire suite of unit tests that are beautifully written and they all pass and your app is still broken.
Jamon Holmgren:
Yeah.
Dima Zaytsev:
People often talk about the testing pyramid. You should have a lot of unit tests, then you should have slightly less integration tests and then end-to-end tests. But reality is you sometimes might flip it around, right? You will only need say five to 10 end-to-end tests to cover majority of your application, whereas it'll take thousands of unit tests to do the same. And even then by the end of the day, if you all your unit test green, you're still not sure whether the app is working. You still like to go straight by hand and test it. Whereas again, and then test if it covers it, that's great, right? You have much more. Assuming it is not flaky of course, because then trust is ruined and you no longer sure that's just by chance that it works this time or maybe that by chance that it failed.
Robin Heinze:
You start ignoring the failures and then your test will be as useless.
Dima Zaytsev:
Yeah, you're becoming complacent, right?
Robin Heinze:
Mm-hmm (affirmative).
Jamon Holmgren:
Yeah. I wonder how much of the testing pyramid stuff is because unit tests are easier to make solid and not flaky. If it was the other way around where unit tests were more flaky and end-to-end were just super solid, I don't think that anybody would write unit tests.
Robin Heinze:
No one would write unit tests.
Jamon Holmgren:
Yeah, it's just about the complexity of writing really good end-to-end tests. And I think-
Robin Heinze:
I was going to say we're simple creatures, Jamon. We get a dopamine hit from a unit test that you write and it goes red and then it goes green and then it stays green forever and-
Jamon Holmgren:
Half the time I swear developers are just playing Sudoku and getting paid for it. They can rationalize this, but it's not really helping. It's having super, super strict TypeScript. Oh, no. I pissed everybody off.
Mazen Chami:
I love super strict TypeScript by the way, side note. Anyways what I was going to say...
Jamon Holmgren:
You like Sudoku too, right?
Mazen Chami:
I do, and I finished an expert one last night. Anyways, back to what I was going to say. I think from my experience, prior to Infinite Red, I spent a lot of time in startups for many years just bouncing back and forth, getting some up off the ground to get acquired and then move on to the next one. That type of thing. And I hate to say it, but tests, and sometimes documentation fell by the wayside. And to be quite frank, Appium and Detox at the time had a very high bar to get it set up initially and most of the time discouraged. But then once, I remember one instance where I allocated time specifically to getting Detox working. I remember just getting it to work where the uploads success and I was like, "Done, I'll come back tomorrow." And then tomorrow, a whole other storm came up and I never got back to my Detox. But hey, my Detox was still successful because I had that one that loaded the app and it was successful.
So I think Maestro is a big game changer and almost magic at this point because it's a quick setup. And using Maestro Studio, which is what I had initially used to get it set up to do one or two things, I think it was a click and then putting in texting in an input field, that took me minutes. I wouldn't even go as far as staying out. I mean, I just talked about Detox taking me a whole day to get to do one test. I wouldn't even call it end-to-end, just a load the app test, took me a day. And this took me a couple minutes and I got it up and running. So that magic makes it much more valuable and will probably help a lot of people move faster and get that much more coverage on their app. It's great. I mean, yeah, magic.
Robin Heinze:
I want to pivot a little bit to the technical side of Maestro and how it actually works. And Mazen correctly described it as magical, which it is, it feels magical. I want to talk about what are some of the technical challenges that you overcame to make this work and to make a product that has such a simple input and does something so complex as controlling an app. What's going on under the hood? Can you explain some of the magic and how you made it all work?
Dima Zaytsev:
Yeah, ironically, most of Maestro cold base is rather simple if you look at it, and of course, yeah, it's coming from a developer, but...
Robin Heinze:
It's irrelative.
Dima Zaytsev:
It's moving data structures around. It sure is like, here's a view element, here's how to find it, Traversetree. So that's all builds on top of the actual magical source, so to say, which are drivers as we call them. And driver is, think of it as a way to communicate with a device. That's not a new idea, we actually took it from Appium as well. Driver tells you what's on the screen right now. You can ask driver to tap on the screen. And the beauty of a driver, it is platform-agnostic, right? We have a single interface and a different implementations for Android, iOS. Might have some special one for React Native at some point perhaps.
Jamon Holmgren:
Is that ADB and IDB? Or are those different?
Dima Zaytsev:
In a way, yes. ADB is or rather Dadb is what we're using for talking to Android device. So, for those who do not know, Dadb is a programmatic access to the same features which ADB and Android Debug Bridge provides. Whereas IDB is a tool built by Facebook actually to replicate some of the same functionality for iOS which approximately are fairly well with some data exceptions here and there.
Robin Heinze:
React Native uses IDB a lot, right? Just for various tooling.
Jamon Holmgren:
Yeah, it does. Yeah.
Robin Heinze:
Launching simulators and stuff.
Jamon Holmgren:
Right.
Robin Heinze:
Yeah.
Jamon Holmgren:
Right. We'll talk about IDB later when it comes to supporting real devices because, spoiler, that's actually the blocker. When did that happen? But besides that, possibly the hardest problem to solve for us was to get the view hierarchy from the device. And you'll discover that neither ADB nor IDB can provide a good enough source of data. So, what we figured out is, well, those tools don't give that to us. However, XCUITest does or Espresso on Android does a UI meter does what if we'll write a test or rather we'll call it a test. I'll push it to the device and we'll run it and this test will never return. It'll just block forever and will start the server at the same time on the device itself. Which allows Maestro to communicate with it and actually pull the UI hierarchy information from the device or a sort of unconventional use of testing frameworks, so to speak.
And that's where most of magic comes from because now having this hierarchy in platform-agnostic way, we can do anything with it. We can traverse it, we can find views in which have a fashion level we would like to, we can decide how the framework itself works. Something we couldn't control, if say we'll just depend on Google's or Apple's work or we can surface it in a web page, right? That's where Maestro Studio actually comes in. It just takes a screenshot and takes the hierarchy and draws it on the screen.
Robin Heinze:
It's so cool. It blows my mind every time. I'm excited to keep using it and I'm glad you guys were able to figure it out because I'm enjoying using it.
Mazen Chami:
I want to go back to your spoiler about the devices, but then I have a more specific question. Do you guys foresee adding the physical device support specifically for hardware stuff like camera, specifically?
Dima Zaytsev:
Having worked with camera in the past, actually several years in my career is such a hard problem to solve how it works. Today it doesn't work, tomorrow it works on this device but not on that device. And the answer is yes. I mean, of course we want to add it and we'll get there as well. So Maestro already supports some physical devices, on Android it does work. Camera specific functionality will add eventually, I'm sure same as geolocation.
Mazen Chami:
Geolocation, yeah.
Dima Zaytsev:
Yeah, Bluetooth as well, perhaps. A lot of things are possible, which you haven't even started exploring. First we want to polish things out so that it works mostly for anyone on any machine. We know that Windows users are having a harder time to set it up. Unfortunately it's not quite as magical yet. So, once all those ravages are addressed, I'm sure we will tap into more sophisticated problems.
Mazen Chami:
Cool. So, basically iOS and Android simulators and then some physical Android devices will work.
Dima Zaytsev:
That's right.
Jamon Holmgren:
So, I was talking with Leland from your team there, and I've been getting into a lot of AI stuff lately, and as a lot of people have.
Robin Heinze:
ChatGPT.
Jamon Holmgren:
And chatGPT and openAI and some of the stuff coming out of Google. And there's a lot of really fun, interesting, very exciting things coming out right now. And I could envision a world where instead of writing YAML, you could write plain text, just click this. You could literally write it like you would talk to someone. How would you write it in English for a manual QA tester? You just typed that out and have them do that. And Leland was like, "Well, we could do that and we could also just have you just push a button and have it explore your app." And I thought that was pretty cool. So, is that something that it would be on the roadmap map? A hundred percent automated testing, have it just explore your app and learn on its own what your app does?
Dima Zaytsev:
Well, that would be a lie to say if I haven't thought about it. But yes, it's all seriousness. That is the next step, I think, right?
Jamon Holmgren:
Mm-hmm (affirmative).
Dima Zaytsev:
As I was mentioning earlier with YouTube application as an example. Yeah, I can describe you how to navigate through the app and YAML is just one format, of course. Yeah, can also do a natural language processing and all that. But ideally, right? If you are a user and I give you an application, YouTube or whatever else, you can probably figure out how to use it without me explaining as well as just generally understand if it's broken or not, right? If it is behaving as intended or not. That's a future where I think Maestro is heading and testing in general, where apart from manually crafted tests for the most business critical use cases, you would have some a crawler which goes through your app understands where can it go or what can it do. You just give it a general context, okay, here's a username and the password and then the rest you figure out.
Jamon Holmgren:
Yeah. Yeah, it might have some guardrails like, "Hey, don't go delete your account or something. But otherwise, here are some general ideas of what you might want to try to achieve with the app." And it just goes and figures out how to do it and then reports back if it runs into roadblocks and things like that. It would be interesting not to get too far a field, but it'd be interesting to even get it to the point where it could detect human user experience issues. It's hard to figure out how to do this and the AI is figuring that out. But this is all way down the road. Obviously, right now you're focused on more the programmer interface, the Maestro Studio side of things. All the AI stuff that I'm thinking about is, who weights out?
Robin Heinze:
I think, like you mentioned, you're trying to get the basics to work really solidly and consistently. And I think that's smart to build this foundation of trust first and get a lot of buy-in from the developers and then you'll have that foundation for when you start doing bigger and better.
Dima Zaytsev:
And for all the listeners out there, as I mentioned, Maestro under the hood, if you exclude the driver, you should never need to touch yourself. It's actually not that complicated to use, right? You can look into source, you can find the Maestro class, it's literally called Maestro class. And it gives you the view hierarchy, it gives you everything that is the screen. You can tap on it, you can do whatever you want. If you just feel like it, maybe you'll be the one building it, right? Maybe you'll figure out how to automate it, how to build the iron rounds and the tools are there, they're ready to be used. YAML is just one way to use it. But under the hood, everything is just calling Maestro class.
Robin Heinze:
Oh yeah, YAML is just a way to communicate a bunch of data.
Mazen Chami:
So, wrapping up this conversation, Dima. What do you think is next from Maestro and Maestro Studio? What is the grand vision that you guys are trying to get to?
Dima Zaytsev:
As I was mentioning before, reliability as a whole. As a former Android engineer, I feel as a second thought for Google and Apple. That's not casting a shade, I think it's just reality. And we would like to close that gap as a whole, not just testing, but other areas as well. So, I want Maestro and Maestro cloud to become a hub for everything reliability related in your application sync, screenshot testing or working network connections, right? Feature flags, dynamic configuration, performance monitoring, and so on and so forth.
Jamon Holmgren:
Yeah, it's a big vision. You all are doing great work though, and you've been very accessible. We've asked questions, you've been very on top of things when we've had any sort of... We had some issues, I think, early with a few things that maybe weren't completely implemented and you all just released them within a week. It was crazy how fast you turned some things around for us, which was really cool. One of our biggest apps actually has already moved to Maestro from Detox. And their Detox test suite was just not working. Literally, they just didn't run it because it wasn't working. It was very, very difficult to get it running. And they were so happy with Maestro, they switched over to it and they're in the process of converting all those tests over. So-
Mazen Chami:
And I can even say that an even bigger client that I'm currently on, I'm in the process of prepping a Maestro demo for them. And I'm very excited for it because I think once they see it... I mean, the first meeting we had about testing was, okay, so what's our Detox strategy? I was like-
Robin Heinze:
Love that.
Mazen Chami:
"Let me blow your minds. Let me blow your minds and show you something else."
Robin Heinze:
Pin in that.
Mazen Chami:
So, they were like, "All right, sure. Give us a demo and we'll go from there." And I'm very excited because when I get to it, they're going to realize the amount of time it took me to get to the demo was so much less than the Detox roadmap that we had planned. So, I'm very excited.
Robin Heinze:
I feel bad for the Detox folks.
Mazen Chami:
It's not casting any shade, it's just there's different tools.
Robin Heinze:
Who do great tool. Yeah, it's a different tool.
Mazen Chami:
Yeah.
Jamon Holmgren:
Yeah. It's a hard problem for one thing. Yeah.
Robin Heinze:
It is. It's an incredibly hard problem.
Jamon Holmgren:
It's a very difficult problem. And as Dima said, standing on the shoulder of Giants, we always have things come out that are based on previous things and it build on those. So, Dima, you're going to have to go to Wix and convince them to start using Maestro internally, that would be a huge cue. But we do have to wrap up here. It's been fantastic, Dima, I really appreciate having you on. And of course, all the best of luck with Maestro and Maestro Studio. If you'd like to nerd out more about Maestro, you can join our Slack community at community.infinite.red. We have over 2000 React Native developers in there. There are already some conversations happening about Maestro in that community. Of course, actually, Dima, where can people communicate with you all over there at Maestro?
Dima Zaytsev:
We have a Slack channel dedicated to Maestro as well with almost a thousand members. So, we can ask questions, we try to reply or go to Github and open an issue, we apply to those as well. But Slack is preferable.
Jamon Holmgren:
Fantastic. And where can people find the mobile.dev team on Twitter, Dima?
Dima Zaytsev:
Our handle is mobile__dev. That's two underscores.
Jamon Holmgren:
Perfect. Yeah, more underscores, more fun. You can find Robin, our new lead software engineer at Infinite Red at Robin_Heinze. She only has one underscore, but she's still cool. You can find Mazen at Mazen Chami, neither Mazen nor I have underscores at Jamon Holmgren. And of course, you can find React Native Radio at React Native R-D-I-O. Thanks so much to our guest, Dima, for joining us today. And as always, thanks to our producer and editor, Todd Werth, our assistant editor and episode release coordinator, Jed Bartausky, our designer Justin Huskey and our guest coordinator, Derek Greenberg. Thanks to our sponsor Chain React, check it out chainreactconf.com. Hopefully we'll get some folks there. Maybe from the Maestro team, that'd be awesome. Special thanks to all of you listening today. Make sure you subscribe on whatever podcasting platform you prefer. Subscribe on all of them. Hey, we like the numbers right? We're well over 5,000 now. I should actually actually figure out what the actual number is now. Robin, do you have a mom joke to send us out in style?
Robin Heinze:
I do. Now this one's courtesy of Gant Laborde, the one and only. How do 37 math petitions ride a bus with 36 seats? They carry the one. Oh, you love them. You know it.
Jamon Holmgren:
All right. See you all next time.
Robin Heinze:
Bye.