Former RNR host Josh Justice along with his coworker Jason Grosz come on to talk about their experience using Detox on a client project.
Former RNR host Josh Justice along with his coworker Jason Grosz come on to talk about their experience using Detox on a client project.
This episode is brought to you by Infinite Red! Infinite Red is a premier React Native design and development agency located in the USA. With five years of React Native experience and deep roots in the React Native community (hosts of Chain React and the React Native Newsletter), Infinite Red is the best choice for your next React Native app.
Helpful Links:
Connect With Us!
Todd Werth:
Welcome back to React Native Radio Podcast, brought to you by Awkward Silence. Episode 264, React Native Testing with Detox.
Jamon Holmgren:
So my two co-hosts, Mazen and Robin, are both out today, but that doesn't mean I am alone here. I actually have two guests, which is really fun and one of whom has actually hosted this podcast in the past, which is really fun as well. And the other guest let me know that he actually has both a tractor and a shed. So this is a perfect intro for me. This could not be better. Jason?
Jason Grosz:
Yes.
Jamon Holmgren:
Before I introduce you, let's talk really quickly about how did you end up with a tractor and a shed? You must live on some acreage somewhere.
Jason Grosz:
Yeah, a little bit. So we bought a house from my in-laws. My father-in-law is a badass mechanic forever. I know nothing about internal combustion. There's many nerd stereotypes that I do not comply with, but that's one I know nothing. So we've got almost two acres in Wisconsin. It's fenced in. We've got a whole lot of mature trees and we've got a 40 foot by 30 foot wick out building.
So it's a metal shed on a slab with 220. We've got a welder and he left behind a whole bunch of gear, one of which is a mid-eighties Simplicity lawn tractor that I just used for hauling stuff around in the yard. The power takeoff doesn't work anymore. It doesn't have a mower deck, it's got chains on it. And it's been hacked up by my father-in-law such that he replaced the engine and the exhaust comes out the front.
So whenever you're driving it, you just get this delightful ingestion of fumes. So it's a little bit ... You don't want to be on it.
Jamon Holmgren:
It sounds like a farm tractor. That is hilarious.
Jason Grosz:
It's great. So needless to say, my inner eight-year-old is just filled with glee every time I get to drive it, right?
Jamon Holmgren:
Absolutely. Does it have a front loader at all or is it just a tractor?
Jason Grosz:
No, it does not. It does not. I wish it did. We would love to buy one of those. It's the usual thing. And then I also have a zero-turn Scag mower. And so those are like a tank. So each wheel operates independently and that also fills me with glee. So I have about a four-hour a week mowing project all summer here in Wisconsin.
Jamon Holmgren:
If you have that much acreage, which I mow around two acres as well, and I think I have three, but some of it is in the woods.
Jason Grosz:
Right, similar with ours.
Jamon Holmgren:
So you might as well make it enjoyable. That's when I listen to podcasts.
Jason Grosz:
There you go.
Jamon Holmgren:
I don't have a commute, so I sitting on my tractor, sitting on my mower, pulling weeds, that sort of thing. That's when I get to actually listen to podcasts.
Jason Grosz:
And for folks who are in drought-stricken areas, I just want to reassure you, I do not water the lawn. I live out in Wisconsin, I water a little when we plant new grass. But we have shade. We have natural, it's this whole thing of water that falls from the sky on a regular basis.
Jamon Holmgren:
Yeah, free water from the sky.
Jason Grosz:
Yeah, yeah, exactly.
Jamon Holmgren:
We have a little too much of that here. I've got some flooding problems sometimes, but deal with it.
Well, I appreciate that, Jason. It's always great to have a fellow tractor owner here on the podcast. It's a rare thing. So it's a lot of fun. I will do intros now. I am Jamon Holmgren, your host, friendly CTO of Infinite Red. I live in the beautiful Pacific Northwest with my wife, four kids, cat and tractor. And I play rec hockey sometimes, which is fun. In fact, my son Cedric just played his first game with me in rec hockey last Saturday. So that was a lot of fun, and I didn't even think he'd be interested in it, to be honest. And he picked it up. So it's going to be a fun season.
Jason Grosz:
That's great. I'm in a band. My main collaborator has three kids. For a while, all three of them were in hockey, and hockey is in competition with me getting to play music with him. And I said, at a certain point I'm like, your kids need to stop winning so much because hockey season goes on freaking forever. It just goes on and on. Yeah, it really does.
Jamon Holmgren:
It's the whole year, really. There's summer leagues, spring, fall, or I mean fall, winter and then spring, which we're in right now.
So I've been talking to Jason Grosz, who is a full stack web developer, been doing this for a couple of decades now. He's been, wow, JavaScript, Ruby, Python, Express, Rails, Django kind of you name it apparently. And also a lot of React lately, which is a lot of fun.
So welcome to the podcast, Jason. And also we have former co-host with me actually back before we even took over React Native Radio, Josh Justice. Josh, welcome back. Josh is a developer, author, speaker and trainer focused on testing, test-driven development refactoring, and evolutionary design. In fact, Josh will be teaching a workshop at Chain React when it is back this May.
So that's going to be awesome. Really happy to have you back on the program and also really happy to have you at Chain React.
Josh Justice:
Yeah, thanks so much. Happy to be here having a great conversation. I've been waiting as long as anyone else for Chain React to be back. I always tell people it's my favorite conference, and I don't only say that when I'm on the podcast with you, just the feel, all the things you said about why you wanted to wait until you could do it in person, totally agree with. The experience is unparalleled. So really looking forward to it.
And the workshop that I taught at last time in 2019 is a great group of about 30 folks. Looking forward to being in the room with folks again this year.
Jason Grosz:
That's right.
Jamon Holmgren:
It was one of our, the attendees said that they really learned a ton at your workshop and really leveled up their testing skills, which is always something I think that all of us need help with pretty much all the time. So that's a great one to jump into. Go to chainreactconf.com. It is one of the sponsors of today's episode, America's best React Native Conference. I don't think I could say it better than Josh did, so I'll have that be the promo, chainreactconf.com.
All right, let's get into our topic for today. And the title is a Detox case study. Now Josh is here to help facilitate the conversation. They are coworkers over there at Test Double, but also Jason Grosz is sort of the one who's been doing most of the hands on here. And so it's going to be, I think a cool discussion around what your experience has been, Jason. And I want to start off before we get into this, and I'm actually going to ask Josh if that's cool because Josh is the trainer here, what's Detox for those of our listeners who have not used it yet?
Josh Justice:
So Detox is an end-to-end testing framework for React Native. And the term end-to-end testing, like many testing terms is used in many different ways. But what it really especially means here is that when Detox is testing your React Native app, it's running your JavaScript layer code, it's running React Natives built-in native code, any third party native code that you have in there as well. And it's actually running on a device, either an iOS simulator, an Android emulator, or a physical Android device. In my understanding, physical iOS devices aren't supported at this time.
So all of those pieces of your app are connecting together and if you're used to Jest tests, well, Jest can be used with Detox but if you're used to React Native testing library tests, those only exercise the JavaScript portion of your app. Really great and valuable as well, but one of the benefits you get from the end-to-end testing is the realism of having all those pieces working together.
Now, Detox calls their framework a gray box testing framework. And what that means, and their documentation goes into detail on that is that they have some knowledge of the internals of your app. One of the main things they do with that is to try to automatically address flake. So Detox knows when your app is making network requests, when animations are happening in React Native, when there's JavaScript timers running and a bunch of other things like that.
And so it's able to ensure that everything is settled before it moves on with the checks, and that can tend to make and then tests a lot more stable and reliable, mitigating one of the problems. One of the downsides of end-to-end tests in a lot of testing libraries is the fact that they can be unreliable and succeed on your machine and fail on CI or succeed on my machine and fail on Jason's machine. So, Detox really tries to address that.
Now, that's one of several different end-to-end testing options on React Native and one of the other ones that's really in the last six months come into the community a lot is Maestro. So that's one that I have not gotten to play with very much yet, but I'm looking into it in advance of the workshop this year and I'm going to be sharing notes on that. I have a ReactNativeTesting.io website where I put a lot of the things that I learn and share about React Native testing. And so we can put the link to that in the show notes, but I'll be sure to get Maestro added onto there with some of my assessment comparing it to Detox and some of the other end-to-end testing tools.
But it's great to have just more options in your tool belt to choose from when it comes to testing. You can take advantage of the pros and the upsides of different testing tools to really fully cover your app. So that's Detox.
Jason Grosz:
That's great. I would love to pair with you on that, Josh, if you get around too, because I heard about Maestro listening to your last podcast and not that my Detox experience was terrible, but it wasn't easy. So I'd be very interested to see what Maestro adds to the mix.
Jamon Holmgren:
Absolutely. And we have another full interview actually coming in. Well, it should be out by now by the time people are listening to this 262 with Dmitry Zaytsev, of mobile.dev around the Maestro framework. It's fantastic though to have a variety of tools because back in the day, we had basically like Appium was sort of the big one and there were a few other options. But they were all kind of hard to use, hard to set up, things like that. So it's really awesome to have options now, and Detox has been really the go-to for a very long time.
Josh Justice:
That actually reminds me of one other thing about Detox that I forgot to share is really developer focused. And so I've worked with QA engineers that have used Appium and are very experienced in that and they do a great job, but that's a very heavy testing stack with a lot of different languages and libraries configured together. Whereas Detox is targeted for developers, it's one tool that you use and get started. You write your test in JavaScript as you're familiar with, typically in Jest, which you're also typically familiar with. And so it's really an end-to-end tool that's targeted towards developers.
Jamon Holmgren:
Very much so. Absolutely. So let's talk about this case study using Detox. So first off, Jason, can you give us the context of the project itself?
Jason Grosz:
Yes. So talk about end-to-end. This is my first React Native project. This is also my first time really working with a client that is in an IoT ecosystem. So I like to make the joke, the T is the problem in the whole IoT. It's all those things out there in the world. So we don't have a great mock for these devices.
So we really did have end-to-end tests that went right to a real physical device that unfortunately means things like, oh, spin up a new branch for your feature branch, become really fricking awkward because you have steps of a human enrolling the thing to be the internet of things thing. So that's painful.
And we did actually have a very stable suite running for a while before we upgraded our way out of Detox compatibility, which is part of the story. That's a teaser. So this is a greenfield mobile app for controlling, doing IoT home control type things and there's a lot of great background there. Are we ready to chat about that a little bit?
Jamon Holmgren:
Absolutely, yeah.
Jason Grosz:
Cool. All right. So this was really a dream project being able to ... In our being consultants, being able to work on a greenfield project is rare. So the manager at my client has great experience and strong technical knowledge that he knows is outdated enough that he doesn't interact with our detailed decisions. But when I started, he said, "I want a fricking pipeline." He's like, "I want, before product is involved, I want you to build me a pipeline that spits a mobile app out the other end and I want to see tests, I want to see linting, and I want you to deliver me a thing that has a button on it before we ever get product involved."
And I was like, "I love you, will you just cut that? You're the best." So that's like a dream. That's a dream task.
Jamon Holmgren:
If you ever get tired of this client, just send them to me, Jason. That sounds like the ideal client. I tell clients when they come to me, the first thing that I like to do is I like to get an icon on your screen. That means we're setting up all this stuff and what's going to happen when you tap the icon is it's probably going to crash, and that's okay, because eventually it's going to be a working app.
But I want to get that icon on your screen. You need to be able to, you pull open your phone and you see that icon and it just makes you feel good because it's like, "Hey, here's a tangible-ish feel like a tactile thing that we're working on." I totally agree. That's a fantastic client.
Jason Grosz:
Yeah, it was great. It was a great way to start a project. And so this is taking my experience with web projects into React Native. This is exactly how I would want to work with the new greenfield web app, same thing. I want to get a piece of texts to the screen and a pipeline that deploys stuff every time I do a push or every time I merge PR and we've got that. So that was great. It was a great grounding and we did almost manage to get this all in place before product was engaged at all.
They made us actually put some user value in the very last thing. We're like, "Oh, okay, I guess we'll do a login. I guess we'll do a login screen for you," and got that. So my very first Detox test was actually testing that the React Native boilerplate came up. So the very first test I wrote was, can you read the text that says here's the link to the React Native documentation? Because of course, why not?
Jamon Holmgren:
So obviously, you eventually have to actually build an app. And when you do that, you're going to need a lot more than just looking for one piece of text. What else did you do to set up your tests at that point?
Jason Grosz:
Again, sort of applying things that I love to do, working with web frameworks and the way I want to use, the position I want an end-to-end testing framework to be is I love doing outside-in testing. So the very first feature that I did, I set up a Detox test that first, before I even had anything on the screen that said, there's going to be a button that says log in. And then I implemented a component that was completely static that had the text log in.
And so then that test passed. I was like, okay, awesome. I have this grounding in reality of you can actually, there's something on the screen. And then I worked my way back. So I started, I think we're maybe spitting out a list of something. And so first I spit out, I wrote a component to spit out a static list of stuff. I got that end-to-end test to pass.
I'm trying to learn just well enough for it to be comfortable to test first. And very, very few people do that, really. So just as an exercise, I'm doing it. And so then the layers, the way I'm thinking about it, we're using React query. And so very often, I'm writing a test for a component that mocks the query and so I can get all of the visual stuff in place on the screen, mock the query, and then work my way back to the data layer and test the React query data stuff in isolation with no JSX involved at all.
I want to get away from ... I want as little as possible JavaScript in my JSX components as possible. And then because of this, and so I'm new to hooks. So because of all the asynchronous rendering stuff and the way you have to deal with async hooks and React testing library, it was super valuable to have that grounding in reality of the deep ... There were many afternoons where the Detox test was passing, but we didn't have the right await for wait, whatever stuff in there to get the Jest tester passed.
Jamon Holmgren:
You mentioned TDD in React Native, and I know Josh is a huge proponent of TDD, but I have to ask Josh, do you do it? A lot of people say they talk the talk. Do you walk the walk? Do you do TDD in React Native?
Josh Justice:
The answer is usually not. So I'm on a current project on React on the Web where the end-to-end test, they're preexisting. It's handled by another team, by QA engineers. And so we're only doing unit tests, but I do often TDD at that level.
On my current side project, it's really interesting. I'm using React Native web to target web and mobile and I'm actually using Cypress for the end-to-end tests. And I'm not doing any unit tests because there's things on the web that kind of make end-to-end testing easier. And so since I'm deploying the same app to both, I can take advantage of Cypress. And this is an app where it's experimental enough that all the internal details are moving around.
And this is something I've heard from folks that avoid unit testing where they're like, "Man, I move stuff around between components and hooks all the time. I would be spending so much effort if I was unit testing there." So for a lot of apps and for my professional apps, I find the unit testing, the component testing lower level totally works fine, but this side project is experimental enough that I'm leaning more on end-to-end tests.
Jamon Holmgren:
The closer you get to the user, the more it's based on what the user flow is going to be and less on the implementation details, of course, which is why we love end-to-end testing.
Jason Grosz:
And there's a great time to talk about that cost benefit and feedback loop, right? That's what it's all about. And so Cypress, part of the reason people just have jumped to it is that it's fast and it's reliable. It's way less flaky than Selenium. And so you can happily have it be sitting there running and making changes and it's all good.
Now, because I'm used to the feedback cycle that you get with the web, Detox ain't quite there. It is not fast enough to make me happy as I'm developing things. And so that's one of your pros and cons.
Josh Justice:
One thing I wanted to throw in before we move on, as Jason mentioned, outside-in testing and starting with an end-to-end test that specifies the feature. So I've got another website about TDD where I have a really brief React Native tutorial on building code outside in. And so that starts with writing. It's a very simple feature, but just to show the process where you write a Detox test, specify the feature, you follow it along, you implement one bit at a time to get past the current test failure or error.
And then in that test we actually step down to a unit test level, which is kind of part of this outside-in test loop that you can often do. So if there's anything that the listeners are curious about, about what Jason means by outside in, we'll put that link in the show notes as well and people can get a taste and see what it's like.
Jamon Holmgren:
Yeah, absolutely. I remember way back in the day. It sounds like, did all three of us do Rails, Ruby way back? Yeah. So we're greybeards here, we go way back. And I remember way back in the day, there's this sort of debate as there always seems to be some debate in the developer community between BDD, behavior-driven development I think, versus TDD, test-driven development.
Now TDD kind of encompasses both I think is what most people think of. But back then, there was kind of a debate about that. Should you start from the outside in? Should you start from the inside out? And there were people kind of entrenched on both sides of this issue. I've always been, well, where do I need the confidence? Where do I need that immediate feedback loop and where do I not? What is the trade-off here, because there is a trade-off. You can write a thousand tests for one function, but why? That just takes way more time.
So that's what I was reminded of when you started talking about that, Josh.
Jason Grosz:
No, that's great. And in speaking of what is enough test coverage, that's one of the things that I'm ... The pain of learning just well enough to test drive components leaves me in a spot where what I focus on is testing the behaviors. And so if I test drive and I write tests that force me to implement the behaviors, I don't have any doubt when I'm done that I've got enough test coverage. I'm not going to write anything else. There's so much boilerplate and so much stuff you have to do to get that to work in the first place. I'm like, okay, I know that a test made me do all the important stuff.
So that's one of the benefits for me is I know when to stop. I know I don't need to test anything else.
Jamon Holmgren:
And at the end of the day, what we really care about is the user's experience. And the thing that you're testing with Detox, with Appium, with Maestro, this is the user experience to some degree. Now, there is still a spot for unit tests, but if you want to hear about all of that stuff, you're going to have to come to Josh's workshop. That's all I'll say.
Jason Grosz:
There you go. There you go. Well, we've got a great example of we knew this and we wanted to get as much logic out of these components. So the kind of thing, we've got a device and you're going to give it ... You got a dropdown and you're going to give it a range of values. You have three different devices and they have different valid starts and stops. Some of them take a value from 10 to 50. Some of them are okay with a value from zero to a hundred, right?
So doing an end-to-end test for that would be so many different conditions. And so that's the kind of thing that we shove into a module and we have a little JavaScript only thing that is, here's your valid range of stuff and then your end-to-end test is just, yeah, there's a dropdown on the screen. You can select a thing and you're off to the races.
Josh Justice:
Speaking of testing behavior, Jason, something you were telling me the other day was about the idea of separating behavior changes and visual changes. Do you want to talk about that for a bit?
Jason Grosz:
Yeah, that's the other thing that I really, again, being a web developer forever, we all care. I'm from back in the day when they used to call us web designers, so I spent a lot of time in my early career chopping up here.
Jamon Holmgren:
Were you ever called a webmaster? That's what I want to know.
Jason Grosz:
I was. I was actually, yes, yes. I like to mockingly say webmeister. That was-
Jamon Holmgren:
Webmeister, there you go.
Jason Grosz:
They also changed that. But it's so easy for us the care. We care about our product, we care about the end users, we care about health and what things look like, and separating the styling from the behavior is another big benefit that I see of actually test driving your behaviors. So I was just gleeful. I worked through one of the other folks at Test Double. We did a feature, we didn't even open up the simulator at all.
So the whole time we got it and we had three buttons on the screen, and we were submitting some data to the server, and we opened it up on the simulator the first time and it looked awful. There was like these huge buttons and they were all overlapping each other and I was like, okay, we did it. We totally succeeded. It works.
So they do that, get that in place and then get a fresh cup of coffee and go spend the time getting your styles in place and adding your themes and getting it all look nice and doing your pixel pushing.
Josh Justice:
Yeah, I totally experienced that as well, both professionally and on side projects. So professionally, I tend to work with designers that have high standards for what they want to look like. And so it's like cool, awesome. It's going to take a little time to get those visuals right. So separating out, building the functionality from the visuals is a big help.
And then on my side projects, I have no visual instincts at all, and so I know what I want it to work. And so I can do that and then I can go over to, I'm using React Native paper or some other very high level component library to give me visuals. And I go through the docs and see what the options are. I see what feels terrible when I put it on a real phone and then I move things around.
So it's in huge projects and in tiny projects, that separation from my wiring is just so, so helpful to be able to do them separately and to iterate on the visuals as long as I need to, knowing that I haven't broken the functionality or I'll get warned if I do.
Jamon Holmgren:
So I'm really curious as you kind of dove into this. You have your app that you're building, you have your strategies around what you're going to do with your testing, and you have the tools themselves. So you have obviously Jest, you have Detox, you have React Native testing library. What were some of the challenges that you encountered along the way when you were integrating all these things together and the workflow?
Jason Grosz:
Yes, so good. So it was a greenfield app and the first set of Detox tests that we wrote, we were asserting on a toast that would come up on the screen and say changes saved successfully. You've successfully updated such and such a value. We got that working, that's great. It was working, the tests were stable and then other things started getting on the screen. And our tests, just like the Detox test just became super unstable and weird because we were testing, we were asserting on a thing that was animated and that faded in and faded out, and guess what, sometimes overlapped other elements on the screen.
So we went through a whole period where we did that and once the app was more real and there were more user focused things of here's the result, full trip from the server of the thing you were changing. I looked at the test and I was like, we're going to just get rid of all those assertions on the toast messages and I have special config for when we're running Detox that sets the toast duration to one millisecond.
So one small change, it's still very realistic, but it enforces the fact no one can even assert on toast messages anymore because they only exist for a millisecond when you're running Detox. And so just doing that, switching the assertions to look at things on the screen instead of those toasts, that got us over one of our bumps in the road.
Jamon Holmgren:
I feel like there's a lesson in that, that sometimes you just have to let practicality win in these things. At the end of the day, you can manually test some things here and there. If your confidence level is fairly high that it's going to continue to work, if you initiate a toast and it generally works, you can probably move on, especially when it's not maybe super mission critical.
So there are moments to just sort of say, okay, let's just move on, let's find a workaround and move on.
Jason Grosz:
So, something I identified that that was a source of a whole lot of our flakes. Now I love the React testing library ethos of assert on text that you see on the screen, try to avoid using test ID or accessibility IDs and work your way down from that. And I love that. And we were doing a lot of Detox by text of a thing, but we've definitely got a bunch of by ID where we just couldn't get the test stable asserting on text on the screen.
Once the app got complicated enough, the same piece of text is on the screen twice. It wasn't happy finding it. So we've got a sliding dial there.
Josh Justice:
Yeah, I was actually talking to Mark Noonan, a friend of mine who works at Cypress about this last year because he attended a workshop that I gave in which I was talking about component testing based on the visual text on the screen. And I confirmed again that Cypress's recommendations are test by test ID. And I was like, "Mark, tell me about that. Is it a disagreement in philosophy? Is it different with end-to-end tests versus component tests?"
Mark is a very reasonable guy and very knowledgeable. So he sent me basically a handcrafted article with all his thoughts on this and a lot of it was it depends. He just really understands working at a testing tool company. He understands the different situations.
I don't have all that in front of me to remember all the wisdom he shared, but one of the things that came out of it was the narrow scope of component tests versus full end-to-end tests. When you've got a component and a component test with it, a lot of times it's going to be fairly small. Even if it's a little bit bigger, but you can see the code in front of you, the JSX. And you kind of know what in all is going on.
But when it is the whole app being tested in integration end to end, if you're on a team, someone can add something totally different that's not in your realm and you may not be aware of all the things changing the app and just the scope of the amount of things that get put on the screen or it's opacity zero, but it's still there. So, there is a fragility there. And so I think there is a reason more so with end-to-end tests to maybe prefer the IDs more, but there's a lot of trade-offs there.
I think I really gained confidence from hearing from someone who works at a testing tool company saying like, "Yeah, you can consider others," and there's a lot of wisdom in the testing library recommendations. And then for end-to-end test, there may be different things. So always think about your context and just like Jamon was saying, don't feel a pressure that you have to do it the one right way. See what you're experiencing and you adjust your approach as you go. That is absolutely the right thing to do as an engineer.
Jason Grosz:
And to talk about history, everything is trade-offs, everything is context. The history and the whole concept of test IDs is there was a whole era in web development where we did our selectors using CSS selectors that were also used for visual things. And so then you have a designer come along and change the markup around a thing or what class it was.
Jamon Holmgren:
It's always the designers, isn't it?
Jason Grosz:
Yeah, it's always those darn designers trying to make stuff look good. So that's a whole other pain. So, test IDs are better than that.
And then I've also had some pain and I honestly, it's complicated enough, I'm not a hundred percent sure on it. I would rather use accessibility labels than test IDs if I could. And in most cases, it's pretty much the same cross platform. I'm remembering there's some weird stuff where the props and the types are different between Android and iOS for what props you have available on common things. And so I've stumbled, you got some knowledge on that, Jamon?
Jamon Holmgren:
I don't have particular knowledge, although we did do an accessibility episode, which we'll link to in the show notes. But I did want to mention that I think that it's fantastic when testing and accessibility line up, so you have incentives in both of those to make those work. And accessibility is very important. It's more and more the tools that we need to come, like accessibility first. It has to be a first class feature and using those accessibility features to enable our end-to-end testing helps ensure that they exist and that they work.
Jason Grosz:
I'm stealing a rant from fellow Test Doubler, Jason Karns. But one of his things is it is a win-win and it's a win for folks that don't need accessibility assistance because if you write your page so that all the important behaviors and all the important information has a label and is accessible, then you're testing the right stuff. You're testing the things that matter to everyone, and if it happens that it gives you an easy hook to find the thing, that's great. It totally is a win-win. It's absolutely a win for everyone.
Jamon Holmgren:
In my notes, it says that you were supporting a legacy app that had custom native integrations including things like biometrics, authentication and stuff. So how did Detox kind of perform in that environment? Or did you integrate Detox at all in that situation?
Jason Grosz:
We didn't. There's a whole other story. I would love to put a pin in that and rewind a little bit. So you asked about tooling and tests. Another sort of interesting thing that I ran across with Detox and with the original setup, I talked about how our manager really wanted a pipeline and everything end to end.
I went in thinking the best thing I'm going to want to do is set up a GitHub Action on Linux and test Android, because Android, Linux, it's easier. The builds are quicker. I'm going to just do that. So we started with that, found out that the built-in GitHub test runners for Linux don't have the virtualization software that you need to actually run a simulator.
I spent a day monkeying around with that. I'm like, okay, I'm not in a position to have a self-hosted runner right now. It is possible to have a VM that has the virtualization stuff that you need to run an emulator. So I said, "Okay, fine, my client's going to just have to pay for some more GitHub dollars." So we switched to doing Android on a Mac runner. It happens that the GitHub provided GitHub Action runners on the Mac do have all the virtualization stuff that you need to run an Android emulator.
Great, we were off to the races. But we were using Mac minutes and it was still a quicker build than the Apple build. And then we just could not get the Android test to be stable. And we'd had other issues with rendering and diceyness with the emulator that we're using.
Jamon Holmgren:
I can feel the pain in your voice. And you know what, I feel that too because I've been in similar situations.
Jason Grosz:
We ended up on running the end-to-end suite on iOS, on a Mac runner in GitHub Actions. But it's all working great or it was, so this gets to your next thing. We are going to be doing the thing where we envelop a legacy mobile app in a React Native app. So my client has a huge mobile app that is currently implemented with all custom stuff and it is React in a WebView.
It's completely custom native layer. They have a completely custom native bridge that's different for Android and for iOS. But they've got all the tooling in place. This has been around long enough. They've got all the tooling in place to run all the JavaScript as a static bundle.
So, 90% of the time when you're working on this legacy app, you can just use a web pack server and just pretend it's a webpage. And that's great, it's great feedback loop it reloads quickly. It's all wonderful. And they actually have a build pipeline that hosts and deploys it as a static bundle on S3.
Their JavaScript has a custom bridge that knows that it's in a webpage. And so if you try to say, "Oh, give me location permissions," or, "Oh, I wish to authenticate biometrically," it has code in there that knows, "Oh, I'm hosted on a webpage." So that's what gave me the confidence to suggest this architectural path of like, well, I know rewrites are dangerous, your other app is huge. This seems really janky. But we think the best path forward is to get this greenfield app working and functional and then in parallel be prepared to host it in a WebView.
So that's where we're headed now. Apparently, I'm not the only person that's been down this path, I think we're not. It just seems really a stretch, but I know it's just ... Their other app is just huge. It's gigantic. And so we have to get them in a position that they can incrementally start pulling screens from the web React version and implementing them at their own pace.
Jamon Holmgren:
Speaking of WebView, I've been involved in the React Native WebView. I'm not as involved anymore, but I think one of the things you mentioned was that dealing with open source, not just WebView but even things like React Native itself with turbo modules and the new architecture. Can you talk a little bit more about the challenges around open source lagging behind in some cases?
Jason Grosz:
I think it's a matter of degree that we deal with in the JavaScript ecosystem. It seems more volatile in React Native land to me than it is in general. But really it's hard dial to twist, and I think we're in a pretty unique situation. So, I've got a Jira Epic with 15 or 20 stories to port their custom native code, and we've got probably a year till we're in production.
And so we reached for turbo modules because we're reading the stuff, we're like, well this is the new architecture. It's called new architecture, why wouldn't you want that? So we're like, okay, before we can start porting this biometric auth, we've got stuff for saving, permissions to save a screenshot to the phone's file system because this IoT app has video of security cam video things. So click a button, save a screenshot.
We've got it all. We know what the code is, we know what we need to do. We're pulling the complicated stuff from the legacy app and wrapping it in a turbo module. So that's what incentivized us to upgrade all the things and get up to 70.4, which is where we are now. So that put us in a weird position. We successfully upgraded to 70.4 about three, four months ago maybe. Only a few libraries had issues. It wasn't terrible, so it's coming along. But unfortunately, this WebView is really critical and it's just in release candidate.
I'm working with the developer, he's been super awesome working with me on it. I'm asking enough questions that he realizes that I care. And so we've been working together on it. So, I think we're close. I think we're close to getting it to work.
Jamon Holmgren:
Yeah, he is amazing. I worked with him on the WebView originally and then he's been maintaining it since. But it's a whole beast. I should do a whole episode just on WebView sometime because that'd be interesting.
Jason Grosz:
And I think you've talked about upgrading, one of your last posts was, the last episode that I listened to was hopefully making things easier to upgrade. So I just did a spike version of upgrading from 704 to 713, and there was just still so many changes. I think 701 is a pretty big change.
So, it's painful. I think it's the usual advice. You're always tweaking the dial of what you write yourself and what you pull from a third-party library and getting those instincts of going and looking at the third-party library and how many developers are on it, how many issues do they have open, take a look at their issues and see how responsive the folks are that is there things sitting there for a year. People saying, "Ah, there's a bug with this," then maybe you tweak the dial and you go look at the code and you say, "How complicated is this? Can I write it myself? Can I fork the repo? What's the path?" It's same decision making that we do first.
Jamon Holmgren:
That's open source.
Jason Grosz:
That's open source. Yep.
Jamon Holmgren:
We were talking before we started recording about how if it didn't exist, you'd have to write it yourself anyway. So just going in there and kind of using that as a starting point, you're already ahead of that.
Jason Grosz:
But we know complexity, complexity is the killer. Paying attention to it and the complexity of managing open source is also a cost, and so you're really, really thinking about all those little decisions. So, back to your question about the native stuff, because this is where we're wedged. As soon as we upgraded to 704 with the fabric renderer, Detox stopped working. Detox tipped over.
I carefully got our CI working. I got the output from the test run. This is where we decided to stop running the Detox test in the pipeline because we had to upgrade to start working on turbo modules. Now I put a note on Detox Wix, all of their issues for get the sample project to support new architecture. And it looks like they've got a release now that does support the new architecture.
So I think the result, I was hoping to have a huzzah at the end of this to say I got it working on our project, but I had a half hour, 45 minutes between things. And I do have it building and installed, but I haven't got, there's changes to the Jest. The Jest test runner isn't integrated in the same way, whatever. I got to monkey around with some configuration files. But I'm hopeful that we can get it back running.
In any case, by my understanding of the way Detox works, I don't think that it can detect completely native things like the little buzzy box that comes up and says, show us your face. I don't think it has a way to do that.
Jamon Holmgren:
Yeah, exactly. And I think this is kind of a common thing with React Native because you aren't just talking about one environment with just JavaScript and HTML and CSS. We're talking about marrying together two different environments that work together in a very unique way.
And I think I counted up the number of languages that you have to deal with when you're working on a React Native app. And it's something like seven or eight, and those are major languages. You've got TypeScript, JavaScript. You have sort of a CSS-like syntax. It's not CSS, but it's there. Obviously, Objective-C, Swift, Java, Kotlin. And then on the build side, there's like Groovy, Ruby, C++ also on the native side now. So yeah, just throw a few more in. Why not?
Jason Grosz:
It's a lot. It's good for you. I'm not A type personality as far as jumping out of planes or riding a motorcycle, but that kind of intellectual challenge is something I just jump into with both feet. Sure, why not? What the hell?
Jamon Holmgren:
Why not? And I'll tell you what, as someone who likes variety and has tended to move on from languages after about five years because I get bored, React Native is, we're working on eight years now and I'm not bored because there's just so many different things to learn. The rabbit hole goes so deep.
Jason Grosz:
That's great. Circling back to TypeScript that you just mentioned, I'd be curious to see the folks out there on the interwebs have any thoughts on this. But another win-win that I've found is we chose early on to set up our project to where our app uses TypeScript, but our Jest tests don't. And so I've had some previous experience with a React project that was TypeScript everywhere and partially, it's because I was new to TypeScript. But I found the boilerplate in the Jest tests to be just a lot.
And being new to TypeScript and a fan of unit testing, I'm also always trying to tease apart what tests can I not write because I've got the safety of TypeScript? What tests can I skip? And so I feel like this is working pretty well, which is the Jest tests are ensuring the runtime behavior. And I can have test data that doesn't comply to the types at all.
And so I will often get things going and ignore the little squiggly red lines in VS code in the actual component and get the test running in JavaScript, and then setting up the types and making all of that happy is a completely separate activity from writing the component tests. And it seems productive to me. It feels good. I have confidence that the thing does what I want it to do, and then I can switch over to getting the confidence that no future developer is going to monkey it up by putting a string instead of a number or object instead of a whatever.
Jamon Holmgren:
So you can't necessarily solve this with Detox, the phase ID thing. Obviously you can manually test it, but nobody wants to do that constantly. So what did you do? How did you solve this? How did you approach it?
Jason Grosz:
Again, we've got a pretty ... The manager on the team is smart. He knows what he's doing. He's explicitly said, "I don't want to have to have a full-time Java developer. I don't want to have a full-time Objective-C developer." We just have to overcome getting this native code that we own into the new app. And then after that, hopefully we're going to tie a bow on it and we're not going to have to write geofences another time. We got it. It works.
This client also has a pretty strong culture of manual testing for better or worse. We've got a great very detail-oriented QA person. And so at first, I was really bothered by the fact, of course the existing native code, the Java and the Objective-C that we're pulling over and taking from completely custom code into a turbo module. There's also no unit tests on the old code. There's really no tests at all.
So that bothered me, but I understand everything's a trade-off and this is not code that's going to be modified a lot. It's not going to live after this. What we've decided to do, our first step is to get this native code that we're porting to work in React Native. And so for each thing that we're porting over, we're making a very simple developer-only test screen.
We've got a route that you can't get to without putting in an Easter egg. You can get the code onto a real phone. And we've got a screen for example, a button that says, is biometrics auth available? Start the biometric auth process, cancel the biometric auth process.
And it spits out JSON to the screen. When you're on Android, it pulls up the native thing that says, "Please give me your fingerprint." If you're on iOS, it pulls up the thing that says, "Show me your face." And then when it succeeds, it spits out from JSON and the QA person can validate that it works.
At first, that bothered me, but fundamentally it's an integration test. It's an integration test between code that's running on a real device plus your JavaScript plus Objective-C and Java and seven other languages. It's a pretty high degree of confidence that if we get those little test screens to work, we've made it as easy as possible for the human that's going to have to test it. And we'll be able to carry that forward after we do the next step, and the next step being create a new bridge from the WebView to React Native in both directions because that's what this existing app expects.
We're going to be using the post message and the evaluate JavaScript methods to talk back and forth between the WebView and React Native layer.
Josh Justice:
Jason, I see a theme in the things you're sharing and it reminds me of what Jamon was saying about trade-offs and testing. And I've started to think about tests in terms of, there's all these different terms end-to-end unit component. But thinking about tests in terms of what does this test integrate with and what does this test isolate from? And you can make that as wide or as narrow as you want in different cases.
And so going wide to end-to-end gives you a lot of overall coverage for things that you can end-to-end test with the tools you have. There's also the question in that world about whether you're mocking out backend API servers and integrated with real ones of those. So that's another whole thing.
But in this case, with this manual developer testing screen, you've found if you narrow it down just to those pieces of preexisting native code, then you've reduced the manual testing effort so far. And so your confidence is way up and your effort to build it and effort to run those manual tests is way down. And so then you rely on React Native testing library automation or vanilla JavaScript automation for things that are a fit to put into vanilla JavaScript or TypeScript.
So yeah, it's really something. As people get under their belt the basics of doing these different types of tests, that's something I try to instill in them is the confidence to try integrating with different things and isolating from different things. And I find that in React Native with the depth of all the things it integrates with, you have to get creative more often of picking and choosing and designing a breadth of a test that fits for your use case and not for your whole app, but for these individual types of tests to answer this type of question.
So yeah, I think the examples you're giving are setting a great example for folks to think through those complicated issues.
Jason Grosz:
That's exactly right. And it's similar, we talked about feedback cycle, feedback loop. As we rolled into this, of course, this is so abstract. As a developer doing it, I needed a way to exercise the code. It served the same purpose that an automated unit test has because how do I get this and actually execute this line of Java that is seven layers deep inside this thing that I moved over?
It has not been just useful for the QA person. So that what we've been doing, there's a whole turbo modules could be a whole other episode. But we've been defining the method in the turbo module JavaScript file that gets you the inputs and the outputs type response, whatever. What arguments does it take? What does it return? Building a test screen and then building a no-op version of that method in the native code.
Going to Objective-C and building a no-op thing that resolves a promise with fargo-bargle, and then you're good. Then you've got that line and you can go to your testing page and click the button and see fargo-bargle come up on the screen and then start working your way back. It's another version of outside-in testing. And then, okay, now I can focus on the Objective-C, which just looks like complete and utter gobbledygook to me every time I look at it.
But I'm learning. I'm learning some things. Square brackets, there's a lot of square brackets in weird places. That's what I'm seeing.
Jamon Holmgren:
All the square brackets. For some reason I kind of like Objective-C, I don't know what it is, I like it. It's weird. I don't know why.
Jason Grosz:
You've been doing native for a while. You sure this isn't Stockholm syndrome?
Jamon Holmgren:
Maybe. I don't know. But it just, yeah, I don't know. That's a whole another thing. We are running out of time here, but thanks so much, Josh and Jason. There's I'm sure much more to say on this subject and of course hearing, I don't know, stories of ... I have my own experiences that I can kind of reflect on as you go through this like, "Yeah, this didn't work, so then I have to try this."
It's just how our industry is. Before we go though, I did want to ask, Josh, what do you plan to cover in your workshop at Chain React? Just so people have kind of an idea.
Josh Justice:
Absolutely. Yeah. So in this workshop we're covering kind of all the several different breadths of tests, as I talked about just there. We're going to be doing end-to-end testing. The plan right now is to do it with Detox, but I'm evaluating Maestro right now. Certainly, we're going to at least talk about Maestro and the trade-offs with Detox. But we might incorporate in it. A day flies by.
One of the pluses though is that the workshop is half specific tools and it's half theory and ideas that translate across any tooling that you use. And so if we do Detox, those principles will translate to Maestro. They'll translate to even better things. If you use React Native testing library, those principles translate to the web and to future tooling as well.
But yeah, so we'll be doing some end-to-end basics introduction. We're going to be doing component testing using React Native testing library. And as we go, there's principles of just how you think about your tests that apply even to vanilla JavaScript tests as well. And it's all couched in terms of a principle that I got. I'm trying to decide if I spoil it or keep it a secret.
But about testing the contract, thinking about answering the question of how do I test? And so often I get the question, how do I test this line of code? And I think there's actually a better mental model that will help you answer that question by thinking about, and it's things we've said all through our discussion today, thinking about from the perspective of the user, thinking about what are the inputs and what are the outputs, thinking about what is the interface rather than the implementation. And so those principles for testing I've seen make a huge impact for folks over time as they get them sunk in.
One thing I want to emphasize as well, in addition to the workshop, another way to connect with me and some other engineers, we call them double agents from Test Double. We're going to be doing some code and coffee office hours at Chain React. And so you'll be able to sign up if you want to talk to me about React Native testing or some of the other things I know about or other folks that are going to be there about React Native, about React, about product design, about thinking about stories and breaking things up, architecting your code, a number of other topics, marketing, creating content, working on your resume. We're going to have agents there that would love to chat with you about any of these things.
And so stay tuned for more details. It'll come out somewhere or the other on Chain React social media. But yeah, so there's two opportunities at Chain React to connect and I would love to connect with anyone. Grab me in the halls as well because I'll be there the whole time. I just love to nerd out about React Native and testing and anything. The heart of the challenge you've got, the better. Let's brainstorm it together.
Jason Grosz:
Nice. That's great. That's great.
Jamon Holmgren:
It's perfect. Test Double is one of our awesome sponsors of Chain React. We love having you folks there. It's just going to make it a rich experience for everybody.
Jason Grosz:
And speaking of variety and 37 different languages, the benefit we haven't talked about or called out enough, part of what's great about TDD and thinking about these seams and what the contract is, is no one can understand all of this at once or keep all these things in their head.
Just be nice to yourself, be kind to yourself. Break these things up into little tiny problems until it seems easy and then break it down a little more and then maybe you'll be leaving behind some code that other humans can actually understand tomorrow or yourself on Tuesday after writing it on Thursday.
Jamon Holmgren:
That is a perfect way to end the episode. Thanks so much, Jason and Josh. If you want to connect with React Native Radio, you can go to Twitter, @ReactNativeRDIO. Josh, I know you're not on Twitter anymore. Where could people find you otherwise?
Josh Justice:
CodingItWrong.com is my personal website. That's what I go by online before I thought about the marketing disadvantages of that. But that links over to my Mastodon social media, LinkedIn and other things. We'll put that in the show notes as well.
Jamon Holmgren:
Awesome. And Jason, how can people connect with you?
Jason Grosz:
I don't know.
Jamon Holmgren:
Come to Chain React.
Jason Grosz:
Yeah, exactly. I'm barely a Twitterer, although I am @agent_0028.
Jamon Holmgren:
No problem.
Jason Grosz:
But barely. So, I will see you if you send a DM.
Jamon Holmgren:
See, this is why you get all this stuff done, is because you're not on there. And you can find me @JamonHolmgren. I've got no real work to do, so I'm on there a lot. Really appreciate everybody coming along with us on this journey. As always, thanks to our producer and editor, Todd Werth; our assistant editor and episode release coordinator, Jed Bartausky; our designer, Justin Huskey; and our guest coordinator, Derek Greenberg.
Thanks to our sponsor, Chain React. Hope to see you all there. And you can also check out infinite.red/react-native for React Native Consulting. A special thanks to everybody listening. Make sure that you subscribe and we will see you all next time.