Testing Architecture for Authentication

[robertjcolley]

Hi Detox Community!

I love the work being done here and I'm looking to get advice on how to solve some problems which a lot of applications will have.

Our app has authentication, HTTPS requests to our server for data, and features which allow the system to build up system state. One of the biggest challenges I've had with Detox is that the tests don't run in a consistent order, and then state building up in the application across tests / user flows causes other tests to be flaky.

I'm wondering if there are any guidelines which the community follows to deal with these issues - maybe its how the tests are structured, the scope of the tests, etc. I'd be happy to share anything we have figured out and want to start a discussion about how to make testing more effective!

[lukebrandonfarrell]

@NS-BOBBY-C Each test suite should be able to run in isolation. We generate a new user with clean state per suite, and then test any state build up in that suite.

[robertjcolley]

How does that affect the test run time? We've gotten to the point where our tests would take an hour or longer to finish

[lukebrandonfarrell]

Sure it would probably take longer, it's stability / time, you gotta find a balance. Not saying that we have found the best way, lots of room for improvement, I like the idea of starting a discussion around this

[robertjcolley]

Thanks! Yea I'm coming from the Cypress world (https://docs.cypress.io/guides/getting-started/testing-your-app#Bypassing-your-UI) and they say that logging in each time is an anti-pattern. I'm sure there is a way to sign in with a detox custom command with the auth providers

[d4vidi]

One of the strongest recommendations that we have involves test isolation. If you want to be able to gain confidence in your tests, reduce flakiness and to easily debug test failures, you really need to eliminate tests residue.

You read about it under the Stable E2E section in @rotemmiz's blog about test stability.

[d4vidi]

That said, in cases such as logging in -- you can test that out in dedicated use cases, and set up a means to mock a logged in account in all others.

[pke]

Interesting read, thanks!

I always wonder about the mock server. First it seems really difficult to mimic everything the real server does (especially how it returns responses) but more importantly its not testing the "real thing" so even if your e2e tests against the mock server are all green how can you know you have covered everything the prod server does? @d4vidi