Our API testing utilizes jest with intricate scenarios. The beforeAll functions are used to establish general helper variables for each test, as well as implement tenant separation. In some cases, the beforeEach functions are employed to set up tenant separation along with default configurations for the test tenant...
For example, a test may appear like this (notably, TypeScript is utilized for writing the tests):
let apiClient: ApiClient;
let tenantId: string;
beforeAll(async () => {
apiClient = await getClientWithCredentials();
});
beforeEach(async () => {
tenantId = await createNewTestTenant();
});
describe('describing complex test scenario', () => {
it('should have some initial state', async () => {
await checkState(tenantId);
});
it('should have some state after performing op1', async () =>{
await op1(tenantId);
await checkStateAfterOp1(tenantId);
});
it('should have some state after performing op2', async () =>{
await op2(tenantId);
await checkStateAfterOp2(tenantId);
});
it('should have some state after performing op1 and op2', async () =>{
await op1(tenantId);
await op2(tenantId);
await checkStateAfterOp1AndOp2(tenantId);
});
it('the order of op1 and op2 should not matter', async () =>{
await op2(tenantId);
await op1(tenantId);
await checkStateAfterOp1AndOp2(tenantId);
});
});
describe('another similar complex scenario', () => {
// ... continuation of similar scenarios
});
The challenge lies in determining the optimal way to share the variables initialized by beforeAll and beforeEach. When executed with the --runInBand option, the above test functions correctly, but issues arise when run in parallel, particularly involving undefined values for tenantId. Serial execution leads to successful outcomes for approximately 50-60% of the tests on an 8-core/16-thread build agent, while quad-core CPUs achieve around 80% success rate. Variations occur based on the level of parallelism.
Exploring potential solutions, references were made to utilizing this for context sharing, which proved ineffective, or encapsulating everything within describe:
An attempt was made at a straightforward approach:
describe('tests', () => {
let apiClient: ApiClient;
let tenantId: string;
beforeAll(async () => {
apiClient = await getClientWithCredentials();
});
beforeEach(async () => {
tenantId = await createNewTestTenant();
});
describe('describing complex test scenario', () => {
it('should have some initial state', async () => {
await checkState(tenantId);
});
// Additional test operations...
});
describe('another similar complex scenario', () => {
// ... continuation of similar scenarios
});
});
Unfortunately, this adjustment did not yield the desired outcome. An aspiration remains to execute the tests in parallel, yet documentation guidance on this aspect is elusive. Any insights or pointers in the right direction would be greatly appreciated.