Running Android Espresso tests on Firebase Test Lab with Flank

Flank it’s an open source API that runs test sharding on Firebase Test Lab, developed by WalmartLabs. Test shards are the number of groups to split the test suite into.

Example: if your test suite has 21 tests and you configure flank to “testShards:3” then your (21 tests) will be split into 3 runs, 21/3 = 7 test per run. Test sharding makes FTL runs quicker. This tutorial shows how to run flank from a MacBook Pro terminal.

Configuration:

  1. Download flank and add to your path.
  2. Add file flank.yml to your project.
  3. Set up gcloud.
  4. Run android example to verify everything is setup correctly.

How to Run:

  1. Configure your flank.yml file values for: device, test timeout, test shards, test runs amount..

2. Open a terminal inside your Android project and run this command:

$ flank.jar firebase test android run

Terminal output example:

Below example flank.yml:

#
# gcloud args match the official gcloud cli
# https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run
gcloud:
# results-bucket: tmp_flank
record-video: true
timeout: 60m # Virtual device
# timeout: 30m # Physical device
async: false

# test and app are the only required args
# Virtual device
# app: /Users/asandoval/dev/android.././googleFlavor-yourApp-x86-debug-269999–2018.09.08–16h29.apk
# test: /Users/asandoval/dev/android/..//androidTest/googleFlavoryourAppFreeX86/debug/googleFlavor-yourApp-x86-debug-androidTest.apk
# Physical device
app: /Users/asandoval/dev/android/../googleFlavor-yourAppFree-armv7a-debug-239999–2018.09.08–17h02.apk
test: /Users/asandoval/dev/android/../googleFlavor-yourApp-armv7a-debug-androidTest.apk

auto-google-login: true
use-orchestrator: false
environment-variables:
clearPackageData: true
directories-to-pull:
— /sdcard/screenshots
performance-metrics: true
test-targets:
— class myApp.ui.actvities.HomeActivityTest#loginTest

device:
# Physical device
— model: walleye
version: 26

# Virtual device
# — model: NexusLowRes
# version: 26

flank:
# test shards — the amount of groups to split the test suite into
# set to -1 to use one shard per test.
testShards: 3
# test runs — the amount of times to run the tests.
# 1 runs the tests once. 10 runs all the tests 10x
testRuns: 1
# # always run — these tests are inserted at the beginning of every shard
# # useful if you need to grant permissions or login before other tests run

software engineer, read, swim, travel. apps: https://goo.gl/rDuuQw https://goo.gl/15GvfE

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store