Agent guide: Testing

Test the action and its result

Inside testthat::test_that(), every pz_expect_*() call is a testthat expectation, and a failure fails the test. Outside testthat, a failure is an error of class paparazzi_expectation_failure that reports the target, the last value seen and how long it waited. A passing expectation returns the context invisibly, so it sits in the middle of a chain.

Open the page with pz_local_page() so it closes when the test ends, pass or fail. Pages opened from an app path stop their server too.

library(paparazzi)

testthat::test_that("adding a task inserts its title first", {
  page <- pz_local_page(pz_example("tasks"), width = 1000, height = 720)
  page |>
    pz_act_type("Repot the fern", target = "#task_title") |>
    pz_act_click("#add_task") |>
    pz_expect_text(
      "Repot the fern",
      target = pz_loc(".task-title", which = "first")
    ) |>
    pz_expect_count(8, target = ".task")
})

The click waits until the Add button can be clicked; the text expectation waits for the save to finish. Those are two different conditions, so expect each step’s result before starting the next.

Choose the expectation that states the behavior

Choose the expectation that states the behavior precisely:

Most element expectations need at least one match, and every match must pass; pz_expect_exists() needs just one. To check one row, target it with match or within. pz_expect_hidden() also passes when the element is absent, so pair it with pz_expect_exists() when presence matters.

testthat::test_that("help starts hidden and can be opened", {
  page <- pz_local_page(pz_example("tasks"))
  page |>
    pz_expect_exists(target = "#help") |>
    pz_expect_hidden(target = "#help") |>
    pz_act_click("#toggle_help") |>
    pz_expect_visible(target = "#help")
})

Match text and values

pz_expect_text() collapses whitespace and trims on both sides. A bare string checks exact, case-sensitive equality with the whole observed text; pass pz_regex() for a pattern, or pz_regex() with fixed = TRUE for a literal substring. One expected value applies to every match; a character vector needs exactly that many matches and compares them in document order.

testthat::test_that("text entry updates the input value", {
  page <- pz_local_page(pz_example("tasks"))
  page |>
    pz_act_type("Buy milk", target = "#task_title") |>
    pz_expect_value("Buy milk", target = "#task_title")
  testthat::expect_identical(pz_get_value(page, target = "#task_title"), "Buy milk")
})

Getters return plain values and end the chain, so compare them with ordinary testthat expectations. pz_get_count() reads once and can return zero; use pz_expect_count() when the count is about to change.

Use pz_regex() to share one pattern between a locator and pz_expect_text(), pz_expect_value(), pz_expect_attr(), pz_expect_url() or pz_expect_title(). The same value applies to every matched element. paparazzi collapses whitespace and trims the text it reads before matching, but leaves regex pattern source unchanged. With fixed = TRUE, it also collapses and trims whitespace in the literal substring. An empty string can match an empty pattern, but an absent attribute or value property never matches. Pass one regex value rather than a vector or list of them, and don’t mix regex values with strings inside one expected vector. paparazzi checks the pattern when the value is used, so an invalid pattern errors at that point, even when nothing matches the target.

pz_with_page(pz_example("tasks"), function(page) {
  label <- pz_regex("^Renew passport$", ignore_case = TRUE)
  passport_title <- pz_loc(".task-title", match = label)
  pz_expect_text(page, label, target = passport_title)
  pz_expect_attr(
    page, pz_loc(".task", match = pz_regex("passport")),
    !!!list(`data-priority` = pz_regex("^(high|normal)$"), draggable = "true")
  )
})

Attribute calls also accept a named list spliced with !!!, as above. One pair’s value can be a bare string while another is a pz_regex(). Each pair accepts a character vector or one regex, not a vector of regexes.

Wait on conditions, not time

Expectations retry until they pass or time out. timeout = NULL uses the page’s default, set with pz_open(timeout = ) or through pz_local_page(). timeout = 0 checks once, for when an earlier step already waited.

When there’s no particular value to expect, wait on a condition instead: pz_wait_for_shiny_idle() for reactive work, pz_wait_for_stable() for settled layout or text, or pz_wait_for_js() for a JavaScript predicate. pz_wait() is a fixed wait that keeps the browser running, recordings included.

pz_with_page(pz_example("tasks"), function(page) {
  page |>
    pz_wait_for_js("document.readyState === 'complete'") |>
    pz_wait_for_stable(target = ".task-list", prop = "rect", for_ms = 200) |>
    pz_expect_count(7, target = ".task", timeout = 0)
})

The default page timeout is ten seconds. For a slow app, raise the timeout rather than adding fixed pauses.

Test navigation between documents

Serve test pages over HTTP so links and history behave as they do on a real site. This example writes two small pages joined by a link, then tests the click, the destination and the way back. It needs httpuv. The function owns the server and the test owns the page.

check_navigation <- function() {
  site <- tempfile("paparazzi-site-")
  dir.create(site)
  withr::defer(unlink(site, recursive = TRUE))
  writeLines(
    '<title>Start</title><a id="next" href="next.html">Continue</a>',
    file.path(site, "index.html")
  )
  writeLines('<title>Next</title><h1>Arrived</h1>', file.path(site, "next.html"))
  server <- pz_serve_static(site)
  withr::defer(server$stop())

  testthat::test_that("the link opens the next document", {
    page <- pz_local_page(server)
    page |>
      pz_act_click("#next") |>
      pz_wait_for_navigation() |>
      pz_expect_url(pz_regex("/next.html", fixed = TRUE)) |>
      pz_expect_text("Arrived", target = "h1") |>
      pz_nav_back() |>
      pz_expect_title("Start")
  })
}
check_navigation()

Navigation resets scopes to the root, so find the destination’s elements again (see Locators and scopes).

Inspect state when a test fails

pz_inspect() prints the page’s state, and getters return specific values to R. A screenshot with an explicit path saves evidence without opening a viewer, and the chain continues.

pz_with_page(pz_example("tasks"), function(page) {
  page |> pz_expect_visible(target = "#new_task")
  pz_inspect(page)
  pz_get_elements(page, target = "#new_task input")
  pz_get_attr(page, "type", target = "#task_title")
  pz_get_style(page, "display", target = "#new_task")
  page |> pz_screenshot(tempfile(fileext = ".png"), frame = "#new_task")
})

For checks no built-in expectation covers, pz_expect_js() takes a JavaScript function that receives each matched element and returns true or false. At the root, the target defaults to the page body. To read a value rather than check one, pz_evaluate() evaluates JavaScript in the page and returns the result; it ends the chain, so assign it instead of piping it.

pz_with_page(pz_example("tasks"), function(page) {
  page |>
    pz_expect_js("el => el.querySelectorAll('.task').length === 7")
})

Browser tests need Chrome or Chromium where chromote can find it. Open a new page in each test so tests don’t share state. A shared Shiny server can save startup time while each page still gets its own app session (see Shiny).