6

I'm using GitHub Actions to do the R CMD Check (see https://github.com/r-lib/actions).

I would like to use snapshot_file in testthat script of the sort:

save_png <- function(code, width = 1000, height = 600) {
  path <- tempfile(fileext = ".png")
  grDevices::png(path, width = width, height = height)
  on.exit(dev.off())
  code
  
  path
}

set.seed(123)
df <- data.frame(y=rnorm(20),x=rnorm(20))

test_that("graphs are correct", {
  expect_snapshot_file(path = save_png(plot(df$y,df$x)), name = "plot1.png")
}

At the moment, the error will always be: Adding new file snapshot "_snaps/gets-test1-example/plot1.png". Alternatively, if I upload my plots generated on my laptop, I will get an error that they are different (ever so slightly)...

Because I am aware that snapshot tests are fragile, I thought of two options:

  • Delete all snapshot files in the _snaps folder that have been generated e.g. on my laptop and then use some form of upload-artefact (https://github.com/actions/upload-artifact) to upload the snaps the first time - and then GitHub actions has something that is generated by the same machine to test it against (I'm thinking something like a dedicated storage place for each OS, where existing files are first loaded to the Workflow machine and then checked)
  • skip all snapshot_files on GitHub Actions (Using skip_on_ci()).

Below is my workflow file, maybe you have an idea how to help me:

on:
  push:
    branches:
      - master
  pull_request:
    branches:
      - master

name: R-CMD-check

jobs:
  R-CMD-check:
    runs-on: ${{ matrix.config.os }}

    name: ${{ matrix.config.os }} (${{ matrix.config.r }})

    strategy:
      fail-fast: false
      matrix:
        config:
          - {os: windows-latest, r: 'release'}
          - {os: macOS-latest, r: 'release'}
          - {os: ubuntu-20.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"}
          - {os: ubuntu-20.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/xenial/latest"}
          - {os: ubuntu-20.04, r: 'devel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"}
          - {os: ubuntu-16.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/xenial/latest"}

    env:
      R_REMOTES_NO_ERRORS_FROM_WARNINGS: true
      RSPM: ${{ matrix.config.rspm }}
    
    defaults:
      run:
        working-directory: gets
    
    steps:
      - uses: actions/checkout@v2

      - uses: r-lib/actions/setup-r@master
        with:
          r-version: ${{ matrix.config.r }}

      - uses: r-lib/actions/setup-pandoc@master

      - name: Query dependencies
        run: |
          install.packages('remotes')
          saveRDS(remotes::dev_package_deps(dependencies = TRUE), "../.github/depends.Rds", version = 2)
          writeLines(sprintf("R-%i.%i", getRversion()$major, getRversion()$minor), "../.github/R-version")
        shell: Rscript {0}

      - name: Cache R packages
        if: runner.os != 'Windows'
        uses: actions/cache@v1
        with:
          path: ${{ env.R_LIBS_USER }}
          key: ${{ runner.os }}-${{ hashFiles('.github/R-version') }}-1-${{ hashFiles('../.github/depends.Rds') }}
          restore-keys: ${{ runner.os }}-${{ hashFiles('../.github/R-version') }}-1-

      - name: Install system dependencies
        if: runner.os == 'Linux'
        run: |
          while read -r cmd
          do
            eval sudo $cmd
          done < <(Rscript -e 'cat(remotes::system_requirements("ubuntu", "20.04","gets"), sep = "\n")')
      - name: Install dependencies
        run: |
          remotes::install_deps(dependencies = TRUE)
          remotes::install_cran("rcmdcheck")
        shell: Rscript {0}

      - name: Check
        env:
          _R_CHECK_CRAN_INCOMING_REMOTE_: false
        run: rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning", check_dir = "check")
        shell: Rscript {0}

      - name: Upload check results
        if: failure()
        uses: actions/upload-artifact@main
        with:
          name: ${{ runner.os }}-r${{ matrix.config.r }}-results
          path: check
2
  • Did you ever find a workable solution for this? I know the snapshot functions are still marked as 'experimental', so maybe the documentation will improve, but at the moment I'm finding it a bit hard to follow. Oct 19, 2021 at 2:18
  • I'm afraid not - I have just continued to run them on my laptop and skip them on ci (GitHub Actions)... Oct 19, 2021 at 7:05

1 Answer 1

0

I think your skip suggestion could work if it's a single OS causing issues: use skip_on_os(os = "windows") for example (https://testthat.r-lib.org/reference/skip.html)

You could also use the variant parameter of expect_snapshot_file() to create a snapshot variant per OS.

test_that("graphs are correct", {
  expect_snapshot_file(
    path = save_png(plot(df$y, df$x)),
    name = "plot1.png",
    variant = Sys.info()[["sysname"]]
  )
})

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.