360

Is there a standardized way in R of measuring execution time of function?

Obviously I can take system.time before and after execution and then take the difference of those, but I would like to know if there is some standardized way or function (would like to not invent the wheel).


I seem to remember that I have once used something like below:

somesysfunction("myfunction(with,arguments)")
> Start time : 2001-01-01 00:00:00  # output of somesysfunction
> "Result" "of" "myfunction"        # output of myfunction
> End time : 2001-01-01 00:00:10    # output of somesysfunction
> Total Execution time : 10 seconds # output of somesysfunction
4
  • 2
    I think you had proc.time on mind cause system.time is one you need.
    – Marek
    Commented Jun 7, 2011 at 8:35
  • 3
    For larger functions, Rprof is nice. It provides a profile of all the processes in a code chunk/function. Commented Aug 24, 2014 at 17:08
  • 54
    New R users finding this question through google: require(microbenchmark) is now (since a couple years ago) the community standard way to time things. times <- microbenchmark( lm(y~x), glm(y~x), times=1e3); example(microbenchmark). This does a statistical comparison of lm vs glm over 1000 tries, rather than system.time testing only once. Commented Apr 24, 2015 at 17:42
  • use res <- microbenchmark(your code1,your code2) and then print(res) to see a table or ggplot2::autoplot(res) to see a boxplot! ref
    – Travis
    Commented Dec 15, 2019 at 12:02

16 Answers 16

328

Another possible way of doing this would be to use Sys.time():

start.time <- Sys.time()
...Relevent codes...
end.time <- Sys.time()
time.taken <- end.time - start.time
time.taken

Not the most elegant way to do it, compared to the answere above , but definitely a way to do it.

5
  • 17
    This is much more memory-efficient, then system.time(), which effectively copies its arguments. It is important when you are dealing with data that barely fit into your RAM. Commented Dec 15, 2015 at 15:21
  • 4
    For people who use Sys.time, please read this for some caveat: Timing R code with Sys.time()
    – Zheyuan Li
    Commented Aug 13, 2018 at 16:20
  • 1
    system.time() was faster for me. I think that answer for system.time() should be accepted! Commented Jun 13, 2019 at 9:42
  • 1
    This is my preferred way to know the time it takes for a long calculation done in parallel on multiple cores. In this case, the wall clock time measured through this call is accurate enough since the computer will be much busier with all cores calculating than doing anything else and the calculations take minutes or hours to complete. This is a very specific use case but worth mentioning. Commented May 17, 2020 at 0:27
  • 2
    For those that like one-liners: s=Sys.time(); <code here> ; Sys.time()-s;. This will print the time difference, along with any output your code might produce.
    – Cole
    Commented Jun 2, 2021 at 5:16
223

The built-in function system.time() will do it.

Use like: system.time(result <- myfunction(with, arguments))

4
  • 1
    Important to know is that system.time() has an argument gcFirst which is TRUE by default. This on the one hand makes the measuring a bit more reproducible but can generate a significant overhead of total run time (which is not measured, off course).
    – jakob-r
    Commented Oct 26, 2016 at 15:29
  • 2
    what unit is this measured in? for example I just ran system.time(result <- myfunction(with, arguments)) and got 187.564 as an output- is that in seconds or what?
    – zsad512
    Commented Sep 14, 2017 at 0:40
  • For people who use system.time, please read this for some caveat: “object not found” and “unexpected symbol” errors when timing R code with system.time().
    – Zheyuan Li
    Commented Aug 13, 2018 at 16:16
  • @zsad512 I am reasonable sure that those are seconds.
    – Tapper
    Commented May 28, 2020 at 1:43
65

As Andrie said, system.time() works fine. For short function I prefer to put replicate() in it:

system.time( replicate(10000, myfunction(with,arguments) ) )
1
  • 31
    You are better of using the microbenchmark package because it doesn't include the overhead of replicate in the timing.
    – hadley
    Commented Jun 7, 2011 at 14:46
56

microbenchmark is a lightweight (~50kB) package and more-or-less a standard way in R for benchmarking multiple expressions and functions:

microbenchmark(myfunction(with,arguments))

For example:

> microbenchmark::microbenchmark(log10(5), log(5)/log(10), times = 10000)
Unit: nanoseconds
           expr min lq    mean median uq   max neval cld
       log10(5)   0  0 25.5738      0  1 10265 10000   a
 log(5)/log(10)   0  0 28.1838      0  1 10265 10000

Here both the expressions were evaluated 10000 times, with mean execution time being around 25-30 ns.

41

A slightly nicer way of measuring execution time, is to use the rbenchmark package. This package (easily) allows you to specify how many times to replicate your test and would the relative benchmark should be.

See also a related question at stats.stackexchange

3
  • 6
    Microbenchmark is even better because it uses higher precision timing functions.
    – hadley
    Commented Jun 7, 2011 at 14:45
  • 4
    @hadley But rbenchmark is more user-friendly in case of comparisons. For me microbenchmark is upgraded system.time. rmicrobenchmark is what we need :)
    – Marek
    Commented Jun 7, 2011 at 15:21
  • 3
    The maintainer of microbenchmark is pretty responsive - I bet he'd add whatever you needed.
    – hadley
    Commented Jun 7, 2011 at 16:03
41

There is also proc.time()

You can use in the same way as Sys.time but it gives you a similar result to system.time.

ptm <- proc.time()
#your function here
proc.time() - ptm

the main difference between using

system.time({ #your function here })

is that the proc.time() method still does execute your function instead of just measuring the time... and by the way, I like to use system.time with {} inside so you can put a set of things...

34

The package "tictoc" gives you a very simple way of measuring execution time. The documentation is in: https://cran.fhcrc.org/web/packages/tictoc/tictoc.pdf.

install.packages("tictoc")
require(tictoc)
tic()
rnorm(1000,0,1)
toc()

To save the elapsed time into a variable you can do:

install.packages("tictoc")
require(tictoc)
tic()
rnorm(1000,0,1)
exectime <- toc()
exectime <- exectime$toc - exectime$tic
23

Although other solutions are useful for a single function, I recommend the following piece of code where is more general and effective:

Rprof(tf <- "log.log", memory.profiling = TRUE)
# the code you want to profile must be in between
Rprof (NULL) ; print(summaryRprof(tf))
3
  • 2
    I didn't know about Rprof until now and it's indeed great! plus it comes with base R so no need for extra package as microbenchmark or profvis.
    – Simon C.
    Commented Jan 18, 2020 at 20:11
  • I wonder if rprof can be visualize as well, like for example if we want to plot the time for each items it profiling?
    – Zawir Amin
    Commented May 19, 2020 at 3:48
  • @ZawirAmin There is a way, just use Rstudio >> profile menu
    – TPArrow
    Commented May 19, 2020 at 9:54
15

Another simple but very powerful way to do this is by using the package profvis. It doesn't just measure the execution time of your code but gives you a drill down for each function you execute. It can be used for Shiny as well.

library(profvis)

profvis({
  #your code here
})

Click here for some examples.

0
14

You can use MATLAB-style tic-toc functions, if you prefer. See this other SO question

Stopwatch function in R

1
  • Was about to add proc.time() … I like the cute name better. =) Commented Sep 26, 2014 at 6:53
8

You can use Sys.time(). However, when you record the time difference in a table or a csv file, you cannot simply say end - start. Instead, you should define the unit:

f_name <- function (args*){
start <- Sys.time()
""" You codes here """
end <- Sys.time()
total_time <- as.numeric (end - start, units = "mins") # or secs ... 
}

Then you can use total_time which has a proper format.

3

Several answers mention taking the difference of two Sys.time()s, ie.

start <- Sys.time()
## ... code here ... ##
end <- Sys.time()
end - start

This prints the result in human-readable format, such as "time difference of 2 secs". However, since the unit can vary (from "secs" to "mins" to "days"), it is less useful, say, to compare multiple runtimes on equal footing with this method if their units differ.

For non-interactive purposes, it is preferred to specify the unit of time.

Specifically, Sys.time() returns a POSIXct object. Taking the difference of two POSIXcts give an object of class difftime, which has a "units" attribute. The `-` operation, in particular, is defined to use difftime() when used with a POSIXct. That is,

time2 - time1

is equivalent to

difftime(time2, time1)

To specify the units attribute, add a units= argument, eg.

difftime(time2, time1, units="secs")

In summary, one can use Sys.time() with difftime() to measure runtimes with a specified unit (secs, mins, etc.), ie.

start <- Sys.time()
## ... code here ... ##
end <- Sys.time()
difftime(end, start, units="secs")
2

Compiling from all anwsers above I came up to use these simplified tic toc functions

tic <- function(){ start.time <<- Sys.time() }
toc <- function(){ round(Sys.time() - start.time) }

to be used as:

tic()
Sys.sleep(3)
toc()

and which prints:

Time difference of 3 secs

2

Based on bench package website:

bench::mark() from package bench is used to benchmark one or a series of expressions, we feel it has a number of advantages over alternatives.

  • Always uses the highest precision APIs available for each operating system (often nanoseconds).
  • Tracks memory allocations for each expression.
  • Tracks the number and type of R garbage collections per expression iteration.
  • Verifies equality of expression results by default, to avoid accidentally benchmarking inequivalent code.
  • Has bench::press(), which allows you to easily perform and combine benchmarks across a large grid of values.
  • Uses adaptive stopping by default, running each expression for a set amount of time rather than for a specific number of iterations.
  • Expressions are run in batches and summary statistics are calculated after filtering out iterations with garbage collections. This allows you to isolate the performance and effects of garbage collection on running time (for more details see Neal 2014).

The times and memory usage are returned as custom objects which have human readable formatting for display (e.g. 104ns) and comparisons (e.g. x$mem_alloc > "10MB").

There is also full support for plotting with ggplot2 including custom scales and formatting.

Use:

bench::mark(log10(5))
#> # A tibble: 1 × 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 log10(5)      212ns    274ns  2334086.        0B        0

Created on 2021-08-18 by the reprex package (v2.0.1)

0
library(rbenchmark)

sleep_func <- function() { Sys.sleep(0.5) }

benchmark(sleep_func())

out:

 test replications elapsed relative user.self sys.self user.child sys.child

1 sleep_func()          100   50.08        1      0.02        0         NA        NA
0

Only base package that shows difference in time with units. Units are displayed automatically (secs, mins, ...)

t1 <- Sys.time()
rnorm(10000,0,1)
t2 <- Sys.time()
sprintf("Time took %.2f %s", t2-t1, units(difftime(t2, t1)))

Where %.2f controls the number of decimal places.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.