105

I have seen many examples of ARM (automatic resource management) on the web for Scala. It seems to be a rite-of-passage to write one, though most look pretty much like one another. I did see a pretty cool example using continuations, though.

At any rate, a lot of that code has flaws of one type or another, so I figured it would be a good idea to have a reference here on Stack Overflow, where we can vote up the most correct and appropriate versions.

4
  • Would this question generate more answers if it wasn't a community wiki? Note sure if voted answers in community wiki award reputation...
    – huynhjl
    Commented Feb 7, 2010 at 17:07
  • 2
    unique references can add another level of safety to ARM to ensure that references to resources are returned to the manager before close() is called. thread.gmane.org/gmane.comp.lang.scala/19160/focus=19168
    – retronym
    Commented Feb 14, 2010 at 8:55
  • @retronym I think the uniqueness plugin will be quite a revolution, more so than continuations. And, in fact, I think this is one thing in Scala that is quite likely to find itself ported to other languages in a not too distant future. When this comes out, let's be sure to edit the answers accordingly. :-) Commented Feb 15, 2010 at 0:23
  • 1
    Because I need to be able to nest multiple java.lang.AutoCloseable instances, each of which depends upon the prior one successfully instantiating, I finally hit upon a pattern that has been very useful for me. I wrote it up as an answer on similar StackOverflow question: stackoverflow.com/a/34277491/501113 Commented Jan 22, 2016 at 20:00

10 Answers 10

75
+550

Chris Hansen's blog entry 'ARM Blocks in Scala: Revisited' from 3/26/09 talks about about slide 21 of Martin Odersky's FOSDEM presentation. This next block is taken straight from slide 21 (with permission):

def using[T <: { def close() }]
    (resource: T)
    (block: T => Unit) 
{
  try {
    block(resource)
  } finally {
    if (resource != null) resource.close()
  }
}

--end quote--

Then we can call like this:

using(new BufferedReader(new FileReader("file"))) { r =>
  var count = 0
  while (r.readLine != null) count += 1
  println(count)
}

What are the drawbacks of this approach? That pattern would seem to address 95% of where I would need automatic resource management...

Edit: added code snippet


Edit2: extending the design pattern - taking inspiration from python with statement and addressing:

  • statements to run before the block
  • re-throwing exception depending on the managed resource
  • handling two resources with one single using statement
  • resource-specific handling by providing an implicit conversion and a Managed class

This is with Scala 2.8.

trait Managed[T] {
  def onEnter(): T
  def onExit(t:Throwable = null): Unit
  def attempt(block: => Unit): Unit = {
    try { block } finally {}
  }
}

def using[T <: Any](managed: Managed[T])(block: T => Unit) {
  val resource = managed.onEnter()
  var exception = false
  try { block(resource) } catch  {
    case t:Throwable => exception = true; managed.onExit(t)
  } finally {
    if (!exception) managed.onExit()
  }
}

def using[T <: Any, U <: Any]
    (managed1: Managed[T], managed2: Managed[U])
    (block: T => U => Unit) {
  using[T](managed1) { r =>
    using[U](managed2) { s => block(r)(s) }
  }
}

class ManagedOS(out:OutputStream) extends Managed[OutputStream] {
  def onEnter(): OutputStream = out
  def onExit(t:Throwable = null): Unit = {
    attempt(out.close())
    if (t != null) throw t
  }
}
class ManagedIS(in:InputStream) extends Managed[InputStream] {
  def onEnter(): InputStream = in
  def onExit(t:Throwable = null): Unit = {
    attempt(in.close())
    if (t != null) throw t
  }
}

implicit def os2managed(out:OutputStream): Managed[OutputStream] = {
  return new ManagedOS(out)
}
implicit def is2managed(in:InputStream): Managed[InputStream] = {
  return new ManagedIS(in)
}

def main(args:Array[String]): Unit = {
  using(new FileInputStream("foo.txt"), new FileOutputStream("bar.txt")) { 
    in => out =>
    Iterator continually { in.read() } takeWhile( _ != -1) foreach { 
      out.write(_) 
    }
  }
}
5
  • 2
    There are alternatives, but I don't meant to imply there's something wrong with that. I just want all those answers here, on Stack Overflow. :-) Commented Feb 8, 2010 at 10:46
  • 5
    Do you know if there is something like this in the standard API? Seems like a chore to have to write this for myself all the time. Commented Oct 15, 2014 at 10:23
  • Been a while since this was posted but the first solution does not close the inner stream if the out constructor throws which probably won't happen here but there are other cases where this can be bad. The close can also throw. No distinction between fatal exceptions either. The second one has code smells everywhere and has zero advantages over the first. You even loose the actual types so would be useless for something like a ZipInputStream.
    – steinybot
    Commented May 19, 2016 at 10:04
  • How do you recommend to do this if the block returns an iterator ? Commented Mar 9, 2020 at 13:52
  • Returning something: def using[R, T <: {def close(): Unit}](resource: T)(fnc: T => R): R = { ... } Commented Jun 16, 2023 at 8:33
63

I've just recently deployed the scala-arm library for automatic resource management. You can find the documentation here: https://github.com/jsuereth/scala-arm/wiki

This library supports three styles of usage (currently):

  1. Imperative/for-expression:

    import resource._
    for(input <- managed(new FileInputStream("test.txt")) {
    // Code that uses the input as a FileInputStream
    }
    
  2. Monadic-style

    import resource._
    import java.io._
    val lines = for { input <- managed(new FileInputStream("test.txt"))
                      val bufferedReader = new BufferedReader(new InputStreamReader(input)) 
                      line <- makeBufferedReaderLineIterator(bufferedReader)
                    } yield line.trim()
    lines foreach println
    
  3. Delimited Continuations-style

Here's an "echo" tcp server:

import java.io._
import util.continuations._
import resource._
def each_line_from(r : BufferedReader) : String @suspendable =
  shift { k =>
    var line = r.readLine
    while(line != null) {
      k(line)
      line = r.readLine
    }
  }
reset {
  val server = managed(new ServerSocket(8007)) !
  while(true) {
    // This reset is not needed, however the  below denotes a "flow" of execution that can be deferred.
    // One can envision an asynchronous execuction model that would support the exact same semantics as below.
    reset {
      val connection = managed(server.accept) !
      val output = managed(connection.getOutputStream) !
      val input = managed(connection.getInputStream) !
      val writer = new PrintWriter(new BufferedWriter(new OutputStreamWriter(output)))
      val reader = new BufferedReader(new InputStreamReader(input))
      writer.println(each_line_from(reader))
      writer.flush()
    }
  }
}

The code makes uses of a Resource type-trait, so it's able to adapt to most resource types. It has a fallback to use structural typing against classes with either a close or dispose method. Please check out the documentation and let me know if you think of any handy features to add.

1
  • 1
    Yes, I saw this. I want to look over the code, to see how you accomplish some things, but I'm way too busy right now. Anyway, since the goal of the question is to provide a reference to reliable ARM code, I'm making this the accepted answer. Commented Aug 9, 2010 at 15:04
18

Here's James Iry solution using continuations:

// standard using block definition
def using[X <: {def close()}, A](resource : X)(f : X => A) = {
   try {
     f(resource)
   } finally {
     resource.close()
   }
}

// A DC version of 'using' 
def resource[X <: {def close()}, B](res : X) = shift(using[X, B](res))

// some sugar for reset
def withResources[A, C](x : => A @cps[A, C]) = reset{x}

Here are the solutions with and without continuations for comparison:

def copyFileCPS = using(new BufferedReader(new FileReader("test.txt"))) {
  reader => {
   using(new BufferedWriter(new FileWriter("test_copy.txt"))) {
      writer => {
        var line = reader.readLine
        var count = 0
        while (line != null) {
          count += 1
          writer.write(line)
          writer.newLine
          line = reader.readLine
        }
        count
      }
    }
  }
}

def copyFileDC = withResources {
  val reader = resource[BufferedReader,Int](new BufferedReader(new FileReader("test.txt")))
  val writer = resource[BufferedWriter,Int](new BufferedWriter(new FileWriter("test_copy.txt")))
  var line = reader.readLine
  var count = 0
  while(line != null) {
    count += 1
    writer write line
    writer.newLine
    line = reader.readLine
  }
  count
}

And here's Tiark Rompf's suggestion of improvement:

trait ContextType[B]
def forceContextType[B]: ContextType[B] = null

// A DC version of 'using'
def resource[X <: {def close()}, B: ContextType](res : X): X @cps[B,B] = shift(using[X, B](res))

// some sugar for reset
def withResources[A](x : => A @cps[A, A]) = reset{x}

// and now use our new lib
def copyFileDC = withResources {
 implicit val _ = forceContextType[Int]
 val reader = resource(new BufferedReader(new FileReader("test.txt")))
 val writer = resource(new BufferedWriter(new FileWriter("test_copy.txt")))
 var line = reader.readLine
 var count = 0
 while(line != null) {
   count += 1
   writer write line
   writer.newLine
   line = reader.readLine
 }
 count
}
2
  • Doesn't using(new BufferedWriter(new FileWriter("test_copy.txt"))) suffer from problems when the BufferedWriter constructor fails? every resource should be wrapped in a using block...
    – Jaap
    Commented Aug 28, 2013 at 18:30
  • @Jaap This is the style suggested by Oracle. BufferedWriter doesn't throw checked exceptions, so if any exception is thrown, the program is not expected to recover from it. Commented Aug 28, 2013 at 19:15
17

For now Scala 2.13 has finally supported: try with resources by using Using :), Example:

val lines: Try[Seq[String]] =
  Using(new BufferedReader(new FileReader("file.txt"))) { reader =>
    Iterator.unfold(())(_ => Option(reader.readLine()).map(_ -> ())).toList
  }

or using Using.resource avoid Try

val lines: Seq[String] =
  Using.resource(new BufferedReader(new FileReader("file.txt"))) { reader =>
    Iterator.unfold(())(_ => Option(reader.readLine()).map(_ -> ())).toList
  }

You can find more examples from Using doc.

A utility for performing automatic resource management. It can be used to perform an operation using resources, after which it releases the resources in reverse order of their creation.

3
  • Could you please add the Using.resource variant as well? Commented Jun 11, 2019 at 19:58
  • @DanielC.Sobral, sure, just added it.
    – chengpohi
    Commented Jun 12, 2019 at 1:59
  • How would you write this for Scala 2.12? Here is a similar using method: def using[A <: AutoCloseable, B](resource: A) (block: A => B): B = try block(resource) finally resource.close()
    – Mike Slinn
    Commented Jul 9, 2019 at 0:51
7

I see a gradual 4 step evolution for doing ARM in Scala:

  1. No ARM: Dirt
  2. Only closures: Better, but multiple nested blocks
  3. Continuation Monad: Use For to flatten the nesting, but unnatural separation in 2 blocks
  4. Direct style continuations: Nirava, aha! This is also the most type-safe alternative: a resource outside withResource block will be type error.
4
  • 1
    Mind you, CPS in Scala are implemented through monads. :-) Commented Feb 8, 2010 at 10:47
  • 1
    Mushtaq, 3) You can do resource management in a monad that isn't the monad of continuation 4) Resource management using my withResources/resource delimited continuations code is no more (and no less) type safe than "using." It is still possible to forget to manage a resource that needs it. compare using(new Resource()) { first => val second = new Resource() //oops! // use resources } // only first gets closed withResources { val first = resource(new Resource()) val second = new Resource() // oops! // use resources... } // only first gets closed
    – James Iry
    Commented Feb 12, 2010 at 15:30
  • 2
    Daniel, CPS in Scala is like CPS in any functional language. It's delimited continuations that use a monad.
    – James Iry
    Commented Feb 12, 2010 at 15:33
  • James, thanks for explaining it well. Sitting in India I could only wish I was there for your BASE talk. Waiting to see when you put those slides online :) Commented Feb 13, 2010 at 1:28
6

There is light-weight (10 lines of code) ARM included with better-files. See: https://github.com/pathikrit/better-files#lightweight-arm

import better.files._
for {
  in <- inputStream.autoClosed
  out <- outputStream.autoClosed
} in.pipeTo(out)
// The input and output streams are auto-closed once out of scope

Here is how it is implemented if you don't want the whole library:

  type Closeable = {
    def close(): Unit
  }

  type ManagedResource[A <: Closeable] = Traversable[A]

  implicit class CloseableOps[A <: Closeable](resource: A) {        
    def autoClosed: ManagedResource[A] = new Traversable[A] {
      override def foreach[U](f: A => U) = try {
        f(resource)
      } finally {
        resource.close()
      }
    }
  }
1
  • This is pretty nice. I took something similar to this approach but defined a map and flatMap method for the CloseableOps instead of foreach so that for comprehensions wouldn't yield a traversable. Commented Aug 10, 2017 at 18:31
1

How about using Type classes

trait GenericDisposable[-T] {
   def dispose(v:T):Unit
}
...

def using[T,U](r:T)(block:T => U)(implicit disp:GenericDisposable[T]):U = try {
   block(r)
} finally { 
   Option(r).foreach { r => disp.dispose(r) } 
}
1

Another alternative is Choppy's Lazy TryClose monad. It's pretty good with database connections:

val ds = new JdbcDataSource()
val output = for {
  conn  <- TryClose(ds.getConnection())
  ps    <- TryClose(conn.prepareStatement("select * from MyTable"))
  rs    <- TryClose.wrap(ps.executeQuery())
} yield wrap(extractResult(rs))

// Note that Nothing will actually be done until 'resolve' is called
output.resolve match {
    case Success(result) => // Do something
    case Failure(e) =>      // Handle Stuff
}

And with streams:

val output = for {
  outputStream      <- TryClose(new ByteArrayOutputStream())
  gzipOutputStream  <- TryClose(new GZIPOutputStream(outputStream))
  _                 <- TryClose.wrap(gzipOutputStream.write(content))
} yield wrap({gzipOutputStream.flush(); outputStream.toByteArray})

output.resolve.unwrap match {
  case Success(bytes) => // process result
  case Failure(e) => // handle exception
}

More info here: https://github.com/choppythelumberjack/tryclose

0

Here is @chengpohi's answer, modified so it works with Scala 2.8+, instead of just Scala 2.13 (yes, it works with Scala 2.13 also):

def unfold[A, S](start: S)(op: S => Option[(A, S)]): List[A] =
  Iterator
    .iterate(op(start))(_.flatMap{ case (_, s) => op(s) })
    .map(_.map(_._1))
    .takeWhile(_.isDefined)
    .flatten
    .toList

def using[A <: AutoCloseable, B](resource: A)
                                (block: A => B): B =
  try block(resource) finally resource.close()

val lines: Seq[String] =
  using(new BufferedReader(new FileReader("file.txt"))) { reader =>
    unfold(())(_ => Option(reader.readLine()).map(_ -> ())).toList
  }
0

While Using is OK, I prefer the monadic style of resource composition. Twitter Util's Managed is pretty nice, except for its dependencies and its not-very-polished API.

To that end, I've published https://github.com/dvgica/managerial for Scala 2.12, 2.13, and 3.0.0. Largely based on the Twitter Util Managed code, no dependencies, with some API improvements inspired by cats-effect Resource.

The simple example:

import ca.dvgi.managerial._
val fileContents = Managed.from(scala.io.Source.fromFile("file.txt")).use(_.mkString)

But the real strength of the library is composing resources via for comprehensions.

Let me know what you think!

Not the answer you're looking for? Browse other questions tagged or ask your own question.