78

I need a mature HTTP client library that is idiomatic to scala, concise in usage, simple semantics. I looked at the Apache HTTP and the Scala Dispatch and numerous new libraries that promise an idiomatic Scala wrapping. Apache HTTP client sure demands verbosity, while Dispatch was easily confusing.

What is a suitable HTTP client for Scala usage?

4
  • 1
    I'd suggest sticking with Dispatch. Sure, its operatoriness isn't to everyone's taste (it's not to mine), but it's not really as bad as it looks at first, and I'd guess that it's going to stay the most popular Scala option for a while. Sep 8, 2012 at 21:26
  • 4
    This might be a bit off topic, but I think we should have a fork of dispatch with plain English method names. This would not be hard to do and maintain
    – Kim Stebel
    Sep 8, 2012 at 21:26
  • 3
    @KimStebel Dispatch 0.9.x can be used with plain English methods alone. Sep 9, 2012 at 0:53
  • 1
    A similar question is stackoverflow.com/questions/11719373/…
    – Rick-777
    Sep 9, 2012 at 14:11

11 Answers 11

29

I did a comparison of most major HTTP client libraries available

Dispatch, and a few others libraries, are not maintained anymore. The only serious ones currently are spray-client and Play! WS.

spray-client is a bit arcane in its syntax. play-ws is quite easy to use :

(build.sbt)

libraryDependencies += "com.typesafe.play" %% "play-ws" % "2.4.3"

(basic usage)

val wsClient = NingWSClient()
wsClient
  .url("http://wwww.something.com")
  .get()
  .map { wsResponse =>
    // read the response
}
3
  • 2
    Dispatch had a reboot: github.com/dispatch/reboot So it is a viable candidate again. If you can get over the cryptic function 'names', it is a very pleasant library.
    – mvherweg
    Jan 11, 2017 at 21:37
  • Doesn't Play WS depend on Akka?
    – cdmckay
    Aug 9, 2017 at 0:20
  • "spray-client is a bit arcane" - I dig arcane syntax personally, but it can result in some IDE glitches. That said I've only ever run into one syntactical glitch with IntelliJ IDEA and spray-can spec testing.
    – Coder Guy
    Dec 20, 2018 at 15:40
28

I've recently started using Dispatch, a bit arcane (great general intro, serious lack of detailed scenario/use-case based docs). Dispatch 0.9.1 is a Scala wrapper around Ning's Async Http Client; to fully understand what going on requires introducing one's self to that library. In practice, the only thing I really had to look at was the RequestBuilder - everything else falling nicely into my understanding of HTTP.

I give the 0.9 release a solid thumbs up (so far!) on getting the job done very simply.. once you get past that initial learning curve.

Dispatch's Http "builder" is immutable, and seems to work well in a threaded environment. Though I can't find anything in docs to state that it is thread-safe; general reading of source suggests that it is.

Do be aware that the RequestBuilder's are mutable, and therefore are NOT thread-safe.

Here are some additional links I've found helpful:

7
  • 2
    Note that the periodic table of operators refer to the previous incarnation of Dispatch, and most of that has been cut off from 0.9.x, and probably won't come back. Sep 9, 2012 at 0:52
  • Thank you for that clarification. What little I've used has worked nicely: url builder shortcuts, POST, '<<' Sep 9, 2012 at 1:52
  • Yes, the request DSL is there -- it's the response that was dumped, aside things like as.String and as.xml.Elem, which are not symbols. Sep 9, 2012 at 3:30
  • Please help me with my new question stackoverflow.com/questions/12342062/… Sep 9, 2012 at 19:09
  • 12
    Why so many people recommend it? The DSL is hard to understand, the document is poor, the examples are few, I can't make a simplest working demo with it.
    – Freewind
    May 30, 2013 at 6:52
21

A little late to the party here, but I've been impressed with spray-client.

It's got a nice DSL for building requests, supports both sync and async execution, as well as a variety of (un)marshalling types (JSON, XML, forms). It plays very nicely with Akka, too.

2
  • 4
    spray client unfortunately states that the chunked requests and responses are not supported. So how could you download or upload very large files in a system with limited memory? (i.e. android & scala) Jul 3, 2013 at 11:08
  • 2
    Spray now depends on Akka.
    – user1804599
    Dec 30, 2014 at 18:35
15

sttp is the Scala HTTP library we've all been waiting for!

It has a fluent DSL for forming and executing requests (code samples from their README):

val request = sttp
  .cookie("session", "*!@#!@!$")
  .body(file) // of type java.io.File
  .put(uri"http://httpbin.org/put")
  .auth.basic("me", "1234")
  .header("Custom-Header", "Custom-Value")
  .response(asByteArray)

It supports synchronous, asynchronous, and streaming calls via pluggable backends, including Akka-HTTP (formerly Spray) and the venerable AsyncHttpClient (Netty):

implicit val sttpHandler = AsyncHttpClientFutureHandler()
val futureFirstResponse: Future[Response[String]] = request.send()

It supports scala.concurrent.Future, scalaz.concurrent.Task, monix.eval.Task, and cats.effect.IO - all the major Scala IO monad libraries.

Plus it has a few additional tricks up its sleeve:

val test = "chrabąszcz majowy" val testUri: Uri = uri"http://httpbin.org/get?bug=$test"

  • It supports encoders/decoders for request bodies/responses e.g. JSON via Circe:

import com.softwaremill.sttp.circe._ val response: Either[io.circe.Error, Response] = sttp .post(uri"...") .body(requestPayload) .response(asJson[Response]) .send()

Finally, it's maintained by the reliable folks at softwaremill and it's got great documentation.

0
14

Two Six years after originally responding to this post, I would have a different answer.

I've been using akka-http, a collaboration between the spray and akka teams. It's backed by Lightbend, tightly aligned with the akka async environment... it's the right tool for this job.

4
10

Having had some unhappy experiences with the Apache client, I set about writing my own. The built-in HttpURLConnection is widely asserted to be buggy. But that's not my experience of it. In fact, the reverse has been so, the Apache client having a somewhat problematic threading model. Since Java6 (or 5?), HttpURLConnection has provided efficient HTTP1.1 connections with essentials like keep-alive being built in, and it handles concurrent usage without fuss.

So, to compensate for the inconvenient API offered by HttpURLConnection, I set about writing a new API in Scala, as an open-source project. It's just a wrapper for HttpURLConnection, but unlike HttpURLConnection, it aims to be easy to use. Unlike Apache client, it should fit easily into an existing project. Unlike Dispatch, it should be easy to learn.

It's called Bee Client

My apologies for the shameless plug. :)

4
  • I'm trying to add it as Maven dependency, no luck. Maybe I must add another repository first? Jun 14, 2013 at 11:56
  • Out of curiosity, have you been able to use keep-alive with HttpUrlConnection on a per-connection basis, or is there really no other way than the global system property? Jun 10, 2014 at 11:00
  • The Config object has a keepAlive boolean: it controls whether the connections are to be closed or kept-alive on a per-connection basis (bigbeeconsultants.co.uk/docs/bee-client/latest/…).
    – Rick-777
    Jun 10, 2014 at 17:11
  • Is this lib still maintained? The link to the source repo is bust and there doesn't seem to be a build for any versions of scala later than 2.11. Sep 14, 2020 at 10:55
5

Besides Dispatch there is not much out there. scalaz had a attempt at building a functional http client. But it is outdated for a while an no version of it exists in the scalaz7 branch. Additionally there is a useful wrapper of the ning async-http-client within the playframework. There your can do calls like:

WS.url("http://example.com/feed").get()
WS.url("http://example.com/item").post("content")

You can use this API as inspiration if you don't use play! in your project and dislike the Dispatch API.

4

Spray

You really should consider using Spray. In my opinion it has a bit of tricky syntax, but it is still pretty usable if you aim to build a high-performance http client. The main advantage of using Spray is that it is based on the akka actor library, which is extremely scalable and powerful. You can scale out your http client to several machines by only changing conf files.

Moreover few month ago Spray join Typesafe, and as I understand it will become a part of the basic akka distribution. proof

Play2

Another option is the Play2 WS lib usage (doc). As far as I know it is still not separated from the Play distribution, but due to its extremely simplicity it is worth it to spend some time attaching the whole Play framework to get that part. There are some issues with providing configuration to it, so this is not great for drop-and-use cases. However, we have used it in few non Play-based projects and everything was fine.

3

ScalaJ-Http is a very simple synchronous http client

https://github.com/scalaj/scalaj-http

I'd recommend it if you need a no-ceremony barebones Scala client.

1

Surprised that no one mentioned finagle here. It is super simple to use:

import com.twitter.finagle.{Http, Service}
import com.twitter.finagle.http
import com.twitter.util.{Await, Future}

object Client extends App {
  val client: Service[http.Request, http.Response] = Http.newService("www.scala-lang.org:80")
  val request = http.Request(http.Method.Get, "/")
  request.host = "www.scala-lang.org"
  val response: Future[http.Response] = client(request)
  Await.result(response.onSuccess { rep: http.Response =>
    println("GET success: " + rep)
  })
}

See quick start guid for more detail: https://twitter.github.io/finagle/guide/Quickstart.html

0

I've used Dispatch, Spray Client and the Play WS Client Library...None of them were simply to use or configure. So I created a simpler HTTP Client library which lets you perform all the classic HTTP requests in simple one-liners.

See an example:

import cirrus.clients.BasicHTTP.GET

import scala.concurrent.Await
import scala.concurrent.duration._

object MinimalExample extends App {

  val html = Await.result(Cirrus(GET("https://www.google.co.uk")), 3 seconds)

  println(html)
}

... produces ...

<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en-GB">...</html>

The library is called Cirrus and is available via Maven Central

libraryDependencies += "com.github.godis" % "cirrus_2.11" % "1.4.1"

The documentation is available on GitHub

https://github.com/Godis/Cirrus

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.