In Java, How to compose an HTTP request message and send it to an HTTP web server?
-
java.sun.com/javase/6/docs/api/java/net/HttpURLConnection.html In particular, getHeaderField, getHeaderFieldKey, and getContent– Federico klez CullocaCommented Aug 31, 2009 at 22:37
-
33There's a mini tutorial here at SO.– BalusCCommented Oct 18, 2010 at 12:56
-
You could use the JSoup lib (jsoup.org) . It does exactly what you ask! Document doc = Jsoup.connect("en.wikipedia.org"). get(); (from the site). A more pythonic way for java.– user2007447Commented Jul 26, 2015 at 13:12
11 Answers
You can use java.net.HttpUrlConnection.
Example (from here), with improvements. Included in case of link rot:
public static String executePost(String targetURL, String urlParameters) {
HttpURLConnection connection = null;
try {
//Create connection
URL url = new URL(targetURL);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
connection.setRequestProperty("Content-Length",
Integer.toString(urlParameters.getBytes().length));
connection.setRequestProperty("Content-Language", "en-US");
connection.setUseCaches(false);
connection.setDoOutput(true);
//Send request
DataOutputStream wr = new DataOutputStream (
connection.getOutputStream());
wr.writeBytes(urlParameters);
wr.close();
//Get Response
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
StringBuilder response = new StringBuilder(); // or StringBuffer if Java version 5+
String line;
while ((line = rd.readLine()) != null) {
response.append(line);
response.append('\r');
}
rd.close();
return response.toString();
} catch (Exception e) {
e.printStackTrace();
return null;
} finally {
if (connection != null) {
connection.disconnect();
}
}
}
-
25Putting some actual code into this answer will help avoid link rot...– CypherCommented Sep 25, 2014 at 16:19
-
3
-
Yes, a lot has changed in the ten years since this answer was given. Not everyone has moved on from JDK8 to 9 and beyond.– duffymoCommented Jun 28, 2019 at 12:44
-
How to put some header content in the request with this way of doing, please ?– nonozorCommented Sep 20, 2022 at 14:00
import java.net.*;
import java.io.*;
public class URLConnectionReader {
public static void main(String[] args) throws Exception {
URL yahoo = new URL("http://www.yahoo.com/");
URLConnection yc = yahoo.openConnection();
BufferedReader in = new BufferedReader(
new InputStreamReader(
yc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();
}
}
-
1The strange thing is that some servers will reply you back with strange ? characters (which seems like an encoding error related to request headers but not) if you don't open an output stream and flush it first. I have no idea why this happens but will be great if someone can explain why?– GorkyCommented Jan 18, 2013 at 8:33
-
1
-
103This is way too much line noise to send an HTTP request imo. Contrast to Python's requests library:
response = requests.get('http://www.yahoo.com/')
; something of similar brevity should be possible in Java. Commented Jul 12, 2014 at 19:09 -
25@leo-the-manic that's because Java is supposed to be a lower level language (than python) and allows (forces) the programmer to handle the details underneath rather than assuming "sane" defaults (i.e. buffering, character encoding, etc.). It is possible to get something as succinct, but then you lose lots of the flexibility of the more barebones approach.– fortranCommented Feb 17, 2015 at 23:54
-
15@fortran Python has equally low-level options to accomplish the same thing as above.– UserCommented Mar 18, 2017 at 4:46
I know others will recommend Apache's http-client, but it adds complexity (i.e., more things that can go wrong) that is rarely warranted. For a simple task, java.net.URL
will do.
URL url = new URL("http://www.y.com/url");
InputStream is = url.openStream();
try {
/* Now read the retrieved document from the stream. */
...
} finally {
is.close();
}
-
6That doesn't help if you want to monkey with request headers, something that's particularly useful when dealing with sites that will only respond a certain way to popular browsers.– JhericoCommented Aug 31, 2009 at 22:57
-
43You can monkey with request headers using URLConnection, but the poster doesn't ask for that; judging from the question, a simple answer is important.– ericksonCommented Sep 1, 2009 at 3:26
Apache HttpComponents. The examples for the two modules - HttpCore and HttpClient will get you started right away.
Not that HttpUrlConnection is a bad choice, HttpComponents will abstract a lot of the tedious coding away. I would recommend this, if you really want to support a lot of HTTP servers/clients with minimum code. By the way, HttpCore could be used for applications (clients or servers) with minimum functionality, whereas HttpClient is to be used for clients that require support for multiple authentication schemes, cookie support etc.
-
3FWIW, our code started with java.net.HttpURLConnection, but when we had to add SSL and work around some of the weird use cases in our screwy internal networks, it became a real headache. Apache HttpComponents saved the day. Our project currently still uses an ugly hybrid, with a few dodgy adapters to convert java.net.URLs to the URIs HttpComponents uses. I refactor those out regularly. The only time HttpComponents code turned out significantly more complicated was for parsing dates from a header. But the solution for that is still simple. Commented Dec 13, 2012 at 7:52
-
1
Here's a complete Java 7 program:
class GETHTTPResource {
public static void main(String[] args) throws Exception {
try (java.util.Scanner s = new java.util.Scanner(new java.net.URL("http://example.com/").openStream())) {
System.out.println(s.useDelimiter("\\A").next());
}
}
}
The new try-with-resources will auto-close the Scanner, which will auto-close the InputStream.
-
@Ska There is no unhandled exception.
main()
throws Exception, which encompasses the MalformedURLException and the IOException.– jerzyCommented Dec 22, 2017 at 9:33 -
Scanner actually is not very optimized when it comes to performance. Commented Mar 14, 2019 at 14:44
Google java http client has nice API for http requests. You can easily add JSON support etc. Although for simple request it might be overkill.
import com.google.api.client.http.GenericUrl;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpResponse;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.javanet.NetHttpTransport;
import java.io.IOException;
import java.io.InputStream;
public class Network {
static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
public void getRequest(String reqUrl) throws IOException {
GenericUrl url = new GenericUrl(reqUrl);
HttpRequest request = HTTP_TRANSPORT.createRequestFactory().buildGetRequest(url);
HttpResponse response = request.execute();
System.out.println(response.getStatusCode());
InputStream is = response.getContent();
int ch;
while ((ch = is.read()) != -1) {
System.out.print((char) ch);
}
response.disconnect();
}
}
-
-
Sorry, that should have been
HTTP_TRANSPORT
, I've edited the answer.– TombartCommented Feb 10, 2014 at 15:57 -
why is HttpResponse not AutoClosable? What is the difference from this and to working with Apache's CloseableHttpClient? Commented May 12, 2016 at 23:57
-
The benefit is the API, which makes it personal preference really. Google's library uses Apache's library internally. That said, I like Google's lib. Commented Jun 24, 2016 at 16:53
-
Simple or complex requests doesn't matter, readability is the king.– JohanCommented Aug 16, 2023 at 2:17
This will help you. Don't forget to add the JAR HttpClient.jar
to the classpath.
import java.io.FileOutputStream;
import java.io.IOException;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.methods.PostMethod;
public class MainSendRequest {
static String url =
"http://localhost:8080/HttpRequestSample/RequestSend.jsp";
public static void main(String[] args) {
//Instantiate an HttpClient
HttpClient client = new HttpClient();
//Instantiate a GET HTTP method
PostMethod method = new PostMethod(url);
method.setRequestHeader("Content-type",
"text/xml; charset=ISO-8859-1");
//Define name-value pairs to set into the QueryString
NameValuePair nvp1= new NameValuePair("firstName","fname");
NameValuePair nvp2= new NameValuePair("lastName","lname");
NameValuePair nvp3= new NameValuePair("email","[email protected]");
method.setQueryString(new NameValuePair[]{nvp1,nvp2,nvp3});
try{
int statusCode = client.executeMethod(method);
System.out.println("Status Code = "+statusCode);
System.out.println("QueryString>>> "+method.getQueryString());
System.out.println("Status Text>>>"
+HttpStatus.getStatusText(statusCode));
//Get data as a String
System.out.println(method.getResponseBodyAsString());
//OR as a byte array
byte [] res = method.getResponseBody();
//write to file
FileOutputStream fos= new FileOutputStream("donepage.html");
fos.write(res);
//release connection
method.releaseConnection();
}
catch(IOException e) {
e.printStackTrace();
}
}
}
-
1Seriously, I really like Java, but what's the matter with that stupid
NameValuePair
list or array. Why not a simpleMap<String, String>
? So much boilerplate code for such simple use cases...– JoffreyCommented Sep 3, 2014 at 11:59 -
6@Joffrey Maps by definition have 1 key per value, means:
A map cannot contain duplicate keys
! But HTTP Parameters can have duplicate keys.– BenCommented Dec 26, 2016 at 20:47
You may use Socket for this like
String host = "www.yourhost.com";
Socket socket = new Socket(host, 80);
String request = "GET / HTTP/1.0\r\n\r\n";
OutputStream os = socket.getOutputStream();
os.write(request.getBytes());
os.flush();
InputStream is = socket.getInputStream();
int ch;
while( (ch=is.read())!= -1)
System.out.print((char)ch);
socket.close();
-
-
@CuriousGuy look at this link programmers.stackexchange.com/questions/29075/…– laksysCommented Sep 18, 2016 at 1:16
-
3It seems to be even easier and more straight forward than the other solutions. Java makes things more complicated than it should be. Commented Jun 18, 2018 at 14:11
If you are using Java 11 or newer (except on Android), instead of the legacy HttpUrlConnection class, you can use Java 11 new HTTP Client API.
An example GET request:
var uri = URI.create("https://httpbin.org/get?age=26&isHappy=true");
var client = HttpClient.newHttpClient();
var request = HttpRequest
.newBuilder()
.uri(uri)
.header("accept", "application/json")
.GET()
.build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.statusCode());
System.out.println(response.body());
The same request executed asynchronously:
var responseAsync = client
.sendAsync(request, HttpResponse.BodyHandlers.ofString())
.thenApply(HttpResponse::body)
.thenAccept(System.out::println);
// responseAsync.join(); // Wait for completion
An example POST request:
var request = HttpRequest
.newBuilder()
.uri(uri)
.version(HttpClient.Version.HTTP_2)
.timeout(Duration.ofMinutes(1))
.header("Content-Type", "application/json")
.header("Authorization", "Bearer fake")
.POST(BodyPublishers.ofString("{ title: 'This is cool' }"))
.build();
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
For sending form data as multipart (multipart/form-data
) or url-encoded (application/x-www-form-urlencoded
) format, see this solution.
See this article for examples and more information about HTTP Client API.
For Java standard library HTTP server, see this post.
There's a great link about sending a POST request here by Example Depot::
try {
// Construct data
String data = URLEncoder.encode("key1", "UTF-8") + "=" + URLEncoder.encode("value1", "UTF-8");
data += "&" + URLEncoder.encode("key2", "UTF-8") + "=" + URLEncoder.encode("value2", "UTF-8");
// Send data
URL url = new URL("http://hostname:80/cgi");
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
// Get the response
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
// Process line...
}
wr.close();
rd.close();
} catch (Exception e) {
}
If you want to send a GET request you can modify the code slightly to suit your needs. Specifically you have to add the parameters inside the constructor of the URL. Then, also comment out this wr.write(data);
One thing that's not written and you should beware of, is the timeouts. Especially if you want to use it in WebServices you have to set timeouts, otherwise the above code will wait indefinitely or for a very long time at least and it's something presumably you don't want.
Timeouts are set like this conn.setReadTimeout(2000);
the input parameter is in milliseconds
New HTTP Client in Java 11+
Java 11 brought a new HTTP client. Contained in the new module java.net.http
. For more info, see:
- JEP 321: HTTP Client API
- JEP 110: HTTP/2 Client (Incubator)
- Java HTTP Client project page, with links to tutorials & presentations.
Using the new HTTP Client is easy. The framework provides convenient fluent syntax.
First create the client object.
HttpClient client = HttpClient.newBuilder( )
.version( HttpClient.Version.HTTP_1_1 )
.build( ) ;
Set up the request.
HttpRequest request = HttpRequest.newBuilder( )
.uri( URI.create( "http://worldtimeapi.org/api/timezone/UTC" ) )
.timeout( Duration.ofMinutes( 1 ) )
.GET( )
.build( );
Hit the server.
HttpResponse < String > response = client.send( request , HttpResponse.BodyHandlers.ofString( ) );
Extract the response parts.
response.statusCode( )
response.body( )
Full example code:
package work.basil.example.web;
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
public class ExHttpClient
{
public static void main ( String[] args )
{
// JEP 321: HTTP Client API
// https://openjdk.org/jeps/321
try (
HttpClient client = HttpClient.newBuilder( )
.version( HttpClient.Version.HTTP_1_1 )
.build( ) ;
)
{
// Example HTTP server: http://worldtimeapi.org/
HttpRequest request = HttpRequest.newBuilder( )
.uri( URI.create( "http://worldtimeapi.org/api/timezone/UTC" ) )
.timeout( Duration.ofMinutes( 1 ) )
.GET( )
.build( );
HttpResponse < String > response = client.send( request , HttpResponse.BodyHandlers.ofString( ) );
System.out.println( response.statusCode( ) );
System.out.println( response.body( ) );
} catch ( IOException | InterruptedException e )
{
throw new RuntimeException( e );
}
}
}
When run:
200
{"utc_offset":"+00:00","timezone":"UTC","day_of_week":3,"day_of_year":248,"datetime":"2024-09-04T18:51:19.336832+00:00","utc_datetime":"2024-09-04T18:51:19.336832+00:00","unixtime":1725475879,"raw_offset":0,"week_number":36,"dst":false,"abbreviation":"UTC","dst_offset":0,"dst_from":null,"dst_until":null,"client_ip":"71.212.7.38"}
HTTP Server
By the way, Java 18+ comes with a basic implementation of a HTTP Web server. See JEP 408: Simple Web Server. Caveat: This implementation is not meant for use in production; its purpose is learning and experimenting.
This simple HTTP server can be run from command-line in a console using a tool named jwebserver
. Or you can run the server from Java code.
So you could create your own little server to test your client code.