123

We use GSLB for geo-distribution and load-balancing. Each service is assigned a fixed domain name. Through some DNS magic, the domain name is resolved into an IP that's closest to the server with least load. For the load-balancing to work, the application server needs to honor the TTL from DNS response and to resolve the domain name again when cache times out. However, I couldn't figure out a way to do this in Java.

The application is in Java 5, running on Linux (Centos 5).

8 Answers 8

97

Per Byron's answer, you can't set networkaddress.cache.ttl or networkaddress.cache.negative.ttl as System Properties by using the -D flag or calling System.setProperty because these are not System properties - they are Security properties.

If you want to use a System property to trigger this behavior (so you can use the -D flag or call System.setProperty), you will want to set the following System property:

-Dsun.net.inetaddr.ttl=0

This system property will enable the desired effect. "0" (Zero) here means disable DNS caching at the JVM level, thus the original TTL value from the DNS will be consulted and used.

But be aware: if you don't use the -D flag when starting the JVM process and elect to call this from code instead:

java.security.Security.setProperty("networkaddress.cache.ttl" , "0")

This code must execute before any other code in the JVM attempts to perform networking operations.

This is important because, for example, if you called Security.setProperty in a .war file and deployed that .war to Tomcat, this wouldn't work: Tomcat uses the Java networking stack to initialize itself much earlier than your .war's code is executed. Because of this 'race condition', it is usually more convenient to use the -D flag when starting the JVM process.

If you don't use -Dsun.net.inetaddr.ttl=0 or call Security.setProperty, you will need to edit $JRE_HOME/lib/security/java.security and set those security properties in that file, e.g.

networkaddress.cache.ttl = 0
networkaddress.cache.negative.ttl = 0

But pay attention to the security warnings in the comments surrounding those properties. Only do this if you are reasonably confident that you are not susceptible to DNS spoofing attacks.

It is worth noting that the Oracle JDK 8 docs recommended approach is to to use networkaddress.cache.ttl and networkaddress.cache.negative.ttl as already described in the official documentation here. These two properties cannot be set at the command line but should be set using setProperty() or utilize the security policy file.

7
  • 2
    The FQN is java.security.Security (at least in jdk7) Dec 23, 2013 at 14:37
  • 1
    Just a comment, those security warnings are mostly in relation to security managers and remote loading. For any normal server application which trusts the DNS to some extend reducing the TTL is fine. (However I dont think 0 is a good minimum and the default of 30s for non-security managers is just fine in most cases).
    – eckes
    Aug 14, 2015 at 16:14
  • 3
    Does the System Property work with the OpenJDK as well or is it Oracle specific?
    – mhlz
    Nov 10, 2016 at 16:18
  • 7
    Never checking the DNS again after the first lookup doesn't protect you from spoofing attacks, it makes a spoofing attack permanent instead of temporary.
    – kbolino
    Sep 11, 2020 at 21:31
  • 1
    The java.security file is located in $JRE_HOME/conf/security/java.security since at least java 11 reference
    – jebeaudet
    Oct 4, 2022 at 16:38
76

Java has some seriously weird dns caching behavior. Your best bet is to turn off dns caching or set it to some low number like 5 seconds.

networkaddress.cache.ttl (default: -1)
Indicates the caching policy for successful name lookups from the name service. The value is specified as as integer to indicate the number of seconds to cache the successful lookup. A value of -1 indicates "cache forever".

networkaddress.cache.negative.ttl (default: 10)
Indicates the caching policy for un-successful name lookups from the name service. The value is specified as as integer to indicate the number of seconds to cache the failure for un-successful lookups. A value of 0 indicates "never cache". A value of -1 indicates "cache forever".

7
  • 8
    Note: this doesn't disable all DNS caching in your OS. Just disables Java's own broken in-memory caching in the library. You can simply set these properties on the command line when you invoke the JVM.
    – Nelson
    Aug 10, 2009 at 19:57
  • 4
    I don't know that "broken" is valid. Java (for security reasons) caches DNS entries forever, or until the JVM is restarted, whichever comes first. This (from what I can tell) was by design. The settings can be made in the java.security policy file, or at the command line. The settings are different for each. Reference: rgagnon.com/javadetails/java-0445.html
    – Milner
    Feb 3, 2010 at 15:27
  • 4
    Note that you can't set these as System properties (i.e using the -D flags or System.setProperty) because they're not system properties - they're Security properties. Jun 20, 2013 at 16:26
  • 7
    This documentation is slightly different in 1.7. Specifically, the cache forever now only happens when a security manager is present: "The default behavior is to cache forever when a security manager is installed, and to cache for an implementation specific period of time, when a security manager is not installed." docs.oracle.com/javase/7/docs/technotes/guides/net/… Jun 12, 2014 at 12:11
  • 2
    @Michael see System.getSecurityManager(). Docs for Java 8: docs.oracle.com/javase/8/docs/api/java/lang/…
    – gesellix
    Feb 28, 2017 at 20:24
22

This has obviously been fixed in newer releases (SE 6 and 7). I experience a 30 second caching time max when running the following code snippet while watching port 53 activity using tcpdump.

/**
 * http://stackoverflow.com/questions/1256556/any-way-to-make-java-honor-the-dns-caching-timeout-ttl
 *
 * Result: Java 6 distributed with Ubuntu 12.04 and Java 7 u15 downloaded from Oracle have
 * an expiry time for dns lookups of approx. 30 seconds.
 */

import java.util.*;
import java.text.*;
import java.security.*;

import java.net.InetAddress;
import java.net.UnknownHostException;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;

public class Test {
    final static String hostname = "www.google.com";
    public static void main(String[] args) {
        // only required for Java SE 5 and lower:
        //Security.setProperty("networkaddress.cache.ttl", "30");

        System.out.println(Security.getProperty("networkaddress.cache.ttl"));
        System.out.println(System.getProperty("networkaddress.cache.ttl"));
        System.out.println(Security.getProperty("networkaddress.cache.negative.ttl"));
        System.out.println(System.getProperty("networkaddress.cache.negative.ttl"));

        while(true) {
            int i = 0;
            try {
                makeRequest();
                InetAddress inetAddress = InetAddress.getLocalHost();
                System.out.println(new Date());
                inetAddress = InetAddress.getByName(hostname);
                displayStuff(hostname, inetAddress);
            } catch (UnknownHostException e) {
                e.printStackTrace();
            }
            try {
                Thread.sleep(5L*1000L);
            } catch(Exception ex) {}
            i++;
        }
    }

    public static void displayStuff(String whichHost, InetAddress inetAddress) {
        System.out.println("Which Host:" + whichHost);
        System.out.println("Canonical Host Name:" + inetAddress.getCanonicalHostName());
        System.out.println("Host Name:" + inetAddress.getHostName());
        System.out.println("Host Address:" + inetAddress.getHostAddress());
    }

    public static void makeRequest() {
        try {
            URL url = new URL("http://"+hostname+"/");
            URLConnection conn = url.openConnection();
            conn.connect();
            InputStream is = conn.getInputStream();
            InputStreamReader ird = new InputStreamReader(is);
            BufferedReader rd = new BufferedReader(ird);
            String res;
            while((res = rd.readLine()) != null) {
                System.out.println(res);
                break;
            }
            rd.close();
        } catch(Exception ex) {
            ex.printStackTrace();
        }
    }
}
10
  • 16
    Yes, Java 1.5 had a default value of infinite caching. Java 1.6 and 1.7 have a 30 second default.
    – Michael
    Mar 19, 2013 at 21:15
  • 7
    The documentation for 1.7 indicates this may only be true in the case where a security manager is not present: "The default behavior is to cache forever when a security manager is installed, and to cache for an implementation specific period of time, when a security manager is not installed." docs.oracle.com/javase/7/docs/technotes/guides/net/… Jun 12, 2014 at 12:10
  • 1
    @Michael care to share the source for that information?
    – rustyx
    Oct 25, 2014 at 21:38
  • 4
    @rustyx Oracle's 1.6 and 1.7 JDK has this in jre/lib/security/java.security for networkaddress.cache.ttl: "# default value is forever (FOREVER). For security reasons, this # caching is made forever when a security manager is set. When a security # manager is not set, the default behavior is to cache for 30 seconds." So applets and apps deployed via Java Web Start still cache forever otherwise it is 30 seconds.
    – Michael
    Oct 27, 2014 at 17:44
  • 1
    Here's a code pointer to OpenJDK 8's java.security, which says that without a security manager the TTL is 30s: hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/f940e7a48b72/src/share/… . I tested this on Mac OS X and Ubuntu 14.04.
    – tro
    Mar 28, 2018 at 14:42
19

To expand on Byron's answer, I believe you need to edit the file java.security in the %JRE_HOME%\lib\security directory to effect this change.

Here is the relevant section:

#
# The Java-level namelookup cache policy for successful lookups:
#
# any negative value: caching forever
# any positive value: the number of seconds to cache an address for
# zero: do not cache
#
# default value is forever (FOREVER). For security reasons, this
# caching is made forever when a security manager is set. When a security
# manager is not set, the default behavior is to cache for 30 seconds.
#
# NOTE: setting this to anything other than the default value can have
#       serious security implications. Do not set it unless 
#       you are sure you are not exposed to DNS spoofing attack.
#
#networkaddress.cache.ttl=-1 

Documentation on the java.security file here.

2
  • 5
    To add to this, when using tomcat6 I had to modify my lib/security file, as setting networkaddress.cache.ttl or sun.net.inetaddr.ttl either programmatically or via the JAVA_OPTS variable did not work.
    – bramp
    Jul 20, 2011 at 2:40
  • 1
    @bramp Thanks bro, I am also facing the same problem and resolved by using your comment and answers +1 for comment and answer. Oct 12, 2015 at 11:20
8

To summarize the other answers, in <jre-path>/lib/security/java.security you can set the value of the property networkaddress.cache.ttl to adjust how DNS lookups are cached. Note that this is not a system property but a security property. I was able to set this using:

java.security.Security.setProperty("networkaddress.cache.ttl", "<value>");

This can also be set by the system property -Dsun.net.inetaddr.ttl though this will not override a security property if it is set elsewhere.

I would also like to add that if you are seeing this issue with web services in WebSphere, as I was, setting networkaddress.cache.ttl will not be enough. You need to set the system property disableWSAddressCaching to true. Unlike the time-to-live property, this can be set as a JVM argument or via System.setProperty).

IBM has a pretty detailed post on how WebSphere handles DNS caching here. The relevant piece to the above is:

To disable address caching for Web services, you need to set an additional JVM custom property disableWSAddressCaching to true. Use this property to disable address caching for Web services. If your system typically runs with lots of client threads, and you encounter lock contention on the wsAddrCache cache, you can set this custom property to true, to prevent caching of the Web services data.

2

According to the official oracle java properties, sun.net.inetaddr.ttl is Sun implementation-specific property, which "may not be supported in future releases". "the preferred way is to use the security property" networkaddress.cache.ttl.

1

So I decided to look at the java source code because I found official docs a bit confusing. And what I found (for OpenJDK 11) mostly aligns with what others have written. What is important is the order of evaluation of properties.

InetAddressCachePolicy.java (I'm omitting some boilerplate for readability):


String tmpString = Security.getProperty("networkaddress.cache.ttl");
if (tmpString != null) {
   tmp = Integer.valueOf(tmpString);
   return;
}
...
String tmpString = System.getProperty("sun.net.inetaddr.ttl");
if (tmpString != null) {
   tmp = Integer.valueOf(tmpString);
   return;
}
...
if (tmp != null) {
  cachePolicy = tmp < 0 ? FOREVER : tmp;
  propertySet = true;
} else {
  /* No properties defined for positive caching. If there is no
  * security manager then use the default positive cache value.
  */
  if (System.getSecurityManager() == null) {
    cachePolicy = 30;
  }
}

You can clearly see that the security property is evaluated first, system property second and if any of them is set cachePolicy value is set to that number or -1 (FOREVER) if they hold a value that is bellow -1. If nothing is set it defaults to 30 seconds. As it turns out for OpenJDK that is almost always the case because by default java.security does not set that value, only a negative one.

#networkaddress.cache.ttl=-1 <- this line is commented out
networkaddress.cache.negative.ttl=10

BTW if the networkaddress.cache.negative.ttl is not set (removed from the file) the default inside the java class is 0. Documentation is wrong in this regard. This is what tripped me over.

1
  • I'd say that the documentation is correct in that the default value is 10 seconds. After you remove the config from java.security file it's no longer original default behavior. Jan 27, 2022 at 8:15
0

I had the same issue. By default JVM caches the results of DNS lookup forever. Setting networkaddress.cache.ttl to 0 helps, but it doesn't "honor" the DNS record TTL.

I used https://github.com/dnsjava/dnsjava library.

String remoteIp = org.xbill.DNS.Address.getByName(remoteHostname).getHostAddress();

This does honor the DNS record TTL regardless of networkaddress.cache.ttl setting.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.