4

I've been reading through documentation for some time. I can see examples for the JavaScript and Go SDK that show how to load the config file by setting the AWS_SDK_LOAD_CONFIG environment variable to a truthy value. The documentation is here and here respectively.

However, for my requirements, I must use Java. I can't find an equivalent reference in the Java SDK. Which leads me to assume three things.

  1. Java's SDK doesn't use this variable
    • I'm pretty sure this might be the case, as just trying it didn't seem to get it to work.
    • Update: checking out both the Java SDK and Java SDK V2 and searching with ack -i "AWS_SDK_LOAD_CONFIG" shows that neither project use this variable.
  2. Java's SDK uses a different variable
    • I think this is un-likely, as it would not be uniform with the other two SDKs.
  3. Java's SDK expects you to do this programmatically.
    • Seems the most likely, yet I can't find how to do this. I must be using the wrong key-words or be overlooking something to get this behavior.

For clarity the profile I need to load is sbx, which lives in my config, but has no adjacent value in the credentials file. Here is my ~/.aws/config file:

[profile shared]
output = json
region = us-west-2
adfs_config.ssl_verification = True
adfs_config.role_arn = ....
adfs_config.adfs_host = ....
adfs_config.adfs_user = ....

[profile sbx]
role_arn = ... (this is different from the adfs_config.role_arn above)
source_profile = shared
region = us-west-2

and ~/.aws/credentials file: (this file is automatically populated with the aws-adfs command.

[shared]
aws_access_key_id = ....
aws_secret_access_key = ....
aws_session_token = ....
aws_security_token = ....
8
  • "the config file". What config file? Feb 28, 2018 at 20:50
  • @CardinalSystem I've updated the title. If you follow the links I provided you will see that I'm referring to the ~/.aws/config file.
    – Niko
    Feb 28, 2018 at 20:52
  • What are you trying to do here? Is it to simply load credentials/region info for a specific profile, or is it to override the location of the ~/.aws/config file?
    – jarmod
    Feb 28, 2018 at 21:33
  • @jarmod Thanks for pointing that out. It's the former. The profile I need is in the config and specifies a role that I need to assume as well as depend on another profile.
    – Niko
    Feb 28, 2018 at 21:35
  • Isn't [profile shared] should be [shared]?
    – alpert
    Feb 28, 2018 at 22:02

3 Answers 3

5

I figured out the answer. The problem to this issue is that the credentials file is loaded by default, but it doesn't always have all the information available from the config file. We need both to be loaded and flattened.

AWS already provides ProfileAssumeRoleCredentialsProvider which allows us to assume a Role from a profile. Once we provide it all the information it needs, it can assume the Role without issue (assuming your token is current)

/**
 * @author Paul Nelson Baker
 * @see <a href="https://github.com/paul-nelson-baker/">GitHub</a>
 * @see <a href="https://www.linkedin.com/in/paul-n-baker/">LinkedIn</a>
 * @since 2018-11
 */
public class CredentialsChain {

  public static final AWSCredentialsProviderChain CREDENTIALS_PROVIDER_CHAIN;

  static {
      AllProfiles allProfiles = flattenConfigurationFiles(
              DEFAULT_CONFIG_LOCATION_PROVIDER.getLocation(), // ~/.aws/config
              DEFAULT_CREDENTIALS_LOCATION_PROVIDER.getLocation() // ~/.aws/credentials
      );
      String currentProfileName = AwsProfileNameLoader.INSTANCE.loadProfileName();
      BasicProfile currentProfile = allProfiles.getProfile(currentProfileName);
      STSProfileCredentialsService profileCredentialsService = new STSProfileCredentialsService();
      // We stick our merged profile provider first, but we still want the default behavior to apply
      // so create a new chain with the default chain as the tail provider.
      CREDENTIALS_PROVIDER_CHAIN = new AWSCredentialsProviderChain(
              new ProfileAssumeRoleCredentialsProvider(profileCredentialsService, allProfiles, currentProfile),
              new DefaultAWSCredentialsProviderChain()
      );
  }

  private static AllProfiles flattenConfigurationFiles(File firstFile, File... additionalFiles) {
      // Utilize the AWS SDK to load the actual profile objects
      List<ProfilesConfigFile> allProfileConfigFiles = Stream.concat(Stream.of(firstFile), Arrays.stream(additionalFiles))
              .map(ProfilesConfigFile::new).collect(Collectors.toList());
      // Process each file one by one, look at their profiles, and place their values into a single map
      // Duplicate profiles will now have the single key/value pairs.
      Map<String, Map<String, String>> buildingMap = new LinkedHashMap<>();
      for (ProfilesConfigFile currentConfigFile : allProfileConfigFiles) {
          for (Entry<String, BasicProfile> currentProfile : currentConfigFile.getAllBasicProfiles().entrySet()) {
              // Some profiles are prefixed with "profile " so we want to cull it so we're actually merging the correct data
              String currentProfileName = currentProfile.getKey().replaceAll("^profile\\s+", "");
              if (!buildingMap.containsKey(currentProfileName)) {
                  buildingMap.put(currentProfileName, new LinkedHashMap<>());
              }
              Map<String, String> profileKeyValuePairs = buildingMap.get(currentProfileName);
              for (Entry<String, String> overridingEntry : currentProfile.getValue().getProperties().entrySet()) {
                  profileKeyValuePairs.put(overridingEntry.getKey(), overridingEntry.getValue());
              }
          }
      }
      // Take the results, and convert them to AWS SDK Types
      Map<String, BasicProfile> finalResult = new LinkedHashMap<>();
      for (Entry<String, Map<String, String>> currentFinalProfile : buildingMap.entrySet()) {
          String currentProfileName = currentFinalProfile.getKey();
          finalResult.put(currentProfileName, new BasicProfile(currentProfileName, currentFinalProfile.getValue()));
      }
      return new AllProfiles(finalResult);
  }

  private CredentialsChain() {
  }
}
1
  • 4
    Another example using ProfileAssumeRoleCredentialsProvier here (found it from this link)
    – dutoitns
    Dec 26, 2018 at 14:58
0

For the Java SDK the variable name seems to be AWS_CONFIG_FILE: https://github.com/aws/aws-sdk-java/blob/4734de6fb0f80fe5768a6587aad3b9d0eaec388f/aws-java-sdk-core/src/main/java/com/amazonaws/SDKGlobalConfiguration.java#L233

3
  • Seems like a logical thing to assume, but it didn't produce any change when I tried just now. Both with the relative and full path don't seem to actually trigger its use. In fact I can't find where it's actually used when I do more acks for AWS_CONFIG_FILE or AWS_CONFIG_FILE_ENV_VAR
    – Niko
    Feb 28, 2018 at 21:34
  • Thank you, that's good information to have. I don't need to tell it where it is, so much as tell it to start using it. Like I mentioned I've tried pointing it to the file with both the full and relative paths.
    – Niko
    Feb 28, 2018 at 21:49
-2

If you need to specify a particular profile within the configuration file you'll need to use ProfileCredentialsProvider:

(I'm not entering this code in an IDE, so there may be typos; feel free to edit):

ProfileCredentialsProvider creds = new ProfileCredentialsProvider("myProfile");
AmazonS3 s3Client = new AmazonS3ClientBuilder()
                    .withCredentials(creds)
                    .build()

This will (I believe) use the default file location, ~/.aws/credentials. There's also a constructor that lets you pass a path to the file.

If you can get away with the default profile, you can just use the static defaultClient() function that all of the builders expose. This has the benefit that you can override using explicit environment variables for the creds, or retrieve from an instance profile.

2
  • 3
    Unfortunately, this doesn't load the config file but reads from the credentials file. It cannot find the sbx entry and dies with java.lang.IllegalArgumentException: No AWS profile named 'sbx'.
    – Niko
    Feb 28, 2018 at 21:48
  • (In case you missed it, I updated my question to show my config/credentials files. I think I submitted my update about the same time as your question so you might not have had the important details necessary)
    – Niko
    Feb 28, 2018 at 21:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.