Java Awssdk S3 S3Client Upload object

Library
Upload file to S3Bucket with S3Client
Upload Stream, String, byte[]
Object Metadata
Object Tags
Redirect URL
S3Waiter Example
Object Retention

In Java Awssdk S3, there are two ways for you to upload files to S3 Bucket: using S3Client or S3TransferManager.

S3Client

S3Client provides a basic way to transfer objects to and from Amazon S3 with high performance and reliability, using the "Multipart Upload API" and "byte-range fetches" of Amazon AWS.

S3TransferManager

The S3 Transfer Manager is a high-level transfer utility built on top of the S3Client. It provides a simple API to allow you to transfer files and directories between your application and Amazon S3.

The S3TransferManager also enables you to monitor a transfer’s progress in real-time, as well as pause the transfer for execution at a later time.

In this article, I will guide you using S3Client to upload files to S3 Bucket and other related examples.

First we create the MyUtils class with a utility method to create the S3Client object, which is used in the examples of this article.

MyUtils.java

import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;

public class MyUtils {  

	public static S3Client createS3Client(Region region) {
		AwsCredentials credentials = AwsBasicCredentials.create("accessKeyId", "secretAccessKey");
		AwsCredentialsProvider credentialsProvider = StaticCredentialsProvider.create(credentials);

		S3Client client = S3Client.builder() //
				.credentialsProvider(credentialsProvider) //
				.region(region) //
				.build();

		return client;
	} 
}

To create an S3Client object you need to create an AwsCredentialsProvider object, this object provides credentials that allow you to interact with AWS. See the article below to create an AwsCredentialsProvider suitable for your purposes.
Java Awssdk Common Credentials Providers
Create AWS accessKeyId/secretAccessKey

1. Library

<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/s3 -->
<dependency>
  <groupId>software.amazon.awssdk</groupId>
  <artifactId>s3</artifactId>
  <version>2.21.10</version>
</dependency>

<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/s3-transfer-manager -->
<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>s3-transfer-manager</artifactId>
    <version>2.21.10</version>
</dependency>

<!-- Log Library -->
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-api -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>2.0.9</version>
</dependency>

<!-- Log Library -->
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>2.0.9</version> 
</dependency>

2. Upload file to S3Bucket with S3Client

A simple example using S3Client to upload a file to S3 Bucket.

S3ClientUploadExample1.java

package org.o7planning.java_14201_awssdk_s3;

import java.io.File;

import org.o7planning.awssdks3.utils.MyUtils;

import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;

public class S3ClientUploadExample1 {
	// EU (Frankfurt) - Germany.
	private static final Region myRegion = Region.EU_CENTRAL_1;
	private static final String bucketName = "test1.o7planning.org";

	public static void main(String[] args) {
		S3Client s3Client = MyUtils.createS3Client(myRegion);

		PutObjectRequest putObject = PutObjectRequest.builder() //
				.bucket(bucketName) //
				.key("statics/images/sample.png") //
				.build();

		File localFile = new File("/Volumes/New/Test/pngFile.png");

		// This example uses RequestBody.fromFile
		// to avoid loading the whole file into memory.
		s3Client.putObject(putObject, RequestBody.fromFile(localFile));

		System.out.println("Done!");
	}
}

In case your Bucket is ACL enabled, you can manage access to each individual object. For example, set public read permission for an object:

PutObjectRequest putObject = PutObjectRequest.builder() //
		.bucket(bucketName) //
		.key("statics/images/sample.png") //
		.acl(ObjectCannedACL.PUBLIC_READ) // (**)
		.build();

See more about ACL to see how it works.
AWS S3 ACL

3. Upload Stream, String, byte[]

You can also upload String, byte[] or a Stream of predetermined length to S3 Bucket. Take a look at the static "Factory" methods of the RequestBody class below:

RequestBody static methods

public static RequestBody fromInputStream(InputStream inputStream, long contentLength)  
public static RequestBody fromString(String contents, Charset cs) 
public static RequestBody fromString(String contents)  
public static RequestBody fromBytes(byte[] bytes)  
public static RequestBody fromByteBuffer(ByteBuffer byteBuffer) 
public static RequestBody fromRemainingByteBuffer(ByteBuffer byteBuffer)  
public static RequestBody empty()  
public static RequestBody fromContentProvider(ContentStreamProvider provider, long contentLength, String mimeType)  
public static RequestBody fromContentProvider(ContentStreamProvider provider, String mimeType)

For example, upload a String into an object on the S3 Bucket:

PutObjectRequest putObject = PutObjectRequest.builder() //
		.bucket(bucketName) //
		.key("readme.txt") // 
		.build();

RequestBody dataToUpload = RequestBody.fromString("Some Content..."); 
s3Client.putObject(putObject, dataToUpload);

4. Object Metadata

When uploading an object to S3 Bucket with S3Client you can set Metadata for this object.

By convention, user-defined Metadata must have the prefix "x-amz-meta-", for example:

x-amz-meta-myval-1
x-amz-meta-myval-2

S3ClientUploadMetadataExample1.java

package org.o7planning.java_14201_awssdk_s3;

import java.io.File;
import java.util.HashMap;
import java.util.Map;

import org.o7planning.awssdks3.utils.MyUtils;

import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.HeadObjectRequest;
import software.amazon.awssdk.services.s3.model.HeadObjectResponse;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;

public class S3ClientUploadMetadataExample1 {
	// EU (Frankfurt) - Germany.
	private static final Region myRegion = Region.EU_CENTRAL_1;
	private static final String bucketName = "test1.o7planning.org";

	public static void main(String[] args) {
		S3Client s3Client = MyUtils.createS3Client(myRegion);

		Map<String, String> userMetaData = new HashMap<>();
		userMetaData.put("x-amz-meta-myVal", "A Test Value");

		PutObjectRequest putObject = PutObjectRequest.builder() //
				.bucket(bucketName) //
				.key("statics/images/sample2.png") //
				.metadata(userMetaData) //
				.build();

		File localFile = new File("/Volumes/New/Test/pngFile.png");

		// This example uses RequestBody.fromFile
		// to avoid loading the whole file into memory.
		s3Client.putObject(putObject, RequestBody.fromFile(localFile));

		System.out.println("Upload Done!\n");

		showMetadatas(s3Client);
	}

	private static void showMetadatas(S3Client s3Client) {
		HeadObjectRequest headObjectRequest = HeadObjectRequest.builder() //
				.bucket(bucketName).key("statics/images/sample2.png")//
				.build();

		HeadObjectResponse headObjectResponse = s3Client.headObject(headObjectRequest); 
		System.out.println("Content Type: " + headObjectResponse.contentType()); 
		Map<String, String> userDefineMetadatas = headObjectResponse.metadata();

		userDefineMetadatas.forEach((k, v) -> {
			System.out.println(k + " : " + v);
		});
	}
}

Output:

Upload Done!

Content Type: image/png
myval : A Test Value

Results seen on S3 Bucket:

5. Object Tags

"TAGS" are additional information attached to an object, each TAG is a pair of values and each object is allowed to have a maximum of 10 such value pairs.

What are object tags in Amazon S3?

S3ClientUploadTagsExample.java

package org.o7planning.java_14201_awssdk_s3;

import java.io.File;
import java.util.ArrayList;
import java.util.List;

import org.o7planning.awssdks3.utils.MyUtils;

import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectTaggingRequest;
import software.amazon.awssdk.services.s3.model.GetObjectTaggingResponse;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.Tag;
import software.amazon.awssdk.services.s3.model.Tagging;

public class S3ClientUploadTagsExample {
	private static final Region myRegion = Region.EU_CENTRAL_1;
	private static final String bucketName = "test1.o7planning.org";

	public static void main(String[] args) {
		S3Client s3Client = MyUtils.createS3Client(myRegion);

		Tag authorTag = Tag.builder() //
				.key("author").value("Smith").build();

		Tag languageTag = Tag.builder() //
				.key("language").value("English").build();

		List<Tag> tags = new ArrayList<>();
		tags.add(authorTag);
		tags.add(languageTag);

		Tagging tagging = Tagging.builder().tagSet(tags).build();

		PutObjectRequest putObject = PutObjectRequest.builder() //
				.bucket(bucketName) //
				.key("java/java-tutorial.pdf") //
				.tagging(tagging) //
				.build();

		File localFile = new File("/Volumes/New/Test/Java-Tutorial.pdf");

		s3Client.putObject(putObject, RequestBody.fromFile(localFile));
		System.out.println("Upload Done!\n");

		showTags(s3Client);
	}

	private static void showTags(S3Client s3Client) {
		GetObjectTaggingRequest request = GetObjectTaggingRequest.builder() //
				.bucket(bucketName) //
				.key("java/java-tutorial.pdf") //
				.build();

		GetObjectTaggingResponse response = s3Client.getObjectTagging(request);
		List<Tag> tags = response.tagSet();

		tags.forEach(tag -> {
			System.out.println(tag.key() + " : " + tag.value());
		});
	}
}

Output:

Upload Done!

author : Smith
language : English

Results seen on S3 Bucket:

6. Redirect URL

If the Static Website feature is enabled in your Bucket, objects can be accessed via a URL. You can also set up an object's URL to redirect to another URL.

URL: https://yourdomain.com/article/java-tutorial-old
---> https://yourdomain.com/article/java-tutorial-new

S3ClientUploadRedirectExample1.java

package org.o7planning.java_14201_awssdk_s3;

import org.o7planning.awssdks3.utils.MyUtils;

import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;

public class S3ClientUploadRedirectExample1 {
	// EU (Frankfurt) - Germany.
	private static final Region myRegion = Region.EU_CENTRAL_1;
	private static final String bucketName = "test1.o7planning.org";

	public static void main(String[] args) {
		S3Client s3Client = MyUtils.createS3Client(myRegion);

		PutObjectRequest putObject = PutObjectRequest.builder() //
				.bucket(bucketName) //
				.key("article/java-tutorial-old") //
				.websiteRedirectLocation("/article/java-tutorial-new") //
				.build();

		// Upload Empty Data
		s3Client.putObject(putObject, RequestBody.empty());

		System.out.println("Done!");
	}
}

After running the above example, you can see the results on the S3 Bucket:

Object Key: article/java-tutorial-old
Metadata Key: x-amz-website-redirect-location
Metadata Value: /article/java-tutorial-new

The object URL can also redirect to an external URL (Of another website).

PutObjectRequest putObject = PutObjectRequest.builder() //
		.bucket(bucketName) //
		.key("go/digitalocean/promotion") //
		.websiteRedirectLocation("https://m.do.co/c/9fdc6836d334") //
		.build();

7. S3Waiter Example

If you are uploading an object on Thread-1, and Thread-2 is waiting for this object to appear on the Bucket. Let's use S3Waiter in Thread-2.

S3WaiterExample1.java

package org.o7planning.java_14201_awssdk_s3;

import java.io.File;

import org.o7planning.awssdks3.utils.MyUtils;

import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.core.waiters.WaiterResponse;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.HeadObjectRequest;
import software.amazon.awssdk.services.s3.model.HeadObjectResponse;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.waiters.S3Waiter;

public class S3WaiterExample1 {
	// EU (Frankfurt) - Germany.
	private static final Region myRegion = Region.EU_CENTRAL_1;
	private static final String bucketName = "test1.o7planning.org";

	private static String objectKey = "statics/videos/sample-15s.mp4";

	public static void main(String[] args) {
		MyUploadThread uploadThread = new MyUploadThread();
		uploadThread.start();

		SubscriberThread subscriberThread = new SubscriberThread();
		subscriberThread.start();
	}

	static class MyUploadThread extends Thread {
		@Override
		public void run() {
			S3Client s3Client = MyUtils.createS3Client(myRegion);

			PutObjectRequest putObject = PutObjectRequest.builder() //
					.bucket(bucketName) //
					.key(objectKey) //
					.build();
			System.out.println("Upload a Video 11.7MB (MyUploadThread)");

			// A Video 11.7MB
			File localFile = new File("/Volumes/New/Test/sample-15s.mp4");

			// This example uses RequestBody.fromFile
			// to avoid loading the whole file into memory.
			s3Client.putObject(putObject, RequestBody.fromFile(localFile));

			System.out.println("Upload Done! (MyUploadThread)");
		}
	}

	static class SubscriberThread extends Thread {
		@Override
		public void run() {
			S3Client s3Client = MyUtils.createS3Client(myRegion);

			System.out.println("Wait until video exists! (SubscriberThread)");
			S3Waiter waiter = s3Client.waiter();
			HeadObjectRequest requestWait = HeadObjectRequest.builder() //
					.bucket(bucketName) //
					.key(objectKey) //
					.build();

			WaiterResponse<HeadObjectResponse> waiterResponse = waiter.waitUntilObjectExists(requestWait);

			waiterResponse.matched().response().ifPresent(System.out::println);
		}
	}
}

Output:

Wait until video exists! (SubscriberThread)
Upload a Video 11.7MB (MyUploadThread)
Upload Done! (MyUploadThread)
HeadObjectResponse(AcceptRanges=bytes, LastModified=2023-11-11T08:08:16Z, ContentLength=11916526, ETag="aa355e698dca652d60c0bd39931714a4", ContentType=video/mp4, ServerSideEncryption=AES256, Metadata={})

Java Awssdk S3 S3Waiter

8. Object Retention

"Object Retention" is an object that is temporarily protected for a set period of time. During the time it is protected you cannot modify or delete it. In other words, "Object Retention" is a "Write-Once-Read-Many (WORM)" object.

S3Client allows uploading objects with "Retention" feature, provided S3 Bucket has "Object Lock" mode enabled. If you are interested, you can see more details in the article below:

Java Awssdk S3 Object Retention

Java Awssdk S3 S3Client Upload object

1. Library

2. Upload file to S3Bucket with S3Client

3. Upload Stream, String, byte[]

4. Object Metadata

5. Object Tags

6. Redirect URL

7. S3Waiter Example

8. Object Retention

Amazon Web Services Tutorials

Amazon Web Services Tutorials

Newest Articles