Read, Write and Append File Amazon AWS S3

Amazon Web Services Logo

The problem are we need to append text and contain in the file directly. As AWS S3 does not support these function yet, not like local System.IO append file, so i came out this solution. Im using these more for logging purpose so thats why i need to read, write and append file in Amazon AWS S3 directly so all my logging keep increase directly in AWS S3 bucket.

First step is read the file, if not exist yet, create the file, create content and save in AWS S3

if file already exist, read the file(not download), get the response, append with new content, and save it back to AWS S3. Which it will replace with the latest.

So, these are snippet how to append file in Amazon S3 Bucket. Hoping will help you and actually is for my future reference.

Create File
To create file, we use PutObjectRequest(). Here code snippet

var client = new AmazonS3Client("accessKeyId", "secretKey", "regionEndpoint");
S3FileInfo s3fileinfo = new Amazon.S3.IO.S3FileInfo(client, bucketName, fileName);

  PutObjectRequest request = new PutObjectRequest();
  request.BucketName ="your-bucket-name";
  request.Key= "your-file-name";
  request.ContentType = "text/plain";
  request.ContentBody = "hello world";
  client.PutObject(request);

Read File
To read file, we use GetObjectRequest. Here code snippet

GetObjectRequest xrequest = new GetObjectRequest
 {
 BucketName = "your-bucket-name",
 Key = "your-file-name"
 };

 GetObjectResponse response = client.GetObject(xrequest);
 Stream responseStream = response.ResponseStream;
 StreamReader reader = new StreamReader(responseStream);
  {
   responseBody = reader.ReadToEnd() + Environment.NewLine ;
   client.PutObject(request);
  }

Write and Append File
To write and append, we read the file, get the response and concat all string include new string. Here code snippet

request.ContentBody = string.Concat(responseBody, "new line of hello world");
client.PutObject(request);

Full Code Snippet

PutObjectRequest request = new PutObjectRequest();

request.BucketName = "your-bucket-name";
request.Key= "your-file-name";
request.ContentType = "text/plain";

var client = new AmazonS3Client("accessKeyId", "secretKey", "regionEndpoint");

S3FileInfo s3fileinfo = new Amazon.S3.IO.S3FileInfo(client, bucketName, fileName);
if (s3fileinfo.Exists) // checking if file exist
 {
 string responseBody = "";
  {
   GetObjectRequest xrequest = new GetObjectRequest
     {
       BucketName = bucketName,
       Key = fileName
     };

    GetObjectResponse response = client.GetObject(xrequest);
    Stream responseStream = response.ResponseStream;
    StreamReader reader = new StreamReader(responseStream);
     {
     responseBody = reader.ReadToEnd() + Environment.NewLine ;

     request.ContentBody = string.Concat(responseBody, "new line of hello world");
     client.PutObject(request); 
     }
  }
 }
 else
 {
 request.ContentBody = "new content";
 client.PutObject(request);
 }

I’m refer these two amazing url and tutorial during my coding for this so i able to coming out with the code. Hopefully this code snippet can help to solve problem in how to read, write or append file in AWS S3 directly

http://docs.ceph.com/

Using Amazon S3 with the AWS.NET API Part 6: S3 in Big Data II

There also some programming post that you might be interested