Web Development Agency | Digital Marketing
Websites media

How Websites media files synchronize to Amazon S3?

Here are the steps need to be followed to synchronize the media files between web development and Amazon S3.

Step: 1 Allocate separate space in AWS

First we need to allocate separate space in AWS for each user who is registering in to the website. Initially we should get AWS credentials like key, canonical id, canonical name, secret, username etc. from AWS account.

Separate space for each user in AWS can be achieved by bucket configuration. Bucket name should be unique. If same bucket is already associated with any one of account in AWS then it will throw an exception. Here Bucket name is formed in terms of AWS username and with unique registration ID of customer.

Allocate separate space in AWS - eGrove Systems

For Integration of AWS kit available for PHP we can download and extract in root folders. Kit name AWSSDKforPHP

After Registration we will create bucket with registered id in AWS.

Create bucket with registered id in AWS

This will create unique bucket in AWS. Ap-southeast-2 specifies the region in which buckets needs to be accessed. While creating buckets, if region is empty then it will create 400 error and shows invalidlocationconstraint error.

Unique Bucket Creation

CrossDomain.xml

Each bucket has separate crossdomain.xml this is used while uploading or downloads a file in AWS. Crossdomain is used for allowing access control.

<?xml version=”1.0″?>
<!DOCTYPE cross-domain-policy SYSTEM “https://www.macromedia.com/xml/dtds/cross-domain-policy.dtd”>
<cross-domain-policy>
<allow-access-from domain=”localhost” secure=”false” />
<allow-access-from domain=”*” secure=”false” />
<allow-http-request-headers-from domain=”*” headers=”*”/>
</cross-domain-policy>

Crossdomain can be accessed with its own bucket URL. Example https://xxxx-xxxxxx-xx.s3-ap-southeast-2.amazonAWS.com/crossdomain.xml

Bucketname: xxxx-xxxxxx-xx
Region: ap-southeast-2
Host: s3.amazon.com
Filename: crossdomain.xml

If we use this endpoint URL then files can be uploaded immediately after bucket creation. Suppose if we used only s3.amazon.com then for files upload it will be delayed for some 2 hours from the time of bucket creation.

Ex- https://xxxx-xxxxxx-xx.s3.amazonAWS.com/crossdomain.xml

Because by default AWS has redirection (307) for some time delay if we accessed host URL instead of region URL in file upload. So for better performance we should call the region endpoint URL.

Crossdomain files in s3 bucket

The above code will upload crossdomain files in s3 bucket.

Folders in AWS s3

In each bucket we have create separate project for particular user. Separate folders are created for each project with its project id. Project folders are unique.

Creating Project in each bucket

Creating Project in each bucket similar to crossdomain.xml

The above code will create project in each bucket with its project id. We can call these files similar to crossdomain.xml for example:

https://xxxx-xxxxxx-xx.s3-ap-southeast-2.amazonAWS.com/project-1/project_files/Publish Files Folder/d1.txt

Similarly we can save multiple folders in each bucket and can be downloaded with its unique access endpoint.

Login to AWS it will list out all the buckets available in your AWS account. Selects your particular bucket and click open and then to folders. In this image.

Allbuckets – List out all buckets

xxxx-xxxxxx-xx – Bucket Name

Project-59 – Project with its id ( To differentiate multiple projects is differentiated with its own project id)

In properties tab we can set a classes and other properties and click save.

See the Upload files in bucket

Configuration for File Upload

Configuration for File Upload

The above code will be used for file upload will send to AWS with generated signature for AWS, Policy, acl access, bucket, and request id, file path etc

$(“#uploader2”).plupload({

buttons: {

browse: true,
start: false,
stop: false

},

runtimes : ‘flash’,//html5,silverlight
url : ‘https://<?php echo $bucket; ?>.s3-ap-southeast-2.amazonAWS.com/’,
multipart: true,
multipart_params: {
“key”: “<?php echo addslashes($fullupload_path).$senduid ?>${filename}”,
‘Filename’: ‘${filename}’, //adding this to keep consistency across the runtimes
‘acl’: ‘public-read’,
‘Content-Type’: ”,
‘AWSAccessKeyId’ : ‘<?php echo $accessKeyId; ?>’,
‘policy’: ‘<?php echo $policy; ?>’,
‘signature’: ‘<?php echo $signature; ?>’,
‘x-amz-meta-uid’:'<?php echo $this->session->userdata(‘uid’); ?>’
},
file_data_name: ‘file’,
filters : {
// Maximum file size
max_file_size : ‘5000mb’,
// Specify what files to browse for
mime_types: [
{title : “upload files”, extensions : “*”}
] },
dragdrop: true,
autostart : true,
// Flash settings
flash_swf_url : ‘../../plupload/js/Moxie.swf’,
init:{

FileFiltered: function(up,file){
//console.log(file[‘name’]);
},
FileUploaded: function(up, file, info) {
//Allow Download Permisssion
var allow_download = $(“#”+file.id).find(“#allow_download_”+file.id).val();
var pid = $(“#filepid”).val();
var fileurl = $(“#fileurl”).val();
$(‘.loading-overlay’).addClass(“loading”);
$.ajax({
complete: function() {$(‘.loading-overlay’).removeClass(“loading”); }, //Hide spinner
url : ‘<?php echo base_url();?>project/fileuploadednotification’,
data:{allow_download:allow_download,filename:file[‘name’],filesize:file[‘size’],pid:pid,fileurl:fileurl},
type : “POST”,
dataType : ‘json’,
success : function(data) {

//alert(data.meta.code);
if(data.meta.code == “200”)
{

//alert(data.result);

}

},
beforeSend: function(XMLHttpRequest){}, //$.blockUI();
cache: false,
error : function(data) {
//alert(“error occur…”);
}

});}} });

The above code is used for file upload. Pulpupload is like a jQuery library and it is used to upload a file to AWS. It has special documentation for file uploading in AWS server.

The above code will check for crossdomain access in AWS bucket will make a request to url ( https://xxx-xxxxx-xx.s3-ap-southeast-2.amazonAWS.com/crossdomain.xml ) once it got success 200 request it will allow you to upload a file.

Else it shows http error and not allows uploading a file. Endpoint URL will be more specific and related to particular bucket like

https://<?php echo $bucket; ?>.s3-ap-southeast-2.amazonAWS.com/’,

Here ap-southeast-2 will be your region; you should change according to your region id. For region and its specific endpoint URL refer AWS documentation.

Here I will attach Request and Response for Bucket Creation and Crossdomain.xml and file upload.

Bucket Creation Request

[header] => Array

(

[x-amz-id-2] => 0zT0sOYzXwCvmCI5rxSu7h1+6qOAtwQht6D+pMukpO7cqoMkqCkfG35n7yXNet28Ci0Eu443JeQ=
[x-amz-request-id] => 7AC02DDB266E3290
[date] => Thu, 17 Mar 2016 04:02:28 GMT
[location] => https://xxx-xxxxx-xx.s3.amazonAWS.com/
[content-length] => 0
[server] => AmazonS3
[_info] => Array

(

[url] => https://xxx-xxxxx-xx.s3.amazonAWS.com/
[content_type] =>
[http_code] => 200
[header_size] => 273
[request_size] => 655
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 0.599251
[namelookup_time] => 0.082067
[connect_time] => 0.089523
[pretransfer_time] => 0.111582
[size_upload] => 198
[size_download] => 0
[speed_download] => 0
[speed_upload] => 330
[download_content_length] => 0
[upload_content_length] => 198
[starttransfer_time] => 0.599165
[redirect_time] => 0
[certinfo] => Array
(
)
[redirect_url] =>
[method] => PUT

)
[x-AWS-request-url] => https://xxx-xxxxx-xx.s3.amazonAWS.com/
[x-AWS-redirects] => 0
[x-AWS-stringtosign] => PUT

Ixgb0XZlX2l05jyO8O+KpA==
application/xml
Thu, 17 Mar 2016 04:02:26 GMT
x-amz-acl:private
/xxx-xxxxx-xx/

[x-AWS-requestheaders] => Array
(

[Content-MD5] => Ixgb0XZlX2l05jyO8O+KpA==
[Content-Type] => application/xml
[Date] => Thu, 17 Mar 2016 04:02:26 GMT
[x-amz-acl] => private
[Authorization] => AWS AKIAJYSPWK3F2LRHWWLQ:vreNHsuO/zTe5n1elvofBPvczKI=
[Expect] =>

)
[x-AWS-requestbody] => <?xml version=”1.0″ encoding=”UTF-8″?>
<CreateBucketConfiguration xmlns=”https://s3.amazonAWS.com/doc/2006-03-01/”><LocationConstraint>ap-southeast-2</LocationConstraint></CreateBucketConfiguration>

)
[body] =>
[status] => 200

)

CrossDomain.xml Request

Array
(

[header] => Array
(

[x-amz-id-2] => Gsr9d1MFGenGZJA7nMjd089bhg44WNmwmcO3DAVBHG8mNvV1tX0OEkUMFhLkzGaqNpV3QcTtBcQ=
[x-amz-request-id] => 28278A5FEAEB0745
[date] => Thu, 17 Mar 2016 04:02:29 GMT
[etag] => “f488f4dc371f9b58dd002da16a1b1c42”
[content-length] => 0
[server] => AmazonS3
[_info] => Array
(

[url] => https://xxx-xxxxx-xx.s3-ap-southeast-2.amazonAWS.com/crossdomain.xml
[content_type] => application/xml
[http_code] => 200
[header_size] => 665
[request_size] => 1011
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 1
[total_time] => 1.289578
[namelookup_time] => 0.114481
[connect_time] => 0.331342
[pretransfer_time] => 0.771969
[size_upload] => 327
[size_download] => 0
[speed_download] => 0
[speed_upload] => 253
[download_content_length] => 0
[upload_content_length] => 327
[starttransfer_time] => 1.010364
[redirect_time] => 0.038524
[certinfo] => Array
(
)
[redirect_url] =>
[method] => PUT

)
[x-AWS-request-url] => https://xxx-xxxxx-xx.s3.amazonAWS.com/crossdomain.xml
[x-AWS-redirects] => 0
[x-AWS-stringtosign] => PUT

text/xml
Thu, 17 Mar 2016 04:02:27 GMT
x-amz-acl:public-read
x-amz-storage-class:STANDARD
/xxx-xxxxx-xx/crossdomain.xml

[x-AWS-requestheaders] => Array
(

[Content-Type] => text/xml
[Date] => Thu, 17 Mar 2016 04:02:27 GMT
[Expect] => 100-continue
[x-amz-acl] => public-read
[x-amz-storage-class] => STANDARD
[Authorization] => AWS AKIAJYSPWK3F2LRHWWLQ:0njsfWoJaN0hOpg7DuK0HCLP8lg=

)

)
[body] =>
[status] => 200

)

File Upload

File Upload

Bucket Deletion:

Similarly, we can delete the bucket programmatically in AWS by calling delete_bucket(bucketname).  Once bucket is deleted we can again create bucket with same name; in same regions without any exceptions.

Yassar Saleh