Author:
erics , September 9th, 2021
The aws s3 sync command is slow and painful! I needed a more efficient way to sync to large buckets (prod to dev). Finally settled on RClone: https://rclone.org/docs/
cd { extracted zip dir }
. / rclone config
cat ~ / . config / rclone / rclone . conf
During rclone config I called remote “s3” ;-}
. / rclone sync - v -- progress -- fast - list -- checksum s3 : wyzaerd - demo - prod s3 : wyzaerd - demo - dev
Categories: How-To's , Technology Tags: AWS , Bucket , howto , rclone , rsync , S3 , s3s3 , s3s3mirror , Sync , tips
| No comments
Author:
erics , July 28th, 2021
SUMMARY: Needed to create an AWS IAM Policy to allow a user access to two buckets and their associated CDN’s in WordPress W3 Total Cache
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
{
"Version" : "2012-10-17" ,
"Statement" : [
{
"Sid" : "VisualEditor0" ,
"Effect" : "Allow" ,
"Action" : "s3:ListBucket" ,
"Resource" : "arn:aws:s3:::wyzaerd-demo-prod"
} ,
{
"Sid" : "VisualEditor1" ,
"Effect" : "Allow" ,
"Action" : [
"s3:PutObject" ,
"s3:GetObject" ,
"s3:DeleteObject" ,
"s3:PutObjectAcl"
] ,
"Resource" : "arn:aws:s3:::wyzaerd-demo-prod/*"
} ,
{
"Sid" : "VisualEditor2" ,
"Effect" : "Allow" ,
"Action" : "s3:ListBucket" ,
"Resource" : "arn:aws:s3:::wyzaerd-demo-dev"
} ,
{
"Sid" : "VisualEditor3" ,
"Effect" : "Allow" ,
"Action" : [
"s3:PutObject" ,
"s3:GetObject" ,
"s3:DeleteObject" ,
"s3:PutObjectAcl"
] ,
"Resource" : "arn:aws:s3:::wyzaerd-demo-dev/*"
} ,
{
"Sid" : "VisualEditor4" ,
"Effect" : "Allow" ,
"Action" : [
"s3:ListAllMyBuckets" ,
"cloudfront:ListDistributions"
] ,
"Resource" : "*"
}
]
}
Categories: How-To's , Technology Tags: Amazon , AWS , Bucket , CDN , CloudFront , howto , IAM , JSON , Policy , S3 , tips , User , W3 Total Cache
| No comments
Author:
erics , May 31st, 2012
1. You need Ruby, rubygems and the right_aws gem. yum install ruby rubygems gem update –system gem install right_aws 2. Replace any values inside curly braces {} below with real values and remove the curly braces. YMMV
require 'rubygems'
require 'right_aws'
S3ID = "{S3 ID GOES HERE}"
S3KEY = "{S3 KEY GOES HERE}"
SRCBUCKET = "{your-s3-bucket-1}"
DESTBUCKET = "{your-s3-bucket-2}"
s3 = RightAws :: S3Interface . new ( S3ID , S3KEY )
s3 . incrementally_list_bucket ( SRCBUCKET ) do | h |
h [ : contents ] . each do | o |
puts ( 'Copying ' + o [ : key ] )
s3 . copy ( SRCBUCKET , o [ : key ] , DESTBUCKET , o [ : key ] )
end
end
puts ( 'Done.' )
Categories: How-To's , Technology Tags: Amazon , AWS , Bucket , Copy , gem , right_aws , Ruby , rubygems , S3
| No comments