Copy Last 24 Hours Files From Windows Server to AWS S3
In this blog, we will copy last 24 hours files from local windows server to AWS S3.
Let's start to achieve this:
Note: For this script, we need to install AWS CLI on local Windows machine and we need configure IAM user credentials with S3 get and put object permission.
Step 1: Create IAM user and download the access key and secret key credentials for AWS CLI configuration.
Step 2: Create a policy with S3 put and get permission, which will help script to fetch files details and copy paste it locally.
1{
2 "Version":"2012-10-17",
3 "Statement":[
4 {
5 "Effect":"Allow",
6 "Action": "s3:ListAllMyBuckets",
7 "Resource":"arn:aws:s3:::*"
8 },
9 {
10 "Effect":"Allow",
11 "Action":["s3:ListBucket","s3:GetBucketLocation"],
12 "Resource":"arn:aws:s3:::awsBucketName"
13 },
14 {
15 "Effect":"Allow",
16 "Action":[
17 "s3:PutObject",
18 "s3:PutObjectAcl",
19 "s3:GetObject",
20 "s3:GetObjectAcl",
21 "s3:DeleteObject"
22 ],
23 "Resource":"arn:aws:s3:::awsBucketName/*"
24 }
25 ]
26}
Step 3: Attach policy with IAM user.
Step 4: Install AWS CLI on windows server and configure credentials using below command:
1aws configure --profile my-s3
Step 5: Copy the below code and paste it in the notepad.
1echo "Start at" > "E:\JsonSale\script-log.txt"
2date >> "E:\JsonSale\script-log.txt"
3$files = Get-ChildItem -Path "Y:\" -Filter "*.zip" | Where-Object { $_.CreationTime -gt (Get-Date).AddHours(-1) } |Select-Object -ExpandProperty Name
4foreach ($file in $files) {
5 echo $file >> "E:\JsonSale\script-log.txt"
6 $Command = & 'C:\Program Files\Amazon\AWSCLIV2\aws.exe' s3 cp "Y:\$file" "s3://source-bucket/Input/$file" --profile profile-s3
7 echo $Command >> "E:\JsonSale\script-log.txt"
8}
9echo "End at" >> "E:\JsonSale\script-log.txt"
10date >> "E:\JsonSale\script-log.txt"
Step 6: Save the file with ".ps1" extension.
**.ps1** extension is a Powershell file extension.
You can also configure the script in windows scheduler to automate this. Use this link to know more.