Skip to main content

curl command to send file to aws s3 bucket

 

Hello Techie's,

Do you want to send file to AWS S3 Bucket using curl command ?

Well, you are in the right place..!

‐‐---------------------------------------------------------------------

bucket= your Bucket name
files_location=/home/test.tar.gz
now_time=$(date +"%H%M%S")
contentType="application/x-compressed-tar"
dateValue=`date -R`
# your key goes here..
S3Key=
# your secrets goes here..
s3Secret=
function pushToS3()
{
  files_path=$1
  for file in $files_path
  do
    fname=$(basename $file)
    logInfo "Start sending $fname to S3"
    resource="/${bucket}/${now_date}/${fname}_${now_time}"
    stringToSign="PUT\n\n${contentType}\n${dateValue}\n${resource}"
    signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`
    curl -X PUT -T "${file}" \
     -H "Host: ${bucket}.s3.amazonaws.com" \
     -H "Date: ${dateValue}" \
     -H "Content-Type: ${contentType}" \
     -H "Authorization: AWS ${s3Key}:${signature}" \
      https://${bucket}.s3.amazonaws.com/${now_date}/${fname}_${now_time}
     logInfo "$fname has been sent to S3 successfully."
  done
}
pushToS3 $files_location
‐----------------------------------------------------------------------
Labels:

Comments

Post a Comment

Popular posts from this blog

How to create a job in autosys using sample jil file

  Hello Techie's I know you are new to Autosys and want to know How to Create Job in Autosys using a jil file, Don't worry we are here with wonderful example and sample jil file. Before going into the sample jil, you need some script to run via Autosys. you can use the sample powershell script written in our blog  How to call stored procedure using powershell scripting using try and catch (vichietechie.blogspot.com)   or if you have your own script, you can use that as well. Problem Statement: 1. you need to Run particular script on sheduled time. 2. you need to know whether the script executed without errors. 2. If executed with errors the sheduled job should send notification. Prerequisties: 1. you must have CA Workload Automation Tool installed in your machine. 2. you must know the hostname & Uid of that machine. 3. you should have script to run in autosys. what is the difference between box and job in autosys ? A box is used to organize and control process flow of...
Post a Comment
Read more

Curl command to check if file exists in s3 bucket

 Hello Techie's,  we can use below Shell script to check if particular file present in s3 bucket. # -------------------------------------------------------------------- #s3 Bucket Credentials #-------------------------------------------------------------------- s3_access_key=XXXXXXXXXXX s3_secret_key=XXXXXXXXXXX host=s3_bucket_api_url bucket=s3_bucket_name folder_name=name_of_the folder_in_bucket file_name=name_of_the_file_to_check #-------------------------------------------------------------------- #curl command variables #-------------------------------------------------------------------- dateValue=`date -R` contentType="application/xml" filepath="/${bucket}/${folder_name}/${file_name} signature_string="GET\n\${contentType}\n${dateValue}\n${filepath}" signature_hash=`echo -en ${signature_string} | openssl sha1 -hmac ${s3_secret_key} -binary | base64` #-------------------------------------------------------------------- #Curl command to check if file exi...
3 comments
Read more

How to link image pull secret to service account in openshift

Hello Techie's, If your services are running in ECS - Openshift container platform and you need to edit the credentials of the Image pull secret, please follow the below steps. What is Image Pull Secret ? It is used to pull an image from a private container image registry or repository to the deployment. How does the yaml file of Image Pull secret looks ? apiVersion: v1 kind: Secret metadata:   ...   name: secret_name   ... data:   .dockerconfigjson: eyJodHRwczovL2luZGV4L ... J0QUl6RTIifX0= type: kubernetes.io/dockerconfigjson Problem Statement: Let's say, you are updating the existing Image Pull Secret values -  for example, changing docker credentials of your repository or updating the uid password from 8 character password to 15 character.  As you make changes in exising secret, Deployments or build which you trigger will obviously fail.  To overcome that you need to link the updated Image Pull secret to the service account which are using that secr...
Post a Comment
Read more