Skip to main content

Curl command to check if file exists in s3 bucket

 Hello Techie's,

 we can use below Shell script to check if particular file present in s3 bucket.


#--------------------------------------------------------------------

#s3 Bucket Credentials

#--------------------------------------------------------------------

s3_access_key=XXXXXXXXXXX

s3_secret_key=XXXXXXXXXXX

host=s3_bucket_api_url

bucket=s3_bucket_name

folder_name=name_of_the folder_in_bucket

file_name=name_of_the_file_to_check

#--------------------------------------------------------------------

#curl command variables

#--------------------------------------------------------------------

dateValue=`date -R`

contentType="application/xml"

filepath="/${bucket}/${folder_name}/${file_name}

signature_string="GET\n\${contentType}\n${dateValue}\n${filepath}"

signature_hash=`echo -en ${signature_string} | openssl sha1 -hmac ${s3_secret_key} -binary | base64`

#--------------------------------------------------------------------

#Curl command to check if file exists

#--------------------------------------------------------------------

FILE=$((($(curl -X GET \

-H "Host: ${host}" \

-H "Date: ${dateValue}" \

-H "Content-Type: ${contentType}" \

-H "Authorization: Aws ${s3_access_key}:${signature_hash}" \

--silent -I https://${host}/${bucket}/${folder_name}/${file_name} \

| grep -E "^HTTP" \

| awk -F " " '{print $2}') == 200)) \

&& echo "file exists")

#--------------------------------------------------------------------

#Check If File exists

#--------------------------------------------------------------------

if [ "${FILE}" = "file exists" ]; then

echo "file is already present in s3 bucket folder"

else

echo "file is not present in s3 bucket folder"

fi

#-------------------------------------------------------------------


Comments

Post a Comment

Popular posts from this blog

How to create a job in autosys using sample jil file

  Hello Techie's I know you are new to Autosys and want to know How to Create Job in Autosys using a jil file, Don't worry we are here with wonderful example and sample jil file. Before going into the sample jil, you need some script to run via Autosys. you can use the sample powershell script written in our blog  How to call stored procedure using powershell scripting using try and catch (vichietechie.blogspot.com)   or if you have your own script, you can use that as well. Problem Statement: 1. you need to Run particular script on sheduled time. 2. you need to know whether the script executed without errors. 2. If executed with errors the sheduled job should send notification. Prerequisties: 1. you must have CA Workload Automation Tool installed in your machine. 2. you must know the hostname & Uid of that machine. 3. you should have script to run in autosys. what is the difference between box and job in autosys ? A box is used to organize and control process flow of...

How to link image pull secret to service account in openshift

Hello Techie's, If your services are running in ECS - Openshift container platform and you need to edit the credentials of the Image pull secret, please follow the below steps. What is Image Pull Secret ? It is used to pull an image from a private container image registry or repository to the deployment. How does the yaml file of Image Pull secret looks ? apiVersion: v1 kind: Secret metadata:   ...   name: secret_name   ... data:   .dockerconfigjson: eyJodHRwczovL2luZGV4L ... J0QUl6RTIifX0= type: kubernetes.io/dockerconfigjson Problem Statement: Let's say, you are updating the existing Image Pull Secret values -  for example, changing docker credentials of your repository or updating the uid password from 8 character password to 15 character.  As you make changes in exising secret, Deployments or build which you trigger will obviously fail.  To overcome that you need to link the updated Image Pull secret to the service account which are using that secr...