I am struggling to upload object on s3 bucket using curl and openssl(which I have to use as required). Please find below my script for upload.

#!/bin/bash -e

# Upload a file to AWS S3.
AWS_ACCESS_KEY_ID="abc"
AWS_SECRET_ACCESS_KEY="xyzxyz"
file="test.txt"
bucket="teamfelgen"
prefix="test"
region="us-east-1"
timestamp=$(date -u "+%Y-%m-%d %H:%M:%S")
signed_headers="date;host;x-amz-acl;x-amz-content-sha256;x-amz-date"

iso_timestamp=$(date -ud "${timestamp}" "+%Y%m%dT%H%M%SZ")
  date_scope=$(date -ud "${timestamp}" "+%Y%m%d")
  date_header=$(date -ud "${timestamp}" "+%a, %d %h %Y %T %Z")

payload_hash() {
  local output=$(shasum -ba 256 "$file")
  echo "${output%% *}"
}

canonical_request() {
  echo "PUT"
  echo "/${prefix}/${file}"
  echo ""
  echo "date:${date_header}"
  echo "host:s3.company-rook.com"
  echo "x-amz-acl:public-read"
  echo "x-amz-content-sha256:$(payload_hash)"
  echo "x-amz-date:${iso_timestamp}"
  echo ""
  echo "${signed_headers}"
  printf "$(payload_hash)"
}

canonical_request_hash() {
  local output=$(canonical_request | shasum -a 256)
  echo "${output%% *}"
}

string_to_sign() {
  echo "AWS4-HMAC-SHA256"
  echo "${iso_timestamp}"
  echo "${date_scope}/${region}/s3/aws4_request"
  printf "$(canonical_request_hash)"
}

signature_key() {
  local secret=$(printf "AWS4${AWS_SECRET_ACCESS_KEY?}" | hex_key)
  local date_key=$(printf ${date_scope} | hmac_sha256 "${secret}" | hex_key)
  local region_key=$(printf ${region} | hmac_sha256 "${date_key}" | hex_key)
  local service_key=$(printf "s3" | hmac_sha256 "${region_key}" | hex_key)
  printf "aws4_request" | hmac_sha256 "${service_key}" | hex_key
}

hex_key() {
  xxd -p -c 256
}

hmac_sha256() {
  local hexkey=$1
  openssl dgst -binary -sha256 -mac HMAC -macopt hexkey:${hexkey}
}

signature() {
  string_to_sign | hmac_sha256 $(signature_key) | hex_key | sed "s/^.* //"
}

curl -vk \
  -T "${file}" \
  -H "Authorization: AWS4-HMAC-SHA256 Credential=${AWS_ACCESS_KEY_ID?}/${date_scope}/${region}/s3/aws4_request,SignedHeaders=${signed_headers},Signature=$(signature)" \
  -H "Date: ${date_header}" \
  -H "x-amz-acl: public-read" \
  -H "x-amz-content-sha256: $(payload_hash)" \
  -H "x-amz-date: ${iso_timestamp}" \
  "https://s3.company-rook.com/${bucket}/${prefix}/${file}"

The problem here is to calculate signature for authorization. Even though if I calculate hash for example given on AWS v4 test suite that is incorrect.

Please find Below how I calculate hash. Copied canonical request from amazon example. stored in get-vanilla-query-order-key-case.creq file.

GET
/
Param1=value1&Param2=value2
host:example.amazonaws.com
x-amz-date:20150830T123600Z

host;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Now if I calculate hash, it is different from what amazon is calculating. PFB 2 approaches to calculate.

C:\project\script>openssl dgst -sha256 get-vanilla-query-order-key-case.creq
SHA256(get-vanilla-query-order-key-case.creq)= 9e487d40177520aed8763d24c77c7179f67622debbd2b8188d93138ba6748ade

$ shasum -a 256 get-vanilla-query-order-key-case.creq
9e487d40177520aed8763d24c77c7179f67622debbd2b8188d93138ba6748ade *get-vanilla-query-order-key-case.creq

Need your suggestions what is going wrong here.

Note : if I run awscli command in debug and use the signature this curl is working.

New contributor
Piyush Khetan is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

Piyush Khetan is a new contributor. Be nice, and check out our Code of Conduct.
 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.