AWS | the broadcast

Today i wanted to stop using hard-coded S3-Credentials in my scripts running on EC2-Instances.

Create an IAM-Role

Services
IAM
Roles
[Create Role]

Select type of trusted entity

AWS-Service
EC2
[Next: Permissions]

Attach permissions policies

AmazonS3ReadOnlyAccess

Review

Role name: READ_S3

Attach Role to EC2-Instance

Services
EC2
Instances
Instance Settings

attach IAM-Role

READ_S3

Prepare Linux Setup
Within the EC2-Instance to which the IAM-Role has been attached.

ubuntu@ip-10-2-0-193:~$ sudo apt-get install -y python-pip
ubuntu@ip-10-2-0-193:~$ sudo apt install -y awscli
ubuntu@ip-10-2-0-193:~$ pip install boto3
ubuntu@ip-10-2-0-193:~$ pip install requests

Check Access to „meta-data“ => „IAM“ => „Security-Credentials“
The Script has to know the name of the role to use.

ubuntu@ip-10-2-0-193:~$ curl http://169.254.169.254/latest/meta-data/iam/security-credentials/READ_S3
{
  "Code" : "Success",
  "LastUpdated" : "2018-02-27T20:16:38Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ASIAI5EHDXGCAQBR7AA",
  "SecretAccessKey" : "6qWNxrTDU2FTynEkkJHl4pADQ4/xdQqgr89tF8x",
  "Token" : "FQoDYXdzEMX//////////wEaDBVs9pF5ec5XdDPiFCK3A5zquC32puTTqR9jV2BiViEBtagFXJ13++wq+lSUoCxL7sZwSCwa9njCWppjv5ShoKdqfkAWWivecZ18Px8SLoO7T83pZCeyri+WKKVdzNrt5tvrftxxlRVMvj+/Hy683KJi4GeheOEAC0XuiNwFC0IMM5xwJatdzXlann6I/A1zLSC5p/iydHPug9YvgEn3+cwTa2CCBNjzk4Sh0NsGkdhVggjZNcYe/jWSxdcCJJNc5cSTz76z2G+vFppbWnL0Rz4iKGRVXXs81upazvtbheffVWApUiL0ULhS2jrxEzFioPNX73AGlwaNM9jD0Zk5N2BSQuQE9yxyKcetkcCZQ85B4pkmxgldPTStQng+zeYv/rAZjyURFHGHTPhOLaHnnZyn89EfjThZ8rYhq1RJpu8Gqx9Z58Jclyj0YiHQe6IfF77N8PBLc0lGu7iT3yc6cp3n2kdsmuK3N3+WmRQI0G+OMJvMSx6eH31IVgwF+nEwkl0epW13D+3RQEdaCwm9b5wV9UFBkXtI4XPgF08gS2qwiyRO25ztuWD1USq9ZchrmJIOMHO4BX1XpOmf0M9drpbmIYbP4EoovnW1AU=",
  "Expiration" : "2018-02-28T02:48:10Z"

Fetch the IAM-Role-Credentials using python.requests()
Attention: recreated the Role in the meantime, so the credentials have been modified.

ubuntu@ip-10-2-0-193:~$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import requests
>>> import boto3

>>> r = requests.get('http://169.254.169.254/latest/meta-data/iam/security-credentials/READ_S3')
>>> cred = r.json()
>>> print cred
{u'Code': u'Success', u'LastUpdated': u'2018-02-27T20:33:14Z', u'AccessKeyId': u'ASIAIRKJFTEBQLSFR6A', u'SecretAccessKey': u'USXXJb1zoCa13FBRrUR5AWwaH0ymo+af8DmW7eQ', u'Token': u'FQoDYXdzEMb//////////wEaDGxRlk0fbmRHnfL8mSK3A0tyJcQ2JgXY+fEnjFioduAnU/FRhth+DhLHzJWnwSw0/isO/tP72wFkneklld1JtAkRkRxH9SjRwqWKpybtW6cEITEksP976Ym7sjxoT+NglC4KNk9DfaRgvJgB8EqDqgG1oZHoJ91O+US+vxPlLAifh7Fvepuw+2nmwPDpOIPz7H6Q2pAr/i7AEncyMGCwZejVhnY5lk8+aDmYMqS7ymfe1n/DyjwEfiYH4kFJplOtubXMsc+rlTg2KtMSovMRX4h+SrdH6bpVpT4DeIvivyH5ABFL66YMyW76P8GGdaGtb8ohxQZ3zeR1gGYs5WEBcziZVEGJDEopKLSUgqgkC1dUFLk0XLfB8vaEGjNsaaT3gaRm2AWAEVonLFyUr+uWvdxptOmFPIMUwUlNitXJkqG44fQEuOw5oDHANJ7zZtPYrWQUhmJzHubJZbvJ313GlPfUQzs7bkDBxoBXIgKIPMpK09uxh48iEByLdqoHyAtkL4qWddQVywuEgEH7Qw/X1/20ngaXqVVSjEa3hu/98EvxKOoushMpeLiG1JyxZJR5nQGH/YgxC17t2ipzsrWPepPkZfz5aAo3IDX1AU=', u'Expiration': u'2018-02-28T02:50:12Z', u'Type': u'AWS-HMAC'}
>>>

Use the credentials to create a Session-Object

>>> session = boto3.Session(
...     aws_access_key_id=cred["AccessKeyId"],
...     aws_secret_access_key=cred["SecretAccessKey"],
...     aws_session_token=cred["Token"],
... )
>>>

Read all S3-Buckets, print out their attributes („Name“)

>>> s3 = session.resource('s3', region_name='us-east-1')
>>>
>>> for b in s3.buckets.all():
...   print b
...
s3.Bucket(name='allones')
>>>

Is this all?

Only one Bucket?

Yes – and it’s located in Frankfurt.

Further reading:
Python API for Amazon S3: Examples

Kategorie: AWS

Amazon AWS – Use IAM-role to authenticate python-script on EC2-Instance to S3-API