Accessing DynamoDB Local from boto3

Question

I am doing AWS tutorial Python and DynamoDB. I downloaded and installed DynamoDB Local. I got the access key and secret access key. I installed boto3 for python. The only step I have left is setting up authentication credentials. I do not have AWS CLI downloaded, so where should I include access key and secret key and also the region?

Do I include it in my python code?

Do I make a file in my directory where I put this info? Then should I write anything in my python code so it can find it?

Answer 1

You can try passing the accesskey and secretkey in your code like this:

import boto3
session = boto3.Session(
    aws_access_key_id=ACCESS_KEY,
    aws_secret_access_key=SECRET_KEY,
)
client = session.client('dynamodb')
OR
dynamodb = session.resource('dynamodb')

Answer 2

From the AWS documentation:

Before you can access DynamoDB programmatically or through the AWS Command Line Interface (AWS CLI), you must configure your credentials to enable authorization for your applications. Downloadable DynamoDB requires any credentials to work, as shown in the following example.

AWS Access Key ID: "fakeMyKeyId"

AWS Secret Access Key:"fakeSecretAccessKey"

You can use the aws configure command of the AWS CLI to set up credentials. For more information, see Using the AWS CLI.

So, you need to create an .aws folder in yr home directory. There create the credentials and config files. Here's how to do this:

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html

Answer 3

If you want to write portable code and keep in the spirit of developing 12-factor apps, consider using environment variables. The advantage is that locally, both the CLI and the boto3 python library in your code (and pretty much all the other offical AWS SDK languages, PHP, Go, etc.) are designed to look for these values.

An example using the official Docker image to quickly start DynamoDB local:

# Start a local DynamoDB instance on port 8000
docker run -p 8000:8000 amazon/dynamodb-local

Then in a terminal, set some defaults that the CLI and SDKs like boto3 are looking for. Note that these will be available until you close your terminal session.

# Region doesn't matter, CLI will complain if not provided
export AWS_DEFAULT_REGION=us-east-1

# Set some dummy credentials, dynamodb local doesn't care what these are
export AWS_ACCESS_KEY_ID=abc
export AWS_SECRET_ACCESS_KEY=abc

You should then be able to run the following (in the same terminal session) if you have the CLI installed. Note the --endpoint-url flag.

# Create a new table in DynamoDB Local
aws dynamodb create-table \
  --endpoint-url http://127.0.0.1:8000 \
  --table-name tmp \
  --attribute-definitions AttributeName=id,AttributeType=S \
  --key-schema AttributeName=id,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST

You should then able to list out the tables with:

aws dynamodb list-tables --endpoint-url http://127.0.0.1:8000

And get a result like:

{
  "TableNames": [
    "tmp"
  ]
}

So how do we get the endpoint-url that we've been specifying in the CLI to work in Python? Unfortunately, there isn't a default environment variable for the endpoint url in the boto3 codebase, so we'll need to pass it in when the code runs. The docs for .NET and Java are comprehensive but for Python, they are a bit more elusive. From the boto3 github repo and also see this great answer, we need to create a client or resource with the endpoint_url keyword. In the below, we're looking for a custom environment variable called AWS_DYNAMODB_ENDPOINT_URL. The point being that if specified, it will be used, otherwise will fall back to whatever the platform default is, making your code portable.

# Run in the same shell as before
export AWS_DYNAMODB_ENDPOINT_URL=http://127.0.0.1:8000

# file test.py
import os
import boto3

# Get environment variable if it's defined
# Make sure to set the environment variable before running
endpoint_url = os.environ.get('AWS_DYNAMODB_ENDPOINT_URL', None)
# Using (high level) resource, same keyword for boto3.client
resource = boto3.resource('dynamodb', endpoint_url=endpoint_url)
tables = resource.tables.all()
for table in tables:
    print(table)

Finally, run this snippet with

# Run in the same shell as before
python3 test.py

# Should produce the following output:
# dynamodb.Table(name='tmp')