tl;dr: Increase your functions memory to at least 1024MB, see update 2
I was curious, so I did some measurements. I created a script that creates a big boi item with pretty much exactly 400KB in size in a fresh table.
Then I test two reads from Python - one with the resource API and the other with the lower level client - eventually consistent reads in both cases.
Here's what I measured:
Reading Big Boi from a Table Resource took 0.366508s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.301585s and consumed 50.0 RCUs
If we extrapolate from the RCUs, the item it read was about 50 * 2 * 4KB = 400 KB
in size (eventually consistent reads consume 0.5 RCUs).
I ran it a few times locally from Germany against eu-central-1
(Frankfurt, Germany) and the highest latency I saw was about 900ms. (This is without DAX.)
As a result of that I think you should show us how you did your measurements.
import uuid
from datetime import datetime, timedelta
import boto3
import boto3.dynamodb.conditions as conditions
TABLE_NAME = "big-boi-test"
BIG_BOI_PK = "f0ba8d6c"
TABLE_RESOURCE = boto3.resource("dynamodb").Table(TABLE_NAME)
DDB_CLIENT = boto3.client("dynamodb")
def create_table():
DDB_CLIENT.create_table(
AttributeDefinitions=[{"AttributeName": "PK", "AttributeType": "S"}],
TableName=TABLE_NAME,
KeySchema=[{"AttributeName": "PK", "KeyType": "HASH"}],
BillingMode="PAY_PER_REQUEST"
)
def create_big_boi_item() -> str:
# based on calculations here: https://zaccharles.github.io/dynamodb-calculator/
template = {
"PK": {
"S": BIG_BOI_PK
},
"bigBoi": {
"S": ""
}
} # This is 16 bytes
big_boi = "X" * (1024 * 400 - 16)
template["bigBoi"]["S"] = big_boi
return template
def store_big_boi():
big_bio = create_big_boi_item()
DDB_CLIENT.put_item(
Item=big_bio,
TableName=TABLE_NAME
)
def get_big_boi_with_table_resource():
start = datetime.now()
response = TABLE_RESOURCE.get_item(
Key={"PK": BIG_BOI_PK},
ReturnConsumedCapacity="TOTAL"
)
end = datetime.now()
seconds = (end - start).total_seconds()
capacity_units = response["ConsumedCapacity"]["CapacityUnits"]
print(f"Reading Big Boi from a Table Resource took {seconds}s and consumed {capacity_units} RCUs")
def get_big_boi_with_client():
start = datetime.now()
response = DDB_CLIENT.get_item(
Key={"PK": {"S": BIG_BOI_PK}},
ReturnConsumedCapacity="TOTAL",
TableName=TABLE_NAME
)
end = datetime.now()
seconds = (end - start).total_seconds()
capacity_units = response["ConsumedCapacity"]["CapacityUnits"]
print(f"Reading Big Boi from a Client took {seconds}s and consumed {capacity_units} RCUs")
if __name__ == "__main__":
# create_table()
# store_big_boi()
get_big_boi_with_table_resource()
get_big_boi_with_client()
Update
I did the same measurements again with an item that looks more like the one you're using, I'm still below 1000ms on average no matter which way I request them:
Reading Big Boi from a Table Resource took 1.492829s and consumed 50.0 RCUs
Reading Big Boi from a Table Resource took 0.871583s and consumed 50.0 RCUs
Reading Big Boi from a Table Resource took 0.857513s and consumed 50.0 RCUs
Reading Big Boi from a Table Resource took 0.769432s and consumed 50.0 RCUs
Reading Big Boi from a Table Resource took 0.690172s and consumed 50.0 RCUs
Reading Big Boi from a Table Resource took 0.670099s and consumed 50.0 RCUs
Reading Big Boi from a Table Resource took 0.633489s and consumed 50.0 RCUs
Reading Big Boi from a Table Resource took 0.605999s and consumed 50.0 RCUs
Reading Big Boi from a Table Resource took 0.598635s and consumed 50.0 RCUs
Reading Big Boi from a Table Resource took 0.606553s and consumed 50.0 RCUs
Reading Big Boi from a Client took 1.66636s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.921605s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.831735s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.707082s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.668602s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.648401s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.5695s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.592073s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.611436s and consumed 50.0 RCUs
Reading Big Boi from a Client took 0.553827s and consumed 50.0 RCUs
Average latency over 10 requests with the table resource: 0.7796304s
Average latency over 10 requests with the client: 0.7770621s
This is what the item looks like:

Here is the full test-script for you to verify:
import statistics
import uuid
from datetime import datetime, timedelta
import boto3
import boto3.dynamodb.conditions as conditions
TABLE_NAME = "big-boi-test"
BIG_BOI_PK = "NestedBoi"
TABLE_RESOURCE = boto3.resource("dynamodb").Table(TABLE_NAME)
DDB_CLIENT = boto3.client("dynamodb")
def create_table():
DDB_CLIENT.create_table(
AttributeDefinitions=[{"AttributeName": "PK", "AttributeType": "S"}],
TableName=TABLE_NAME,
KeySchema=[{"AttributeName": "PK", "KeyType": "HASH"}],
BillingMode="PAY_PER_REQUEST"
)
def create_big_boi_item() -> str:
# based on calculations here: https://zaccharles.github.io/dynamodb-calculator/
template = {
"PK": {
"S": "NestedBoi"
},
"bigBoiContainer": {
"M": {
"bigBoiList": {
"L": [
]
}
}
}
} # 43 bytes
item = {
"M": {
"t": {
"S": "1614712316"
},
"a": {
"S": "product_view"
},
"i": {
"S": "1275"
}
}
} # 36 bytes
number_of_items = int((1024 * 400 - 43) / 36)
for _ in range(number_of_items):
template["bigBoiContainer"]["M"]["bigBoiList"]["L"].append(item)
return template
def store_big_boi():
big_bio = create_big_boi_item()
DDB_CLIENT.put_item(
Item=big_bio,
TableName=TABLE_NAME
)
def get_big_boi_with_table_resource():
start = datetime.now()
response = TABLE_RESOURCE.get_item(
Key={"PK": BIG_BOI_PK},
ReturnConsumedCapacity="TOTAL"
)
end = datetime.now()
seconds = (end - start).total_seconds()
capacity_units = response["ConsumedCapacity"]["CapacityUnits"]
print(f"Reading Big Boi from a Table Resource took {seconds}s and consumed {capacity_units} RCUs")
return seconds
def get_big_boi_with_client():
start = datetime.now()
response = DDB_CLIENT.get_item(
Key={"PK": {"S": BIG_BOI_PK}},
ReturnConsumedCapacity="TOTAL",
TableName=TABLE_NAME
)
end = datetime.now()
seconds = (end - start).total_seconds()
capacity_units = response["ConsumedCapacity"]["CapacityUnits"]
print(f"Reading Big Boi from a Client took {seconds}s and consumed {capacity_units} RCUs")
return seconds
if __name__ == "__main__":
# create_table()
# store_big_boi()
n_experiments = 10
experiments_with_table_resource = [get_big_boi_with_table_resource() for i in range(n_experiments)]
experiments_with_client = [get_big_boi_with_client() for i in range(n_experiments)]
print(f"Average latency over {n_experiments} requests with the table resource: {statistics.mean(experiments_with_table_resource)}s")
print(f"Average latency over {n_experiments} requests with the client: {statistics.mean(experiments_with_client)}s")
If I increase n_experiments, it tends to get even faster, probably because DDB caches internally.
Still: can't reproduce.
Update 2
After learning you're running Lambda functions, I ran the tests again inside of Lambda with different memory configurations.
Memory |
n_experiments |
average time with resource |
average time with client |
128MB |
10 |
6.28s |
5.06s |
256MB |
10 |
3.26s |
2.61s |
512MB |
10 |
1.62s |
1.33s |
1024MB |
10 |
0.84s |
0.68s |
2048MB |
10 |
0.52s |
0.43s |
4096MB |
10 |
0.51s |
0.41s |
As mentioned in the comments, CPU and Network performance scale with the amount of Memory you assign to a function.
You can solve your problem by throwing money at it :-)