awspydk

Async boto3 with Autogenerated JIT Data Classes

Motivation

This library is forked from an internal project that works with a lot of backend AWS APIs, and I got tired of having to constantly parse the returned responses before being able to work with them for (most of the time), a few seconds. Any API-driven application that uses boto3 tends to suffer from being syncronous only. So this library solves a few major problems:

Enables both sync and async from the same client.
Client can be called implicitly without needing initialization, i.e. you can directly use AwsClient without needing to initialize.
Dynamically generates class functions based on the boto3.client child functions, and allows you to call them. This is useful in repl or ipython environments where type-hints are always helpful, especially when the names are so long.
Automatically initializes the aws client if its not initialized from the defaults, simply by calling it.
Translates all results into Automatically Generated Dataclasses through lazycls.
- Can be disabled by setting aws.config.AutoCls = False

Quickstart

pip install --upgrade awspydk

from aws import AwsClient

# Sync Method
buckets = AwsClient.v1.s3_list_buckets(as_cls=True)

# Async Method
buckets = await AwsClient.v1.async_s3_list_buckets(as_cls=True)

"""
Both yield the same results.
The underlying classes are auto-generated from Pydantic BaseModels, so anything you can do with Pydantic Models, you can do with these.

{
    'Buckets': [
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 8, 25, 16, 42, 46, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 17, 54, 56, tzinfo=tzutc()), Name='...',
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 3, 4, 20, 10, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 1, 20, 50, 33, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 4, 2, 28, tzinfo=tzutc()), Name='...')
    ],
    'Owner': AwsS3Owner(DisplayName='...', ID='...')
}
"""

## Change Regions
AwsClient.reset(region='us-west-1')

## Change the defaut clients created
from aws.config import DefaultClients

## Modify to only create ec2 client
DefaultClients = {
    'ec2': 'ec2'
}

## Reset implicitly
AwsClient.reset()


BotoKwargs = {
    'AWS_PROFILE': ...,
}

## Reset Explicitly
AwsClient.reset(clients=DefaultClients, boto_kwargs=BotoKwargs)

Client Defaults

These are found in aws.config

AwsRegion = envToStr('AWS_REGION', 'us-east-1')
AutoCls = envToBool('AWSSDK_AUTOCLS', 'true')

## These are the default clients that will be autogenerated.
## Key is the shorthand, value is the actual AWS API Name in boto3
DefaultClients = {
    'ec2': 'ec2', 
    'ecr': 'ecr', 
    'r53' :'route53', 
    'acm': 'acm',
    'elb': 'elb',
    'elbv2': 'elbv2',
    'asg': 'autoscaling',
    's3': 's3'
}

## These are the default resources that will be autogenerated.
## Key is the shorthand, value is the actual AWS Resource API Name in boto3
DefaultResources = {
    'Ec2': 'ec2',
    'S3': 's3',
    'Iam': 'iam'
}

# These are the default filter args for querying
DefaultFilterArgs = {
    'string_only': True,
    'remove_null': True
}

Comments

async s3 sync

Hello, Not an issue exactly but I'm wondering if I can pick your brain on how I might use this (or what other options there are ) for fetching s3 bucket data in an async manner. My team is using aws s3 sync with --exclude to get sub sets of folders in buckets but it's quite a lot of data and I'm wondering if this lib could help in some way but I'm not sure how I would use it. Can you give me any guidance?

opened by colin-ryan-airm 1

A modern, easy to use, feature-rich, and async ready API wrapper improved and revived from original discord.py.

A Python API wrapper that is improved and revived from the original discord.py

19 Nov 6, 2021

Revolt.py - An async library to interact with the https://revolt.chat api.

Revolt.py An async library to interact with the https://revolt.chat api. This library will be focused on making bots and i will not implement anything