Python Aws-cdk-pipelines-datalake-infrastructure Resources

Quiet Riot 🎶 C'mon, Feel The Noise 🎶 An enumeration tool for scalable, unauthenticated validation of AWS principals; including AWS Acccount IDs, roo

ZenML is an extensible, open-source MLOps framework to create production-ready machine learning pipelines. It has a simple, flexible syntax, is cloud and tool agnostic, and has interfaces/abstractions that are catered towards ML workflows.

logdata-anomaly-miner This tool parses log data and allows to define analysis pipelines for anomaly detection. It was designed to run the analysis wit

raspberry-solar-mon Home solar infrastructure (with Peimar Inverter) monitoring based on Raspberry Pi 3 B+ using Grafana, InfluxDB, Custom Python Coll

Webhook Listener A Python Lambda Webhook Listener - Generates a permanent URL on created assets. See doc on Generating a Permanent URL: https://www.co

Streamz helps you build pipelines to manage continuous streams of data. It is simple to use in simple cases, but also supports complex pipelines that involve branching, joining, flow control, feedback, back pressure, and so on.

Sceptre About Sceptre is a tool to drive AWS CloudFormation. It automates the mundane, repetitive and error-prone tasks, enabling you to concentrate o

CTF infrastructure deployment automation tool. Focus on the challenges. Mirrored from

Amazon Personalize Filter Rotation This project contains the source code and supporting files for deploying a serverless application that provides aut

SDK: Overview of the Kubeflow pipelines service Kubeflow is a machine learning (ML) toolkit that is dedicated to making deployments of ML workflows on

🚀 Welcome to AWS Copilot Workshop In this workshop, you'll learn how to build, release and operate your containerised applications to Amazon ECS and

s3-credentials A tool for creating credentials for accessing S3 buckets For project background, see s3-credentials: a tool for creating credentials fo

ETL Pipeline for AWS Project Description In this project, ETL pipeline is build on data warehouse hosted on AWS Redshift. The data is loaded from S3 t

AWS Lambda Fast API Fast API starter application compatible with API Gateway and Lambda Function. How to deploy it? Terraform AWS Lambda API is a reus

Python + AWS Lambda Hands On Python Criada em 1990, por Guido Van Rossum. "Bala de prata" (quase). Muito utilizado em: Automatizações - Selenium, Beau

overview img-proof (IPA) provides a command line utility to test images in the Public Cloud (AWS, Azure, GCE, etc.). With img-proof you can now test c

Introduction AthenaCLI is a command line interface (CLI) for the Athena service that can do auto-completion and syntax highlighting, and is a proud me

Cloudkeeper Housekeeping for Clouds! Table of contents Overview Docker based quick start Cloning this repository Component list Contact License Overvi

Jetson Multicamera Pipelines Easy-to-use realtime CV/AI pipelines for Nvidia Jetson Platform. This project: Builds a typical multi-camera pipeline, i.

Quiet Riot 🎶 C'mon, Feel The Noise 🎶 An enumeration tool for scalable, unauthenticated validation of AWS principals; including AWS Acccount IDs, roo

ec2-bastion-sshconfig This script will interate over instances found by ec2.py and if those instances are not publically accessible it will search the

AWS-Transitgateway-Route-Propagation This Lambda will Pull propagated routes from TGW and update VPC route table. Tested on python 3.8 Lambda AWS INST

chalice-twitter-demo Demo to explain how to use AWS Chalice to connect to twitter API and change profile picture at scheduled times. Video Demo Click

AWS Lambda Shared Layers This repository is hosting the code from the following blog post: AWS Lambda & Shared layers for Python. The goal of this rep

AWS Org Inventory Dumps to CSV all the resources in an organization's member accounts. Set your environment's AWS_PROFILE and AWS_DEFAULT_REGION varia

WMigrate A tool for migrating AWS Workmail Users and Groups cross region and cross accounts. It also creates user and group aliases and adds the users

PipeChain Motivation PipeChain is a utility library for creating functional pipelines. Let's start with a motivating example. We have a list of Austra

Research PACS on AWS Challenge to solve Solution presentation Deploy the solution Further reading Releases License Challenge to solve The rise of new

MyTech® - Your Technology Django e-commerce website with Advanced Features and SEO Friendly Images and Prices are only used for Demo purpose and does

🎵 A repository for manually annotating files to create labeled acoustic datasets for machine learning.

Table of Contents What is GeoNode? Try out GeoNode Install Learn GeoNode Development Contributing Roadmap Showcase Most useful links Licensing What is

Boto3 - The AWS SDK for Python Boto3 is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python, which allows Python developers to wri

ec2-imagebuilder-ami-share CloudFormation template and CDK stack that contains a CustomResource with Lambda function to allow the setting of the targe

Unit Testing Interactions with Amazon Web Services (AWS) Unit testing AWS interactions with pytest and moto. These examples demonstrate how to structu

How The New York Times can increase Engagement on Facebook Using machine learning to understand characteristics of news content that garners "high" Fa

Nixtla Nixtla is an open-source time series forecasting library. We are helping data scientists and developers to have access to open source state-of-

Caylent Security Catalyst Reference Architecture Examples This repository contains solutions for Caylent's Security Catalyst. The Security Catalyst is

QSIprep: Preprocessing and analysis of q-space images Full documentation at https://qsiprep.readthedocs.io About qsiprep configures pipelines for proc

This repository contains the Lambda function to redrive sqs events from source to destination queue while controlling maxRetry per event.

AWS SQS event redrive Lambda This repository contains one simple AWS Lambda function in Python to redrive AWS SQS events from source queue to destinat

Cryptocurrency Prices Overview Instalação Repositório Configuração CI/CD Roadmap Testes Overview A ideia deste projeto é aplicar o conteúdo estudado s

Security Hub Controls CLI A CLI tool to disable and enable security standards controls in AWS Security Hub. It is designed to work together with AWS S

AWS infrastructure to ship CloudTrail logs from S3 to Kinesis This repository contains a Terraform module to ship CloudTrail logs stored in a S3 bucke

What is This? This repository is an opinonated project template for using aws-cdk, Chalice and React in concert. Where aws-cdk and Chalice are in Pyth

SysBindings Daemon Little toolkit for control the sysctl/sysfs bindings on Kubernetes Cluster on the fly and without unnecessary restarts of cluster o

aws-sso-sync-okta A small script to migrate or synchronize users & groups from Okta to AWS SSO Changelog Version Remove hardcoded values on variables

MONAI Deploy App SDK offers a framework and associated tools to design, develop and verify AI-driven applications in the healthcare imaging domain.

Server-and-Client-File-Transfer Python codes for the server and client end that facilitates file transfers. I will be using an AWS EC2 instance as the

AWS DeepRacer Overview The AWS DeepRacer Evo vehicle is a 1/18th scale Wi-Fi enabled 4-wheel ackermann steering platform that features two RGB cameras

AWS Interactive CLI - Allows you to execute a complex AWS commands by chaining one or more other AWS CLI dependency

MLOps template with examples for Data pipelines, ML workflow management, API development and Monitoring.

slashbot Slashbot makes it easy to create slash commands using AWS Lambda functions. These can be handy for creating a secure way to execute automated

Using AWS Batch jobs to bulk copy/sync files in S3

HiSim is a Python package for simulation and analysis of household scenarios using modern components as alternative to fossil fuel based ones.

Using AWS CDK Pipelines and AWS Lambda for multi-branch pipeline management and infrastructure deployment. This project shows how to use the AWS CDK P

AWStanding Easily load variables from AWS Parameter store into environment variables. Why to AWStanding? Because it handles AWS pagination so the amou

Zappa makes it super easy to build and deploy server-less, event-driven Python applications (including, but not limited to, WSGI web apps) on AWS Lambda + API Gateway. Think of it as "serverless" web hosting for your Python apps. That means infinite scaling, zero downtime, zero maintenance - and at a fraction of the cost of your current deployments!

Amazon Web Services (AWS) publishes its current IP address ranges in JSON format. Python v3 provides an ipaddress module in the standard library that allows you to create, manipulate, and perform operations on IPv4 and IPv6 addresses and networks. Wouldn't it be nice if you could work with the AWS IP address ranges like native Python objects?

aws-automation-plugins This repository contains ready to deploy automations on AWS. How-To All projects in this repository contain a deploy.sh file wh

This repository contains code to containerize and deploy a GPU docker on AWS for summarization task. Find a detailed blogpost here Youtube Video Versi

Write maintainable, production-ready pipelines using Jupyter or your favorite text editor. Develop locally, deploy to the cloud. ☁️

LIVE001 - AWS Lambda para Ingerir Dados de APIs Fazer o deploy de uma função lambda com infraestrutura como código Lambda vai numa API externa e extra

troposphere - Python library to create AWS CloudFormation descriptions

備考・注意事項 現在 α 版で、まだ実験的なプロダクトです。通常利用には耐えないでしょうし、サポートもできません。 安定しているとは到底言いがたい品質ですが、それでも構わない方のみ導入してください。 使い方などの説明も用意できていないため、自力でトラブルに対処できるエンジニアの方以外には現状おすすめ

Este repo ilustra un proceso sencillo de automatización de transformación y modelado de datos, a través de un pipeline utilizando Luigi. Stack princip

Data lineage made simple, reliable, and automated. Effortlessly track the flow of data, understand dependencies and analyze impact. Features Visualiza

Wonk is a tool for combining a set of AWS policy files into smaller compiled policy sets.

Automate activities in Control Tower provisioned AWS accounts Table of contents Introduction Architecture Prerequisites Tools and services Usage Clean

Rockpool Rockpool is a Python package for developing signal processing applications with spiking neural networks. Rockpool allows you to build network

Simple DDL Parser Build with ply (lex & yacc in python). A lot of samples in 'tests/. Is it Stable? Yes, library already has about 5000+ usage per day

The flowchart from the AWS IAM policy evaluation documentation page, as of 2021-09-12, and dating back to at least 2018-12-27, is the following: The f

tokeks A zero-dependency Python library for getting the Kubernetes token of a AWS EKS cluster. No AWS CLI, third-party client or library (boto3, botoc

Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions

Recommended AWS CDK project structure for Python applications The project implements a user management backend component that uses Amazon API Gateway,

Pokemon Catches Data Pipeline This is a sample project to practice end-to-end data project; Terraform is used to deploy infrastructure; Kafka is the t

A Python framework for creating reproducible, maintainable and modular data science code.

FwordCTF 2021 You can find here the source code of the challenges I wrote (Web and Bash) in FwordCTF 2021 and the source code of the platform with our

Earth Observation API Deploy a STAC API and a dynamic mosaic tiler API using AWS CDK.

txtai executes machine-learning workflows to transform data and build AI-powered semantic search applications.

AWS External Account Scanner Xenos, is Greek for stranger. AWSXenos will list all the trust relationships in all the IAM roles, and S3 buckets, in an

Using AWS's API Gateway + Lambda + Python to run a simple websocket application. For learning/testing. The AWS Resources seemed overly complex and were missing some critical gotchas in setting up a system like this.

aws-lambda-scheduler aws-lambda-scheduler lets you call any existing AWS Lambda Function you have in the future. This functionality is achieved by dyn

🧞 InfraGenie InfraGenie is allows you to split out your infrastructure project into separate independent pieces, each with its own terraform state. T

Lorien: A Unified Infrastructure for Efficient Deep Learning Workloads Delivery Lorien is an infrastructure to massively explore/benchmark the best sc

Aqueduct Framework for creating efficient data processing pipelines. Contact Feel free to ask questions in telegram t.me/avito-ml Key Features Increas

Konomi 備考・注意事項 現在 α 版で、まだ実験的なプロダクトです。通常利用には耐えないでしょうし、サポートもできません。 安定しているとは到底言いがたい品質ですが、それでも構わない方のみ導入してください。 使い方などの説明も用意できていないため、自力でトラブルに対処できるエンジニアの方以外に

Deploy an inference API on AWS (EC2) using FastAPI Docker and Github Actions To learn more about this project: medium blog post The goal of this proje

Arduino + Raspberry Pi + Unity3D + Cloud + Hyperledger Our Mission: Keep it simple, leave no one behind. Blockchain-Enabled Smart Sensor Framework usi

CPOST (Clara Pipeline Operator Sizing Tool) Tool to measure resource usage of Clara Platform pipeline operators Cpost is a tool that will help you run

A suite of utilities for AWS Lambda Functions that makes tracing with AWS X-Ray, structured logging and creating custom metrics asynchronously easier

AWSTraining This repository are used to give class about AWS by Marco Antonio Pereira Linkedin: https://www.linkedin.com/in/marcoap To see the types o

xrpl-price-persist-oracle-sam This is a XRPL Oracle that publishes external data into the XRPL. This Oracle was inspired by XRPL-Labs/XRPL-Persist-Pri

AWS SQS With Lambda notification 서버 구축을 위한 Poc TODO serverless를 통해 sqs 관련 리소스(람다, sqs) 배포 가능한 템플릿 작성 및 배포 poc차원에서 간단한 rest api 호출을 통한 sqs fifo 큐에 메시지

CDK Pipelines for Data Lake Infrastructure Deployment This solution helps you deploy data lake infrastructure on AWS using CDK Pipelines. This is base

aws-iot-shadow-rest-api HTTP Calls to Amazon Web Services Rest API for IoT Core Shadow Actions 💻 🌐 💡 This simple script implements the following aw

AWS_Subdomain_Takeover_Detector Purpose The purpose of this automation is to detect misconfigured Route53 entries which are vulnerable to subdomain ta

Directory Backup Moved to S3 (Pyscript) Description Here it's a python script that needs to use this script simply create a directory backup and moved

A Python library to utilize AWS API Gateway's large IP pool as a proxy to generate pseudo-infinite IPs for web scraping and brute forcing.

Heimdall Cloud Checker 🇧🇷 About Visius is a Brazilian cybersecurity startup that follows the signs of the crimson thunder ;) 🎸 ! As we value open s

spacy-huggingface-hub: Push your spaCy pipelines to the Hugging Face Hub This package provides a CLI command for uploading any trained spaCy pipeline