37 Repositories
Python gcp Libraries
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
Streamify A data pipeline with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more! Description Objective The project will stre
RAD Lab enables users to deploy infrastructure on Google Cloud Platform (GCP) to support specific use cases. Infrastructure is created and managed through Terraform in conjunction with support scripts written in Python. The templates, code, and documentation for each use case are bundled into modules.
RAD Lab RAD Lab enables users to deploy infrastructure on Google Cloud Platform (GCP) to support specific use cases. Infrastructure is created and man
Sample code and notebooks for Vertex AI, the end-to-end machine learning platform on Google Cloud
Google Cloud Vertex AI Samples Welcome to the Google Cloud Vertex AI sample repository. Overview The repository contains notebooks and community conte
Compares and analyzes GCP IAM roles.
gcp-iam-analyzer I wrote this to help in my day to day working in GCP. A lot of the time I am doing role comparisons to see which role has more permis
This example implements the end-to-end MLOps process using Vertex AI platform and Smart Analytics technology capabilities
MLOps with Vertex AI This example implements the end-to-end MLOps process using Vertex AI platform and Smart Analytics technology capabilities. The ex
Template repo for a GCP-hosted REST API with automatic API versioning and custom domain mapping
Python + Poetry REST API with FastAPI, hosted on GCP This template will get you ready to deploy a FastAPI app in Google Cloud with automatic API versi
A bot for PDF for doing Many Things....
Telegram PDF Bot A Telegram bot that can: Compress, crop, decrypt, encrypt, merge, preview, rename, rotate, scale and split PDF files Compare text dif
An image classification app boilerplate to serve your deep learning models asap!
Image 🖼 Classification App Boilerplate Have you been puzzled by tons of videos, blogs and other resources on the internet and don't know where and ho
Data-science-on-gcp - Source code accompanying book: Data Science on the Google Cloud Platform, Valliappa Lakshmanan, O'Reilly 2017
data-science-on-gcp Source code accompanying book: Data Science on the Google Cloud Platform, 2nd Edition Valliappa Lakshmanan O'Reilly, Jan 2022 Bran
Checkov is a static code analysis tool for infrastructure-as-code.
Checkov - Prevent cloud misconfigurations during build-time for Terraform, Cloudformation, Kubernetes, Serverless framework and other infrastructure-as-code-languages with Checkov by Bridgecrew.
Terraform wrapper to manage state across multiple cloud providers(AWS, GCP, and Azure)
Terraform Remote State Manager(tfremote) tf is a python package for managing terraform remote state for: Google(Gcloud), AWS, and Azure. It sets a def
Pythonic event-processing library based on decorators
Process Events In Style This library aims to simplify the common pattern of event processing. It simplifies the process of filtering, dispatching and
Cookiecutter to create a Google Function. Powered by Poetry, GitHub actions, and Google Cloud Platform
Cookiecutter Google Function Cookiecutter template for a Google Function. Powered by Poetry, and GitHub actions. Quickstart Install the latest Cookiec
Scout Suite - an open source multi-cloud security-auditing tool,
Description Scout Suite is an open source multi-cloud security-auditing tool, which enables security posture assessment of cloud environments. Using t
Pulumi - Developer-First Infrastructure as Code. Your Cloud, Your Language, Your Way 🚀
Pulumi's Infrastructure as Code SDK is the easiest way to create and deploy cloud software that use containers, serverless functions, hosted services,
Security Monkey monitors AWS, GCP, OpenStack, and GitHub orgs for assets and their changes over time.
NOTE: Security Monkey is in maintenance mode and will be end-of-life in 2020. For AWS users, please make use of AWS Config. For GCP users, please make
Demonstrate a Dataflow pipeline that saves data from an API into BigQuery table
Overview dataflow-mvp provides a basic example pipeline that pulls data from an API and writes it to a BigQuery table using GCP's Dataflow (i.e., Apac
Deploy your apps on any Cloud provider in just a few seconds
The simplest way to deploy your apps in the Cloud Deploy your apps on any Cloud providers in just a few seconds ⚡ Qovery Engine is an open-source abst
GCP Scripts and API Client Toolss
GCP Scripts and API Client Toolss Script Authentication The scripts and CLI assume GCP Application Default Credentials are set. Credentials can be set
Full ELT process on GCP environment.
Rent Houses Germany - GCP Pipeline Project: The goal of the project is to extract data about house rentals in Germany, store, process and analyze it u
A collection of online resources to help you on your Tech journey.
Everything Tech Resources & Projects About The Project Coming from an engineering background and looking to up skill yourself on a new field can be di
A job launching library for docker, EC2, GCP, etc.
doodad A library for packaging dependencies and launching scripts (with a focus on python) on different platforms using Docker. Currently supported pl
Building house price data pipelines with Apache Beam and Spark on GCP
This project contains the process from building a web crawler to extract the raw data of house price to create ETL pipelines using Google Could Platform services.
A collection of online resources to help you on your Tech journey.
Everything Tech Resources & Projects About The Project Coming from an engineering background and looking to up skill yourself on a new field can be di
Cloudkeeper is “housekeeping for clouds” - find leaky resources, manage quota limits, detect drift and clean up.
Cloudkeeper Housekeeping for Clouds! Table of contents Overview Docker based quick start Cloning this repository Component list Contact License Overvi
sysctl/sysfs settings on a fly for Kubernetes Cluster. No restarts are required for clusters and nodes.
SysBindings Daemon Little toolkit for control the sysctl/sysfs bindings on Kubernetes Cluster on the fly and without unnecessary restarts of cluster o
Goblet is an easy-to-use framework that enables developers to quickly spin up fully featured REST APIs with python on GCP
GOBLET Goblet is a framework for writing serverless rest apis in python in google cloud. It allows you to quickly create and deploy python apis backed
Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions
Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions
gcp-doctor - Diagnostics for Google Cloud Platform
gcp-doctor is a command-line diagnostics tool for GCP customers. It finds and helps to fix common issues in Google Cloud Platform projects. It is used to test projects against a wide range of best-practices and frequent mistakes, based on the troubleshooting experience of the Google Cloud Support team.
Tutorial to pretrain & fine-tune a 🤗 Flax T5 model on a TPUv3-8 with GCP
Pretrain and Fine-tune a T5 model with Flax on GCP This tutorial details how pretrain and fine-tune a FlaxT5 model from HuggingFace using a TPU VM ava
Tool to check whether a GCP bucket is public or not.
Tool to check publicly accessible GCP bucket. Blog https://justm0rph3u5.medium.com/gcp-inspector-auditing-publicly-exposed-gcp-bucket-ac6cad55618c Wha
Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.
deep_autoviml Build keras pipelines and models in a single line of code! Table of Contents Motivation How it works Technology Install Usage API Image
gcptree - Like the unix tree command but for GCP Org Heirarchy
gcptree Like the unix tree command but for GCP Org Heirarchy. For a note on coloring, the org node is green, folders and blue, and projects that are n
Ethereum ETL lets you convert blockchain data into convenient formats like CSVs and relational databases.
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions.
tfquery: Run SQL queries on your Terraform infrastructure. Query resources and analyze its configuration using a SQL-powered framework.
🌩️ tfquery 🌩️ Run SQL queries on your Terraform infrastructure. Ask questions that are hard to answer 🚀 What is tfquery? tfquery is a framework tha
Policy and data administration, distribution, and real-time updates on top of Open Policy Agent
⚡ OPAL ⚡ Open Policy Administration Layer OPAL is an administration layer for Open Policy Agent (OPA), detecting changes to both policy and policy dat
A Blazing fast Security Auditing tool for Kubernetes
A Blazing fast Security Auditing tool for kubernetes!! Basic Overview Kubestriker performs numerous in depth checks on kubernetes infra to identify th