Skip to content

ivan-bilan/The-Microservices-Pandect

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

47 Commits
 
 
 
 
 
 
 
 

Repository files navigation

The-Microservices-Pandect

This pandect (πανδέκτης is Ancient Greek for encyclopedia) was created to help you find and understand almost anything related to Microservices that is available online.

Note Quick legend on available resource types:

⭐ - GitHub repository with the number of stars

📙 - resource you can read, usually a blog post or a paper

🗂️ - a collection of additional resources

🔱 - non-open source tool, framework or paid service

🎥️ - a resource you can watch

🎙️ - a resource you can listen to

Table of Contents

📇 Main Section 🗃️ Sub-sections Sample
Essential Reading General Resources, Platforms
Podcasts -
YouTube Channels -
Observability Alerting, Visualizing
Deployment General Tools, CI/CD
Cost Optimization -
Stateful Workloads -
Serverless Tools and Frameworks
Security -
Learning Resources Kubernetes, DevOps
Infrastructure as Code -
Other Topics Streaming, Testing

Essential-Reading

General Resources

Platforms

  • 🔱 AWS - Amazon Web Services on-demand cloud computing platform
  • 🔱 Azure - cloud computing service created by Microsoft
  • 🔱 Google Cloud Platform - GCP - suite of cloud computing services from Google
  • 🔱 OpenStack - free, open standard cloud computing platform
  • 🔱 Digital Ocean - DigitalOcean provides developers with cloud services
  • 🔱 Linode - cloud hosting company that provides virtual private servers
  • 🔱 Okteto - tool to develop applications on Kubernetes

Stories from the Industry

Compilations & Resource Collections

Roadmaps

From the Monolith to Microservices

Project Examples

  • CNCF Projects App - project example created with only CNCF graduated projects [GitHub, 143 stars]

Podcasts 🔙 Back to the Table of Contents


  • 🎙️ Cloudcast - independent Cloud Computing podcast [Years: 2011 - now, Status: active]
  • 🎙️ PodCTL - podcast focused on Cloud-native applications (by Red Hat) [Years: 2017 - now, Status: active]
  • 🎙️ Kubernetes Podcast - Kubernetes Podcast from Google [Years: 2018 - now, Status: active]
  • 🎙️ Data Engineering Podcast - Data management, microservices, ETL and more [Years: 2017 - now, Status: active]
  • 🎙️ The Secure Developer - A podcast about security for developers [Years: 2017 - now, Status: active]
  • 🎙️ APIs you won't hate - podcast about building and designing APIs [Years: 2019 - now, Status: active]

Youtube-Channels

  • 🎥 Continuous Delivery - Continuous Delivery Pipelines and Processes [Youtube, 57k Subscribers]
  • 🎥 CNCF - Cloud Native Computing Foundation - provides educational and informative content on cloud native computing [Youtube, 65k Subscribers]
  • 🎥 Snyk - build cloud native applications securely [Youtube, 2k Subscribers]
  • 🎥 CloudBeesTV - cloud conferences from the end-to-end automated software delivery company [Youtube, 9k Subscribers]
  • 🎥 Containers from the Couch - learning resources on Container Services [Youtube, 4k Subscribers]
  • 🎥 GOTO Conferences - GOTO is a software development content and events platform [Youtube, 230k Subscribers]

Observability

🔙 Back to the Table of Contents

General Monitoring

  • Prometheus - open-source systems monitoring and alerting toolkit
  • Prometheus Federation
  • kubewatch - Watch k8s events and trigger Handlers [GitHub, 2416 stars]
  • cortex - multi-tenant, long term Prometheus [GitHub, 4910 stars]

Error Monitoring

Alerting

Logging

  • loki - horizontally-scalable, highly-available, multi-tenant log aggregation system [GitHub, 17580 stars]

Visualizing

  • grafana - observability and data visualization platform [GitHub, 52288 stars]

Deployment

🔙 Back to the Table of Contents

General Deployment Tools

  • kubespray - Deploy a Production Ready Kubernetes Cluster [GitHub, 13125 stars]

Zero Downtime Deploys

There are industry proven strategies to make sure your deploys don't cause any downtime.

  • 𝐁𝐥𝐮𝐞/𝐆𝐫𝐞𝐞𝐧 𝐃𝐞𝐩𝐥𝐨𝐲𝐬 - deploy your change to a subset of nodes and redirect some traffic to them, while still using the old version for the remaining traffic. Once you have validated that the traffic to the new nodes is correct, take out the nodes with the old version and redirect all of your requests to new nodes. Power down nodes with the old version.
  • 𝐑𝐞𝐝/𝐁𝐥𝐚𝐜𝐤 𝐃𝐞𝐩𝐥𝐨𝐲𝐬 - similar to the above with one distinct difference: you direct all of your traffic to the new nodes, old nodes are kept alive just in case but don't receive any requests. After validation, they are removed.
  • 𝐂𝐚𝐧𝐚𝐫𝐲 𝐃𝐞𝐩𝐥𝐨𝐲𝐬 - in this deployment strategy, you release a new version of your microservice or ML model to a defined subset of users (i.e. 25%) and keep it alive for a prolonged time to make sure everything works as expected. You could technically use this also for A/B testing of your new version.
  • 𝐑𝐨𝐥𝐥𝐢𝐧𝐠 𝐃𝐞𝐩𝐥𝐨𝐲𝐬 - the simplest option, release your new version of the microservice incrementally node by node or in batches. It's slow, but easy to revert back if needed.
General
Tools:
  • flagger - Canary, A/B Testing and Blue/Green deployments for Kubernetes [GitHub, 3943 stars]

CI/CD

General
  • 🔱 Tekton - open-source framework for creating CI/CD systems
GitOps:
  • flux - The GitOps Kubernetes operator [GitHub, 6951 stars]
  • 🔱 ArgoCD - A declarative, GitOps continuous delivery tool for Kubernetes

Cost-Optimization

🔙 Back to the Table of Contents

General
AWS
  • 🔱 Spot Fleet [AWS Services]
  • AutoSpotting - open source spot market automation tool for EC2 [GitHub, 2115 stars]
Autoscaling
  • keda - Kubernetes-based Event Driven Autoscaling [GitHub, 5648 stars]
General Tools
  • Komiser - Multi-cloud environment inspector for costs and security [GitHub, 2866 stars]
  • Infracost - Cloud cost estimates for Terraform in your CLI and pull requests [GitHub, 8185 stars]

Stateful-Workloads

🔙 Back to the Table of Contents

Databases and Operators

General
  • 🗂 OperatorHub.io - OperatorHub.io is a resource for the Kubernetes community to find and share Operators
In-memory
  • redis - Redis is an open source, in-memory data structure store
MySQL
  • 🔱 MariaDB - MariaDB Server: The open source relational database
  • vitess - Canary, A/B Testing and Blue/Green deployments for Kubernetes [GitHub, 3943 stars]
PostgreSQL
OLAP - Online Analytical Processing
Object Storage
  • 🔱 Ceph - implements object storage on a single distributed computer cluster

Serverless

🔙 Back to the Table of Contents

General

Examples and Learning Resources

Tools & Frameworks

  • serverless - Serverless Framework using AWS Lambda, Azure Functions, Google CloudFunctions [GitHub, 43750 stars]
  • Chalice - Python Serverless Microframework for AWS [GitHub, 9297 stars]
  • OpenFaaS - Serverless Functions Made Simple [GitHub, 22320 stars]
  • Up - deploy infinitely scalable serverless apps, apis, and sites [GitHub, 8638 stars]
  • Dapr - portable, serverless, event-driven runtime for stateless and stateful microservices [GitHub, 19789 stars]
  • Nuclio - High-Performance Serverless event and data processing platform [GitHub, 4649 stars]

Security

🔙 Back to the Table of Contents

General Tools

  • kubesploit - Cross-platform post-exploitation HTTP/2 Command & Control server [GitHub, 940 stars]
  • 🔱 consul - Consul automates networking for simple and secure application delivery
  • Komiser - Multi-cloud environment inspector for costs and security [GitHub, 2866 stars]
  • tfsec - Security scanner for your Terraform code [GitHub, 5315 stars]

Security Audit Tools

  • kube-bench - Checks usage of security best practices as defined in the CIS Kubernetes Benchmark [GitHub, 5306 stars]
  • Prowler - security tool to perform AWS security best practices assessments [GitHub, 6957 stars]
  • ScoutSuite - Multi-Cloud Security Auditing Tool [GitHub, 6957 stars]

Secrets

  • sealed-secrets - A Kubernetes controller and tool for one-way encrypted Secrets [GitHub, 5597 stars]
  • 🔱 Vault - Manage Secrets and Protect Sensitive Data
  • aws-vault - securely store and access AWS credentials in development environments [GitHub, 6927 stars]
  • SOPS: Secrets OPerationS - Simple and flexible tool for managing secrets on any platform [GitHub, 11257 stars]

Protocols

Auth, API Gateways etc.

  • Grant - OAuth Proxy [GitHub, 3743 stars]
  • Dex - OpenID Connect (OIDC) identity and OAuth 2.0 provider [GitHub, 7536 stars]
  • Kong - Cloud-Native API Gateway [GitHub, 33318 stars]
  • Gloo Edge - Kubernetes-native API Gateway Built on Envoy [GitHub, 3601 stars]

Other

  • Awesome WAF - Everything about web-application firewalls (WAF) [GitHub, 4759 stars]
  • PENTESTING-BIBLE - Learn ethical hacking [GitHub, 11035 stars]

Learning Resources

🔙 Back to the Table of Contents

Kubernetes

AWS

Kafka

DevOps

  • DevOps Guide - from basic to advanced with Interview Questions and Notes [GitHub, 5423 stars]
  • DevOps Exercises - questions and exercises on technical topics related to DevOps and SRE [GitHub, 33396 stars]

Docker

  • Docker Curriculum - comprehensive tutorial on getting started with Docker [GitHub, 4892 stars]

Infrastructure as Code

🔙 Back to the Table of Contents

Infrastructure as Code Tools

  • Terraform - open-source infrastructure as code software tool for consistent CLI workflow
  • 🔱 CloudFormation - cloud provisioning with infrastructure as code for AWS
  • 🔱 Azure Resource Manager - manage your app resources on Azure
  • 🔱 Cloud Deployment Manager - create and manage cloud resources on GCP with simple templates
  • HashiCorp Vagrant / [GitHub, 22046 stars]
  • 🔱 CFEngine - automate your infrastructure, security & compliance
  • Ansible - automation across open hybrid cloud deployments
  • 🔱 CHEF - Policy-Based Configuration Management Automation Architecture
  • Pulumi - Modern Infrastructure as Code. Any cloud, any language [GitHub, 14215 stars]

Additional Tooling

  • Terraformer - CLI tool to generate terraform files from existing infrastructure (reverse Terraform) [GitHub, 8970 stars]
  • Checkov - static code analysis tool for infrastructure-as-code [GitHub, 4890 stars]

Examples and Learning Resources

Other

🔙 Back to the Table of Contents

Streaming Frameworks / Engines

Effective Containerization

  • distroless - Language focused docker images, minus the operating system [GitHub, 14182 stars]

Load Shedding

Load Shedding is used to prevent your microservices from an outage by reducing non-essential requests.

Learning Resources
Tools
  • Concurrency Limits - TCP congestion control to auto-detect concurrency limits for services [GitHub, 2836 stars]

Testing

General
Tooling
  • Terratest - Go library to write automated tests for your infrastructure code [GitHub, 6494 stars]
  • Serverless Offline - Emulate AWS λ and API Gateway locally [GitHub, 4747 stars]
  • Moto - easily mock out tests based on AWS infrastructure [GitHub, 6186 stars]
  • LocalStack - fully functional local AWS cloud stack [GitHub, 44.6k stars]

PaaS - Platform-as-a-service

  • Empire - PaaS built on top of Amazon EC2 Container Service with Heroku like workflow [GitHub, 2672 stars]

Container Network Interface (CNI)

  • CNI - networking for Linux containers [GitHub, 4517 stars]

Kafka

  • strimzi - Apache Kafka running on Kubernetes [GitHub, 3545 stars]

Documenting Architectural Design


License CC0

Attributions

Resources

  • All linked resources belong to original authors

Icons

Fonts


The Pandect Series also includes

     

About

A comprehensive reference for all topics related to building and maintaining microservices

Topics

Resources

License

Stars

Watchers

Forks

Languages