Continuous Security Group Rule Change Detection & Response at scale

Overview

Introduction

Get notified of Security Group Changes across all AWS Accounts & Regions in an AWS Organization, with the ability to respond/revert those changes with a single button click from a Slack Channel. This is made easy and possible with the recent announcement of CloudTrail Lake, which helps aggregate CloudTrail logs from all accounts/regions in a queryable(if that's a word :P) format.

The infrastructure needed for this project is deployed as a CDK Application, which deploys a CodeCommit repository and a CodeBuild stage which synthesizes the cdk app to cloudformation template and deploys in the target environment.

NOTE Security Group Rule Changes are allowed by default. Meaning changes are reverted only when explicitly denied by user interaction through slack channel interactive Deny button.

How it works on the high level

Prerequisites

  • Appropriate IAM Roles and trust relationships within the AWS Organization and member accounts, FleetAccess.

An ideal setup would be like this

  • CloudTrail Lake setup - Tutorial to enable cloudtrail lake
  • A role in the Organization account with the ability to invoke start_query and get_query_results and trusted by the hub-001 role in the Security account, where the lambda functions run from.
  • Slack App setup with the API Gateway endpoint updated. The Signing secret from the app should be updated in the config file to be used by the lambda function to verify requests from slack. The app enabled with features - Incoming Webhooks, Interactivity.
  • Usage of the latest available boto3 library as API calls related to cloudtrail lake is new and only available on very recent versions, thus boto3 is packaged along with lambda functions.
  • Have Cloud Development Kit installed with npm install cdk. https://github.com/aws/aws-cdk

Components

  • Lambda Functions
    • revertsg-1 - Triggered every 10 mins by cloudwatch event rule.
    • revertsg-2 - Invoked by API Gateway.
  • API Gateway to receive requests from Slack and proxy to revertsg-2
  • Dynamodb to hold security group rule change details.
  • CloudWatch Event Rule time based to trigger revertsg-1 every 10 mins.

Workflow

Security Group Change Detection & Response

  • CloudWatch Event Time based rule will trigger lambda revertsg-1, every 10 mins.
  • Lambda function revertsg-1 will assume role cloudtrail-lake-read-role in the organization account and run query to fetch events with event name AuthorizeSecurityGroupIngress in the last 20 mins. There is an over lap so that events which were
  • Query results are gathered and new security group rule changes are added to a dynamodb table secgrouprequests and also details are sent to a slack channel in an interactive message with the ability to either ignore or deny this change.
  • Slack interaction invokes API Gateway which in turn invokes revertsg-2 with all the headers and body proxied.
  • Security group rule changes are allowed by default, so,
    • If the user clicks on Approve(well technically it's already approved :P), revertsg-2 does the same and responds back with the user name who ignored this change event.
    • If the user clicks Deny, meaning to revert the change, revertsg-2 will,
      • Read dynamodb table with the cloudtrail requestid, get that specific event details, assume spoke-001 role on that account from the security account as hub-001, invokes the revoke_security_group_ingress API call, responds with the messaged as denied with the user name.

Usage

  • Git clone https://github.com/raajheshkannaa/green-stone
  • Update config.py in these 4 places, because both the CDK App and the source code depends on various constants from this configuration file.
AUTOMATION_ACCOUNT = '
   
    ' # Where our automation is run
ORG_ACCOUNT = '
    
     ' # This is where CloudTrail Lake is setup.
#CLOUDTRAIL_LAKE_READ_ROLE = '
     
      '
CLOUDTRAIL_LAKE_READ_ROLE = 'cloudtrail-lake-read-role' # This is the role name used if deployed using FleetAccess - https://github.com/raajheshkannaa/fleet-access 
HOOK_URL = '
      
       ' # https://api.slack.com/messaging/webhooks
SIGNINGSECRET = 
        
       
      
     
    
   
  • Once the above details are updated, run cdk ls to confirm the stacks are good.
  • Run cdk synth to make sure templates synthesize without errors.
  • With the Security Account credentials in the local terminal using export AWS_PROFILE= , run cdk deploy or use cdk deploy --profile security-account.
  • CDK will deploy the CSGDRRPipelineStack pipeline stack, which in turn creates the CodeCommit repository, CodePipeline with necessary IAM permissions, Stages of deployment. At end of deployment, in the terminal cdk would print out the API Gateway url which needs to be updated in the Slack App in the interactivity section.
  • The first time you deploy, the pipeline runs and fails, because we haven't pushed our code yet.
  • So, now gather the codecommit repository details which was created and git push the cdk app to the repository which has the cloudformation stacks to be deployed by the pipeline.
  • Once code is pushed, pipeline picks it up automatically and begins deployment, its an amazing feeling to look at this in action, with so minimal effort.
  • The automation is triggered every 10 mins and if there are any Security Group changes, that would show up in the Slack Channel for which the webhook was configured earlier.

Considerations

  • CloudTrail events are delayed by up to 2-3 mins sometimes before it gets delivered to the cloudtrail lake. Timings are adjusted accordingly for this project, with the CloudWatch Rule and also the event times` for the cloudtrail lake query.
  • Security Group Rule Changes are allowed by default, to make sure Security doesn't add friction to the operations with rest of the organization. Meaning changes are reverted only when explicitly denied by user interaction through slack channel interactive Deny button.
  • Because this is a new service/feature from AWS, Lambda's boto3 library is not updated with the ability to invoke these API calls, so I've packaged a recent version of boto3 part of the lambda function. This could have been a Lambda layer, but hey, doesn't matter after couple weeks or a month.
You might also like...
Code of Classification Saliency-Based Rule for Visible and Infrared Image Fusion

CSF Code of Classification Saliency-Based Rule for Visible and Infrared Image Fusion Tips: For testing: CUDA_VISIBLE_DEVICES=0 python main.py For trai

PyExplainer: A Local Rule-Based Model-Agnostic Technique (Explainable AI)
PyExplainer: A Local Rule-Based Model-Agnostic Technique (Explainable AI)

PyExplainer PyExplainer is a local rule-based model-agnostic technique for generating explanations (i.e., why a commit is predicted as defective) of J

Rule Based Classification Project

Kural Tabanlı Sınıflandırma ile Potansiyel Müşteri Getirisi Hesaplama İş Problemi: Bir oyun şirketi müşterilerinin bazı özelliklerini kullanaraknseviy

Rule Based Classification Project For Python

Rule-Based-Classification-Project (ENG) Business Problem: A game company wants to create new level-based customer definitions (personas) by using some

Rule-based Customer Segmentation
Rule-based Customer Segmentation

Rule-based Customer Segmentation Business Problem A game company wants to create level-based new customer definitions (personas) by using some feature

Rule based classification A hotel s customers dataset

Rule-based-classification-A-hotel-s-customers-dataset- Aim: Categorize new customers by segment and predict how much revenue they can generate This re

PyExplainer: A Local Rule-Based Model-Agnostic Technique (Explainable AI)
PyExplainer: A Local Rule-Based Model-Agnostic Technique (Explainable AI)

PyExplainer PyExplainer is a local rule-based model-agnostic technique for generating explanations (i.e., why a commit is predicted as defective) of J

A rule-based log analyzer & filter

Flog 一个根据规则集来处理文本日志的工具。 前言 在日常开发过程中,由于缺乏必要的日志规范,导致很多人乱打一通,一个日志文件夹解压缩后往往有几十万行。 日志泛滥会导致信息密度骤减,给排查问题带来了不小的麻烦。 以前都是用grep之类的工具先挑选出有用的,再逐条进行排查,费时费力。在忍无可忍之后决

Official implement of Paper:A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sening  images
Official implement of Paper:A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sening images

A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images 深度监督影像融合网络DSIFN用于高分辨率双时相遥感影像变化检测 Of

Owner
Raajhesh Kannaa Chidambaram
Raajhesh Kannaa Chidambaram
PyTorch Implementation for AAAI'21 "Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection"

UMS for Multi-turn Response Selection Implements the model described in the following paper Do Response Selection Models Really Know What's Next? Utte

Taesun Whang 47 Nov 22, 2022
MLOps will help you to understand how to build a Continuous Integration and Continuous Delivery pipeline for an ML/AI project.

page_type languages products description sample python azure azure-machine-learning-service azure-devops Code which demonstrates how to set up and ope

null 1 Nov 1, 2021
Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel order of RGB and BGR. Simple Channel Converter for ONNX.

scc4onnx Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel

Katsuya Hyodo 16 Dec 22, 2022
object detection; robust detection; ACM MM21 grand challenge; Security AI Challenger Phase VII

赛题背景 在商品知识产权领域,知识产权体现为在线商品的设计和品牌。不幸的是,在每一天,存在着非法商户通过一些对抗手段干扰商标识别来逃避侵权,这带来了很高的知识产权风险和财务损失。为了促进先进的多媒体人工智能技术的发展,以保护企业来之不易的创作和想法免受恶意使用和剽窃,因此提出了鲁棒性标识检测挑战赛

null 65 Dec 22, 2022
Code for paper: Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks

Group-CAM By Zhang, Qinglong and Rao, Lu and Yang, Yubin [State Key Laboratory for Novel Software Technology at Nanjing University] This repo is the o

zhql 98 Nov 16, 2022
BC3407-Group-5-Project - BC3407 Group Project With Python

BC3407-Group-5-Project As the world struggles to contain the ever-changing varia

null 1 Jan 26, 2022
A tight inclusion function for continuous collision detection

Tight-Inclusion Continuous Collision Detection A conservative Continuous Collision Detection (CCD) method with support for minimum separation. You can

Continuous Collision Detection 89 Jan 1, 2023
Official Implementation and Dataset of "PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency", CVPR 2021

Portrait Photo Retouching with PPR10K Paper | Supplementary Material PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask an

null 184 Dec 11, 2022
Trading and Backtesting environment for training reinforcement learning agent or simple rule base algo.

TradingGym TradingGym is a toolkit for training and backtesting the reinforcement learning algorithms. This was inspired by OpenAI Gym and imitated th

Yvictor 1.1k Jan 2, 2023
GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognition

GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognition

Xinyan Zhao 29 Dec 26, 2022