AWS SQS event redrive Lambda

Overview

AWS SQS event redrive Lambda

This repository contains one simple AWS Lambda function in Python to redrive AWS SQS events from source queue to destination queue. This script can be triggered periodically via cloudwatch as well if source queue contains too many events and you dont want to redrive them altogether.

  • batch_size can be passed as input which will move only X events in one execution cycle. Default batch size is 10 if no value is passed.
  • This script ensures that if same event is getting moved into DLQ again and again, it wont get retried more than 3 times.

Disclaimer: This script adds one additional field 'RetryCounter' to keep track of number of retries in SQS's MessageAttributes. In 99.9% of cases, ideally this shouldn't cause any break in consumer code, because this is just an additional field added in attributes (NOT in body), but please confirm the impact once before usage. None of the existing data of SQS event is touched upon.

Setup

  1. Create/Open AWS account
  2. Go to Lambda service
  3. Click on create new Lambda
  4. Give a suitable name to your lambda function.
  5. Choose python 3.6 or above as 'Runtime'
  6. Click on 'Create Function'
  7. Copy and paste the code present in 'SQSEventsRedrive.py' file in this repository in the Lambda.
  8. Click on 'Deploy' to deploy this code in lambda server.
  9. Click on Test and provide the input to the lambda function as explained in below section and submit the request.
  10. You should see the data getting moved from source to destination queue.
  11. In case of any issues, you can check the logs in cloudwatch by clicking on Existing lambda main page -> Monitoring -> Cloudwatch Logs -> see recent file and recent logs there.

Sample Input

{
  "source_queue_url" : "AWS source sqs queue URL",
  "destination_queue_url" : "AWS destination sqs queue URL",
  "batch_size": 10
}

Note:

  • Only 'batch_size' is optional here. queue URLs are mandatory fields.
  • Resource URLs are expected in the input, not the ARNs.
You might also like...
AWS-serverless-starter - AWS Lambda serverless stack via Serverless framework
AWS-serverless-starter - AWS Lambda serverless stack via Serverless framework

Serverless app via AWS Lambda, ApiGateway and Serverless framework Configuration

Automated AWS account hardening with AWS Control Tower and AWS Step Functions
Automated AWS account hardening with AWS Control Tower and AWS Step Functions

Automate activities in Control Tower provisioned AWS accounts Table of contents Introduction Architecture Prerequisites Tools and services Usage Clean

Implement backup and recovery with AWS Backup across your AWS Organizations using a CI/CD pipeline (AWS CodePipeline).
Implement backup and recovery with AWS Backup across your AWS Organizations using a CI/CD pipeline (AWS CodePipeline).

Backup and Recovery with AWS Backup This repository provides you with a management and deployment solution for implementing Backup and Recovery with A

SQS + Lambda를 활용한 문자 메시지 및 이메일, Voice call 호출을 간단하게 구현하는 serverless 템플릿
SQS + Lambda를 활용한 문자 메시지 및 이메일, Voice call 호출을 간단하게 구현하는 serverless 템플릿

AWS SQS With Lambda notification 서버 구축을 위한 Poc TODO serverless를 통해 sqs 관련 리소스(람다, sqs) 배포 가능한 템플릿 작성 및 배포 poc차원에서 간단한 rest api 호출을 통한 sqs fifo 큐에 메시지

Criando Lambda Functions para Ingerir Dados de APIs com AWS CDK

LIVE001 - AWS Lambda para Ingerir Dados de APIs Fazer o deploy de uma função lambda com infraestrutura como código Lambda vai numa API externa e extra

A python library for creating Slack slash commands using AWS Lambda Functions

slashbot Slashbot makes it easy to create slash commands using AWS Lambda functions. These can be handy for creating a secure way to execute automated

Building and deploying AWS Lambda Shared Layers

AWS Lambda Shared Layers This repository is hosting the code from the following blog post: AWS Lambda & Shared layers for Python. The goal of this rep

AWS Lambda Fast API starter application

AWS Lambda Fast API Fast API starter application compatible with API Gateway and Lambda Function. How to deploy it? Terraform AWS Lambda API is a reus

A Python AWS Lambda Webhook listener that generates a permanent URL when an asset is created in Contentstack.

Webhook Listener A Python Lambda Webhook Listener - Generates a permanent URL on created assets. See doc on Generating a Permanent URL: https://www.co

Owner
null
DIAL(Did I Alert Lambda?) is a centralised security misconfiguration detection framework which completely runs on AWS Managed services like AWS API Gateway, AWS Event Bridge & AWS Lambda

DIAL(Did I Alert Lambda?) is a centralised security misconfiguration detection framework which completely runs on AWS Managed services like AWS API Gateway, AWS Event Bridge & AWS Lambda

CRED 71 Dec 29, 2022
aws-lambda-scheduler lets you call any existing AWS Lambda Function you have in a future time.

aws-lambda-scheduler aws-lambda-scheduler lets you call any existing AWS Lambda Function you have in the future. This functionality is achieved by dyn

Oğuzhan Yılmaz 57 Dec 17, 2022
Python + AWS Lambda Hands OnPython + AWS Lambda Hands On

Python + AWS Lambda Hands On Python Criada em 1990, por Guido Van Rossum. "Bala de prata" (quase). Muito utilizado em: Automatizações - Selenium, Beau

Marcelo Ortiz de Santana 8 Sep 9, 2022
Aws-lambda-requests-wrapper - Request/Response wrapper for AWS Lambda with API Gateway

AWS Lambda Requests Wrapper Request/Response wrapper for AWS Lambda with API Gat

null 1 May 20, 2022
Lambda-function - Python codes that allow notification of changes made to some services using the AWS Lambda Function

AWS Lambda Function This repository contains python codes that allow notificatio

Elif Apaydın 3 Feb 11, 2022
Live Coding - Mensageria na AWS com Amazon SNS e Amazon SQS

Live Coding - Mensageria na AWS com Amazon SNS e Amazon SQS Repositório para o Live Coding do dia 08/12/2021 Serviços utilizados Amazon SNS Amazon SQS

Cassiano Ricardo de Oliveira Peres 3 Mar 1, 2022
SSH-Restricted deploys an SSH compliance rule (AWS Config) with auto-remediation via AWS Lambda if SSH access is public.

SSH-Restricted SSH-Restricted deploys an SSH compliance rule with auto-remediation via AWS Lambda if SSH access is public. SSH-Auto-Restricted checks

Adrian Hornsby 30 Nov 8, 2022
A suite of utilities for AWS Lambda Functions that makes tracing with AWS X-Ray, structured logging and creating custom metrics asynchronously easier

A suite of utilities for AWS Lambda Functions that makes tracing with AWS X-Ray, structured logging and creating custom metrics asynchronously easier

Amazon Web Services - Labs 1.9k Jan 7, 2023
POC de uma AWS lambda que executa a consulta de preços de criptomoedas, e é implantada na AWS usando Github actions.

Cryptocurrency Prices Overview Instalação Repositório Configuração CI/CD Roadmap Testes Overview A ideia deste projeto é aplicar o conteúdo estudado s

Gustavo Santos 3 Aug 31, 2022