Cloud-native SIEM for intelligent security analytics for your entire enterprise.

Overview

Microsoft Sentinel

Welcome to the Microsoft Sentinel repository! This repository contains out of the box detections, exploration queries, hunting queries, workbooks, playbooks and much more to help you get ramped up with Microsoft Sentinel and provide you security content to secure your environment and hunt for threats. You can also submit to issues for any samples or resources you would like to see here as you onboard to Microsoft Sentinel. This repository welcomes contributions and refer to this repository's wiki to get started. For questions and feedback, please contact [email protected]

Resources

We value your feedback. Here are some channels to help surface your questions or feedback:

  1. General product specific Q&A - Join in the Microsoft Sentinel Tech Community conversations
  2. Product specific feature requests - Upvote or post new on Microsoft Sentinel feedback forums
  3. Report product or contribution bugs - File a GitHub Issue using Bug template
  4. General feedback on community and contribution process - File a GitHub Issue using Feature Request template

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

Add in your new or updated contributions to GitHub

Note: If you are a first time contributor to this repository, General GitHub Fork the repo guidance before cloning or Specific steps for the Sentinel repo.

General Steps

Brand new or update to a contribution via these methods:

Pull Request

  • After you push your changes, you will need to submit the Pull Request (PR)
  • Details about the Proposed Changes are required, be sure to include a minimal level of detail so a review can clearly understand the reason for the change and what he change is related to in the code.
  • After submission, check the Pull Request for comments
  • Make changes as suggested and update your branch or explain why no change is needed. Resolve the comment when done.

Pull Request Detection Template Structure Validation Check

As part of the PR checks we run a structure validation to make sure all required parts of the YAML structure are included. For Detections, there is a new section that must be included. See the contribution guidelines for more information. If this section or any other required section is not included, then a validation error will occur similar to the below. The example is specifically if the YAML is missing the entityMappings section:

A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:00.95]     Kqlvalidations.Tests.DetectionTemplateStructureValidationTests.Validate_DetectionTemplates_HaveValidTemplateStructure(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [FAIL]
  X Kqlvalidations.Tests.DetectionTemplateStructureValidationTests.Validate_DetectionTemplates_HaveValidTemplateStructure(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [104ms]
  Error Message:
   Expected object to be <null>, but found System.ComponentModel.DataAnnotations.ValidationException with message "An old mapping for entity 'AccountCustomEntity' does not have a matching new mapping entry."

Pull Request Kql Validation Check

As part of the PR checks we run a syntax validation of the kql queries defined in the template. If this check fails go to Azure Pipeline (by pressing on the errors link on the checks tab in your PR) Azurepipeline In the pipeline you can see which test failed and what is the cause: Pipeline Tests Tab

Example error message:

A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:01.81]     Kqlvalidations.Tests.KqlValidationTests.Validate_DetectionQueries_HaveValidKql(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [FAIL]
  X Kqlvalidations.Tests.KqlValidationTests.Validate_DetectionQueries_HaveValidKql(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [21ms]
  Error Message:
   Template Id:fa0ab69c-7124-4f62-acdd-61017cf6ce89 is not valid Errors:The name 'SymantecEndpointProtection' does not refer to any known table, tabular variable or function., Code: 'KS204', Severity: 'Error', Location: '67..93',The name 'SymantecEndpointProtection' does not refer to any known table, tabular variable or function., Code: 'KS204', Severity: 'Error', Location: '289..315'

If you are using custom logs table (a table which is not defined on all workspaces by default) you should verify your table schema is defined in json file in the folder Azure-Sentinel\.script\tests\KqlvalidationsTests\CustomTables

Example for table tablexyz.json

{
  "Name": "tablexyz",
  "Properties": [
    {
      "Name": "SomeDateTimeColumn",
      "Type": "DateTime"
    },
    {
      "Name": "SomeStringColumn",
      "Type": "String"
    },
    {
      "Name": "SomeDynamicColumn",
      "Type": "Dynamic"
    }
  ]
}

Run Kql Validation Locally

In order to run the kql validation before submitting Pull Request in you local machine:

  • You need to have .Net Core 3.1 SDK installed How to download .Net (Supports all platforms)
  • Open Shell and navigate to Azure-Sentinel\\.script\tests\KqlvalidationsTests\
  • Execute dotnet test

Example of output (in Ubuntu):

Welcome to .NET Core 3.1!
---------------------
SDK Version: 3.1.403

Telemetry
---------
The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry

----------------
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Find out what's new: https://aka.ms/dotnet-whats-new
Learn about the installed HTTPS developer cert: https://aka.ms/aspnet-core-https
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs
Write your first app: https://aka.ms/first-net-core-app
--------------------------------------------------------------------------------------
Test run for /mnt/c/git/Azure-Sentinel/.script/tests/KqlvalidationsTests/bin/Debug/netcoreapp3.1/Kqlvalidations.Tests.dll(.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.

Test Run Successful.
Total tests: 171
     Passed: 171
 Total time: 25.7973 Seconds

Detection schema validation tests

Similarly to KQL Validation, there is an automatic validation of the schema of a detection. The schema validation includes the detection's frequency and period, the detection's trigger type and threshold, validity of connectors Ids (valid connectors Ids list), etc. A wrong format or missing attributes will result with an informative check failure, which should guide you through the resolution of the issue, but make sure to look into the format of already approved detection.

Run Detection Schema Validation Locally

In order to run the kql validation before submitting Pull Request in you local machine:

  • You need to have .Net Core 3.1 SDK installed How to download .Net (Supports all platforms)
  • Open Shell and navigate to Azure-Sentinel\\.script\tests\DetectionTemplateSchemaValidation\
  • Execute dotnet test

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

For information on what you can contribute and further details, refer to the "get started" section on the project's wiki.

Comments
  • Adding Azure DevOps data connector and onboarding scripts

    Adding Azure DevOps data connector and onboarding scripts

    Required items, please complete

    Change(s):

    • Adding data connector for Azure DevOps

    Reason for Change(s):

    • Didn't exist yet

    Version Updated:

    • Required only for Detections/Analytic Rule templates
    • See guidance below

    Testing Completed:

    • Tested and validated see screenshots reamd.md for proof

    image

    Checked that the validations are passing and have addressed any issues that are present:

    • See guidance below
    Connector 
    opened by azurekid 61
  • Digital Shadows solution

    Digital Shadows solution


    Change(s):

    • Initial PR
    • Added solution by Digital Shadows for Azure Sentinel
    • Contains Analytic rule yaml file, Connector Azure function, Playbook JSON file, and Workbook JSON file.

    Reason for Change(s):

    • Initial PR

    Testing Completed:

    • Yes

    Checked that the validations are passing and have addressed any issues that are present:

    • Yes

    Playbook Connector Workbook Solution Detection 
    opened by shikhin-metron 54
  • Cybersixgill Actionable alerts

    Cybersixgill Actionable alerts

    Required items, please complete

    Change(s):

    • Cybersixgill Actionable alerts Solution

    Reason for Change(s):

    • New solution

    Version Updated:

    • N/A

    Testing Completed:

    • Yes

    Checked that the validations are passing and have addressed any issues that are present:

    • Yes
    Playbook Connector Workbook Solution Hunting 
    opened by loginsoft-integrations 45
  • On-Premises Exchange Security Configuration first commit

    On-Premises Exchange Security Configuration first commit

    Change(s):

    • First PR of new solution "ESI - On-Premises Exchange Security Configuration" without package

    Reason for Change(s):

    • Adding new REST API Connector with its parser and the first Workbook

    Version Updated:

    • First version 1.0.0

    Testing Completed:

    • Yes

    Checked that the validations are passing and have addressed any issues that are present:

    • No
    Connector Parser Workbook Solution 
    opened by nlepagnez 44
  • Forescout Host Property Monitor solution - initial version

    Forescout Host Property Monitor solution - initial version

    Change(s):

    • Initial version of solution Forescout Host Property Monitor

    Reason for Change(s):

    • New Solution development for Forescout Host Property Monitor

    Version Updated:

    • No

    Testing Completed:

    • Yes

    Checked that the validations are passing and have addressed any issues that are present:

    • Yes
    Playbook Solution Detection 
    opened by fs-connect 41
  • Adding Mandiant Advantage Threat Intelligence Solution

    Adding Mandiant Advantage Threat Intelligence Solution

    Required items, please complete

    Change(s):

    • Initial commit for adding Mandiant Advantage Threat Intelligence solution

    Reason for Change(s):

    • Adding Mandiant Advantage Threat Intelligence solution

    Testing Completed:

    • Yes

    Checked that the validations are passing and have addressed any issues that are present:

    • Need Help
    Playbook Solution 
    opened by chrishultin 40
  • Esi   exchange configuration evolution

    Esi exchange configuration evolution

    Required items, please complete

    Change(s):

    • CriticalCmdletsUsageDetection.yaml
    • ServerOrientedWithUserOrientedAdministration.yaml
    • Data Connectors/ESI-ExchangeAdminAuditLogEvents.json
    • Data Connectors/ESI-ExchangeOnPremisesCollector.json
    • Data Connectors/ESI-ExchangeOnlineCollector.json
    • Data Connectors/azuredeploy_ESI_ExchangeOnlineCollector_Automation.json
    • Parser/ExchangeAdminAuditLogs.txt
    • Parser/ExchangeConfiguration.txt
    • Sample-Data/ESI-ExchangeAdminAuditLogs-SampleData.json
    • SolutionMetadata.json
    • Watchlists/ExchangeVIPs.json
    • Workbooks/Microsoft Exchange Least Privilege with RBAC - Online.json
    • Workbooks/Microsoft Exchange Least Privilege with RBAC.json
    • Workbooks/Microsoft Exchange Search AdminAuditLog.json
    • Workbooks/Microsoft Exchange Security Monitoring.json
    • Workbooks/Microsoft Exchange Security Review - Online.json
    • Workbooks/Microsoft Exchange Security Review.json

    Reason for Change(s):

    • Finalizing the solution before packaging it

    Version Updated:

    • No

    Testing Completed:

    • Yes

    Checked that the validations are passing and have addressed any issues that are present:

    • Yes
    Connector Parser Workbook WatchList Analytic Rules 
    opened by nlepagnez 38
  • add WithSecure Elements data connector

    add WithSecure Elements data connector

    Required items, please complete

    Change(s):

    • Add WithSecure Elements via Connector data connector

    Reason for Change(s):

    • Add new data connector for WithSecure Elements solution

    Version Updated:

    • No

    Testing Completed:

    • Yes

    Checked that the validations are passing and have addressed any issues that are present:

    • Yes
    Connector 
    opened by HubertSzymanski5 38
  • Fix Atlassian Confluence API parameter

    Fix Atlassian Confluence API parameter

    Required items, please complete

    Change(s):

    • Fix Atlassian Confluence API parameter

    Reason for Change(s):

    • The audit API of confluence takes startDate and endData as a parameter. These two paramters should be in utc timestampe format but the current implementation passed a datetime string. And thus the function app is not retrieving any log from the API. See: https://developer.atlassian.com/cloud/confluence/rest/api-group-audit/#api-wiki-rest-api-audit-get

    Version Updated:

    • N/A

    Testing Completed:

    • Only tested in my personal environment

    Checked that the validations are passing and have addressed any issues that are present:

    • N/A
    Connector 
    opened by nevermoe 37
  • [TrendMicro] Update new version data connector.

    [TrendMicro] Update new version data connector.

    Before submitting this PR please ensure that you have read the following sections and then completed the template below:

    Thank you for your contribution to the Microsoft Sentinel Github repo.

    The code should have been tested in a Microsoft Sentinel environment that does not have any custom parsers, functions or tables, so that you validate no incorrect syntax and execution functions properly.

    Details of the code changes in your submitted PR. Providing descriptions for pull requests ensures, there is context to changes being made and greatly enhances the code review process. Providing associated Issues that this resolves also easily connects the reason.

    Change(s):

    • Updated syntax for XYZ.yaml

    Reason for Change(s):

    • New schema used for XYZ.yaml
    • Resolves ISSUE #1234

    After the submission has been made, please look at the Validation Checks:

    Check that the validations are passing and address any issues that are present. Let us know if you have tried fixing and need help.

    References:

    PR Template


    Description for the PR:

    • Remove custom table, TrendMicro_XDR_CL.
    • Add new custom table: TrendMicro_XDR_WORKBENCH_CL, TrendMicro_XDR_RCA_Task_CL, TrendMicro_XDR_RCA_Result_CL and TrendMicro_XDR_OAT_CL.
    • Add new azure function with time and queue trigger for new custom table.

    Testing Completed: Yes/ No : Yes


    Connector 
    opened by adamhuang97 31
  • FlareSystemsFirework: update to 1.0.1 solution

    FlareSystemsFirework: update to 1.0.1 solution

    Fixes #

    Proposed Changes

    • Added Playbook
    • Changed wizard and instructions in Data Connector as well as Solution deployment (Zip)
    • The changes to the zip are already published in microsoft marketplace listing
    opened by jctaillandier 30
  • NRT rule triggers 50 incidents

    NRT rule triggers 50 incidents

    Describe the bug

    We are currently using the native NRT MFA Rejected by User analytics rule, which should only trigger on the last minute. However last night someone denied a MFA request, but this rule kept triggering every 5 minutes.

    I'm not sure where else to report since we only have Basic support which does not include Sentinel..

    To Reproduce Steps to reproduce the behavior:

    1. Setup NRT MFA Rejected by User AAD analytics rule in Sentinel.
    2. Have someone deny a MFA prompt
    3. See 50 incidents popup

    Expected behavior

    Only the two, same events in all incidents:

    image

    image

    Screenshots If applicable, add screenshots to help explain your problem.

    opened by hazcod 2
  • feat: add Playbooks/Enrich-MalwareBazaar

    feat: add Playbooks/Enrich-MalwareBazaar

    Change(s):

    • add Playbooks/Enrich-MalwareBazaar - https://bazaar.abuse.ch/api/

    Reason for Change(s):

    • Data Enrichment

    Version Updated:

    • N/A

    Testing Completed:

    • Yes

    Checked that the validations are passing and have addressed any issues that are present:

    • Yes
    Playbook 
    opened by juju4 0
  • feat: add Playbooks/Enrich-CIRCL-Intezer-Analyze

    feat: add Playbooks/Enrich-CIRCL-Intezer-Analyze

    Change(s):

    • add Playbooks/Enrich-CIRCL-Intezer-Analyze - https://www.intezer.com/blog/malware-analysis/intezer-analyze-free-community-edition /

    Reason for Change(s):

    • Data Enrichment

    Version Updated:

    • N/A

    Testing Completed:

    • Yes

    Checked that the validations are passing and have addressed any issues that are present:

    • Yes
    Playbook 
    opened by juju4 0
  • feat: add Playbooks/Enrich-CIRCL-hashlookup

    feat: add Playbooks/Enrich-CIRCL-hashlookup

    Change(s):

    • add playbook Enrich-CIRCL-hashlookup using https://circl.lu/services/hashlookup/

    Reason for Change(s):

    • Data enrichment

    Version Updated:

    • N/A

    Testing Completed:

    • Yes

    Checked that the validations are passing and have addressed any issues that are present:

    • Yes
    Playbook 
    opened by juju4 0
Owner
Microsoft Azure
APIs, SDKs and open source projects from Microsoft Azure
Microsoft Azure
Download and archive entire usenet newsgroups over NNTP.

Usenet Archiving Tool This code is for archiving Usenet discussions, not downloading files. Newsgroup posts are saved under the authors name and email

Corey White 2 Dec 23, 2021
Decentralized intelligent voting application.

DiVA Decentralized intelligent voting application. Hack the North 2021. Inspiration Following the previous US election, many voters were fearful that

Ali Shariatmadari 4 Jun 5, 2022
Intelligent Systems Project In Python

Intelligent Systems Project In Python

RLLAB 3 May 16, 2022
Intelligent Employer Profiling Platform.

Intelligent Employer Profiling Platform Setup Instructions Generating Model Data Ensure that Python 3.9+ and pip is installed. Install project depende

Harvey Donnelly 2 Jan 9, 2022
Prophet is a tool to discover resources detailed for cloud migration, cloud backup and disaster recovery

Prophet is a tool to discover resources detailed for cloud migration, cloud backup and disaster recovery

null 22 May 31, 2022
🍏 Make Thinc faster on macOS by calling into Apple's native Accelerate library

?? Make Thinc faster on macOS by calling into Apple's native Accelerate library

Explosion 81 Nov 26, 2022
Python package for reference counting native pointers

refcount master: testing: This package is primarily for managing resources in native libraries, written for instance in C++, from Python. While it boi

CSIRO Hydroinformatics 2 Nov 3, 2022
A PowSyBl and Python integration based on GraalVM native image

PyPowSyBl The PyPowSyBl project gives access PowSyBl Java framework to Python developers. This Python integration relies on GraalVM to compile Java co

powsybl 23 Dec 14, 2022
A python script for combining multiple native SU2 format meshes into one mesh file for multi-zone simulations.

A python script for combining multiple native SU2 format meshes into one mesh file for multi-zone simulations.

MKursatUzuner 1 Jan 20, 2022
HatAsm - a HatSploit native powerful assembler and disassembler that provides support for all common architectures

HatAsm - a HatSploit native powerful assembler and disassembler that provides support for all common architectures.

EntySec 8 Nov 9, 2022
This code makes the logs provided by Fiddler proxy of the Google Analytics events coming from iOS more readable.

GA-beautifier-iOS This code makes the logs provided by Fiddler proxy of the Google Analytics events coming from iOS more readable. To run it, create a

Rafael Machado 3 Feb 2, 2022
Ballcone is a fast and lightweight server-side Web analytics solution.

Ballcone Ballcone is a fast and lightweight server-side Web analytics solution. It requires no JavaScript on your website. Screenshots Design Goals Si

Dmitry Ustalov 49 Dec 11, 2022
Active Transport Analytics Model: A new strategic transport modelling and data visualization framework

{ATAM} Active Transport Analytics Model Active Transport Analytics Model (“ATAM”

ATAM Analytics 2 Dec 21, 2022
Audio-analytics for music-producers! Automate tedious tasks such as musical scale detection, BPM rate classification and audio file conversion.

Click here to be re-directed to the Beat Inspect Streamlit Web-App You are a music producer? Let's get in touch via LinkedIn Fundamental Analytics for

Stefan Rummer 11 Dec 27, 2022
A git extension for seeing your Cloud Build deployment

A git extension for seeing your Cloud Build deployment

Katie McLaughlin 13 May 10, 2022
Security-related flags and options for C compilers

Getting the maximum of your C compiler, for security

null 135 Nov 11, 2022
Programming labs for 6.S060 (Foundations of Computer Security).

6.S060 Labs This git repository contains the code for the labs in 6.S060. In these labs, you will add a series of security features to a photo-sharing

MIT PDOS 10 Nov 2, 2022
This application demonstrates IoTVAS device discovery and security assessment API integration with the Rapid7 InsightVM.

Introduction This repository hosts a sample application that demonstrates integrating Firmalyzer's IoTVAS API with the Rapid7 InsightVM platform. This

Firmalyzer BV 4 Nov 9, 2022
Linux Security and Monitoring Scripts

Linux Security and Monitoring Scripts These are a collection of security and monitoring scripts you can use to monitor your Linux installation for sec

Andre Pawlowski 65 Aug 27, 2022