A curated list of amazingly awesome Cybersecurity datasets

Overview

Awesome-Cybersecurity-Datasets

A curated list of amazingly awesome Cybersecurity datasets.

Please contribute to this list with new datasets by sending me a pull request or by contacting me at @santiagohramos.

Happy learning!

Table of contents

Datasets

Network traffic

  • Unified Host and Network Dataset - The Unified Host and Network Dataset is a subset of network and computer (host) events collected from the Los Alamos National Laboratory enterprise network over the course of approximately 90 days. The host event logs originated from most enterprise computers running the Microsoft Windows operating system on Los Alamos National Laboratory's (LANL) enterprise network. The network event data originated from many of the internal enterprise routers within the LANL enterprise network.
  • Comprehensive, Multi-Source Cyber-Security Events - This data set represents 58 consecutive days of de-identified event data collected from five sources within Los Alamos National Laboratory's corporate, internal computer network.
  • User-Computer Authentication Associations in Time - This anonymized data set encompasses 9 continuous months and represents 708,304,516 successful authentication events from users to computers collected from the Los Alamos National Laboratory (LANL) enterprise network.
  • Canadian Institute for Cybersecurity datasets - Canadian Institute for Cybersecurity datasets are used around the world by universities, private industry and independent researchers.
  • KDD Cup 1999 Data - This database contains a standard set of data to be audited, which includes a wide variety of intrusions simulated in a military network environment.
  • 2017-SUEE-data-set - The data sets contain traffic in and out of the web server of the Student Union for Electrical Engineering (Fachbereichsvertretung Elektrotechnik) at Ulm University. Internal hosts are hosts from within the university network, some of them are cable bound, others connect through one of two wifi services on campus (eduroam and welcome). The data was mixed with attack traffic.
  • CTU-13 Dataset - A Labeled Dataset with Botnet, Normal and Background traffic.
  • PCAP files - Malware Traffic, Network Forensics, SCADA/ICS Network Captures, Packet Injection Attacks / Man-on-the-Side Attacks...
  • pcapt - Big repository of PCAP files.
  • Project Sonar - Project Sonar produces multiple UDP datasets every month. This data is gathered by sending protocol-specific UDP probes across the entire IPv4 address space. The types of probes sent each week continues to expand as the project matures.
  • IoT devices captures - This dataset represents the traffic emitted during the setup of 31 smart home IoT devices of 27 different types (4 types are represented by 2 devices each). Each setup was repeated at least 20 times per device-type.

Malware

  • UNSW-NB15 data set - This data set has nine families of attacks, namely, Fuzzers, Analysis, Backdoors, DoS, Exploits, Generic, Reconnaissance, Shellcode and Worms. The Argus, Bro-IDS tools are utilised and twelve algorithms are developed to generate totally 49 features with the class label.
  • Malware Training Sets - Today (please refers to blog post date) the collected classified datasets is composed by the following samples: APT1 292 Samples, Crypto 2024 Samples, Locker 434 Samples, Zeus 2014 Samples
  • The Drebin Dataset - The dataset contains 5,560 applications from 179 different malware families. The samples have been collected in the period of August 2010 to October 2012 and were made available to us by the MobileSandbox project.
  • Stratosphere IPS - Malware captures, Normal captures, mixed captures...
  • Microsoft Malware Classification Challenge - You are provided with a set of known malware files representing a mix of 9 different families. Each malware file has an Id, a 20 character hash value uniquely identifying the file, and a Class, an integer representing one of 9 family names.

Software

  • Javascript Vulnerability dataset - Dataset constructed from the vulnerability information in public databases of the Node Security Project and the Snyk platform, and code fixing patches from GitHub.

WebApps

  • West Point NSA Data Sets - Snort Intrusion Detection Log. Domain Name Service Logs. Web Server Logs. Log Server Aggregate Log.
  • Web Attack Payloads - A collection of web attack payloads.
  • Machine-Learning-driven-Web-Application-Firewall - Set of good and bad queries to a web application firewall.
  • Internet-Wide Scan Data Repository - The Censys Projects publishes daily snapshots of what we know about each IPv4 host, Alexa Top Million website, and known X.509 certificate. These datasets contain structured, non-ephemeral JSON records that identify a host's configuration.
  • 500K HTTP Headers - Recently we crawled the Top 500K sites (as ranked by Alexa). Following requests from readers we are making available the HTTP Headers for research purposes.
  • HTTP DATASET CSIC 2010 - The HTTP dataset CSIC 2010 contains thousands of web requests automatically generated. It can be used for the testing of web attack protection systems. It was developed at the Information Security Institute of CSIC (Spanish Research National Council).
  • ISOT datasets - The ISOT Lab has collected through different projects various datasets some of which are available for public sharing. ISOT Web Interactions Dataset (Mouse/Keystroke/Site Actions), ISOT Botnet Dataset...
  • Web Logs Secrepo - Web logs generated by secrepo community and secrepo web application
  • Common Crawl - The Common Crawl corpus contains petabytes of data collected over the last 7 years. It contains raw web page data, extracted metadata and text extractions.
  • Website Classification Dataset - The entire selective archive is manually curated, including classification of sites into a two-tiered subject hierarchy. We have made this manually-generated classification information available as an open dataset, in tab-separated column format.
  • AZSecure-data - The AZSecure-data PORTAL currently provides access to Web forums, Internet phishing websites, Twitter data, and other data.

URLs & Domain Names

  • Malicious URLs Dataset - The data set consists of about 2.4 million URLs (examples) and 3.2 million features.
  • cybercrime-tracker - List of labeled malicious URLs.
  • Malware Domain List - Malware Domain List.
  • ZeuS Tracker - ZeuS Tracker tracks ZeuS Command&Control servers (hosts) around the world and provides you a domain- and a IP-blocklist.
  • Feodo Tracker - List of Feodo botnet C&C servers
  • Ransomware Tracker - Ransomware Tracker offers various types of blocklists that allows you to block Ransomware botnet C&C traffic.
  • URLhaus - URLhaus is a project from abuse.ch with the goal of sharing malicious URLs that are being used for malware distribution.
  • Alexa Top 1 Million - CSV dataset with the most popular sites by Alexa.
  • OpenDNS Top Domains List - The OpenDNS Top Domains List is the top 10,000 domain names our resolvers all over the globe are receiving queries for, sorted by popularity.
  • The Majestic Million - The million domains we find with the most referring subnets.
  • StopForumSpam - The data provided here represents what we believe will only ever ben used to abuse. IP Addresses, domains and usernames listed here will be returned in API results as "blacklisted".

Host

  • The ADFA Intrusion Detection Datasets - This dataset provides a contemporary Linux dataset for evaluation by traditional HIDS. This dataset provides a contemporary Windows dataset for evaluation by HIDS.
  • Unified Host and Network Dataset - The Unified Host and Network Dataset is a subset of network and computer (host) events collected from the Los Alamos National Laboratory enterprise network over the course of approximately 90 days. The host event logs originated from most enterprise computers running the Microsoft Windows operating system on Los Alamos National Laboratory's (LANL) enterprise network. The network event data originated from many of the internal enterprise routers within the LANL enterprise network.
  • Public Security Log Sharing Site - This site contains various free shareable log samples from various systems, security and network devices, applications, etc. The logs are collected from real systems, some contain evidence of compromise and other malicious activity. Wherever possible, the logs are NOT sanitized, anonymized or modified in any way (just as they came from the logging system).
  • Aktaion2 Data - The project is meant to be a learning/teaching tool on how to blend multiple security signals and behaviors into an expressive framework for intrusion detection.

Email

Fraud

  • Credit Card Fraud - The datasets contains transactions made by credit cards in September 2013 by european cardholders. This dataset presents transactions that occurred in two days, where we have 492 frauds out of 284,807 transactions. The dataset is highly unbalanced, the positive class (frauds) account for 0.172% of all transactions.

Honeypots

  • DDS Dataset Collection - A tar/gzip CSV file from a collection of AWS honeypots. A zip CSV file of domains and a high level classification of dga or legit along with a subclass of either legit, cryptolocker, gox or newgoz.
  • Threat_Research - Centralized repository to dump threat research data gathered from my network of honeypots.

Binaries

  • The ember dataset - The ember dataset is a collection of 1.1 million sha256 hashes from PE files that were scanned sometime in 2017. This repository makes it easy to reproducibly train the benchmark model, extend the provided feature set, or classify new PE files with the benchmark model.

Phishing

  • Phishing Websites Data Set - In this dataset, we shed light on the important features that have proved to be sound and effective in predicting phishing websites. In addition, we propose some new features.

Passwords

MISC

  • SecRepo - Samples of Security Related Data.
  • PANDA SHARE - This site stores recordings of executions produced by the PANDA dynamic analysis platform. The goal is to make dyanamic analysis repeatable. Any analysis dynamic analysis, run on the same replay, will produce the same results.
  • SHERLOCK - The dataset is essentially a massive time-series dataset spanning nearly every single kind of software and hardware sensor that can be sampled from a Samsung Galaxy S5 smartphone, without root privileges. The dataset contains over 600 billion data points in over 10 billion data records.
  • WerdLists - Wordlists, Dictionaries and Other Data Sets for Writing Software Security Test Cases.
You might also like...
Password List Maker

Red-Key Red-Key Password List Maker Version 1.1.2 Created By FireKing255 -=Features=- Create Random Password List Create Password List Create Password

Fast python tool to test apache path traversal CVE-2021-41773 in a List of url

CVE-2021-41773 Fast python tool to test apache path traversal CVE-2021-41773 in a List of url Usage :- create a live urls file and use the flag "-l" p

This program will brute force any Instagram account you send it its way given a list of proxies.

Instagram Bruter This program will brute force any Instagram account you send it its way given a list of proxies. NOTICE I'm no longer maintaining thi

List of S3 Hacks

s3-leaks List of AWS S3 Leaks Feel free to send in a PR if you know of other leaks Date Description Notes Aug2020 S3 bucket mess up exposed 182GB of s

 Simple Python 3 script to detect the
Simple Python 3 script to detect the "Log4j" Java library vulnerability (CVE-2021-44228) for a list of URL with multithreading

log4j-detect Simple Python 3 script to detect the "Log4j" Java library vulnerability (CVE-2021-44228) for a list of URL with multithreading The script

Python script that sends CVE-2021-44228 log4j payload requests to url list

scan4log4j Python script that sends CVE-2021-44228 log4j payload requests to url list [VERY BETA] using Supply your url list to urls.txt Put your payl

Simple Python 3 script to detect the
Simple Python 3 script to detect the "Log4j" Java library vulnerability (CVE-2021-44228) for a list of URL with multithreading

log4j-detect Simple Python 3 script to detect the "Log4j" Java library vulnerability (CVE-2021-44228) for a list of URL with multithreading The script

Password List Creator Simple !

Password List Creator Simple !

Uses Sharphound, Bloodhound and Neo4j to produce an actionable list of attack paths for targeted remediation.
Uses Sharphound, Bloodhound and Neo4j to produce an actionable list of attack paths for targeted remediation.

GoodHound ______ ____ __ __ / ____/___ ____ ____/ / / / /___ __ ______ ____/ / / / __/ __ \/ __ \/ __

Comments
  • Awesome Lint

    Awesome Lint

    Are you interested to have this list connected with Sindre's 'masterlist' of awesome lists? If so, I would be open to making a PR for linting it, and making it ready to PR to https://github.com/sindresorhus/awesome

    opened by NewAlexandria 1
  • Currentness / relevancy question

    Currentness / relevancy question

    As someone who hasn't used these kinds of data in real operations, how can I/anyone know whether these datasets are meaningfully up-to-date?

    Or, is that question not as relevant as having the historical existence of such public data  — and from there deciding how to use it, or whether to look for replacements?

    opened by NewAlexandria 0
Owner
null
MITMSDR for INDIAN ARMY cybersecurity hackthon

There mainly three things here: MITMSDR spectrum Manual reverse shell MITMSDR Installation Clone the project and run the setup file: ./setup One of th

null 2 Jul 26, 2022
OSINT Cybersecurity Tools

OSINT Cybersecurity Tools Welcome to the World of OSINT: An ongoing collection of awesome tools and frameworks, best security software practices, libr

Paul Veillard, P. Eng 7 Jul 1, 2022
Seamless deployment and management of cybersecurity solutions 🏗️

Description ??️ Background ???? Vision ?? Concepts ?? Solutions' Lifecycle. Operations ⭕ Functionalities ?? Supported Cybersecurity Solutions ?? Insta

MutableSecurity 36 Nov 10, 2022
Mert Güvençli 142 Jan 5, 2023
Consolidating and extending hosts files from several well-curated sources. You can optionally pick extensions to block pornography, social media, and other categories.

Take Note! With the exception of issues and PRs regarding changes to hosts/data/StevenBlack/hosts, all other issues regarding the content of the produ

Steven Black 22.1k Jan 2, 2023
This is python script that will extract the functions call in all used DLL in an executable and then provide a mapping of those functions to the attack classes defined and curated malapi.io.

F2Amapper This is python script that will extract the functions call in all used DLL in an executable and then provide a mapping of those functions to

Ajit Kumar 3 Sep 3, 2022
Password list generator for password spraying - prebaked with goodies

Generates permutations of Months, Seasons, Years, Sports Teams (NFL, NBA, MLB, NHL), Sports Scores, "Password", and even Iterable Keyspaces of a specified size.

Casey Erdmann 65 Dec 22, 2022
zip-brute Zip File Password Cracking with Using Password List

Zip brute is a python script that cracks zip that are password protected using a wordlist dictionary.

AnonyminHack5 13 Nov 3, 2022
A small Python Script To get all levels of subdomains from a list

getlevels A small Python Script To get all levels of subdomains Easily get 1st level, 2nd level, 3rd level, 4th level .... nth level subdomains Usag

null 9 Feb 15, 2022
DependConfusion-X Tool is written in Python3 that scans and monitors list of hosts for Dependency Confusion

DependConfusion-X Tool is written in Python3 which allows security researcher/bug bounty hunter to scan and monitor list of hosts for Dependency Confusion.

Ali Fathi Ali Sawehli 4 Dec 21, 2021