A high-performance DNS stub resolver for bulk lookups and reconnaissance (subdomain enumeration)

Overview

MassDNS

A high-performance DNS stub resolver

MassDNS is a simple high-performance DNS stub resolver targeting those who seek to resolve a massive amount of domain names in the order of millions or even billions. Without special configuration, MassDNS is capable of resolving over 350,000 names per second using publicly available resolvers.

Contributors

Compilation

Clone the git repository and cd into the project root folder. Then run make to build from source. If you are not on Linux, run make nolinux. On Windows, the Cygwin packages gcc-core, git and make are required.

Usage

Usage: ./bin/massdns [options] [domainlist]
  -b  --bindto           Bind to IP address and port. (Default: 0.0.0.0:0)
      --busy-poll        Use busy-wait polling instead of epoll.
  -c  --resolve-count    Number of resolves for a name before giving up. (Default: 50)
      --drop-group       Group to drop privileges to when running as root. (Default: nogroup)
      --drop-user        User to drop privileges to when running as root. (Default: nobody)
      --extended-input   Input names are followed by a space-separated list of resolvers.
                         These are used before falling back to the resolvers file.
      --filter           Only output packets with the specified response code.
      --flush            Flush the output file whenever a response was received.
  -h  --help             Show this help.
      --ignore           Do not output packets with the specified response code.
  -i  --interval         Interval in milliseconds to wait between multiple resolves of the same
                         domain. (Default: 500)
  -l  --error-log        Error log file path. (Default: /dev/stderr)
      --norecurse        Use non-recursive queries. Useful for DNS cache snooping.
  -o  --output           Flags for output formatting.
      --predictable      Use resolvers incrementally. Useful for resolver tests.
      --processes        Number of processes to be used for resolving. (Default: 1)
  -q  --quiet            Quiet mode.
      --rand-src-ipv6    Use a random IPv6 address from the specified subnet for each query.
      --rcvbuf           Size of the receive buffer in bytes.
      --retry            Unacceptable DNS response codes.
                         (Default: All codes but NOERROR or NXDOMAIN)
  -r  --resolvers        Text file containing DNS resolvers.
      --root             Do not drop privileges when running as root. Not recommended.
  -s  --hashmap-size     Number of concurrent lookups. (Default: 10000)
      --sndbuf           Size of the send buffer in bytes.
      --status-format    Format for real-time status updates, json or ansi (Default: ansi)
      --sticky           Do not switch the resolver when retrying.
      --socket-count     Socket count per process. (Default: 1)
  -t  --type             Record type to be resolved. (Default: A)
      --verify-ip        Verify IP addresses of incoming replies.
  -w  --outfile          Write to the specified output file instead of standard output.

Output flags:
  L - domain list output
  S - simple text output
  F - full text output
  B - binary output
  J - ndjson output

Advanced flags for the domain list output mode:
  0 - Include NOERROR replies without answers.

Advanced flags for the simple output mode:
  d - Include records from the additional section.
  i - Indent any reply record.
  l - Separate replies using a line feed.
  m - Only output reply records that match the question name.
  n - Include records from the answer section.
  q - Print the question.
  r - Print the question with resolver IP address, Unix timestamp and return code prepended.
  s - Separate packet sections using a line feed.
  t - Include TTL and record class within the output.
  u - Include records from the authority section.

Advanced flags for the ndjson output mode:
  e - Write a record for each terminal query failure.

For a detailed description of the command line interface, please consult the man page using man ./doc/massdns.1.

Example

Resolve all AAAA records from domains within domains.txt using the resolvers within resolvers.txt in lists and store the results within results.txt:

$ ./bin/massdns -r lists/resolvers.txt -t AAAA domains.txt > results.txt

This is equivalent to:

$ ./bin/massdns -r lists/resolvers.txt -t AAAA -w results.txt domains.txt

Example output

By default, MassDNS will output response packets in text format which looks similar to the following:

;; Server: 77.41.229.2:53
;; Size: 93
;; Unix time: 1513458347
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51298
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 0

;; QUESTION SECTION:
example.com. IN A

;; ANSWER SECTION:
example.com. 45929 IN A 93.184.216.34

;; AUTHORITY SECTION:
example.com. 24852 IN NS b.iana-servers.net.
example.com. 24852 IN NS a.iana-servers.net.

The resolver IP address is included in order to make it easier for you to filter the output in case you detect that some resolvers produce bad results.

Resolving

The repository includes the file resolvers.txt consisting of a filtered subset of the resolvers provided by the subbrute project. Please note that the usage of MassDNS may cause a significant load on the used resolvers and result in abuse complaints being sent to your ISP. Also note that the provided resolvers are not guaranteed to be trustworthy. The resolver list is currently outdated with a large share of resolvers being dysfunctional.

MassDNS's DNS implementation is currently very sporadic and only supports the most common records. You are welcome to help changing this by collaborating.

PTR records

MassDNS includes a Python script allowing you to resolve all IPv4 PTR records by printing their respective queries to the standard output.

$ ./scripts/ptr.py | ./bin/massdns -r lists/resolvers.txt -t PTR -w ptr.txt

Please note that the labels within in-addr.arpa are reversed. In order to resolve the domain name of 1.2.3.4, MassDNS expects 4.3.2.1.in-addr.arpa as input query name. As a consequence, the Python script does not resolve the records in an ascending order which is an advantage because sudden heavy spikes at the name servers of IPv4 subnets are avoided.

Reconnaissance by brute-forcing subdomains

Perform reconnaissance scans responsibly and adjust the -s parameter to not overwhelm authoritative name servers.

Similar to subbrute, MassDNS allows you to brute force subdomains using the included subbrute.py script:

$ ./scripts/subbrute.py lists/names.txt example.com | ./bin/massdns -r lists/resolvers.txt -t A -o S -w results.txt

As an additional method of reconnaissance, the ct.py script extracts subdomains from certificate transparency logs by scraping the data from crt.sh:

$ ./scripts/ct.py example.com | ./bin/massdns -r lists/resolvers.txt -t A -o S -w results.txt

The files names.txt and names_small.txt, which have been copied from the subbrute project, contain names of commonly used subdomains. Also consider using Jason Haddix' subdomain compilation with over 1,000,000 names.

Screenshots

Screenshot

Security

MassDNS does not require root privileges and will therefore drop privileges to the user called "nobody" by default when being run as root. If the user "nobody" does not exist, MassDNS will refuse execution. In this case, it is recommended to run MassDNS as another non-privileged user. The privilege drop can be circumvented using the --root argument which is not recommended. Also note that other branches than master should not be used in production at all.

Practical considerations

Performance tuning

MassDNS is a simple single-threaded application designed for scenarios in which the network is the bottleneck. It is designed to be run on servers with high upload and download bandwidths. Internally, MassDNS makes use of a hash map which controls the concurrency of lookups. Setting the size parameter -s hence allows you to control the lookup rate. If you are experiencing performance issues, try adjusting the -s parameter in order to obtain a better success rate.

Rate limiting evasion

In case rate limiting by IPv6 resolvers is a problem, have a look at the freebind project including packetrand, which will cause each packet to be sent from a different IPv6 address from a routed prefix.

Result authenticity

If the authenticity of results is highly essential, you should not rely on the included resolver list. Instead, set up a local unbound resolver and supply MassDNS with its IP address. In case you are using MassDNS as a reconnaissance tool, you may wish to run it with the default resolver list first and re-run it on the found names with a list of trusted resolvers in order to eliminate false positives.

In case you are enumerating subdomains for a single name, e.g. for example.com, you may want to simply leave out third-party resolvers. In this case, you can directly probe the authoritative nameservers like so:

$ ./bin/massdns -r <(./scripts/auth-addrs.sh example.com) --norecurse -o Je example-com-subdomains.txt > results.txt

Todo

  • Prevent flooding resolvers which are employing rate limits or refusing resolves after some time
  • Implement bandwidth limits
  • Employ cross-resolver checks to detect DNS poisoning and DNS spam (e.g. Level 3 DNS hijacking)
  • Add wildcard detection for reconnaissance
  • Improve reconnaissance reliability by adding a mode which re-resolves found domains through a list of trusted (local) resolvers in order to eliminate false positives
  • Detect optimal concurrency automatically
  • Parse the command line properly and allow the usage/combination of short options without spaces
Comments
  • many fake records

    many fake records

    Hi,

    I was running your tool on my vps , it was running fine for quite sometime but yesterday I installed it on new VPS and I am getting weird result. there is one weird server(142.54.173.92) which is sending fake records , this server does not listed on resolvers.txt(using from your repo). It looks like something is wrong. I have tried to traceroute this ip but last 10 hopes are not visible and i have also tried to see code massdns but it is in compiled format.

    I have copied your repo from my local system to vps wihtout using make command and once i received weird result , i delete entire folder and clone from online using git clone command and use make command .......still i receive this weird records from this weird server ...... It looks like two possible scenario , one of the resolver from list is sending records to this fake dns server(142.54.173.92) which is sending fake records as result. second could be somewhere in my vps , this records are stored due to using existing copy of the folder and re installation is not removing it .....i grep dns but i could not find any susceptible file

    ./subbrute.py names.txt example.com | ./bin/massdns -r resolvers.txt -t A -a -o -c 100 -w 18_dec_1.txt -

    [18_dec_1.txt]

    Note : i have changed result to example.com in above command and output file attached. I really appreciate if you can check on your side that if you are receiving this server in output or something wrong with massdns compiled file or can you provide me any lead to resolve this issue ?

    Thanks, Anis

    opened by yam23 16
  • Installation error

    Installation error

    HI,

    I am getting the error below while compiling the file after cloning from git. Please help.

    $ make mkdir -p bin cc -O3 -std=c11 -Wall -fstack-protector-strong *.c -o bin/massdns -ldl In file included from main.c:7: ./massdns.h:7:10: fatal error: 'sys/epoll.h' file not found #include <sys/epoll.h> ^~~~~~~~~~~~~ 1 error generated. make: *** [all] Error 1

    opened by h1hakz 12
  • Emtpy results.txt MACOSX

    Emtpy results.txt MACOSX

    Hello,

    I installed massdns via brew in MacosX. It installed successfully, but when i run any command the results.txt is empty. I have tried the --root option as well.

    massdns -r resolvers.txt -t AAAA -w results.txt urls.txt massdns -r resolvers.txt -t AAAA -w results.txt urls.txt --root massdns -r resolvers.txt -t AAAA -w results.txt urls.txt --root --error-log error.txt --busy-poll

    resolvers.txt contains only 8.8.8.8 urls.txt contains only google.com

    OUTPUT:

    Processed queries: 1 Received packets: 0 Progress: 100.00% (00 h 00 min 25 sec / 00 h 00 min 25 sec) Current incoming rate: 0 pps, average: 0 pps Current success rate: 0 pps, average: 0 pps Finished total: 1, success: 0 (0.00%) Mismatched domains: 0 (0.00%), IDs: 0 (0.00%) Failures: 0: 0.00%, 1: 0.00%, 2: 0.00%, 3: 0.00%, 4: 0.00%, 5: 0.00%, 6: 0.00%, 7: 0.00%, 8: 0.00%, 9: 0.00%, 10: 0.00%, 11: 0.00%, 12: 0.00%, 13: 0.00%, 14: 0.00%, 15: 0.00%, 16: 0.00%, 17: 0.00%, 18: 0.00%, 19: 0.00%, 20: 0.00%, 21: 0.00%, 22: 0.00%, 23: 0.00%, 24: 0.00%, 25: 0.00%, 26: 0.00%, 27: 0.00%, 28: 0.00%, 29: 0.00%, 30: 0.00%, 31: 0.00%, 32: 0.00%, 33: 0.00%, 34: 0.00%, 35: 0.00%, 36: 0.00%, 37: 0.00%, 38: 0.00%, 39: 0.00%, 40: 0.00%, 41: 0.00%, 42: 0.00%, 43: 0.00%, 44: 0.00%, 45: 0.00%, 46: 0.00%, 47: 0.00%, 48: 0.00%, 49: 0.00%, 50: 100.00%, Response: | Success: | Total: OK: | 0 ( 0.00%) | 0 ( 0.00%) NXDOMAIN: | 0 ( 0.00%) | 0 ( 0.00%) SERVFAIL: | 0 ( 0.00%) | 0 ( 0.00%) REFUSED: | 0 ( 0.00%) | 0 ( 0.00%) FORMERR: | 0 ( 0.00%) | 0 ( 0.00%)

    opened by Ncoder2 9
  • Limits can slow massdns?

    Limits can slow massdns?

    Hello! Run as root --root -s 5000, but still uses about 40 mbps (half of the channel). Virtualbox ubuntu 16.04: 64 bit, 12gb ram 3 core, Connection PPPoe.

    Maybe problem with limits: ?

    root@ri:~# ulimit -a
    core file size          (blocks, -c) 0
    data seg size           (kbytes, -d) unlimited
    scheduling priority             (-e) 0
    file size               (blocks, -f) unlimited
    pending signals                 (-i) 64040
    max locked memory       (kbytes, -l) 64
    max memory size         (kbytes, -m) unlimited
    open files                      (-n) 100000
    pipe size            (512 bytes, -p) 8
    POSIX message queues     (bytes, -q) 819200
    real-time priority              (-r) 0
    stack size              (kbytes, -s) 8192
    cpu time               (seconds, -t) unlimited
    max user processes              (-u) 64040
    virtual memory          (kbytes, -v) unlimited
    file locks                      (-x) unlimited
    

    Cheers,

    opened by Roundik 6
  • Possible Encoding Bug

    Possible Encoding Bug

    I am using massdns compiled on this commit:

    d8b911d26fe6af80ef2d2647220e9a48ac832d3f

    I am using ndjson output mode. And when I run massdns for RRSIG record type, I get this value. Domain name is swordsec.com

     {'asset_name_or_value':
     {'fqdn': 'swordsec.com.',
     'record_type': 'RRSIG',
     'value': '.\\x0d\\x02\\x00\\x00\\x0e\\xcd`\\x1e\\x96\\x88`\\x1b\\xd7.\\x86\\xc9\\x08swordsec\\x03com\\x002\\xec\\xad\\x05\\x140\\x06\\x12\\x9c\\x07s\\xd3\\x89v\\xdf\\xabI^\\xb6\\x0at\\x8a\\xe1\\xa0\\xc691\\x9a\\x06\\x90\\x83\\xf5\\xb0\\x13\\xc4d\\x9d\\x99\\x05+\\x9c4ei\\x0e \\xb0S\\x99\\x8f\\xd4]f\\xc3\\xcc\\xc2W_\\xf6!\\xbdA\\x01\\xdb',
     'resolver': '64.6.64.6:53',
     'status': 'NOERROR'},
     time_now': '2021-02-05 12:15:54.256611'}
    
    

    As you can see there is "swordsec\x03com" in value, so I guess the correct encoding would result "swordsec.com" in that part. I think there might be a encoding error. I also noticed similar encoding in DNSKEY and CDNSKEY records too.

     {'asset_name_or_value': 
    {'fqdn': 'swordsec.com.',
     'record_type': 'DNSKEY',
     'value': '\\x00.\\x0d\\xa0\\x93.\\x11,\\xf9\\x13\\x88\\x18\\xcd/\\xea\\xe9p\\xeb\\xbdMj0\\xf6.\\x8c%\\xb3%\\xa3\\x9a\\xbb\\xc5.\\x11\\x97\\xaa\\x09\\x82\\x83\\xe5\\xaa\\xf4!\\x17|*\\xa5\\xd7\\x14\\x99*\\x99W\\xd1\\xbc\\xc1\\x8f\\x98\\xcdq\\xf1\\xf1\\x80ke\\xe1H',
     'resolver': '77.88.8.8:53',
     'status': 'NOERROR'},
     ''time_now': '2021-02-05 12:15:54.295338'}
    
    {'asset_name_or_value': {
    'fqdn': 'swordsec.com.',
     'record_type': 'CDNSKEY',
     'value':"\\x01.\\x0d\\x99\\xdb.\\xc1L\\xab\\xdc3\\xd6\\xd7}\\xa6:/\\x15\\xf7\\x11\\x12XO#N\\x8d\\x1d\\xc4(\\xe3\\x9e\\x8aJ\\x97\\xe1\\xaa'\\x1aU]\\xc9\\x07\\x01\\xe1~*LKo\\x12.|2\\xd4OJ\\xc0+\\xd8\\x94\\xcf-.\\xe7w\\x8a\\x19", 
    'resolver': '8.8.4.4:53', 
    'status': 'NOERROR'},
    'time_now': '2021-02-05 12:15:54.268398'}
    
    
    opened by srknzl 4
  • Poor performance on Digital Ocean droplet

    Poor performance on Digital Ocean droplet

    Massdns used to work perfectly for at least 1 year on Digital Ocean Droplets (DOD) with performance ~22k pps. Recently after I apt update && apt upgrade my Ubuntu machine on DOD the massdns became unusable with performance ~50-300 pps. Now massdns often completely freezes SSH connection so hard reset via DOD cPanel is needed. It has never happened before.

    Currently I'm using massdns from my home connection (150Mb/s) with the same public resolvers which I used at DOD and I'm getting ~8-10k pps. I tried to create fresh DOD and clean install massdns but it didn't help. I even opened support ticket a explained the problem to the DO support but got response that all networking regarding my DOD is fully functional.

    Does anyone have similar issues on latest Ubuntu machines or/and DODs?

    opened by MMquant 4
  • Feedback & list of issues I am facing

    Feedback & list of issues I am facing

    Greetings!! Thanks for this wonderful tool. I am facing following issues while using this tool:

    1. The tool is not scanning all domains (when given list of 10000 domains hardly scans 1000 of them which I verified by grep "QUESTION SECTION" in output file generated with -o F switch). Although above problem is solved for now by using -s 1000(default is 10000) so I think you should change the default value to 1000.
    2. No explicit option to grep Domain having NXDOMAIN status since it is required for finding domains vulnerable to subdomain takeover(like Azure). Although above problem can be solved by : ./massdns -q -t A -r /opt/resolvers.txt domain_list.txt -o F -w output_massdns.txt cat output_massdns.txt | grep NXDOMAIN -A 7 | grep CNAME But it is not efficient.
    3. Getting different result when using -o F and then -w and only using -w (without any output flag) Although result should come same.So please tell me if -o F append some extra details which are drop when we are not using it.
    4. Include Option which do below operation: Take list of domains and output only those domain names whose IP can be resolved. (Should also exclude those domains which have status NO ERROR but only have CNAME and no IP) Since this list can be passed further to httprobe(or CURL) or other tools which do some active scanning.(As list fetched above will assure that the domains' IP recored exist).
    opened by mansoorr123 4
  • Benchmark on ZDNS with MassDNS?

    Benchmark on ZDNS with MassDNS?

    ZDNS was another tool to lookup dns records which was developed by ZMap Team. Have you ever do some comparison on ZMap with MassDNS. I wonder which is better for subdomain enumerations.

    opened by jkryanchou 4
  • Massdns does not respect concurrent lookups parameter

    Massdns does not respect concurrent lookups parameter

    Hi there,

    When setting a low number of concurrent lookups with a short list of resolvers, massdns doesn't cap the concurrent lookups and use a much higher throughput.

    For example, the following command : massdns -t A -s 10 -r resolvers.txt -o S -w output.txt domains.txt

    with a "resolvers.txt" file containing 1 resolver will execute around 500 concurrent lookups. The same command with a "resolvers.txt" file containing about 30 resolvers will execute around 150 concurrent lookups.

    If I increase the --hashmap-size number, the multiplication factor remains the same. If I use the whole list of resolvers from https://public-dns.info/, the number of concurrent lookups I've set is respected.

    I suspect this is related to how the hashmap works and I'm not sure it's easily fixable.

    opened by h49nakxs 3
  • Add epoch timestamps to responses in NDJSON output format (app start only, rx only, or rx+tx)

    Add epoch timestamps to responses in NDJSON output format (app start only, rx only, or rx+tx)

    I have a workflow where I often have batches of massdns NDJSON files and batches of data from various other tools. The challenge I'm having right now is ensuring that the most recent DNS responses are used when merging/joining these datasets when there are duplicate questions with different answers- due to a change in the zone file that occurred some time between the two sessions that generated the batches. In some cases, massdns NDJSON files have been concatenated, or don't have accurate mtime/ctime. So I'm kind of helpless there

    The simple solution to this is in line with what other similar tools do- stamp an epoch timestamp into the record

    For my purposes, it would be a huge help even if it was just a fixed time that was retrieved once and stamped into every NDJSON row, grabbed with time() at startup

    The most correct and/or powerful/broadly useful implementation would probably be to have both a tx and rx stamp, as two logically separate fields. Something like:

    {
      "name": "www.bah.com.",
      "type": "A",
      "class": "IN",
      "status": "NOERROR",
      "tx_ts": 1614783271,
      "rx_ts": 1614783273,
      "data": {
        "answers": [
          {
            "ttl": 300,
            "type": "A",
            "class": "IN",
            "name": www.bah.com,
            "data": "1.2.3.4"
          }
        ],
      },
      "resolver": "1.2.3.4:53"
    }
    

    I realize that capturing the transmit stamp may require thoughtfulness to avoid impacting the transmission rate, and I also need to re-familiarize myself with the massdns code before saying this will be simple. But I put a little bit of thought into it as I was writing this up.

    A naive implementation would be continuously calling time(), which would result in a continuous flood of system calls. Not really a good solution and may have a significant performance impact. That's not what I would like to do.

    I'm guessing that the best solution would be one that avoids system calls/interrupts entirely, either by directly making use of architecture-specific instructions (e.g. rdtsc for x86/x86_64) or a libc wrapper for the same. I think (need to check) that some or all of the "clock" types in glibc clock_gettime() utilize architecture-specific instructions instead of system calls. I think most (if not all) are accessible from userspace. FWIW, I have access to ARM and PPC64 machines to check the portability of whatever seems to be a good solution, to avoid breaking people on weird architectures

    I don't expect any CPU instructions to return epoch time, but if epoch time is captured once at the start of the application, the ticks can just scaled (from nanoseconds, milliseconds, or whatever) to seconds and then subtracted from that initial epoch value without any measurable impact to performance

    I think the choices are (in order of least effort, least invasiveness to most):

    1. This is a dumb idea. Do nothing
    2. Add a single static timestamp to every NDJSON row, representing the time the process started
    3. Add a dynamically retrieved timestamp to every NDJSON row, generated at the time the response is received
    4. Add both a transmit and receive timestamp to every NDJSON row, generated before transmit and after receive

    I think #4 is doable. #2 solves my problem "well enough". I'm happy to take a shot at any of them.

    Thoughts on this?

    As always, I appreciate the time you put into this, it's a very useful tool for quite a few workflows I have

    (BTW- I'll feel very silly if this data is already captured and just not stamped into the row- I just didn't have time to look through the source before entering this)

    opened by mzpqnxow 3
  • Errors when processing large file

    Errors when processing large file

    I'm struggling to find the cause of two errors that I get when I try to run massdns on a file of generated subdomains. Most of the subdomains are being resolved but I get the error "Error sending: Bad address" and "Error sending: Address family not supported by protocol" seemingly randomly. I've changed the hashmap-size and interval but still get this error. The subdomains being resolved all appear to be valid and were generated with alt-dns. Is there anyway to find out which subdomains exactly are causing this error?

    I've also tried running it on a Ubuntu VM and Windows, both successfully compile and run, but give the same errors.

    opened by yr-or 3
  • Epoll failure on interrupted system call

    Epoll failure on interrupted system call

    I ran massdns for several hours to resolve large number of domains, and it quit prematurely with

    DEBUG Epoll failure: Interrupted system call
    

    This comes from https://github.com/blechschmidt/massdns/blob/master/src/main.c#L2013.

    I'm not expert on this, but maybe epoll_wait() failures due to signal interrupts could be ignored?

    At least that the understanding I got from e.g. https://stackoverflow.com/a/6870391

    opened by suola 3
  • No output stdout of servfails

    No output stdout of servfails

    I am trying to understand why massdns it isn't giving any output to stdout.

    I run the same exact command in one server and it runs correctly, but running it in another server gives no outout to stdout (I think it is because the version of massdns).

    I can't get massdns to print me servfail results.

    echo servfail-domain.com | massdns -r google -s 2000 --flush -o J -c 5 2> /dev/null gives no output, while in my other server I get a json containing the servfail domain.

    {"name":...

    edit: Just confirmed that it is because it was an old version of massdns. I was able to reproduce by getting the executable to the new vps, the last version of massdns won't output anythng. Any idea why it isn't working anymore in newest versions?

    opened by marcelo321 1
  • Surprising high number of lost lookups: about 20% false negatives

    Surprising high number of lost lookups: about 20% false negatives

    Hey there,

    I noticed quite a lot of lost lookups with massdns. In the example below, it roughly misses about 1900 names out of 10k domains.

    I am not sure what these false negatives come from though, apologies for not trying to find where the bug is in massdns.

    I made a screen recording https://asciinema.org/a/415235 of the reproduction steps below:

    $ docker run --rm -it --entrypoint sh blechschmidt/massdns
    /massdns # apk --no-cache add git alpine-sdk curl ipython py3-dnspython
    /massdns # git pull
    /massdns # make
    /massdns # cat > /resolvers.txt <<EOF # See https://gist.github.com/seb-elttam/af28008b092eb5bcdfede8565c55147e#file-mtresolver-py-L18
    1.1.1.1
    1.0.0.1
    8.8.8.8
    8.8.4.4
    9.9.9.10
    149.112.112.10
    94.140.14.140
    94.140.14.141
    64.6.64.6
    64.6.65.6
    77.88.8.8
    77.88.8.1
    74.82.42.42
    EOF
    
    /massdns # curl -s http://s3.amazonaws.com/alexa-static/top-1m.csv.zip | unzip -p - | cut -d, -f2- \
     | head -n 10k  > /domains.txt
    
    /massdns # time ./bin/massdns -s 10000 -q -r /resolvers.txt -o S -w /out.txt /domains.txt; \
     cat /out.txt | grep ' A [0-9]' | cut -d' ' -f1 | sort -u | wc -l
    real    0m 7.07s
    user    0m 0.09s
    sys     0m 0.21s
    7534
    
    /massdns # time ./bin/massdns -s 1000 -q -r /resolvers.txt -o S -w /out.txt /domains.txt; \
     cat /out.txt | grep ' A [0-9]' | cut -d' ' -f1 | sort -u | wc -l
    real	0m 5.03s
    user	0m 0.10s
    sys	0m 0.18s
    7911
    
    /massdns # git clone https://gist.github.com/seb-elttam/af28008b092eb5bcdfede8565c55147e /gist && cd /gist
    /gist # ipython
    Python 3.9.5 (default, May 12 2021, 20:44:22) 
    Type 'copyright', 'credits' or 'license' for more information
    IPython 7.23.1 -- An enhanced Interactive Python. Type '?' for help.
    
    In [1]: from mtresolver import *
       ...: r = resolve_hostnames('/domains.txt', 1000)
       ...: len(r)
    06:38:03 DEBUG enter
    06:38:21 DEBUG exit
    Out[1]: 9847
    In [2]: from mtresolver import *
       ...: r = resolve_hostnames('/domains.txt', 10000)
       ...: len(r)
    06:39:07 DEBUG enter
    06:39:25 DEBUG exit
    Out[2]: 9850
    
    opened by ghost 3
  • Trying to understand the output

    Trying to understand the output

    Hi,

    Thanks for this cool program :) I was trying to do a "dig" for each domain manually, and it was bloomin slow. So I'm using:

    ./massdns/bin/massdns -r massdns/lists/resolvers.txt -o S -t A massdns/domains-to-do.txt > results.txt

    Which finishes with:

    Processed queries: 239005
    Received packets: 405301
    Progress: 100.00% (00 h 00 min 57 sec / 00 h 00 min 57 sec)
    Current incoming rate: 24 pps, average: 7054 pps
    Current success rate: 1 pps, average: 4160 pps
    Finished total: 239004, success: 239002 (100.00%)
    Mismatched domains: 17138 (4.29%), IDs: 0 (0.00%)
    Failures: 0: 19.28%, 1: 16.90%, 2: 13.38%, 3: 10.62%, 4: 8.31%, 5: 6.52%, 6: 5.10%, 7: 3.94%, 8: 3.12%, 9: 2.47%, 10: 2.01%, 11: 1.59%, 12: 1.27%, 13: 1.02%, 14: 0.82%, 15: 0.67%, 16: 0.57%, 17: 0.45%, 18: 0.35%, 19: 0.31%, 20: 0.24%, 21: 0.20%, 22: 0.15%, 23: 0.13%, 24: 0.10%, 25: 0.08%, 26: 0.06%, 27: 0.05%, 28: 0.05%, 29: 0.04%, 30: 0.04%, 31: 0.02%, 32: 0.02%, 33: 0.02%, 34: 0.01%, 35: 0.02%, 36: 0.01%, 37: 0.01%, 38: 0.01%, 39: 0.01%, 40: 0.01%, 41: 0.00%, 42: 0.00%, 43: 0.00%, 44: 0.00%, 45: 0.00%, 46: 0.00%, 47: 0.00%, 48: 0.00%, 49: 0.00%, 50: 0.00%,
    Response: | Success:               | Total:
    OK:       |       229527 ( 96.04%) |       242872 ( 60.73%)
    NXDOMAIN: |         1643 (  0.69%) |         1676 (  0.42%)
    SERVFAIL: |         7832 (  3.28%) |         9740 (  2.44%)
    REFUSED:  |            0 (  0.00%) |       145641 ( 36.42%)
    FORMERR:  |            0 (  0.00%) |            0 (  0.00%)
    

    My file has 239,005 domains in - but I see quite a lot of SERVEFAIL and NXDOMAIN errors (over 9k of them). What do those mean? I'm going to run a test to compare what is outputted, vs what I'm passing in - but the outputted results.txt file is only 229,641 lines, which makes me think quite a few are missing - especially when a lot of domains have multiple lines:

    aradhana.org.uk. A 185.230.63.107
    aradhana.org.uk. A 185.230.63.186
    aradhana.org.uk. A 185.230.63.171
    

    Thanks in advance!

    Andy

    opened by youradds 2
  • Different results in output -o S and -o J (simple and json)

    Different results in output -o S and -o J (simple and json)

    Hello,

    After 2 hours of testing I found out that i am getting different results from simple output that json output and it is getting me crazy.

    So i am trying to get NOERROR responses like:

    ...-o J 2> /dev/null | jq -r 'select(.status=="NOERROR") .name' and I get like 50 results. with only 6 being valid (only 6 of the total are valid).

    Running the same command but like -o S 2> /dev/null | awk '{print $1}' | anew and I get only the 6 valid, and the command is working as expected.

    Then I analyzed the json output and noticed that I am getting a looot of noerror responses that are incorrect, but for some reason that isn't happening with simple output.

    I made the tests against same hosts and with the same resolvers countless of times and getting really different responses.

    opened by marcelo321 3
  • No output created when using --ignore

    No output created when using --ignore

    cat hostfile | dnsgen - | massdns -r ~/tools/massdns-1.0.0/lists/resolvers.txt --ignore NXDOMAIN --ignore SERVFAIL --ignore REFUSED -t A -o S -w resolved-hosts

    This results in nothing being saved. All the OK should be in the output. massdns is running fine but nothing is being saved

    I located the error to the -o S option for simple output.

    opened by math-ematics 2
Releases(v1.0.0)
  • v1.0.0(Oct 8, 2021)

    With the release of version 1.0.0, MassDNS switches to semantic versioning. Compared to the previous release, response code filtering using --ignore and --filter is new. Additionally, a bug in the NDJSON output was fixed by commit 0b507651cce50e1e3d4073a7956fbc82c8fd8efb. Furthermore, the handling of unconventional DNS names was improved (23ee79a0a410ff326f23a44b83e934d77181e23c and related commits).

    Source code(tar.gz)
    Source code(zip)
  • v0.3(Mar 29, 2020)

    This version of MassDNS uses a custom DNS implementation in order to improve performance by minimizing memory allocations. For this reason, MassDNS no longer depends on ldns. In addition, MassDNS now supports outputting results as NDJSON.

    Source code(tar.gz)
    Source code(zip)
Owner
B. Blechschmidt
B. Blechschmidt
Dome - Subdomain Enumeration Tool. Fast and reliable python script that makes active and/or passive scan to obtain subdomains and search for open ports.

DOME - A subdomain enumeration tool Check the Spanish Version Dome is a fast and reliable python script that makes active and/or passive scan to obtai

Vadi 329 Jan 1, 2023
Subdomain enumeration,Web scraping and finding usernames automation script written in python

Subdomain enumeration,Web scraping and finding usernames automation script written in python

Syam 12 Nov 22, 2022
MayorSec DNS Enumeration Tool

MayorSecDNSScan MSDNSScan is used to identify DNS records for target domains and check for zone transfers. There really isn't much special about it, a

Joe Helle 68 Dec 12, 2022
Python DNS Lookup: The Domain Name System (DNS) is basically the phonebook of the Internet

-Python-DNS-Lookup- ✨ ?? Python DNS Lookup ✨ ?? The Domain Name System (DNS) is

Ronnie Atuhaire 2 Feb 14, 2022
Yuyu Scanner is a Web Reconnaissance & Web Analysis Scanner to find assets and information about targets.

Yuyu Scanner Yuyu Scanner is a Web Reconnaissance & Web Analysis Scanner to find assets and information about targets. installation ! run as root

Justakazh 20 Nov 24, 2022
A toolkit for web reconnaissance, it's fast and easy to use.

A toolkit for web reconnaissance, it's fast and easy to use. File Structure httpsuite/ main.py init.py db/ db.py init.py subdomains_db directories_db

whoami security 22 Jul 22, 2022
A gui application used for network reconnaissance while pentesting

netrecon A gui application used for network reconnaissance while pentesting

Krisna Pranav 4 Sep 3, 2022
CloudFlare reconnaissance, tries to uncover the IP behind CF.

CloudFlare reconnaissance, tries to uncover the IP behind CF.

Neospace 8 Dec 3, 2021
Fast subdomain scanner, Takes arguments from a Json file ("args.json") and outputs the subdomains.

Fast subdomain scanner, Takes arguments from a Json file ("args.json") and outputs the subdomains. File Structure core/ colors.py db/ wordlist.txt REA

whoami security 4 Jul 2, 2022
A simple subdomain scanner in python

Subdomain-Scanner A simple subdomain scanner in python ✨ Features scans subdomains of a domain thats it! ??‍♀️ How to use first download the scanner.p

Portgas D Ace 2 Jan 7, 2022
A Tool for subdomain scan with other tools

ReconTracer A Tool for subdomain scan with other tools ReconTracer Find subdomains by using another amazing sources!. Obs: In a close future recontrac

null 15 Dec 18, 2021
Advanced subdomain scanner, any domain hidden subdomains

little advanced subdomain scanner made in python, works very quick and has options to change the port u want it to connect for

Nano 5 Nov 23, 2021
SubFind - Subdomain Finder Tools

SubFind (Subdomain Finder Tools) Info Tools Result Of Subdomain Command In Termi

LangMurpY 2 Jan 25, 2022
Just your basic port scanner - with multiprocessing capabilities & further nmap enumeration.

Just-Your-Basic-Port-Scanner Just your basic port scanner - with multiprocessing capabilities & further nmap enumeration. Use at your own discretion,

Edward Zhou 0 Nov 6, 2021
Transparent proxy server that works as a poor man's VPN. Forwards over ssh. Doesn't require admin. Works with Linux and MacOS. Supports DNS tunneling.

sshuttle: where transparent proxy meets VPN meets ssh As far as I know, sshuttle is the only program that solves the following common case: Your clien

null 9.4k Jan 4, 2023
A python package with tools to read and postprocess the output of the channel DNS-solver (davecats/channel), as well as its associated postprocessing tools.

Python tools for davecats/channel A python package with tools to read and postprocess the output of the channel dns solver, as well as its associated

Andrea Andreolli 1 Dec 13, 2021
A collection of over 5.1 million sub-domains and assets belonging to public bug bounty programs, compiled into a repo, for performing bulk operations.

?? Public Bug Bounty Targets Data By BugBountyResources A collection of over 5.1M sub-domains and assets belonging to bug bounty targets, all put in a

Bug Bounty Resources 87 Dec 13, 2022
An experimental script to perform bulk parsing of arbitrary file features with YARA and console logging.

RonnieColemanYARAParser This script is named after Ronnie Coleman, and peforms bulk lifts on arbitary file features using YARA console logging. Requir

Steve 20 Dec 13, 2022
DNS hijacking via dead records automation tool

DeadDNS Multi-threaded DNS hijacking via dead records automation tool How it works 1) Dig provided subdomains file for dead DNS records. 2) Dig the fo

null 45 Dec 20, 2022