ADExplorerSnapshot.py is an AD Explorer snapshot ingestor for BloodHound.

Last update: Dec 23, 2022

Overview

ADExplorerSnapshot.py

ADExplorerSnapshot.py is an AD Explorer snapshot ingestor for BloodHound.

AD Explorer allows you to connect to a DC and browse LDAP data. It can also create snapshots of the server you are currently attached to. This tool allows you to convert those snapshots to BloodHound-compatible JSON files.

Limitations

The ingestor only supports offline information collection from the snapshot file and won't interact with systems on the network. That means features like session and localadmin collection are not available. GPO/OU collection is missing. The ingestor processes all data it possibly can from the snapshot (including ACLs). You will only have the data available of the LDAP/DC that you ran the snapshot against.

Installation

git clone https://github.com/c3c/ADExplorerSnapshot.py.git
cd ADExplorerSnapshot.py
pip3 install --user .

Usage

usage: ADExplorerSnapshot.py [-h] snapshot

ADExplorer snapshot ingestor for BloodHound

positional arguments:
  snapshot

optional arguments:
  -h, --help  show this help message and exit

Notes

Making snapshots in AD Explorer is more network-intensive than the traditional BloodHound ingestors as it attempts to retrieve all objects it can from the LDAP.

ADExplorerSnapshot.py will create caches of information for quicker lookups while processing the data. Especially when processing larger snapshots (e.g. 4GB+) you will also need to have sufficient RAM available. In my tests, about half of the snapshot file size was required in RAM.

The library was tested with a number of data sets, please create an issue report if you run into problems.

The AD Explorer snapshot parser is implemented as its own module, which could also be used individually.

The format in which snapshots are stored by AD Explorer is proprietary and led to a fun reverse engineering journey. A 010 editor template is included in this repository, which I used for iteratively mapping out the contents of the snapshot into structs.

License and credits

This code is licensed under the MIT license and makes use of code that is also licensed under the MIT license.

ADExplorerSnapshot.py relies on the following projects:

BloodHound.py (the Python BloodHound ingestor); for processing LDAP data.
dissect.cstruct (C-style binary struct parser): for parsing the binary snapshot data.

Credits:

Cedric Van Bockhaven (Deloitte) for implementation
Marat Nigmatullin (Deloitte) for the idea

Comments

KeyError: 'ms-mcs-admpwd'

Hi there,

Thanks for the awesome tool!

Happy New Year to you and I hope you have a good 2022 so far!

I encountered the following error when using the tool.

Would you be able to assist on this?

└─# python3 ADExplorerSnapshot.py 2Jan2022.dat 
[*] Server: Corp.local
[*] Time of snapshot: 2022-01-02T11:59:17
[*] Mapping offset: XXXXXX
[*] Object count: XXXXXXX
[+] Parsing properties: XXXXXXX
[+] Parsing classes:XXXXXXX
[+] Parsing object offsets: XXXXXXX
[+] Preprocessing objects: XXXXXXX sids, XXXXXXX computers, XXXXXXX domains with XXXXXXX DCs
[▇] Collecting data: 31006/44901 (XXXXXXX users, XXXXXXX groups, XXXXXXX computers, XXXXXXX trusts)
Traceback (most recent call last):
  File "/root/Desktop/Tools/ADExplorerSnapshot.py/ADExplorerSnapshot.py", line 2, in <module>
    adexpsnapshot.main()
  File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init_.py", line 603, in main
    ADExplorerSnapshot(fh, log)
  File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init.py", line 92, in __init_
    self.process()
  File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init_.py", line 158, in process
    ret = fun(obj)
  File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init_.py", line 293, in processComputers
    aces = self.parse_acl(computer, 'computer', ADUtils.get_entry_property(entry, 'nTSecurityDescriptor', raw=True))
  File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init_.py", line 475, in parse_acl
    aces = self._parse_acl_cached(haslaps, entrytype, acl)
  File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init_.py", line 466, in _parse_acl_cached
    _, aces = parse_binary_acl(fake_entry, entrytype, acl, self.objecttype_guid_map)
  File "/root/.local/lib/python3.9/site-packages/bloodhound/enumeration/acls.py", line 142, in parse_binary_acl
    if ace_object.acedata.get_object_type().lower() == objecttype_guid_map['ms-mcs-admpwd']:
  File "/usr/lib/python3/dist-packages/requests/structures.py", line 54, in _getitem_
    return self._store[key.lower()][1]
KeyError: 'ms-mcs-admpwd'

bug

opened by chengteck 2

KeyError: 'category'

I have encountered a problem

python3 ADExplorerSnapshot.py tests/data/Server.dat [*] Server: ***.com [*] Time of snapshot: [*] Mapping offset: 0x2853a5 [*] Object count: 3632 [+] Parsing properties: 1499 [+] Parsing classes: 269 [+] Parsing object offsets: 3632 [+] Restored pre-processed information from data cache [v] Collecting data: 3169/3632 (0 users, 0 groups, 0 computers, 0 trusts) Traceback (most recent call last): File "/opt/anaconda3/lib/python3.8/site-packages/dissect/cstruct/types/instance.py", line 16, in getattr return self._values[attr] KeyError: 'category'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "ADExplorerSnapshot.py", line 2, in adexpsnapshot.main() File "ADExplorerSnapshot.py/adexpsnapshot/init.py", line 603, in main ADExplorerSnapshot(fh, log) File "ADExplorerSnapshot.py/adexpsnapshot/init.py", line 92, in init self.process() File "ADExplorerSnapshot.py/adexpsnapshot/init.py", line 158, in process ret = fun(obj) File "ADExplorerSnapshot.py/adexpsnapshot/init.py", line 364, in processUsers if not (('user' in entry.classes and 'person' == entry.category) or 'msds-groupmanagedserviceaccount' in entry.classes): File "ADExplorerSnapshot.py/adexpsnapshot/parser/classes.py", line 59, in getattr return getattr(self._data, attr) File "/opt/anaconda3/lib/python3.8/site-packages/dissect/cstruct/types/instance.py", line 18, in getattr raise AttributeError("Invalid attribute: %r" % attr) AttributeError: Invalid attribute: 'category'

How do I need to solve this problem？Thanks
bug

opened by Alex888-li 2
Fix user primaryGroupID

Fixes #6

get_primary_membership from bloodhound.enumeration.memberships can't handle the list object returned from entry['attributes']['primary_membership'].

This PR allows the tool to return the correct primaryGroupID by defining a temporary dictionary that the get_primary_membership function can handle.

opened by frenetic00 1
ADExplorerSnapshot doesn't map primaryGroupID

The script used to import ADExplorer snapshots in to Bloodhound doesn't pick up on the PrimaryGroupID attribute and therefore doesn't map the relation. This results in a users not showing as part of certain groups.

opened by N7WEra 1
'data' is not defined

The dictionary on the following line is not defined:

https://github.com/c3c/ADExplorerSnapshot.py/blob/33708fa990d400ffd0d26d928dab33e66bf1a732/adexpsnapshot/init.py#L275

Pull requests coming your way.

Another issue is that the following attribute is not used, do you want to keep in in there for future use? https://github.com/c3c/ADExplorerSnapshot.py/blob/33708fa990d400ffd0d26d928dab33e66bf1a732/adexpsnapshot/init.py#L225
bug

opened by OlafHaalstra 1
Commandline in documentation screenshot is outdated

In the program the -v flag is currently not used.

Furthermore I would like to suggest to increase the commandline options by specifying an input file and output directory for more granular control in the case of dealing with multiple files.
documentation enhancement

opened by OlafHaalstra 0
ERROR: Failed building wheel for ADExplorerSnapshot

Can anyone help me resolve this error?

┌──(kali㉿kali)-[~/Bloodhound/ADExplorerSnapshot.py] └─$ sudo pip install --user . [sudo] password for kali: Processing /home/kali/Bloodhound/ADExplorerSnapshot.py Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting dissect.cstruct>=2.0 Downloading dissect.cstruct-2.0-py3-none-any.whl (30 kB) Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from ADExplorerSnapshot==0.1.dev42+g590b947) (2.27.1) Collecting pwntools>=4.5.0 Downloading pwntools-4.8.0-py2.py3-none-any.whl (11.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.7/11.7 MB 6.2 MB/s eta 0:00:00 Collecting frozendict Downloading frozendict-2.3.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (113 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 113.2/113.2 KB 13.2 MB/s eta 0:00:00 Collecting bloodhound>=1.2.0 Downloading bloodhound-1.4.0-py3-none-any.whl (70 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.1/70.1 KB 7.1 MB/s eta 0:00:00 Requirement already satisfied: ldap3!=2.5.0,!=2.5.2,!=2.6,>=2.5 in /usr/lib/python3/dist-packages (from bloodhound>=1.2.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.8.1) Requirement already satisfied: dnspython in /usr/lib/python3/dist-packages (from bloodhound>=1.2.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.2.0) Requirement already satisfied: impacket>=0.9.17 in /usr/lib/python3/dist-packages (from bloodhound>=1.2.0->ADExplorerSnapshot==0.1.dev42+g590b947) (0.9.24) Requirement already satisfied: future in /usr/lib/python3/dist-packages (from bloodhound>=1.2.0->ADExplorerSnapshot==0.1.dev42+g590b947) (0.18.2) Requirement already satisfied: pyasn1>=0.4 in /usr/lib/python3/dist-packages (from bloodhound>=1.2.0->ADExplorerSnapshot==0.1.dev42+g590b947) (0.4.8) Requirement already satisfied: sortedcontainers in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.1.0) Collecting rpyc Downloading rpyc-5.2.3-py3-none-any.whl (71 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.3/71.3 KB 13.3 MB/s eta 0:00:00 Collecting unicorn>=1.0.2rc1 Downloading unicorn-2.0.0-py2.py3-none-manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.1/16.1 MB 7.5 MB/s eta 0:00:00 Requirement already satisfied: pysocks in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (1.7.1) Collecting colored-traceback Downloading colored-traceback-0.3.0.tar.gz (3.8 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: python-dateutil in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.8.1) Requirement already satisfied: pyserial>=2.7 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (3.5) Requirement already satisfied: six>=1.12.0 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (1.16.0) Requirement already satisfied: paramiko>=1.15.2 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.8.1) Collecting psutil>=3.3.0 Downloading psutil-5.9.1-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (282 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 282.0/282.0 KB 7.5 MB/s eta 0:00:00 Collecting intervaltree>=3.0 Downloading intervaltree-3.1.0.tar.gz (32 kB) Preparing metadata (setup.py) ... done Collecting ropgadget>=5.3 Downloading ROPGadget-6.8-py3-none-any.whl (31 kB) Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (21.3) Requirement already satisfied: pip>=6.0.8 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (22.0.2) Collecting pathlib2 Downloading pathlib2-2.3.7.post1-py2.py3-none-any.whl (18 kB) Collecting capstone>=3.0.5rc2 Downloading capstone-5.0.0rc2-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (2.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/2.8 MB 7.0 MB/s eta 0:00:00 Collecting pyelftools>=0.2.4 Downloading pyelftools-0.29-py2.py3-none-any.whl (174 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 174.3/174.3 KB 8.3 MB/s eta 0:00:00 Requirement already satisfied: pygments>=2.0 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.11.2) Requirement already satisfied: mako>=1.0.0 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (1.1.3) Collecting plumbum Downloading plumbum-1.7.2-py2.py3-none-any.whl (117 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.8/117.8 KB 5.6 MB/s eta 0:00:00 Building wheels for collected packages: ADExplorerSnapshot, intervaltree, colored-traceback Building wheel for ADExplorerSnapshot (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for ADExplorerSnapshot (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [2012 lines of output] .. . . . . note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for ADExplorerSnapshot Building wheel for intervaltree (setup.py) ... done
Created wheel for intervaltree: filename=intervaltree-3.1.0-py2.py3-none-any.whl size=26119 sha256=c5adb48c91f8ba94ee8d0b46ad72abead60c58045e46b7efee84df4988efb963 Stored in directory: /root/.cache/pip/wheels/fa/80/8c/43488a924a046b733b64de3fac99252674c892a4c3801c0a61 Building wheel for colored-traceback (setup.py) ... done Created wheel for colored-traceback: filename=colored_traceback-0.3.0-py3-none-any.whl size=4622 sha256=e603e3140cca08f23dc046af7df4e3d50cf4db37ba4f9bd267970339ebbe8281 Stored in directory: /root/.cache/pip/wheels/10/49/bd/750e09783fb038570efede2d03819a7141fc2350de51daf575 Successfully built intervaltree colored-traceback Failed to build ADExplorerSnapshot ERROR: Could not build wheels for ADExplorerSnapshot, which is required to install pyproject.toml-based projects

opened by mc702 0
Fix: avoid sid=none in JSON output
I had an issue with the output files. Bloodhound would fail when importing. The message in the dev console said something like Cannot read property of null.

I found that the computers.json contained lines like this:

"AllowedToDelegate": [ "S-1-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", null ],

This patch makes sure to add sids to the AllowedToDelegate list only if they are not Null (or empty).

I reckon this happens when the SID is from another domain.
opened by AdrianVollmer 1
User-created Machine Accounts are not parsed properly as Computers
User-created machine accounts (e.g. with new-adcomputer cmdlet) are not included in the parsed output from AD Explorer snapshot. Per source code, it appears the object is not parsed correctly due to a lack of the "dnshostname" attribute for Computer objects created this way.

https://github.com/c3c/ADExplorerSnapshot.py/blob/590b94735db3cf18e6cc5e7512937192a1080715/adexpsnapshot/init.py#L205

To replicate :

Create an object with "New-ADComputer" (notice the dnshostname attribute is missing)

Open in ADExplorer, create snapshot

Run parser, check the computer.json file
opened by 0x0db5 0

Owner

GitHub

Ingest openldap data into bloodhound

Bloodhound for Linux Ingest a dumped OpenLDAP ldif into neo4j to be visualized in Bloodhound. Usage: ./ldif_to_neo4j.py ./sample.ldif | cypher-shell -