Hello,

I need your help on two points:

1. I want to record a PCAP file using the decoder.

I try the following command:

decode -d writer -i INTERFACE -o pcap FILE NAME

He gives me the following error message:

WARNING:writer:rawHandler() got an unexpected keyword argument 'smac'

Do you know what it is?

2. The Dshell guide says that you should copy the following databases into the GeoIP folder:

GeoIP.dat, GeoIPv6.dat, GeoIPASNum.dat, GeoIPASNumv6.dat

The linked URL "https://dev.maxmind.com/geoip/geoip2/geolite2/" only has the GeoLite2 databases left:

GeoLite2 City, ... Country, ... ASN

These contain databases, but no longer the files mentioned above.

Does this still work?

Hey! I hope this message finds you well.

So, I'm trying to run some plugins to live capture on a well configured interface and I got no results.

The Dshell prompt is presented again after I execute the decode command.

Dshell> decode -i ens5f0 -d search --search_expression UPDATE Dshell>

Any help?

Best regards.

This may just be personal preference, but the line-break added to the end of each blob in followstream output makes typical flows more difficult to read (for me). Without the line-break, the output more closely approximates that of Wireshark's follow stream output.

README.md states:

1. Configure pygeoip by moving the MaxMind data files (GeoIP.dat, GeoIPv6.dat, GeoIPASNum.dat, GeoIPASNumv6.dat) to /share/GeoIP/"

Should that be /home/[username]/share/GeoIP

or

sudo mkdir /usr/local/share/GeoIP/ ?

It is a bit unclear in README.md.

Pull #99 created a case where, if a decoder (such as xor) internally creates a Connection (e.g. for use with downstream chained decoders), the new object's nextoffset member references NoneType values instead of integers. And within the context of normal chained decoder operation, there is no condition wherein IPHandler would be called with a SYN flag to establish the natural starting offsets.

Prior to #99, the default values were 0, so the internal object would function, albeit with artificial sequence numbers.

My proposed solution is to manually (in the decoder) set the starting values of nextoffset to match the values of the "parent" Connection.

Note: This condition doesn't impact other chainable decoders (such as country filters or grep) because those decoders don't create a new Connection object. They simply select which connections to pass downstream vs not.

For reference, the following error was observed running xor+followstream, leading to the identification of this bug:

ARNING:xor:unsupported operand type(s) for +: 'NoneType' and 'int'
WARNING:xor:unsupported operand type(s) for +: 'NoneType' and 'int'
WARNING:xor:unsupported operand type(s) for +: 'NoneType' and 'int'

make file for Linux System packaging , at least friendlier DESTDIR=/opt/Dshell

may be some to-do's per say ,

but with SRPM spec from others or debian control spec , and or my ebuild
MV Makefile Makefile.orginal mv ALT-Makefile Makefile Emake ALL

also can add on third party Plugins /decoders via packaging and call a script to build or rebuild decoders allowing $USER/.dshell/logs etc to be made might be an improvement if not running as sudo/su.

Make Dshell use avalible to more users.

Hi

I can't seem to get dshell working with pcaps saved to disk or traffic from an interface. In wireshark the traffic has the pppoe layer above the ethernet layer, and another layer above the pppoe one, which wireshark calls 'Point-to-point Protocol' and is 2 bytes in length.

I've started dshell with --strip= all numbers from 1 to 6, with nothing happening, the pcap definitely has dns traffic in it. --strip requires an int, so what should I give it?

Many thanks

I haven't do the network data capture and analysis for quite a while. mainly because being lazy. is it possible to include those pcap files mentioned in the README file ?

DestDir isnt cleanly defined in makefile so thus its a royal pest to package DSHELL on many distros....

gentoo uses sandbox /var/portage/$Packagename/$package-version/build , D$ would fake the root based on DestDir's else i have to force {$ROOT} to force make a root-fs tree in sandbox (and it works about as well as a cluster-F####) {9999 is customary for Live-git version } once built then it gets copied over to actual install tree and temp paths removed. unfortunately .dshellrc dshell-decode dshell gets real paths and has to get fixed. basically it jumps out of security sandbox so it doesn't package well as is.
however python most of all DISTUTILS packages goes up without a hitch. some require nominal patching to behave...

A: distro or system Agnostic is the power of distutils https://docs.python.org/2/distutils/setupscript.html ie python setup.py /Dshell/share/GeoIP copy if exsits /usr/share/GeoIP file symlinks etc.
B: makes for Easier Agnostic Packaging by Linux distros , adds consistent directories for 3rd parity add on modules. C: Can define USER or SYSTEM mode installs default-sysntem .dshell > /etc/skel/ so thus it is copied over to many users on login (usr /home/$username/Dshell-path/....... Symlink /opt/bin/Dshell or /usr/bin/Dshell/ dshell-decode dshell @/usr/bin Dshell/docs to /usr/share/doc/Dshell as many of the docs are dynamically generated. d: RPM/DEB /MSI ETC are a nice added bonus feature of python distutils , In theory Dshell could be just as easily patched for windows Boxes with a good and proper Python setup. and or even MACOS etc natives. as add-on modules that do added packaging, RPM or Deb , etc could be added on to the main modules , and then system packages generated for users conveniences latter. E: find any missing docs or apply updates. new modules etc.

https://pythonhosted.org/setuptools/python3.html https://github.com/pypa/sampleproject https://docs.python.org/2.0/dist/creating-rpms.html https://ghantoos.org/2008/10/19/creating-a-deb-package-from-a-python-setuppy/ http://cyrille.rossant.net/create-a-standalone-windows-installer-for-your-python-application/

[install] prefix=/usr/bin/Dshell install_lib=//usr/bin/Dshell/lib install_scripts=/usr/bin/Dshell/bin etc.

[bdist_wininst] prefix=/c:/Dshell install_lib=/some/lib/path install_scripts=/some/bin/path

This is a new output module for elasticsearch. It works standalone with --output elasticout,host=ES_HOST,index=INDEXNAME, but I find it also useful to extend the class. This allows for more centralized control over the cluster and dynamic/deterministic index names and doc IDs.

This is a utility to split pcap files by ip src/dst pair or tcp/udp stream using Dshell's PCAPWriter() to write the output files. This is sometimes useful when trying to process pcap in dshell that contains numerous and interleaved really large streams.

There seems to be an issue with installing pcapy: https://github.com/helpsystems/pcapy/issues/73 which is preventing installation of DShell.

Collecting pcapy
  Downloading pcapy-0.11.4.tar.gz (37 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-nqz_zlei/pcapy_e1a8e2d5b3794862af12f48d4e4fdfdc/setup.py", line 49, in <module>
          save_init_posix = sysconfig._init_posix
      AttributeError: module 'distutils.sysconfig' has no attribute '_init_posix'
      [end of output]

It appears unlikely that pcapy will release a fix for the issue since the latest commit was back in 2019. A possible work around is to use pcapy-ng (https://github.com/stamparm/pcapy-ng/) instead pcapy. Update the pcapy to pcapy-ng in setup.py

    install_requires=[
        "geoip2",
        "pcapy-ng",
        "pypacker",
    ],

It appears that the blob_handler() function isn't getting called at certain critical junctures. My understanding was that the core code would call this function (if defined in a plugin) as it processed packets, every time the stream changed direction. It actually looks like the blob_handlers aren't called until the connection closes and the blobs are formed/iterated.

• Moves parsing of TLS extensions to a common function, invoked in parsing both the ClientHello and ServerHello.
• Includes SSL/TLS version string in primary output and kw values

The only invocation of setfilter() on the capture device (pcapy.Reader class) is based on the initial_bpf from the first plugin on the chain.

Because of this, any efforts to expand the filter are moot. Narrowing of the filter seems effective through manipulating compiled bpf filters on the plugin objects, but only the packets pulled from the wire or file (governed by the pcapy.Reader filter) are ever passed to the feed_plugin_chain function.

It seems we may need a mechanism to update the Reader filter when bpf filters are changed in the plugin chain. But this is not trivial, because recompiling bpf happens in the plugin object and the instantiated Reader appears only in decode.py.

I initially noticed this because the automatic vlan wrapper wasn't working with any plugin on vlan tagged PCAP files, but it has potential effects also in chained plugins and plugins that dynamically alter their bpf filters.