Generic ASN.1 library for Python


ASN.1 library for Python

PyPI Python Versions Build status Coverage Status GitHub license

This is a free and open source implementation of ASN.1 types and codecs as a Python package. It has been first written to support particular protocol (SNMP) but then generalized to be suitable for a wide range of protocols based on ASN.1 specification.


  • Generic implementation of ASN.1 types (X.208)
  • Standards compliant BER/CER/DER codecs
  • Can operate on streams of serialized data
  • Dumps/loads ASN.1 structures from Python types
  • 100% Python, works with Python 2.7 and 3.5+
  • MT-safe
  • Contributed ASN.1 compiler Asn1ate

Why using pyasn1

ASN.1 solves the data serialisation problem. This solution was designed long ago by the wise Ancients. Back then, they did not have the luxury of wasting bits. That is why ASN.1 is designed to serialise data structures of unbounded complexity into something compact and efficient when it comes to processing the data.

That probably explains why many network protocols and file formats still rely on the 30+ years old technology. Including a number of high-profile Internet protocols and file formats.

Quite a number of books cover the topic of ASN.1. Communication between heterogeneous systems by Olivier Dubuisson is one of those high quality books freely available on the Internet.

The pyasn1 package is designed to help Python programmers tackling network protocols and file formats at the comfort of their Python prompt. The tool struggles to capture all aspects of a rather complicated ASN.1 system and to represent it on the Python terms.

How to use pyasn1

With pyasn1 you can build Python objects from ASN.1 data structures. For example, the following ASN.1 data structure:

Record ::= SEQUENCE {
  id        INTEGER,
  house [1] INTEGER DEFAULT 0

Could be expressed in pyasn1 like this:

class Record(Sequence):
    componentType = NamedTypes(
        NamedType('id', Integer()),
            'room', Integer().subtype(
                implicitTag=Tag(tagClassContext, tagFormatSimple, 0)
            'house', Integer(0).subtype(
                implicitTag=Tag(tagClassContext, tagFormatSimple, 1)

It is in the spirit of ASN.1 to take abstract data description and turn it into a programming language specific form. Once you have your ASN.1 data structure expressed in Python, you can use it along the lines of similar Python type (e.g. ASN.1 SET is similar to Python dict, SET OF to list):

>>> record = Record()
>>> record['id'] = 123
>>> record['room'] = 321
>>> str(record)

Part of the power of ASN.1 comes from its serialisation features. You can serialise your data structure and send it over the network.

>>> from pyasn1.codec.der.encoder import encode
>>> substrate = encode(record)
>>> hexdump(substrate)
00000: 30 07 02 01 7B 80 02 01 41

Conversely, you can turn serialised ASN.1 content, as received from network or read from a file, into a Python object which you can introspect, modify, encode and send back.

>>> from pyasn1.codec.der.decoder import decode
>>> received_record, rest_of_substrate = decode(substrate, asn1Spec=Record())
>>> for field in received_record:
>>>    print('{} is {}'.format(field, received_record[field]))
id is 123
room is 321
house is 0
>>> record == received_record
>>> received_record.update(room=123)
>>> substrate = encode(received_record)
>>> hexdump(substrate)
00000: 30 06 02 01 7B 80 01 7B

The pyasn1 classes struggle to emulate their Python prototypes (e.g. int, list, dict etc.). But ASN.1 types exhibit more complicated behaviour. To make life easier for a Pythonista, they can turn their pyasn1 classes into Python built-ins:

>>> from pyasn1.codec.native.encoder import encode
>>> encode(record)
{'id': 123, 'room': 321, 'house': 0}

Or vice-versa -- you can initialize an ASN.1 structure from a tree of Python objects:

>>> from pyasn1.codec.native.decoder import decode
>>> record = decode({'id': 123, 'room': 321, 'house': 0}, asn1Spec=Record())
>>> str(record)

With ASN.1 design, serialisation codecs are decoupled from data objects, so you could turn every single ASN.1 object into many different serialised forms. As of this moment, pyasn1 supports BER, DER, CER and Python built-ins codecs. The extremely compact PER encoding is expected to be introduced in the upcoming pyasn1 release.

More information on pyasn1 APIs can be found in the documentation, compiled ASN.1 modules for different protocols and file formats could be found in the pyasn1-modules repo.

How to get pyasn1

The pyasn1 package is distributed under terms and conditions of 2-clause BSD license. Source code is freely available as a GitHub repo.

You could pip install pyasn1 or download it from PyPI.

If something does not work as expected, open an issue at GitHub or post your question on Stack Overflow or try browsing pyasn1 mailing list archives.

Copyright (c) 2005-2020, Ilya Etingof. All rights reserved.

  • IndexError: tuple index out of range when encoding X509 Cert

    IndexError: tuple index out of range when encoding X509 Cert

    I think this one was not reported yet :)

    I used pyasn1 to extract the X509 certificate part of an PKCS#7 Message, here is an example:

    import binascii
    from pyasn1.codec.der.decoder import decode
    from pyasn1.codec.der.encoder import encode
    certhex  = b"3082039506092a864886f70d010702a082038630820382020101310b300906052b0e03021a0500300b06092a864886f70d010701a08202d0308202cc30820289a003020102020466eda548300b06072a8648ce380403050030363110300e06035504061307736961766173683110300e060355040a1307736961766173683110300e06035504031307736961766173683020170d3136303132353037323533305a180f32303534303532353037323533305a30363110300e06035504061307736961766173683110300e060355040a1307736961766173683110300e0603550403130773696176617368308201b83082012c06072a8648ce3804013082011f02818100fd7f53811d75122952df4a9c2eece4e7f611b7523cef4400c31e3f80b6512669455d402251fb593d8d58fabfc5f5ba30f6cb9b556cd7813b801d346ff26660b76b9950a5a49f9fe8047b1022c24fbba9d7feb7c61bf83b57e7c6a8a6150f04fb83f6d3c51ec3023554135a169132f675f3ae2b61d72aeff22203199dd14801c70215009760508f15230bccb292b982a2eb840bf0581cf502818100f7e1a085d69b3ddecbbcab5c36b857b97994afbbfa3aea82f9574c0b3d0782675159578ebad4594fe67107108180b449167123e84c281613b7cf09328cc8a6e13c167a8b547c8d28e0a3ae1e2bb3a675916ea37f0bfa213562f1fb627a01243bcca4f1bea8519089a883dfe15ae59f06928b665e807b552564014c3bfecf492a0381850002818100e9b46a500aabcbb7b42e93477db9f8ac7c819eadd392f4fa2dd7c10973c399d00d45796052bfb376567a518cfe1432a0f845e4be75aa76462335bf7f84b6c634bb6c884b6ae2e7b6f1cc78c3be6f98b65980832c40d9da3d2b047cba98b24c877b2ace7908b0af56fd62367df355e011a0482c9b8751c373a8d131076cd01421a321301f301d0603551d0e04160414eeac32e5f08781bb6f2f392814951b328868ae4c300b06072a8648ce3804030500033000302d0215009209eb12e966aafae281620e7da9743774d7846302144bf36a9634967ca47639bbc10712571b036e13d531818e30818b020101303e30363110300e06035504061307736961766173683110300e060355040a1307736961766173683110300e0603550403130773696176617368020466eda548300906052b0e03021a0500300b06072a8648ce3804010500042e302c02145de23a86f0c2e45dcad93ef8cab675ec56a6ff4b021404c1a9f7f791631ae00d61fe8ae23f502c33fe84"
    pkcs7message = binascii.unhexlify(certhex)
    message, _ = decode(pkcs7message)
    cert = encode(message[1][3])

    With version 0.3.3 this fails due to the following error, it worked fine with the versions before 0.3.1:

    Traceback (most recent call last):
      File "", line 12, in <module>
        cert = encode(message[1][3])
      File "C:\Program Files\Anaconda3\lib\site-packages\pyasn1\codec\der\", line 56, in __call__
        return encoder.Encoder.__call__(self, value, defMode, maxChunkSize, ifNotEmpty=ifNotEmpty)
      File "C:\Program Files\Anaconda3\lib\site-packages\pyasn1\codec\cer\", line 208, in __call__
        return encoder.Encoder.__call__(self, value, defMode, maxChunkSize, ifNotEmpty)
      File "C:\Program Files\Anaconda3\lib\site-packages\pyasn1\codec\ber\", line 483, in __call__
        self, value, defMode, maxChunkSize, ifNotEmpty=ifNotEmpty
      File "C:\Program Files\Anaconda3\lib\site-packages\pyasn1\codec\ber\", line 61, in encode
        encodeFun, value, defMode, maxChunkSize, ifNotEmpty=ifNotEmpty
      File "C:\Program Files\Anaconda3\lib\site-packages\pyasn1\codec\cer\", line 156, in encodeValue
        substrate = encodeFun(value[idx], defMode, maxChunkSize, namedTypes[idx].isOptional) + substrate
      File "C:\Program Files\Anaconda3\lib\site-packages\pyasn1\type\", line 155, in __getitem__
        return self.__namedTypes[idx]
    IndexError: tuple index out of range

    Maybe, it has to do how I use it, maybe it is a real bug.

    By the way, this is how I use it: I also need to remove the first identifier in the DER Coded sequence, which does not look right, but I did not found another way. (If you know a better way to extract the cert from the message please tell me :))

    opened by reox 17
  • Streamed version of ber.decoder.decode

    Streamed version of ber.decoder.decode

    (An updated version of #175):

    I implemented the functionality so that instead of copying strings, it passes a ~~BytesIO object~~ stream whenever possible. It should solve the #174 issue.

    Changes in API:

    • decodeStream in the modules accepts a stream directly
    • decode behaves as it did before, constructing a new BytesIO object and passing it to decodeStream

    Thus, no API should break unless the author decides so (which I suggest as another issue).

    I tested heavily with Python 2.7 and 3.7, CI should care of the rest.

    • test for open() results added. If any other kind of stream is supposed to be used, it can (and should be added).


    • for the tiniest objects (like "false"), the new decode is ~5 % slower.
    • for example objects of 10-50 bytes, it's ~0-2 % faster.
    • for a 2MB file containing many objects, it's ~5 times faster.

    Note: As file object in Python 2.7 does not allow direct setting of attributes, it's first read into memory in this version. A trivial patch, that slows the overall performance by 2-3 %, but avoids pre-reading the file into memory, can be provided.

    What was considered as well (but did not work / was not performing enough):

    • using memoryview (this could potentially work with bytes if the code were refactored a lot)
    • wrapping input in a custom stream-like object that keeps the content and position - the cost of creating objects and accessing attributes was too high and still all streams were not covered.

    Please, let me know what changes are there still to be made. Thanks Jan

    opened by janpipek 15
  • ImportError: cannot import name 'opentype'

    ImportError: cannot import name 'opentype'

    Since the last update of 4 hours ago my CI broke because a module is using your pyasn1 module. I received the following in my stack trace:

      File "/usr/local/lib/python3.5/dist-packages/oauth2client/", line 24, in <module>
        from pyasn1_modules.rfc2459 import Certificate
      File "/usr/local/lib/python3.5/dist-packages/pyasn1_modules/", line 20, in <module>
        from pyasn1.type import opentype
    ImportError: cannot import name 'opentype'

    Is this anything related to the last changes made in this module and a way this is fixable on my side or with a new release of this module?

    opened by DanielMartinus 14
  • Bugfix/emv


    This is a small pile of changes required to make EMV tags be capable of being uniquely identified and parsed more correctly. As far as I can tell, these don't interfere with existing tag encoding/decoding and probably should be applied to DER and CER if applicable.

    opened by mmattice 13
  • Add method equivalent of x[component]=None

    Add method equivalent of x[component]=None

    Add a new method which allows setting a default value to a component. The following lines are equivalent:

    x.setComponentToDefault(comp) x[comp] = None

    but the first one is more explicit and raises fewer eyebrows during code review.

    opened by viraptor 12
  • PyAsn1Error when decoding explicitly tagged subtypes

    PyAsn1Error when decoding explicitly tagged subtypes

    I've got a little wrapper library at that goes a little beyond the ASN.1 specs to providing a canonical presentation.

    Since upgrading to pyasn1 0.3.x my builds fail, and I'm having a hard time figuring out what if anything I'm doing wrong.

    Summarized, I do the following:

    • Encode more complex types as sequences:
        self.MAPPING = univ.Sequence(
          tagSet = univ.Sequence.tagSet.tagExplicitly(
            tag.Tag(tag.tagClassContext, tag.tagFormatConstructed, 0x04)
    • In the mapping case, it's actually a sequence of two item sequences, representing the key and value.
    • When decoding, there's now a PyAsn1Error when the sequence end is reached. Previously iteration would just stop.

    I can circumvent that by catching the error, but that might hide other issues. I can explicitly add the length of the sequence, but my understanding of ASN.1 is that this shouldn't be necessary.

    The error is "Compontent type not defined".

    Do you have any pointers for me?

    opened by jfinkhaeuser 11
  • 0.2.4 release is missing

    0.2.4 release is missing


    According to the CHANGELOG, there is a v0.2.4 release. I suppose it has been out, but it cannot be found in the releases tab.

    opened by MuxZeroNet 9
  • Drop support for EOL Pythons

    Drop support for EOL Pythons

    Python 2.4, 2.5, 2.6, 3.2 and 3.3 are end of life. They are no longer receiving bug fixes, including for security issues. These Pythons went EOL on:

    Python 2.6: 2013-10-29 Python 3.2: 2016-02-20 Python 3.3: 2017-09-29

    For additional details on supported Python versions, see:

    Removing support for EOL Pythons will reduce testing and maintenance resources.

    Using pypinfo, here are the download statistics of the project for the last 30 days, showing limited downloads from EOL Pythons:

    | python_version | percent | download_count | | -------------- | ------: | -------------: | | 2.7 | 90.33% | 7,975,885 | | 3.6 | 4.95% | 437,261 | | 3.5 | 2.99% | 263,637 | | 3.4 | 1.02% | 89,723 | | 2.6 | 0.69% | 60,511 | | 3.7 | 0.02% | 2,024 | | 3.3 | 0.00% | 441 | | 3.2 | 0.00% | 38 | | 3.8 | 0.00% | 9 | | None | 0.00% | 2 |

    By removing support for older Pythons, can remove or update the following workarounds and code:

    • Built-in function bin() is always available.
    • Built-in function callable() is always available.
    • Built-in function all() is always available.
    • The collections.OrderedDict class is always available.
    • The datetime.strptime() method is always available.
    • The platform module is always available.
    • The string.partition() method is always available.
    • The NullHandler class is always available.
    • The method __trunc__ is always available.
    • Can use set literals.
    • Updated unittest is always available. The unittest2 module is unnecessary.

    Users on older versions of Python can continue to download, install, and use older versions of the library that continue to have support for older Pythons.

    opened by jdufresne 9
  • AttributeError: 'property' object has no attribute 'presentTypes'

    AttributeError: 'property' object has no attribute 'presentTypes'

    I'm not sure about this I don't know much of python myself, coming from another script (freeipa) (ipa-server-install)

        from pyasn1_modules import rfc2315, rfc2459
      File "/usr/lib/python2.7/site-packages/pyasn1_modules/", line 67, in <module>
        class DigestedData(univ.Sequence):
      File "/usr/lib/python2.7/site-packages/pyasn1_modules/", line 72, in DigestedData
        namedtype.NamedType('digest', Digest)
      File "/root/.local/lib/python2.7/site-packages/pyasn1/type/", line 159, in __init__
        self.__ambiguousTypes = 'terminal' not in kwargs and self.__computeAmbiguousTypes() or {}
      File "/root/.local/lib/python2.7/site-packages/pyasn1/type/", line 278, in __computeAmbiguousTypes
        ambigiousTypes[idx] = NamedTypes(*partialAmbigiousTypes, **dict(terminal=True))
      File "/root/.local/lib/python2.7/site-packages/pyasn1/type/", line 158, in __init__
        self.__tagToPosMap = self.__computeTagToPosMap()
      File "/root/.local/lib/python2.7/site-packages/pyasn1/type/", line 251, in __computeTagToPosMap
        for _tagSet in tagMap.presentTypes:
    AttributeError: 'property' object has no attribute 'presentTypes'```
    opened by davidsielert 9
  • Default encoding of OctetString

    Default encoding of OctetString


    The default defined encoding of a univ.OctetString seems to be ISO-8859-1 (see the code here). I was wondering why this choice was made, instead of something more universal, like UTF8?

    This default encoding is causing some weird behavior in some programs relying on this library (see here or here).

    Is it possible to change the default encoding to UTF8? If not, why not, and what do you suggest we can do to ensure proper encoding in our programs?



    opened by the-useless-one 7
  • Need

    Need "Questions" enabled on GitHub?

    I was looking at the mailing list archives, and it seems that early 2019 was the last traffic, so getting help there seems unlikely... What about enabling "Questions" for the project?

    opened by pprindeville 0
  • KeyError when processing some PySNMP MIB modules

    KeyError when processing some PySNMP MIB modules

    pyasn1 is used in parsing SNMP MIBS used by the pysnmp package. Some of the MIBS (from python3-pysnmp4-mibs package -, for example) trigger a "KeyError" exception when they use a subtype with "namedValues" as the key. The code above assumes that the object already has all of the kwargs present. It could be more forgiving by checking for the existence of the key and creating a new k/v pair if it doesn't exist instead of automatically assuming it is present and trying to append to it.

    opened by wyllys66 1
  • 0.4.8:  pyasn1-module pytest warnings

    0.4.8: pyasn1-module pytest warnings

    During testing pyasn1-module pythest shows some warnings:

    + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pyasn1-module-0.2.8-6.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pyasn1-module-0.2.8-6.fc35.x86_64/usr/lib/python3.8/site-packages
    + /usr/bin/pytest -ra
    =========================================================================== test session starts ============================================================================
    platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
    benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
    rootdir: /home/tkloczko/rpmbuild/BUILD/pyasn1-modules-0.2.8
    plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, httpbin-1.0.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, case-1.5.3, isort-1.3.0, aspectlib-1.5.2, asyncio-0.15.1, toolbox-0.5, xprocess-0.17.1, aiohttp-0.3.0, checkdocs-2.7.0, mock-3.6.1, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, flaky-3.7.0, hypothesis-6.14.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, Faker-8.9.0, cases-3.6.2
    collected 212 items
    . .                                                                                                                                                                  [  0%]
    tests/ .                                                                                                                                                  [  0%]
    tests/ .                                                                                                                                              [  1%]
    tests/ ..                                                                                                                                             [  2%]
    tests/ .                                                                                                                                              [  2%]
    tests/ ......                                                                                                                                         [  5%]
    tests/ .                                                                                                                                              [  6%]
    tests/ ..                                                                                                                                             [  7%]
    tests/ .                                                                                                                                              [  7%]
    tests/ ...                                                                                                                                            [  9%]
    tests/ ..                                                                                                                                             [  9%]
    tests/ ..                                                                                                                                             [ 10%]
    tests/ .                                                                                                                                              [ 11%]
    tests/ ..                                                                                                                                             [ 12%]
    tests/ ..                                                                                                                                             [ 13%]
    tests/ ..........                                                                                                                                     [ 18%]
    tests/ ..                                                                                                                                             [ 18%]
    tests/ .                                                                                                                                              [ 19%]
    tests/ .                                                                                                                                              [ 19%]
    tests/ ...                                                                                                                                            [ 21%]
    tests/ ...                                                                                                                                            [ 22%]
    tests/ ....                                                                                                                                           [ 24%]
    tests/ ..                                                                                                                                             [ 25%]
    tests/ ..                                                                                                                                             [ 26%]
    tests/ .                                                                                                                                              [ 27%]
    tests/ ..                                                                                                                                             [ 27%]
    tests/ ..........                                                                                                                                     [ 32%]
    tests/ ..                                                                                                                                             [ 33%]
    tests/ ..                                                                                                                                             [ 34%]
    tests/ .                                                                                                                                              [ 35%]
    tests/ .                                                                                                                                              [ 35%]
    tests/ ..                                                                                                                                             [ 36%]
    tests/ ..                                                                                                                                             [ 37%]
    tests/ ...                                                                                                                                            [ 38%]
    tests/ ..                                                                                                                                             [ 39%]
    tests/ ...                                                                                                                                            [ 41%]
    tests/ ..                                                                                                                                             [ 42%]
    tests/ ......                                                                                                                                         [ 45%]
    tests/ ..                                                                                                                                             [ 45%]
    tests/ ..                                                                                                                                             [ 46%]
    tests/ ..                                                                                                                                             [ 47%]
    tests/ .                                                                                                                                              [ 48%]
    tests/ ....                                                                                                                                           [ 50%]
    tests/ ..                                                                                                                                             [ 51%]
    tests/ .                                                                                                                                              [ 51%]
    tests/ .                                                                                                                                              [ 52%]
    tests/ ..                                                                                                                                             [ 53%]
    tests/ ..                                                                                                                                             [ 54%]
    tests/ .                                                                                                                                              [ 54%]
    tests/ ....                                                                                                                                           [ 56%]
    tests/ ..                                                                                                                                             [ 57%]
    tests/ ..                                                                                                                                             [ 58%]
    tests/ ..                                                                                                                                             [ 59%]
    tests/ ...                                                                                                                                            [ 60%]
    tests/ ..                                                                                                                                             [ 61%]
    tests/ ..                                                                                                                                             [ 62%]
    tests/ ..                                                                                                                                             [ 63%]
    tests/ ..                                                                                                                                             [ 64%]
    tests/ .                                                                                                                                              [ 64%]
    tests/ .                                                                                                                                              [ 65%]
    tests/ ..                                                                                                                                             [ 66%]
    tests/ ..                                                                                                                                             [ 67%]
    tests/ ..                                                                                                                                             [ 68%]
    tests/ ..                                                                                                                                             [ 69%]
    tests/ ..                                                                                                                                             [ 70%]
    tests/ ..                                                                                                                                             [ 71%]
    tests/ ..                                                                                                                                             [ 72%]
    tests/ ....                                                                                                                                           [ 73%]
    tests/ ..                                                                                                                                             [ 74%]
    tests/ ......                                                                                                                                         [ 77%]
    tests/ .                                                                                                                                              [ 78%]
    tests/ ..                                                                                                                                             [ 79%]
    tests/ .                                                                                                                                              [ 79%]
    tests/ ..                                                                                                                                             [ 80%]
    tests/ ..                                                                                                                                             [ 81%]
    tests/ .                                                                                                                                              [ 81%]
    tests/ .                                                                                                                                              [ 82%]
    tests/ ..                                                                                                                                             [ 83%]
    tests/ ..                                                                                                                                             [ 84%]
    tests/ ..                                                                                                                                             [ 85%]
    tests/ ..                                                                                                                                             [ 86%]
    tests/ ..                                                                                                                                             [ 87%]
    tests/ .                                                                                                                                              [ 87%]
    tests/ .                                                                                                                                              [ 88%]
    tests/ ...                                                                                                                                            [ 89%]
    tests/ .                                                                                                                                              [ 90%]
    tests/ .                                                                                                                                              [ 90%]
    tests/ ..                                                                                                                                             [ 91%]
    tests/ .                                                                                                                                              [ 91%]
    tests/ .                                                                                                                                              [ 92%]
    tests/ ......                                                                                                                                         [ 95%]
    tests/ ..                                                                                                                                             [ 96%]
    tests/ .                                                                                                                                              [ 96%]
    tests/ ..                                                                                                                                             [ 97%]
    tests/ ...                                                                                                                                            [ 99%]
    tests/ ..                                                                                                                                             [100%]
    ============================================================================= warnings summary =============================================================================
      /usr/lib/python3.8/site-packages/pyasn1/type/ DeprecationWarning: __int__ returned non-int (type SizedInteger).  The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
        return SizedInteger(value).setBitLength(len(value))
    -- Docs:
    ===================================================================== 211 passed, 8 warnings in 9.69s ======================================================================
    opened by kloczek 0
  • Stacktrace when enabling debugging

    Stacktrace when enabling debugging

    _componentValues can be NoValue in some situation, which makes debugging crash:

      File "pyasn1/type/", line 2738, in prettyPrintType
        for idx, componentType in enumerate(self.componentType.values() or self._componentValues):

    I fixed the bug by changing this line:


            enum = self.componentType.values()
            if not enum and not isinstance(self._componentValues,base.NoValue):
                enum = self._componentValues
            for idx, componentType in enumerate(enum):
    opened by ptitdoc 0
  • 0.4.8: pytest warning

    0.4.8: pytest warning

    Just normal build, install and test cycle used on building package from non-root account:

    • " build"
    • " install --root </install/prefix>"
    • "pytest with PYTHONPATH pointing to setearch and sitelib inside </install/prefix>
    + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pyasn1-0.4.8-6.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pyasn1-0.4.8-6.fc35.x86_64/usr/lib/python3.8/site-packages
    + /usr/bin/pytest -ra
    =========================================================================== test session starts ============================================================================
    platform linux -- Python 3.8.9, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
    benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
    rootdir: /home/tkloczko/rpmbuild/BUILD/pyasn1-0.4.8
    plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, httpbin-1.0.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, case-1.5.3, isort-1.3.0, aspectlib-1.5.2, asyncio-0.15.1, toolbox-0.5, xprocess-0.17.1, aiohttp-0.3.0, checkdocs-2.7.0, mock-3.6.1, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, cases-3.6.1, flaky-3.7.0, hypothesis-6.14.0, benchmark-3.4.1, xdist-2.3.0, Faker-8.8.1
    collected 1143 items
    . .                                                                                                                                                                  [  0%]
    tests/ ..                                                                                                                                               [  0%]
    tests/codec/ber/ .................................................................................................................................... [ 11%]
    ..................................................................                                                                                                   [ 17%]
    tests/codec/ber/ .................................................................................................................................... [ 29%]
    ........................................................                                                                                                             [ 34%]
    tests/codec/cer/ ............................                                                                                                         [ 36%]
    tests/codec/cer/ .....................................................................................                                                [ 43%]
    tests/codec/der/ ..........................                                                                                                           [ 46%]
    tests/codec/der/ ...........................................                                                                                          [ 50%]
    tests/codec/native/ .............                                                                                                                     [ 51%]
    tests/codec/native/ ................                                                                                                                  [ 52%]
    tests/compat/ ......                                                                                                                                   [ 53%]
    tests/compat/ ....                                                                                                                                    [ 53%]
    tests/compat/ .............                                                                                                                            [ 54%]
    tests/type/ ...............................................................................................                                              [ 62%]
    tests/type/ .............................................                                                                                          [ 66%]
    tests/type/ .................                                                                                                                       [ 68%]
    tests/type/ ......                                                                                                                                   [ 68%]
    tests/type/ ...                                                                                                                                      [ 69%]
    tests/type/ ...............                                                                                                                               [ 70%]
    tests/type/ ............................................................................................................................................ [ 82%]
    .................................................................................................................................................................... [ 97%]
    .............                                                                                                                                                        [ 98%]
    tests/type/ .....................                                                                                                                      [100%]
    ============================================================================= warnings summary =============================================================================
      /home/tkloczko/rpmbuild/BUILDROOT/python-pyasn1-0.4.8-6.fc35.x86_64/usr/lib/python3.8/site-packages/pyasn1/type/ DeprecationWarning: __int__ returned non-int (type SizedInteger).  The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
        return SizedInteger(value).setBitLength(len(value))
    -- Docs:
    ==================================================================== 1142 passed, 8 warnings in 37.08s =====================================================================
    opened by kloczek 0
  • 0.4.8: sphinx warning

    0.4.8: sphinx warning

    + /usr/bin/python3 build_sphinx -b man --build-dir build/sphinx
    running build_sphinx
    Running Sphinx v4.0.2
    making output directory... done
    loading intersphinx inventory from
    building [mo]: targets for 0 po files that are out of date
    building [man]: all manpages
    updating environment: [new config] 72 added, 0 changed, 0 removed
    reading sources... [100%] pyasn1/type/useful/utctime
    /home/tkloczko/rpmbuild/BUILD/pyasn1-0.4.8/docs/source/pyasn1/contents.rst:5: WARNING: Duplicate explicit target name: "pyasn1".
    looking for now-outdated files... none found
    pickling environment... done
    checking consistency... done
    writing... python-pyasn1.3 { pyasn1/contents pyasn1/type/base/contents pyasn1/type/base/asn1type pyasn1/type/base/simpleasn1type pyasn1/type/base/constructedasn1type pyasn1/type/base/novalue pyasn1/type/univ/contents pyasn1/type/univ/integer pyasn1/type/univ/boolean pyasn1/type/univ/bitstring pyasn1/type/univ/octetstring pyasn1/type/univ/null pyasn1/type/univ/objectidentifier pyasn1/type/univ/real pyasn1/type/univ/enumerated pyasn1/type/univ/any pyasn1/type/univ/setof pyasn1/type/univ/sequenceof pyasn1/type/univ/set pyasn1/type/univ/sequence pyasn1/type/univ/choice pyasn1/type/char/contents pyasn1/type/char/numericstring pyasn1/type/char/printablestring pyasn1/type/char/teletexstring pyasn1/type/char/t61string pyasn1/type/char/videotexstring pyasn1/type/char/ia5string pyasn1/type/char/graphicstring pyasn1/type/char/visiblestring pyasn1/type/char/iso646string pyasn1/type/char/generalstring pyasn1/type/char/universalstring pyasn1/type/char/bmpstring pyasn1/type/char/utf8string pyasn1/type/useful/contents pyasn1/type/useful/objectdescriptor pyasn1/type/useful/generalizedtime pyasn1/type/useful/utctime pyasn1/type/tag/contents pyasn1/type/tag/tag pyasn1/type/tag/tagset pyasn1/type/tag/tagmap pyasn1/type/constraint/contents pyasn1/type/constraint/singlevalue pyasn1/type/constraint/containedsubtype pyasn1/type/constraint/valuerange pyasn1/type/constraint/valuesize pyasn1/type/constraint/permittedalphabet pyasn1/type/constraint/withcomponents pyasn1/type/constraint/constraintsintersection pyasn1/type/constraint/constraintsunion pyasn1/type/constraint/constraintsexclusion pyasn1/type/namedtype/contents pyasn1/type/namedtype/namedtype pyasn1/type/namedtype/optionalnamedtype pyasn1/type/namedtype/defaultednamedtype pyasn1/type/namedtype/namedtypes pyasn1/type/opentype/contents pyasn1/type/opentype/opentype pyasn1/type/namedval/contents pyasn1/type/namedval/namedval pyasn1/codec/ber/contents pyasn1/codec/cer/contents pyasn1/codec/der/contents pyasn1/codec/native/contents pyasn1/error/contents example-use-case download changelog license } done
    build succeeded, 1 warning.
    opened by kloczek 0
  • INTEGER length error in the tlv

    INTEGER length error in the tlv


    I am trying to create the following structure

     0x30, 0x82, 0x02, 0x12,         ; SEQUENCE (530 Bytes)
        0x30, 0x82, 0x01, 0x0A,     ; SEQUENCE (266 Bytes) (public key) 
            0x02, 0x82, 0x01, 0x01  ; INTEGER  (277 Bytes)
                <257 bytes public key-modulus  >,
            0x02, 0x03,             ; INTEGER  ( 03 Bytes)
                <3 bytes PK-exponent (value: 65537)>,
        0x02, 0x82, 0x01, 0x00,     ; INTEGER  (256 Bytes)
            <256byte Signature { public key  } *singed by CA private key >_
    class publicKey(Sequence):
        componentType = NamedTypes(
            NamedType('exponent', Integer())
    class SrklHost(Sequence):
        componentType = NamedTypes(
            NamedType('signature', Integer())
    ca_key = serialization.load_pem_private_key( ...
    host_key = rsa.generate_private_key(
    host_key_pk = host_key.public_key().public_bytes(serialization.Encoding.DER,format=serialization.PublicFormat.PKCS1)
    chosen_hash = hashes.SHA256()
    hasher = hashes.Hash(chosen_hash, default_backend())
    digest = hasher.finalize()
    sig = ca_key.sign(
    message , _ = decode(host_key_pk)
    pk , _ = decode(host_key_pk, asn1Spec= publicKey())
    signedItem = SrklHost()
    signedItem['hostPk'] = pk
    signedItem['signature'] = int.from_bytes(sig,'big')

    The signedItem message should be in total 534 bytes long But sometimes it return an extra byte resulting to 535 bytes in total

    I can see that in the 0x02, 0x82, 0x01, 0x00, ; INTEGER (256 Bytes) section of signature adds an extra byte Resulting to 0x02 0x82 0x01 0x01 0x00.

    Am I doing something wrong ?

    I am using cryptography io for other cryptography functionality

    Thank you !

    opened by EKA-13 1
  • Encoders based on [Bytes]IO instead of string joining?

    Encoders based on [Bytes]IO instead of string joining?

    Hi Ilya,

    after some time, I finally got some time to follow up on the streaming support we introduced last year (in #174 ). I started thinking how to best inject a BinaryIO into the encoding project. I can give it a try but before changing any code, I realized one (rather positive) thing:

    For huge objects, encoding is much less problematic than decoding. There is some necessary copying of strings on each level of the structure but on the lower levels, only very small strings are copied, theoretically making the total asymptotic time something like O(total_size * depth_of_hierarchy) instead of O(totalsize^2) (which was the case of decoder).

    However in practice, with a toy example, I was not able to even make it worse than linear - Sequence of sequences of 1000x1000 objects cost ~ the same as one sequence of 1,000,000 objects and that cost ~100 times more than one sequence of 10,000 objects. Perhaps some time might be spared by using one output stream but on the expense of passing this object all around in the encoding process. What are your thoughts?

    Best regards, Jan

    opened by janpipek 0
  • Added changes to support PPC64LE

    Added changes to support PPC64LE


    I request you to please review few changes, I have made, builds are passing.


    opened by ezeeyahoo 3
  • Can this support loop through the records

    Can this support loop through the records

    I know most ASN.1 libraries can decode the byte stream and populate the given record type. But in most cases one file could contain thousands of such records one after another. Can this library used to decode each record in turn. Another reason for this question is sometimes between two records there can be several bytes with 0 value (which I don't know exactly why, but I've seen in certain telco related data files). Is there a way to skip through these 0 value bytes and decode any valid records. Thank you

    opened by kmrsh 2
  • v0.4.8(Nov 16, 2019)

    This is a minor feature release:

    • Added ability to combine SingleValueConstraint and PermittedAlphabetConstraint objects into one for proper modeling FROM ... EXCEPT ... ASN.1 clause.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.7(Sep 1, 2019)

    The major feature of this release is SET ... WITH COMPONENTS constraint support.

    • Added WithComponentsConstraint along with related ComponentPresentConstraint and ComponentAbsentConstraint classes to be used with Sequence/Set types representing SET ... WITH COMPONENTS ... like ASN.1 constructs.

    The other important change is that sizeSpec attribute of ASN.1 objects is now deprecated in favor of uniform subtypeSpec, which is now used for all constraints.

    • Deprecate subtypeSpec attributes and keyword argument. It is now recommended to pass ValueSizeConstraint, as well as all other constraints, to subtypeSpec.
    • Added isInconsistent property to all constructed types. This property conceptually replaces verifySizeSpec method to serve a more general purpose e.g. ensuring all required fields are in a good shape. By default this check invokes subtype constraints verification and is run by codecs on value de/serialisation.

    In the bug fixes department we have just one fix:

    • Fixed a design bug in a way of how the items assigned to constructed types are verified. Now if Asn1Type-based object is assigned, its compatibility is verified based on having all tags and constraint objects as the type in field definition. When a bare Python value is assigned, then field type object is cloned and initialized with the bare value (constraints verificaton would run at this moment).

    Complete list of changes can be found in CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.4.6(Jul 31, 2019)

    This release brings support for one overlooked ASN.1 feature:

    • Added previously missing SET OF ANY construct encoding/decoding support.

    As a consequence, SequenceOf/SetOf objects behavior has been made closer to Python list and more consistent with the rest of pyasn1 types (at the expense of potential backward compatibility issues):

    • New elements to SequenceOf/SetOf objects can now be added at any position - the requirement for the new elements to reside at the end of the existing ones (i.e. s[len(s)] = 123) is removed.
    • List-like slicing support added to SequenceOf/SetOf objects.
    • Removed default initializer from SequenceOf/SetOf types to ensure consistent behaviour with the rest of ASN.1 types. Before this change, SequenceOf/SetOf instances immediately become value objects behaving like an empty list. With this change, SequenceOf/SetOf objects remain schema objects unless a component is added or .clear() is called. This change can potentially cause incompatibilities with existing pyasn1 objects which assume SequenceOf/SetOf instances are value objects right upon instantiation. The behaviour of Sequence/Set types depends on the componentType initializer: if no componentType is given, the behavior is the same as SequenceOf/SetOf have. If componentType is given, but neither optional nor defaulted components are present, created instance remains being the schema object. If, however, either optional or defaulted component is present, created instance immediately becomes a value object.
    • Added .reset() method to all constructed types to turn value object into a schema object.

    Also, a couple of minor usability improvements:

    • Added omitEmptyOptionals option which is respected by Sequence and Set encoders. When omitEmptyOptionals is set to True, empty initialized optional components are not encoded. Default is False.
    • Added PyAsn1UnicodeDecodeError/PyAsn1UnicodeDecodeError exceptions to help the caller treating unicode errors happening internally to pyasn1 at the upper layers.
    • Added support for subseconds CER/DER encoding edge cases in GeneralizedTime codec.

    And, of course, bug fixes \o/

    • Fixed 3-digit fractional seconds value CER/DER encoding of GeneralizedTime.
    • Fixed AnyDecoder to accept possible TagMap as asn1Spec to make dumping raw value operational

    Complete list of changes can be found in CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.4.5(Dec 29, 2018)

  • v0.4.4(Jul 26, 2018)

    This is a maintenance release fixing a pair of annoying bugs:

    • Fixed native encoder type map to include all ASN.1 types rather than just ambiguous ones
    • Fixed crash in .prettyPrint of Sequence and Set occurring at OPTIONAL components

    Changes are noted in the CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.4.3(May 23, 2018)

  • v0.4.2(Nov 23, 2017)

    This is an emergency release fixing a regression.

    It fixes a single bug in OctetString/BitString encoders which blows up when you BER-encode ASN.1 structure produced by BER decoder.

    Changes are noted in the CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Nov 23, 2017)

    This release introduces ASN.1 open type support and brings a handful of other fixes and improvements. Most important changes are:

    • ANY DEFINED BY clause support implemented
    • Encoders refactored to take either a value (as ASN.1 object) or a Python value plus ASN.1 schema
    • The ASN.1 types' __repr__() and __str__() implementations reworked for better usability
    • Improved backward-compatibility with older pyasn1 versions
    • Sphinx documentation rearranged, simplified and reworded

    All changes are noted in the CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.7(Oct 4, 2017)

  • v0.3.6(Sep 20, 2017)

    This is mostly a bug fix release. Most important changes are:

    • The __getitem__/__setitem__ behavior of Set/Sequence and SetOf/SequenceOf objects aligned with the canonical Mapping and Sequence protocols in part
    • Fixed crash in ASN.1 encoder when encoding an explicitly tagged component of a Set/Sequence object

    More changes noted in the CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.5(Sep 16, 2017)

    This is mostly a bug fix release. Most important fixes include:

    • Explicit tag encoding optimized to avoid unnecessary copying
    • End-of-octets sentinel encoding optimized
    • Refactored ASN.1 codecs properties to silently enforce proper length and chunk size encoding modes
    • Fixed DER encoder to always produce primitive encoding
    • Fixed crash at SequenceOf native decoder
    • Fixed a couple of crashes when debug mode is enabled

    More changes noted in the CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.4(Sep 7, 2017)

    This is mostly a bug fix release. Most important fixes include:

    • Added missing component-less SEQUENCE/SET objects dict duck-typing support
    • Fixed crash at SEQUENCE and SEQUENCE OF CER encoder when running in schemaless mode
    • Fixed Character types instantiation from OctetString type -- double unicode decoding may have scrambled the data

    More changes noted in the CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.3(Aug 26, 2017)

    This is a bugfix release addressing a handful of regressions introduced in 0.3.2.

    Most importantly:

    • Fixed exponential index size growth bug when building ambiguous NamedTypes tree
    • Fixed constructed types decoding failure at BER codec if running in unguided mode
    • Fixed SetOf ordering at CER/DER encoder

    More changes noted in the CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Aug 4, 2017)

    This is a bugfix release addressing a handful of regressions introduced in 0.3.1.

    Most importantly:

    • Fixed SequenceOf/SetOf types initialization syntax to remain backward compatible with pyasn1 0.2.*
    • Rectified thread safety issues by moving lazy, run-time computation into object initializer.
    • Fixed GeneralizedTime/UTCTime CER/DER codecs to actually get invoked
    • Fixed DER/CER encoders handling optional SEQUENCE/SET fields containing nested SEQUENCE/SET with optional fields.

    More changes noted in the CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(Jul 26, 2017)

    The main theme for this release is API and performance improvements.

    Be aware that by way of usability improvements, a some backward compatibilities were introduced. For the details please refer to the CHANGELOG.

    The highlights:

    • ASN.1 types __init__(), .clone() and .subtype() signatures refactored to expect keyword arguments
    • Many "getters" of ASN.1 types refactored into Python properties
    • SetOf/SequenceOf ASN.1 types better mimic Python list
    • The NamedValues implementation refactored to mimic Python dict
    • The GeneralizedTime and UTCTime types now support to/from Python datetime object conversion
    • Many minor performance improvements and bug fixes

    More changes noted in the CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • 0.2.3(Feb 25, 2017)

    This is a performance improvement release:

    • BitString type implementation rebased onto Python int to behave like an integer and improve serialization performance (100x times!).
    • Integer codecs reimplemented to base on Python's built-in integer serialization codecs to improve serialization performance (~40%).
    • Decoding performance of Set and Choice types improved by caching once computed tags maps.
    • ASN.1 character types refactored to duck-type unicode strings thus making them more convenient in text processing context.
    • Documentation updated at many places.

    More changes noted in the CHANGELOG.

    Source code(tar.gz)
    Source code(zip)
  • v0.2.2(Feb 7, 2017)

    This release fixes a regression in Enumerated ASN.1 type subtyping procedure.

    Besides that, canonical decoders were hardened to reject non-canonical serialization on input.

    More details can be learned from the change log

    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Feb 4, 2017)

    This is a bug fix and usability improvement release:

    • Conversion between Python built-in types and pyasn1 objects implemented
    • Python types emulation improved to make pyasn1 types look more like their Python counterparts
    • BitString type usability improved in many ways
    • Sphinx documentation added

    More changes noted in the CHANGELOG.

    The upcoming release is to be focused on codecs performance improvement.

    Source code(tar.gz)
    Source code(zip)
Ilya Etingof
Sr. Software Engineer at Red Hat; OpenStack hardware provisioning team
Ilya Etingof
FlatBuffers: Memory Efficient Serialization Library

FlatBuffers FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serializ

Google 16.8k Sep 23, 2021
🦉 Modern high-performance serialization utilities for Python (JSON, MessagePack, Pickle)

srsly: Modern high-performance serialization utilities for Python This package bundles some of the best Python serialization libraries into one standa

Explosion 230 Sep 9, 2021
Python wrapper around rapidjson

python-rapidjson Python wrapper around RapidJSON Authors: Ken Robbins <[email protected]> Lele Gaifax <[email protected]> License: MIT License Sta

null 429 Sep 17, 2021
serialize all of python

dill serialize all of python About Dill dill extends python's pickle module for serializing and de-serializing python objects to the majority of the b

The UQ Foundation 1.5k Sep 23, 2021
Python library for serializing any arbitrary object graph into JSON. It can take almost any Python object and turn the object into JSON. Additionally, it can reconstitute the object back into Python.

jsonpickle jsonpickle is a library for the two-way conversion of complex Python objects and JSON. jsonpickle builds upon the existing JSON encoders, s

null 962 Sep 16, 2021
A lightweight library for converting complex objects to and from simple Python datatypes.

marshmallow: simplified object serialization marshmallow is an ORM/ODM/framework-agnostic library for converting complex datatypes, such as objects, t

marshmallow-code 5.7k Sep 24, 2021
Extended pickling support for Python objects

cloudpickle cloudpickle makes it possible to serialize Python constructs not supported by the default pickle module from the Python standard library.

null 1k Sep 23, 2021
simplejson is a simple, fast, extensible JSON encoder/decoder for Python

simplejson simplejson is a simple, fast, complete, correct and extensible JSON <> encoder and decoder for Python 3.3+ with legacy suppo

null 1.4k Sep 24, 2021
MessagePack serializer implementation for Python[Python]

MessagePack for Python What's this MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JS

MessagePack 1.5k Sep 22, 2021
Python bindings for the simdjson project.

pysimdjson Python bindings for the simdjson project, a SIMD-accelerated JSON parser. If SIMD instructions are unavailable a fallback parser is used, m

Tyler Kennedy 474 Sep 24, 2021
Ultra fast JSON decoder and encoder written in C with Python bindings

UltraJSON UltraJSON is an ultra fast JSON encoder and decoder written in pure C with bindings for Python 3.6+. Install with pip: $ python -m pip insta

null 3.4k Sep 24, 2021
Protocol Buffers - Google's data interchange format

Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. Overview Protocol Buffe

Protocol Buffers 50.8k Sep 23, 2021
Crappy tool to convert .scw files to .json and and vice versa.

SCW-JSON-TOOL Crappy tool to convert .scw files to .json and vice versa. How to use Run file with two arguments: python <scw2json or j

Fred31 5 May 14, 2021