FlatBuffers: Memory Efficient Serialization Library

Overview

logo FlatBuffers

Build Status Build status Join the chat at https://gitter.im/google/flatbuffers Discord Chat Twitter Follow

FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serialized data without parsing/unpacking it first, while still having great forwards/backwards compatibility.

Go to our landing page to browse our documentation.

Supported operating systems

  • Windows
  • MacOS X
  • Linux
  • Android
  • And any others with a recent C++ compiler.

Supported programming languages

  • C++
  • C#
  • C
  • Dart
  • Go
  • Java
  • JavaScript
  • Lobster
  • Lua
  • PHP
  • Python
  • Rust
  • TypeScript

and more in progress...

Contribution

To contribute to this project, see CONTRIBUTING.

Licensing

Flatbuffers is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.


Issues
  • Feature RFC: Flatbuffers support for Optional types.

    Feature RFC: Flatbuffers support for Optional types.

    Note: This top comment is heavily edited. It was kept up with the current state until this issue closed.

    Motivation

    Flatbuffers should allow users to choose to use optional types. There has been some interest in distinguishing between default values (which are not stored in the binary) and some notion of None which the user controls.

    Here are some links to previous interest in this idea:

    • #333
    • #3777
    • https://groups.google.com/forum/#!topic/flatbuffers/1hrNtBI0BQI
    • https://github.com/google/flatbuffers/issues/5875#issuecomment-619737916

    Currently, a user can control a field's presence in the binary by specifying "force_defaults" and checking "IsFieldPresent" which is a bit of a hack. This proposal should define proper Flatbuffers optional types, which should be a better way of doing this. Use of this feature is only advisable for new fields, since changing default values is in general backwards-incompatible.

    How do we represent this in the schema file?

    We will specify it like so

    table Monster { mana: int = null; }
    

    This visually implies that optional types are at odds with default values and is "consistent" since the value to the right of the equals sign is what we interpret non-presence to mean.

    Change to Schema Grammar:

    field_decl = ident : type [ = (scalar | null) ] metadata ;

    ~We can add a field tag, e.g. "optional" or "no_default", that triggers this behavior. Hopefully no one is using those tags. Maybe we can make it specifiable to flatc, an "--optional-field-keyword-tag" flag, just in case people are using it and can't stop.~

    How do we represent this in the binary?

    We are going with option (A).

    (A) Non-Presence means None

    Instead of omitting zero-like values, the generated code must store them. Non-presence for optional fields no longer means "whatever the default is," now it means None. You can interpret it as "the default value is None". This also means we cannot specify both specify a non-null default and mark the field as optional.

    Pros:

    • This seems more intuitive.
    • It aligns with the "force_defaults" + "IsFieldPresent" hacky manual approximation of this feature.
    • If Nones are more common than zero-likes then this will have smaller binaries.

    Cons:

    • @aardappel thinks this is harder to implement

      "making presence an indicator would require we pass this special field status down to the field construction code to override the current val == default check, which means slowdown, codegen and runtime changes in all languages.. whereas my "least likely to be used default" trick requires no changes"

    ~(B) Some Sentinel value means None~

    In this scenario, zero-like values are still not stored. Instead we choose some "sentinel" value which we interpret to be None (e.g. int can use int_min and float can use some kind of Nan).

    Pros:

    • @aardappel thinks this is easier to implement

      "it requires the schema parser to set default values for you, and no changes anywhere else"

    • If zero-likes are more common than None then this will have smaller binaries

    Cons:

    • Someone might want to use the sentinel value (Hyrum's law).
      • This can be mitigated by publishing the sentinels and letting users decide whether they need the sentinels.
    • This probably won't work for fields representing raw bits.

    How do we represent this in every language API?

    We'll need to change the type signature of all generated code (building/reading/mutating/object/etc) around the optional type to signal its optional-ness. I think we should use the language's local standard for optional types. Suggestions:

    • Python: Optional[T].
    • Rust: Option<T>.
    • C++17 has std::optional<T> but its not obvious what to use for earlier versions. T* would work.
    • Java: Optional shows up in Java 1.8 and triggers autoboxing, so idk :/

    The exact generated-API for a language should be discussed in the PR implementing this feature in that language.

    Out of scope

    (I'll add links if you make issues for these feature requests)

    • Syntactic types
    • Default values for strings and tables

    TODO

    | task | owner | done |---|---|---| | Change flatc to support schemas with optional types and cause an error if they're used in unsupported languages | @CasperN | #6026 ✅ | Implement optional type API in C++ | @vglavnyy | #6155 ✅ | Implement optional type API in Java | @paulovap | #6212 ✅ | Implement optional type API in Rust | @CasperN | #6034 ✅ | Implement optional type API in Swift | @mustiikhalil | #6038 ✅ | Implement optional type API in lobster | @aardappel | ✅ | Implement optional type API in Kotlin | @paulovap | #6115 ✅ | Implement optional type API in Python | @rw? | Implement optional type API in Go | @rw? | Implement optional type API in C | @mikkelfj | ✅ | Implement optional type API in C# | @dbaileychess | #6217 ✅ | Implement optional type API in Typescript/javacscript | @krojew | #6215 ✅ | Php, Dart, etc ... | ? | Update documentation to advertise this feature | @cneo | #6270 ✅

    [edits]

    • added todo list
    • added points from the discussion to each section.
    • added out of scope section
    • Decision: go with (A) and use = null syntax in schema file (cross out alternatives)
    • Updated TODO list, finished parser, Rust in progress
    • Change to schema grammar, link to swift PR, Note at top
    • Added more languages to the TODO
    • Lobster support 🦞
    • Kotlin and C support
    • Java, C#, TS/JS support and docs, issue closed, no longer editing.
    opened by CasperN 133
  • Fixed array length

    Fixed array length

    This change ports the pull request #3987 by daksenik and idoroshev to the most recent commit and adds support for generation of JSON schema, Java, C# and Python.

    opened by svenk177 101
  • Support for Rust programming language

    Support for Rust programming language

    This pull request include an implementation of the Flatbuffers runtime for the Rust programming language and Rust code generation via the flatc binary.

    Code is well documented and currently passes a full suite of tests. I do however still need to add a sample and do more benchmark testing. In the meantime I want to get this on the projects radar to discuss.

    Thanks!

    opened by josephDunne 76
  • [Swift] Swift implementation 🎉🎉

    [Swift] Swift implementation 🎉🎉

    Good afternoon,

    @mzaks and I hope with this PR we would be able to add an official implementation for swift, it's heavily inspired by the C++ and C# implementations. There are some elements that are still missing such as the code generator and also documentation for the code base, however I found that we can implement those after getting an initial review for the swift code base, and if it lives up to the standards.

    The FlatBuffer class uses apples underlaying UnsafeMutableRawPointer which allows is to write directly to the memory, and lefts a lot of heavy weight from our shoulders, since this implementation comes which the following function storeBytes with takes a value and it's Bytes representation and just adds it to the buffer. UnsafeMutableRawPointer the name is a bit misleading since it's actually safe to use if everything writing the bytes is well structured. and since we don't really allow the user to get to the underlying adding and pushing to the buffer and only allowing them to do so through the builder it would be completely safe to use it.

    The FlatBuffersBuilder class uses the same underlying logic as C++ and C#, but unlike C# it relies on generics to actually add bytes and elements into the buffer. we took a different approach regarding adding the structs into the buffer, since swift uses the storeBytes to actually store the struct into the buffer, and since FlatBuffers do not allow string in Structs we took that into our advantage.

    struct Vec2: Writeable {
        var _x: Float32
        var _y: Float32
        var _z: Float32
        var _c: Color2
    
         init(x: Float32, y: Float32, z: Float32, color: Color2) { _c = color; _x = x; _y = y; _z = z }
    
         static func createVec2(_ bb: FlatBuffersBuilder, v: Vec2) -> Offset<UOffset> {
            return bb.create(struct: v)
        }
    }
    

    the following struct will be able to be directly inserted into the buffer, and we will be implementing a reader struct that will read the Writable struct as shown below. This will have the same concept as the builder Objects in c++. The readers were implemented to follow the C# way of fetching elements from the buffer.

    struct Vec2_Read: Readable {
        private var __p: Struct
        init(_ fb: FlatBuffer, o: Int32) { __p = Struct(bb: fb, position: o) }
        var c: Color2 { return Color2(rawValue: __p.readBuffer(of: Int32.self, at: 12)) ?? .red }
        var x: Float32 { return __p.readBuffer(of: Float32.self, at: 0)}
        var y: Float32 { return __p.readBuffer(of: Float32.self, at: 4)}
        var z: Float32 { return __p.readBuffer(of: Float32.self, at: 8)}
    }
    

    All the test case are passing, and they were created by the flatc code generator for both C++ and C#, and the basic implementation of swift was verified against those two since the code generator isn't implemented yet.

    what's missing: 1- Code Generator 2- Documentation 3- Benchmarking

    when merging this PR we would be closing the following issues, Closes #5504. and I will be rebasing the commit before merging for sure

    opened by mustiikhalil 75
  • Tracking issue: Rust buffer verification

    Tracking issue: Rust buffer verification

    This is a tracking issue to document design and implementation of a verifier system for Rust Flatbuffers. I'm thinking that we can clone the logic from the C++ codebase.

    The benefit is twofold:

    1. Verifiers let users check if data is valid, thereby providing a security check for unknown data.
    2. If a buffer is verified, we can justify using more unsafe pointer access in Rust, thereby removing bounds checking.

    Anyone have thoughts on this? @aardappel

    enhancement rust 
    opened by rw 69
  • Non smart pointers --gen-object-api version

    Non smart pointers --gen-object-api version

    It is needed to allow the users to assign one struct to another.

    I also added implementation for operator= for unions, now you can do: union1 = union2; or union1 = supportedStruct;

    It is also needed for the upcoming (in the (near) future) Qt support.

    opened by bog-dan-ro 68
  • [TS/JS] New gen TS code gen

    [TS/JS] New gen TS code gen

    Modernize TS/JS code gen by doing a reworked TS only code gen.

    TODO

    • [x] Split output into modules (files)
    • [x] Devise a sensible alias/prefix for flatbuffers imports to not collide with user symbols
    • [x] Track needed imports
    • [x] Alias imported symbols if same name from different namespaces
    • [x] Do not generate extra ;
    • [x] Fix wrong static method generation
    • [x] Drop flatbuffers namespace completely
    • [x] var to let or const
    • [x] Remove ns wrapping/prefixing
    • [x] Fix object API
    • [x] Remove obsolete generator options
    • [x] Fix and verify JavaScriptFlexBuffersTest.js
    • [x] Remove jsdoc type annotations

    Documentation

    • [ ] Decide what to do about Compiler.md, JavaScriptUsage.md, Tutorial.md and javascript_sample.sh

    TODO (undecided)

    • [ ] Resolve closer relative paths in imports
    • [ ] Avoid unused imports
    • [ ] Proper indentation
    • [ ] Const correctness
    • [ ] noImplicitAny compliant generated source
    c++ javascript typescript codegen documentation json CI 
    opened by bjornharrtell 66
  • Implement Flatbuffer Verifier for Rust

    Implement Flatbuffer Verifier for Rust

    Tracking issue for the title.

    @rw @krojew @ImmemorConsultrixContrarie

    rust 
    opened by CasperN 61
  • [Python] (scalar) vector reading speedup via numpy

    [Python] (scalar) vector reading speedup via numpy

    Reading vectors with generated Python code is slow. There have been other efforts to speed this up but they appear to have stagnated.

    This PR attempts to make a minimal change that covers a use case I personally hit a lot (copying over large vectors that represent nested flatbuffers). There may be other small changes that could drastically speed up other use cases that are not included here since I am unaware of them :)

    What's the change?

    This PR adds support for accessing a scalar vector as a numpy array of the corresponding type. This is much faster than copying large vectors element-by-element.

    • Update the Table class in Python to have a GetVectorAsNumpy method which returns a zero-copy view (in numpy terminology) into a scalar vector cast as the correct type.
    • Update the python-code-generation code (idl_gen_python.cpp) to generate a method which wraps GetVectorAsNumpy in generated code. This method is named <field name>AsNumpy, which can be compared to the generated name of the method to get the length of a vector, <field name>Length. Attempting to use this method if numpy is not installed will result in an error, but otherwise numpy is optional.
    • Update appveyor CI to run Python tests.
    • Update python docs appropriately.

    See also

    • #4090 Byte data from flatbuffer vector into a Python NumPy array
      • This PR can probably close that issue?
    • #4144 Reading and Writing Binary blobs is incredibly slow with the default API!
      • This PR can probably close that issue?
    • #4152 Python: Add numpy array accessors for vectors [WIP]
      • The PR here and that PR seem to accomplish similar things, but I haven't read enough details to say whether or not the PR here completely supersedes that one.
    • #284 Python: Support Cython build
    • #304 Python: Speedup with cython extension (closed by author due to lack of time)
    opened by kbrose 53
  • Port FlatBuffers to Python.

    Port FlatBuffers to Python.

    Implement code generation and runtime library for Python 2 and 3, derived from the Go implementation. Additionally, the test suite verifies:

    the exact bytes in the Builder buffer during many scenarios, vtable deduplication, and table construction, via a fuzzer derived from the Go implementation.

    opened by rw 50
  • Unify generate_code.{bat|sh} scripts

    Unify generate_code.{bat|sh} scripts

    It's becoming a maintenance burden to keep both the generate_code.sh and generate_code.bat files in sync. We should explore options to make them cross-platforms like using python or even integrating the steps into cmake.

    opened by dbaileychess 0
  • Make clean removes checked in generated files

    Make clean removes checked in generated files

    Running make clean deletes a bunch of files that are purposely checked into the repo. We should avoid deleting them in the clean step.

    deleted:    samples/monster.bfbs
    deleted:    samples/monster_generated.h
    deleted:    tests/arrays_test.bfbs
    deleted:    tests/arrays_test_generated.h
    deleted:    tests/monster_extra_generated.h
    deleted:    tests/monster_test.bfbs
    deleted:    tests/monster_test_bfbs_generated.h
    deleted:    tests/monster_test_generated.h
    deleted:    tests/namespace_test/namespace_test1_generated.h
    deleted:    tests/namespace_test/namespace_test2_generated.h
    deleted:    tests/native_type_test_generated.h
    deleted:    tests/optional_scalars_generated.h
    deleted:    tests/union_vector/union_vector_generated.h
    
    opened by dbaileychess 0
  • Make and generate_code.sh generate same outputs

    Make and generate_code.sh generate same outputs

    Fixes #6843

    There were some differences between the arguments in the CMakeLists.txt file and test/generate_code.sh scripts that made modified files after a clean build.

    The discrepancy occurred in #6613

    c++ 
    opened by dbaileychess 0
  • [C++] Removed 'auto' and added 'const' qualifiers

    [C++] Removed 'auto' and added 'const' qualifiers

    We overuse the auto type deduction keyword in C++ source code, causing extra strain on the reader. This follows the general advice of the Google Style Guide on type deduction.

    So I replaced most instances of auto with the explicit type, I only left auto in places where there are iterators and other complex types.

    I also added const qualifiers to the implementation code, as it helps improve readability in knowing if a variable will be changed or not. I expect to examine all const variable assignments in a future PR to see if we can remove the use of temporaries.

    Some private member fields did no have the _ appended to them, I added it in.

    c++ codegen 
    opened by dbaileychess 0
  • Add support for Apple silicon

    Add support for Apple silicon

    Before and after:

    % file flatc
    flatc: Mach-O 64-bit executable arm64
    % file flatc
    flatc: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
    flatc (for architecture x86_64):        Mach-O 64-bit executable x86_64
    flatc (for architecture arm64): Mach-O 64-bit executable arm64
    
    CI 
    opened by vaind 0
  • Use ArrayBuffer type for JS flexbuffer's toObject method

    Use ArrayBuffer type for JS flexbuffer's toObject method

    Corrects the type of the JS flexbuffers toObject method. It appears the tests didn't catch the mistake since they're in JS. Using the correct type causes a runtime exception: Uncaught TypeError: First argument to DataView constructor must be an ArrayBuffer.

    typescript 
    opened by lourd 0
  • Include subdirectories in published npm JS package

    Include subdirectories in published npm JS package

    Updates the files globs in package.json to include subdirectories in order to also include the flexbuffers directory in the published npm package.

    Updates .gitignore to include the tsc-generated JS files.

    Fixes #6849.

    json 
    opened by lourd 2
  • Missing flexbuffers from npm package

    Missing flexbuffers from npm package

    flexbuffers appears to be broken due to missing files from the flatbuffers npm package. The files reference a flexbuffers directory that isn't there.

    A simple code sandbox demonstrating the module import error https://codesandbox.io/s/stoic-wildflower-dsfmt?file=/src/index.ts:44-102

    Screen Shot 2021-09-20 at 11 34 08 PM

    See the published files here on unpkg https://unpkg.com/browse/[email protected]/ts/flexbuffers.ts

    Relatedly, if it were working, is the recommended way to import flexbuffers from the js, ts, or mjs directories? If it's supposed to be automatically handled, I don't believe that's working, either.

    opened by lourd 0
  • compile error with dart 2.14 (flutter 2.5 sdk) [macos, FlatBuffers 2.0]

    compile error with dart 2.14 (flutter 2.5 sdk) [macos, FlatBuffers 2.0]

    ➜ flatc --version flatc version 2.0.0

    ➜ dart --version Dart SDK version: 2.14.0 (stable) (Mon Sep 6 13:47:59 2021 +0200) on "macos_x64"

    ➜ flutter --version Flutter 2.5.0 • channel stable • https://github.com/flutter/flutter.git Framework • revision 4cc385b4b8 (10 days ago) • 2021-09-07 23:01:49 -0700 Engine • revision f0826da7ef Tools • Dart 2.14.0

    image

    opened by bajizhh 2
  • Name clash in generated Java code  (regression in 2.0.0)

    Name clash in generated Java code (regression in 2.0.0)

    The following .fbs:

    table Vector
    {
        vec: [string];
        sep: string;
    }
    

    generates invalid Java code where the outer class Vector clashes with the inner public static final class Vector:

    // automatically generated by the FlatBuffers compiler, do not modify
    
    package ddlog.__tutorial;
    
    import java.nio.*;
    import java.lang.*;
    import java.util.*;
    import com.google.flatbuffers.*;
    
    @SuppressWarnings("unused")
    public final class Vector extends Table {
      public static void ValidateVersion() { Constants.FLATBUFFERS_2_0_0(); }
      public static Vector getRootAsVector(ByteBuffer _bb) { return getRootAsVector(_bb, new Vector()); }
      public static Vector getRootAsVector(ByteBuffer _bb, Vector obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); }
      public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); }
      public Vector __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; }
    
      public String vec(int j) { int o = __offset(4); return o != 0 ? __string(__vector(o) + j * 4) : null; }
      public int vecLength() { int o = __offset(4); return o != 0 ? __vector_len(o) : 0; }
      public StringVector vecVector() { return vecVector(new StringVector()); }
      public StringVector vecVector(StringVector obj) { int o = __offset(4); return o != 0 ? obj.__assign(__vector(o), 4, bb) : null; }
      public String sep() { int o = __offset(6); return o != 0 ? __string(o + bb_pos) : null; }
      public ByteBuffer sepAsByteBuffer() { return __vector_as_bytebuffer(6, 1); }
      public ByteBuffer sepInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 6, 1); }
    
      public static int createVector(FlatBufferBuilder builder,
          int vecOffset,
          int sepOffset) {
        builder.startTable(2);
        Vector.addSep(builder, sepOffset);
        Vector.addVec(builder, vecOffset);
        return Vector.endVector(builder);
      }
    
      public static void startVector(FlatBufferBuilder builder) { builder.startTable(2); }
      public static void addVec(FlatBufferBuilder builder, int vecOffset) { builder.addOffset(0, vecOffset, 0); }
      public static int createVecVector(FlatBufferBuilder builder, int[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]); return builder.endVector(); }
      public static void startVecVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); }
      public static void addSep(FlatBufferBuilder builder, int sepOffset) { builder.addOffset(1, sepOffset, 0); }
      public static int endVector(FlatBufferBuilder builder) {
        int o = builder.endTable();
        return o;
      }
    
      public static final class Vector extends BaseVector {
        public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; }
    
        public Vector get(int j) { return get(new Vector(), j); }
        public Vector get(Vector obj, int j) {  return obj.__assign(__indirect(__element(j), bb), bb); }
      }
    }
    

    javac complains:

    ddlog/__tutorial/Vector.java:45: error: class Vector is already defined in package ddlog.__tutorial
      public static final class Vector extends BaseVector {
    
    opened by ryzhyk 1
Releases(v2.0.0)
  • v2.0.0(May 10, 2021)

    A summary of what is in this release since 1.12.0:

    • Support for optional scalars in most languages.
    • Rust Verifier, Object API and FlexBuffers support, and many API improvements.
    • Overhaul of TypeScript support.
    • gRPC for many languages brought up to date.
    • C++ compile time field reflection.
    • JS FlexBuffers support.
    • Kotlin FlexBuffers and platform support.
    • Swift Object API and many Swift performance improvements.
    • Lua supports LuaJIT and many more fixes.
    • Python FlexBuffers support.
    • Many other fixes and smaller improvements, 371 commits since 1.12.0

    Note, "2.0" doesn't signify any kind of major overhaul of FlatBuffers, it is merely trying to be more semver compatible, and this release does have breaking changes for some languages much like all releases before it.

    Source code(tar.gz)
    Source code(zip)
    Linux.flatc.binary.clang++-9.zip(1.04 MB)
    Mac.flatc.binary.zip(968.25 KB)
    Windows.flatc.binary.zip(1006.77 KB)
  • v1.12.0(Mar 12, 2020)

    A summary of what is in this release since 1.11.0:

    • An official Swift port! (including gRPC support).
    • A Kotlin port!
    • Object API and native JSON support for C#.
    • Object API and gRPC support for Python.
    • Object API for Go.
    • FlexBuffers for Java.
    • Many other fixes and smaller improvements, 247 commits since 1.11.0
    Source code(tar.gz)
    Source code(zip)
    flatc_windows.zip(902.07 KB)
  • v1.11.0(Apr 24, 2019)

    A summary of what is in this release since 1.10.0:

    • More accurate C++ float handling, NaN handling, limit checking.
    • Java optimization: faster object accessors, faster UTF8 conversion.
    • A lot of Rust port improvements.
    • Vector of union JSON parsing.
    • Parser can now work from binary schemas.
    • Python: numpy vector support.
    • Packaging: Snap & RPM support.
    • Many improvements to our CI, added Bazel CI, dockerized language tests.
    • Many other fixes and smaller improvements, 169 commits since 1.10.0
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(596.76 KB)
  • v1.10.0(Oct 3, 2018)

    A summary of what is in this release since 1.9.0:

    • Rust support!
    • Dart support!
    • Lua support!
    • Lobster support!
    • C++: Improved / more correct verifier, compare operator in generated code, new/improved move operators for many data types, string_view support.
    • Java & C# support vectors of unions.
    • Many other fixes and smaller improvements, 145 commits since 1.9.0
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(539.51 KB)
  • v1.9.0(Apr 5, 2018)

    A summary of what is in this release since 1.8.0:

    • Further optimized C++ memory usage for FlatBufferBuilder: from 3 to 1 dynamic allocations (!)
    • GRPC support in Java & Maven.
    • Build files for Bazel and Conan.
    • Prefixed size support for Java/C#/Python.
    • Typescript docs.
    • Many other fixes and improvements, 80 commits since 1.8.0
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(421.96 KB)
  • v1.8.0(Nov 20, 2017)

    A summary of what is in this release since 1.7.0:

    • Vtable trimming in all language implementations: can reduce binary size 10-20%!
    • Mini-reflection tables (in C++) for debug-print, etc.
    • Improved .proto conversion.
    • Vectors of unions now also supported in JS/TS/PHP/C.
    • Improved namespace handling.
    • Boolean types in FlexBuffers.
    • Python Numpy vector support.
    • Nested FlatBuffer JSON parsing.
    • C++98 (stlport) support for core FlatBuffers and FlexBuffers.
    • JSON Schema output support.
    • Many other fixes and improvements, 90 commits since 1.7.0
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(391.39 KB)
  • v1.7.1(Jun 20, 2017)

    This is a bug fix release.

    • Calling CreateVector in C++ on a vector of Offset types with an explicit template parameter (of that Offset type) of could cause the wrong specialization to be selected in 1.7.0. This should affect very few people, but safer to use 1.7.1 just in-case. flatc from 1.7.0 is compatible with 1.7.1.
    Source code(tar.gz)
    Source code(zip)
  • v1.7.0(Jun 17, 2017)

    A summary of what is in this release since 1.6.0:

    • Typescript support!
    • Improved GRPC API with proper zero-copy path. BREAKS API
    • Fixed many issues with ReleaseBufferPointer functionality BREAKS API
    • FlexBuffers: nested in FlatBuffer accessors, (nested) parsing from JSON, JSON output.
    • Vectors of unions and unions of structs/strings (C++ only so far).
    • Several speed optimizations for larger data types in C++ FlatBufferBuilder.
    • Schema registry functionality.
    • Improvements to the object API.
    • Improved docs: C# API, vector of structs.
    • Many other fixes and improvements, 116 commits since 1.6.0
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(374.72 KB)
  • v1.6.0(Feb 15, 2017)

  • v1.5.0(Jan 3, 2017)

    A summary of what is in this release since 1.4.0:

    • GRPC support for Go.
    • C# accessors now based on structs (generates less garbage).
    • Improved object API (e.g. choice of pointer types, external references).
    • Flatc can test for conformity against older schema.
    • In-place binary search lookups for C# and Java.
    • Documentation improvements.
    • Many other fixes and improvements, 133 commits since 1.4.0
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(239.70 KB)
  • v1.4.0(Aug 16, 2016)

    A summary of what is in this release since 1.3.0:

    • Support for GRPC!
    • C++ now has an object API: read & write to generated C++ objects.
    • Mutation support in Go & JS.
    • Many improvements in name space handling and general code generation.
    • Libfuzzer based tests.
    • Documentation for C in the main project.
    • Many other fixes and improvements, 300 commits since 1.3
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(206.77 KB)
  • v1.3.0(Feb 3, 2016)

    A summary of what is in this release since 1.2.0:

    • Support for PHP!
    • Much improved documentation: a tutorial shared by all languages.
    • API improvements for Python, C#, and others.
    • C++ code can now be built with GCC as low as 4.4, and doesn't rely on exceptions anymore.
    • JSON parser can now skip unknown fields.
    • Benchmark code available in benchmark branch.
    • Many other fixes and improvements, 140+ commits since 1.2
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(179.12 KB)
  • v1.2.0(Oct 28, 2015)

    A summary of what is in this release since 1.1.0:

    • Support for JavaScript and Python!
    • A much improved C# and Go API and implementation.
    • Simple mutable FlatBuffers functionality (C++/Java/C#).
    • New reflection functionality: binary schemas, and more dynamic read/mutate (C++, relflection.h).
    • Much improved .proto parsing.
    • Many other fixes and improvements, 200+ commits since 1.1
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(158.76 KB)
  • v1.1.0(Apr 1, 2015)

    A summary of what is in this release since 1.0.0:

    • An extensive overhaul to the Java API.
    • Out-of-the-box support for C# and Go.
    • An optional verifier to make FlatBuffers practical in untrusted scenarios.
    • .proto parsing for easier migration from Protocol Buffers.
    • Optional manual assignment of field IDs.
    • Dictionary functionality through binary search on a key field.
    • Bug fixes and other improvements thanks to 200+ commits from 28 contributors.
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(107.33 KB)
  • v1.0.3(Oct 22, 2014)

    This is an interim release before 1.1.0, with important bug fixes and a few new features:

    • C# is now supported.
    • flatc is able to convert .proto files to FlatBuffer schemas for easier migration from Protocol Buffers.
    • Java FlatBuffers functionality is now more friendly, with better generated code for tables and vectors in particular.
    • Support for required fields.
    • C++ enums are now strongly typed.
    • Option to generate dependent includes in C++ ... and many smaller fixes.
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(91.12 KB)
  • v1.0.2(Sep 3, 2014)

    This is an interim release before 1.1.0, with important bug fixes and a few new features:

    • Schemas can now include other schemas directly.
    • Schemas can refer to types from unrelated namespaces.
    • JSON parsing & writing supports unicode escape codes (\u).
    • Enum value prefixing in C++ is now an option.
    Source code(tar.gz)
    Source code(zip)
    flatc_windows_exe.zip(79.31 KB)
  • v1.0.1(Aug 18, 2014)

    This is an interim release before 1.1.0, with important bug fixes and a few new features:

    • Initial support for the Go programming language.
    • Verification functionality allows safe access of buffers from untrusted sources.
    • File identification allows convenient magic-number sanity checks when a FlatBuffer is used as a file format.
    • Ability to manually assign field id's in the schema.
    • Improved enum parsing functionality in JSON.
    • bit_flags for enums.
    • Nested FlatBuffer support.
    • Vector iterators.
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Jun 17, 2014)

Owner
Google
Google ❤️ Open Source
Google
🦉 Modern high-performance serialization utilities for Python (JSON, MessagePack, Pickle)

srsly: Modern high-performance serialization utilities for Python This package bundles some of the best Python serialization libraries into one standa

Explosion 230 Sep 9, 2021
A lightweight library for converting complex objects to and from simple Python datatypes.

marshmallow: simplified object serialization marshmallow is an ORM/ODM/framework-agnostic library for converting complex datatypes, such as objects, t

marshmallow-code 5.7k Sep 24, 2021
Python wrapper around rapidjson

python-rapidjson Python wrapper around RapidJSON Authors: Ken Robbins <[email protected]> Lele Gaifax <[email protected]> License: MIT License Sta

null 429 Sep 17, 2021
MessagePack serializer implementation for Python msgpack.org[Python]

MessagePack for Python What's this MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JS

MessagePack 1.5k Sep 22, 2021
serialize all of python

dill serialize all of python About Dill dill extends python's pickle module for serializing and de-serializing python objects to the majority of the b

The UQ Foundation 1.5k Sep 23, 2021
simplejson is a simple, fast, extensible JSON encoder/decoder for Python

simplejson simplejson is a simple, fast, complete, correct and extensible JSON <http://json.org> encoder and decoder for Python 3.3+ with legacy suppo

null 1.4k Sep 24, 2021
Protocol Buffers - Google's data interchange format

Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. https://developers.google.com/protocol-buffers/ Overview Protocol Buffe

Protocol Buffers 50.8k Sep 23, 2021
Generic ASN.1 library for Python

ASN.1 library for Python This is a free and open source implementation of ASN.1 types and codecs as a Python package. It has been first written to sup

Ilya Etingof 187 Sep 20, 2021
Python library for serializing any arbitrary object graph into JSON. It can take almost any Python object and turn the object into JSON. Additionally, it can reconstitute the object back into Python.

jsonpickle jsonpickle is a library for the two-way conversion of complex Python objects and JSON. jsonpickle builds upon the existing JSON encoders, s

null 962 Sep 16, 2021
Extended pickling support for Python objects

cloudpickle cloudpickle makes it possible to serialize Python constructs not supported by the default pickle module from the Python standard library.

null 1k Sep 23, 2021