numpy2tfrecord
Simple helper library to convert a collection of numpy data to tfrecord, and build a tensorflow dataset from the tfrecord.
Installation
$ git clone [email protected]:yonetaniryo/numpy2tfrecord.git
$ cd numpy2tfrecord
$ pip install .
Test
$ pytest -v
How to use
Convert a collection of numpy data to tfrecord
import numpy as np
from numpy2tfrecord import Numpy2Tfrecord
converter = Numpy2Tfrecord()
x = np.arange(100).reshape(10, 10).astype(np.float32) # float array
y = np.arange(100).reshape(10, 10).astype(np.int64) # int array
a = 5 # int
b = 0.3 # float
entry = {"x": x, "y": y, "a": a, "b": b}
converter.add(entry) # add data entry
...
converter.export_to_tfrecord("test.tfrecord") # export to tfrecord
Build a tensorflow dataset from tfrecord
dataset = build_dataset_from_tfrecord("test.tfrecord") # load tfrecord and build tf.data.Dataset
5.7k Jan 9, 2023
1.3k Jan 2, 2023
1.8k Jan 8, 2023
2.6k Jan 4, 2023
92 Dec 30, 2022
34 Dec 27, 2022
3 Sep 23, 2022
102 Dec 17, 2022
1 Jan 10, 2022
349 Aug 6, 2022
2 Feb 16, 2022
506 Jan 8, 2023
4.2k Jan 2, 2023
29 May 25, 2022
653 Dec 30, 2022
2.4k Jan 5, 2023
0 May 6, 2022
12 Jul 23, 2022
4 Dec 14, 2021