In the near future, this repository will be used once again for developing the Keras codebase. For the time being, the Keras codebase is being developed at tensorflow/tensorflow, and any PR or issue should be directed there.
I am trying to save a simple LSTM model for text classification. The input of the model is padded vectorized sentences.
model = Sequential()
model.add(LSTM(40, input_shape=(16, 32)))
model.add(Dense(20))
model.add(Dense(8, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
For saving I'm using the following snippet:
for i in range(50):
from sklearn.cross_validation import train_test_split
data_train, data_test, labels_train, labels_test = train_test_split(feature_set, dummy_y, test_size=0.1, random_state=i)
accuracy = 0.0
try:
with open('/app/accuracy', 'r') as file:
for line in file:
accuracy = float(line)
except Exception:
print("error")
model.fit(data_train, labels_train, nb_epoch=50)
loss, acc = model.evaluate(feature_set, dummy_y)
if acc > accuracy:
model.save_weights("model.h5", overwrite=True)
model.save('my_model.h5', overwrite=True)
print("Saved model to disk.\nAccuracy:")
print(acc)
with open('/app/accuracy', 'w') as file:
file.write('%f' % acc)
But whenever I'm trying to load the same model
from keras.models import load_model
model = load_model('my_model.h5')
I'm getting random accuracy like an untrained model. same result even when trying to load weights separately. If I set the weights
lstmweights=model.get_weights()
model2.set_weights(lstmweights)
like above. It is working if model and model2 are run under same session (same notebook session). If I serialize lstmweights and try to load it from different place, again I'm getting result like untrained model. It seems saving only the weights are not enough. So why model.save is not working. Any known point?
Assume we are trying to learn a sequence to sequence map. For this we can use Recurrent and TimeDistributedDense layers. Now assume that the sequences have different lengths. We should pad both input and desired sequences with zeros, right? But how will the objective function handle the padded values? There is no choice to pass a mask to the objective function. Won't this bias the cost function?
Theano has supported multiple GPUs since v0.8.0. (cf. Using multiple GPUs — Theano 0.8.0 documentation ) Does Keras also support using multiple GPUs?
For example, can I run the below task?
Hey guys,
I was wondering how are the initial internal states in a recurrent layer dealt with? So far it appears they are reset are every run. Is there any way to preserve them?
I'd like to be able to feed a .predict_proba() function data one time step at a time for a time series task, as the points come in, without also feeding the entire history all over again. Is this somehow possible? Thanks
During fine-tuning, if a Batch Normalization layer is frozen it uses the mini-batch statistics. I believe this is incorrect and it can lead to reduced accuracy especially when we use Transfer learning. A better approach in this case would be to use the values of the moving mean and variance.
Changes on this PR: In this PR I update the Batch Normalization layer to use the learned statistics if frozen during training. This is achieved by making the trainable flag part of the computational graph and by depending the behavior of the BN not only on the learning_phase but also on the value of the trainable property.
Brief explanation: Assume we use one of the pre-trained CNNs of Keras and we want to fine-tune it. Unfortunately, we get no guarantees that the mean and variance of our new dataset inside the BN layers will be similar to the ones of the original dataset. As a result, if we fine-tune the top layers, their weights will be adjusted to the mean/variance of the new dataset. Nevertheless, during inference the top layers will receive data which are scaled using the mean/variance of the original dataset. This discrepancy can lead to reduced accuracy.
I understand that this is a significant change that requires thorough review. To faciliate the situation I've documented why making such a change is important and provided detailed comparisons before and after applying the patch on my blog.
EDIT: Since the fix was not merged on master, I maintain unofficial patches available for Keras 2.1.6, Keras 2.2.2 and Keras 2.2.4.
I am doing text classification. Also I am using my pre-trained word embeddings and i have a LSTM layer on top with a softmax at the end.
vocab_size = embeddings.shape[0]
embedding_size = embeddings.shape[1]
model = Sequential()
model.add(Embedding(
input_dim=vocab_size,
output_dim=embedding_size,
input_length=max_length,
trainable=False,
mask_zero=True,
weights=[embeddings]
))
model.add(LSTM(200, return_sequences=False))
model.add(Dropout(0.5))
model.add(Dense(3, activation='softmax', activity_regularizer=activity_l2(0.0001)))
Pretty simple. Now I want to add attention to the model, but i don't know how to do it.
My understanding is that i have to set return_sequences=True so as the attention layer will weigh each timestep accordingly. This way the LSTM will return a 3D Tensor, right?
After that what do i have to do?
Is there a way to easily implement a model with attention using Keras Layers or do i have to write my own custom layer?
If this can be done with the available Keras Layers, I would really appreciate an example.
I am solving an NLP task and I am trying to model it directly as a sequence using different RNN flavors. How can I use my own Word Vectors rather than using an instance of layers.embeddings.Embedding?
There appears to be a bug in the make_predict_function for the tensorflow backend. The following error message appears for me when trying to call model.predict(...)
self._make_predict_function()
File "/usr/local/lib/python3.4/dist-packages/keras/engine/training.py", line 679, in _make_predict_function
**self._function_kwargs)
File "/usr/local/lib/python3.4/dist-packages/keras/backend/tensorflow_backend.py", line 615, in function
return Function(inputs, outputs, updates=updates)
File "/usr/local/lib/python3.4/dist-packages/keras/backend/tensorflow_backend.py", line 589, in __init__
with tf.control_dependencies(self.outputs):
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 3192, in control_dependencies
return get_default_graph().control_dependencies(control_inputs)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 2993, in control_dependencies
c = self.as_graph_element(c)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 2291, in as_graph_element
raise ValueError("Tensor %s is not an element of this graph." % obj)
ValueError: Tensor Tensor("Sigmoid_2:0", shape=(?, 17), dtype=float32) is not an element of this graph.
This does not happen when using the theano backend.
Notes: The model is loaded from json, and is defined as follows:
seq1=Input(dtype='int32',shape=(400,),name='input_text')
seq2=Input(dtype='int32',shape=(20,),name='input_titles')
embeddeding=Embedding(max_features,embedding_dims,dropout=0.3)
encoding_1=embeddeding(seq1)
encoding_2=embeddeding(seq2)
filter_lengths = [1,3,6]
def max_1d(X):
return K.max(X, axis=1)
convs1=[]
convs2=[]
for fl in filter_lengths:
conv1=Convolution1D(nb_filter=nb_filter,
filter_length=fl,
border_mode='valid',
activation='relu',
subsample_length=1)(encoding_1)
conv1=Lambda(max_1d, output_shape=(nb_filter,))(conv1)
convs1.append(conv1)
conv2=Convolution1D(nb_filter=nb_filter,
filter_length=fl,
border_mode='valid',
activation='relu',
subsample_length=1)(encoding_2)
conv2=Lambda(max_1d, output_shape=(nb_filter,))(conv2)
convs2.append(conv2)
m=merge([*convs1,*convs2],mode='concat')
m=Highway(activation='relu')(m)
m=Highway(activation='relu')(m)
m=Dropout(0.5)(m)
hovedkategori_loss=Dense(labsHovedKat.shape[1],activation='sigmoid',name='hovedkategori')(m)
m1=merge([hovedkategori_loss,m],mode='concat')
underkategori_loss=Dense(labsUnderKat.shape[1],activation='sigmoid',name='underkategori')(m1)
model=Model(input=[seq1,seq2],output=[hovedkategori_loss,underkategori_loss])
model.compile(optimizer='adam',loss='binary_crossentropy',metrics={'hovedkategori':'accuracy','underkategori':'accuracy'})
I am loading a model from an hdf5 file with using keras.models.load_model. when i try to predict using that model the following error raises:
'thread._local' object has no attribute 'value'
the code is:
''' from keras.preprocessing.image import img_to_array from keras.models import load_model
model_path = (''/home/pi/classifier/models/model.hdf5'') classifier = load_model(model_path, compile=False) preds = classifier.predict(img_to_array(image)) ''' where image is numpy array read by opencv
Platform: Raspberry Pi 4 OS: Raspbian Buster TF version: 1.13.1 Keras version: 2.3.0
with theano backend (CPU or GPU without cnDNN), I could train reproducible model by
fixed_seed_num = 1234
nunpy.random.seed(fixed_seed_num)
random.seed(fixed_seed_num) # not sure if needed or not
While in pure tensorflow without keras wrapper, it could also be reproducible by
tersorflow.set_random_seed(fixed_seed_num)
Don't know why, but in Keras + tensorflow backend, non of the above could give reproducible training model.
Environment:
BTW: it would be great if keras could expose an unified API for reproducible training. something like:
keras.set_random_seed(fixed_seed_num)
Hi there, I am trying to implement a classification problem with three classes: 0,1 and 2. I would like to fine tune my cost function so that missclassification is weighted some how. In particular, predicting 1 instead of 2 should give twice the cost than predicting 0. writing it in a table format, it should be something like that:
Costs: Predicted: 0 | 1 | 2 __________________________ Actual 0 | 0 | 0.25 | 0.25 1 | 0.25 | 0 | 0.5 2 | 0.25 | 0.5 | 0
I really like keras framework, it would be nice if it is possible to implement it and not having to dig into tensorflow or theano code.
Thanks
Help me currently i am doing at Jupyter notebook anaconda i got invalid syntax when running this code trainer.trainModel() currently i am using Python 3.7.6 TenserflowVer: 2.4.0
from imageai.Detection.Custom import DetectionModelTrainer
trainer = DetectionModelTrainer()
trainer.setModelTypeAsYOLOv3()
trainer.setDataDirectory(data_directory="hololens")
trainer.setTrainConfig(object_names_array=["hololens"], batch_size=4, num_experiments=100, train_from_pretrained_model=("pretrained-yolov3.h5")
trainer.trainModel()
Describe the problem.
Specifying classifier_activation=softmax in ConvNeXt models does not work, instead it returns the model with a linear classifier layer.
Inspecting source code: https://github.com/keras-team/keras/blob/e6784e4302c7b8cd116b74a784f4b78d60e83c26/keras/applications/convnext.py#L524-L526
Definition of Head function:
https://github.com/keras-team/keras/blob/e6784e4302c7b8cd116b74a784f4b78d60e83c26/keras/applications/convnext.py#L327-L348
Find the Colab gist here.
If OK I can open a PR and fix it.
Description of issue (what needs to be changed):. Correct links. https://keras.io/guides/preprocessing_layers/
As it said, the following layers apply random augmentation transforms to a batch of images. They are only active during training.
tf.keras.layers.RandomCrop tf.keras.layers.RandomFlip tf.keras.layers.RandomTranslation tf.keras.layers.RandomRotation tf.keras.layers.RandomZoom tf.keras.layers.RandomHeight tf.keras.layers.RandomWidth tf.keras.layers.RandomContrast
So, in the model, if we use these layers, they will be active (i.e. model.fit / model(..., training=True) during training but will be inactive when we call model.evaluate and model.predict / model(..., training=False). A sample model would be
model = keras.Sequential(
[
tf.keras.layers.RandomHeight,
tf.keras.layers.RandomContrast,
tf.keras.layers.Conv2D,
tf.keras.layers.Dense
...
]
)
Queries
model.predict? We can use model(..., training=True) but (a). It's best suited for small amount of data, and (b). some other function (dropout) might be active.epoch frequency support for TerminateOnNaN callback.
Batch level hook fails when used with distributed strategies, as logs is not of type dict (tensorflow.python.distribute.coordinator.values.RemoteValueImpl)
System Info
Current Behaviour: I implemented a custom layer, and use this layer to build a model. After training it, the original model gave a predicted value A, and the original model is saved as a h5 file. Then I load model from the h5 file, but the loaded model gave a different predicted value B, which means the results are not reproducible. Normally, the two models are supposed to give the same predicted value. The custom layer is as simple as a Dense layer, and I have already locate that the custom layer caused the above problem. Actually, if I comment the line of custom layer, and uncomment the line below it (which is an original tf Dense layer), both the original model and the loaded_model gave the same results.
Standalone code to reproduce the issue https://colab.research.google.com/drive/19_8DqzfC2JadKM9ZykJRLDEcxEkzjrT7?usp=sharing Can also refer to the link where the issue is firstly posted: https://github.com/tensorflow/tensorflow/issues/59041
Relevant log output 2022-12-29 10:33:52.034627: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 2022-12-29 10:33:53.136713: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set 2022-12-29 10:33:53.138072: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1 2022-12-29 10:33:53.198743: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: pciBusID: 0000:3d:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6 coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 23.70GiB deviceMemoryBandwidth: 871.81GiB/s 2022-12-29 10:33:53.198834: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 2022-12-29 10:33:53.203166: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11 2022-12-29 10:33:53.203273: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11 2022-12-29 10:33:53.204328: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10 2022-12-29 10:33:53.204657: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10 2022-12-29 10:33:53.209031: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10 2022-12-29 10:33:53.209857: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11 2022-12-29 10:33:53.210030: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8 2022-12-29 10:33:53.212456: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0 2022-12-29 10:33:53.335970: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-12-29 10:33:53.348062: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set 2022-12-29 10:33:53.349549: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: pciBusID: 0000:3d:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6 coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 23.70GiB deviceMemoryBandwidth: 871.81GiB/s 2022-12-29 10:33:53.349604: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 2022-12-29 10:33:53.349644: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11 2022-12-29 10:33:53.349654: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11 2022-12-29 10:33:53.349664: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10 2022-12-29 10:33:53.349673: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10 2022-12-29 10:33:53.349682: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10 2022-12-29 10:33:53.349691: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11 2022-12-29 10:33:53.349701: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8 2022-12-29 10:33:53.352041: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0 2022-12-29 10:33:53.352076: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 2022-12-29 10:33:53.873305: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix: 2022-12-29 10:33:53.873357: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0 2022-12-29 10:33:53.873366: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N 2022-12-29 10:33:53.877072: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 22430 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:3d:00.0, compute capability: 8.6) /usr/local/lib64/python3.8/site-packages/tensorflow/python/data/ops/dataset_ops.py:3503: UserWarning: Even though the tf.config.experimental_run_functions_eagerly option is set, this option does not apply to tf.data functions. tf.data functions are still traced and executed as graphs. warnings.warn( 2022-12-29 10:33:53.990568: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2) 2022-12-29 10:33:53.997553: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2600000000 Hz 2022-12-29 10:33:54.015124: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11 2022-12-29 10:33:54.679072: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11 2022-12-29 10:33:54.679260: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once. 625/625 [==============================] - 5s 7ms/step - loss: 0.4327 [[0.44245386] [0.6916534 ] [0.49306032] [0.98741436] [0.7631112 ]] [[0.48893338] [0.54947186] [0.40105245] [0.56347597] [0.32270208]]
System information.
TensorFlow version (you are using): 2.9 Are you willing to contribute it (Yes/No) : Yes
Describe the feature and the current behavior/state: Currently, I couldn't find any good implementation for converting label to ordinal-regression/classification label. Either people use for loop to do it or they convert labels element-wise rather than batch-wise.
Describe the feature clearly here. Be sure to convey here why the requested feature is needed. Any brief description about the use-case would help.:
This feature is needed for Ordinal Regression / Classification task. In tf.keras.utils there is to_categorical to convert labels to one_hot labels for classification. But for ordinal regression / classification we need to transform the labels differently. But currently tf.keras doesn't have the support of this feature.
Here's an example of the feature with contrast to one_hot encoding,
Input:
![using a custom layer, loaded_model cannot give the same predicted value compared with original model [results not reproducible]](https://avatars.githubusercontent.com/u/7259514?v=4)
label = np.array([0, 1, 2, 3, 1])
one_hot = tf.keras.utils.to_categorical(label, num_classes=4)
Output:
[[1, 0, 0, 0],
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1],
[0, 1, 0, 0]]
Input:
label = np.array([0, 1, 2, 3, 1])
ordinal = to_ordinal(label, num_classes=4)
Output:
[[0, 0, 0],
[1, 0, 0],
[1, 1, 0],
[1, 1, 1],
[1, 0, 0]]
Will this change the current api? How? This feature currently doesn't have any implementation.
Who will benefit from this feature? All the members of the community. Ordinal regression/classification is quite interesting and getting popular day by day. But it is hardly mentioned in the literature how to transform the label for ordinal regression. I think this feature will be a nice addition to keras for everyone.
Following is the Numpy implementation but the same code can be used for TensorFlow as well,
def to_ordinal(y, num_classes=None):
if not num_classes:
num_classes = np.max(y) + 1
range_values = np.arange(num_classes-1)[None,:]
range_values = np.tile(range_values, [y.shape[0], 1])
ordinal_label = np.where(range_values < y[:, None], 1, 0)
return ordinal_label
Please see the release history at https://github.com/tensorflow/tensorflow/releases/tag/v2.11.0 for more details.
Full Changelog: https://github.com/keras-team/keras/compare/v2.10.0...v2.11.0
Source code(tar.gz)Full Changelog: https://github.com/keras-team/keras/compare/v2.11.0-rc2...v2.11.0-rc3
Source code(tar.gz)Full Changelog: https://github.com/keras-team/keras/compare/v2.11.0-rc1...v2.11.0-rc2
Source code(tar.gz)Please see the release history at https://github.com/tensorflow/tensorflow/releases/tag/v2.11.0-rc1 for more details.
exclude_from_weight_decay to AdamW by @markub3327 in https://github.com/keras-team/keras/pull/16274reset_states not working when invoked within a tf.function in graph mode. by @copybara-service in https://github.com/keras-team/keras/pull/16400f prefix on f-strings fix by @code-review-doctor in https://github.com/keras-team/keras/pull/16459AutoShardPolicy.DATA for TensorLike datasets by @lgeiger in https://github.com/keras-team/keras/pull/16604is_legacy_optimizer to optimizer config to keep saving/loading consistent. by @copybara-service in https://github.com/keras-team/keras/pull/16842tf.keras.preprocessing.image* to tf.keras.utils* by @chunduriv in https://github.com/keras-team/keras/pull/16864distribute_reduction_method in Model. by @kretes in https://github.com/keras-team/keras/pull/16664metrics parameter in compile by @matangover in https://github.com/keras-team/keras/pull/16893Returns section in compute_output_shape by @chunduriv in https://github.com/keras-team/keras/pull/16955Full Changelog: https://github.com/keras-team/keras/compare/v2.10.0...v2.11.0-rc1
Source code(tar.gz)Please see the release history at https://github.com/tensorflow/tensorflow/releases/tag/v2.10.0 for more details.
Full Changelog: https://github.com/keras-team/keras/compare/v2.9.0...v2.10.0
Source code(tar.gz)Please see the release history at https://github.com/tensorflow/tensorflow/releases/tag/v2.10.0-rc3 for more details.
exclude_from_weight_decay to AdamW by @markub3327 in https://github.com/keras-team/keras/pull/16274reset_states not working when invoked within a tf.function in graph mode. by @copybara-service in https://github.com/keras-team/keras/pull/16400f prefix on f-strings fix by @code-review-doctor in https://github.com/keras-team/keras/pull/16459AutoShardPolicy.DATA for TensorLike datasets by @lgeiger in https://github.com/keras-team/keras/pull/16604is_legacy_optimizer to optimizer config to keep saving/loading consistent. by @copybara-service in https://github.com/keras-team/keras/pull/16842is_legacy_optimizer to optimizer config to keep saving/loading … by @qlzh727 in https://github.com/keras-team/keras/pull/16856Full Changelog: https://github.com/keras-team/keras/compare/v2.9.0-rc0...v2.10.0-rc1
Source code(tar.gz)Please see the release history at https://github.com/tensorflow/tensorflow/releases/tag/v2.9.0 for more details.
Full Changelog: https://github.com/keras-team/keras/compare/v2.8.0...v2.9.0
Source code(tar.gz)Full Changelog: https://github.com/keras-team/keras/compare/v2.9.0-rc1...v2.9.0-rc2
Source code(tar.gz)Full Changelog: https://github.com/keras-team/keras/compare/v2.9.0-rc0...v2.9.0-rc1
Source code(tar.gz)Please see https://github.com/tensorflow/tensorflow/blob/r2.9/RELEASE.md for Keras release notes.
tf.keras:
tf.keras.applications.resnet_rs models. This includes the ResNetRS50, ResNetRS101, ResNetRS152, ResNetRS200, ResNetRS270, ResNetRS350 and ResNetRS420 model architectures. The ResNetRS models are based on the architecture described in Revisiting ResNets: Improved Training and Scaling Strategiestf.keras.optimizers.experimental.Optimizer. The reworked optimizer gives more control over different phases of optimizer calls, and is easier to customize. We provide Adam, SGD, Adadelta, AdaGrad and RMSprop optimizers based on tf.keras.optimizers.experimental.Optimizer. Generally the new optimizers work in the same way as the old ones, but support new constructor arguments. In the future, the symbols tf.keras.optimizers.Optimizer/Adam/etc will point to the new optimizers, and the previous generation of optimizers will be moved to tf.keras.optimizers.legacy.Optimizer/Adam/etc.tf.keras.layers.UnitNormalization.tf.keras.regularizers.OrthogonalRegularizer, a new regularizer that encourages orthogonality between the rows (or columns) or a weight matrix.tf.keras.layers.RandomBrightness layer for image preprocessing.tf.keras.utils.disable_interactive_logging() to write the logs to ABSL logging. You can also use tf.keras.utils.enable_interactive_logging() to change it back to stdout, or tf.keras.utils.is_interactive_logging_enabled() to check if interactive logging is enabled.verbose argument of Model.evaluate() and Model.predict() to "auto", which defaults to verbose=1 for most cases and defaults to verbose=2 when used with ParameterServerStrategy or with interactive logging disabled.jit_compile in Model.compile() now applies to Model.evaluate() and Model.predict(). Setting jit_compile=True in compile() compiles the model's training, evaluation, and inference steps to XLA. Note that jit_compile=True may not necessarily work for all models.tf.keras.dtensor namespace. The APIs are still classified as experimental. You are welcome to try it out. Please check the tutoral and guide on https://www.tensorflow.org/ for more details about DTensor.assign_sub when computing moving_average_update by @lgeiger in https://github.com/keras-team/keras/pull/15773keras.utils.register_keras_serializable flows we are expecting users to follow work, and will continue to work with the new design and implementation coming in. by @copybara-service in https://github.com/keras-team/keras/pull/15992classifier_activation argument accessible for DenseNet and NASNet models by @adrhill in https://github.com/keras-team/keras/pull/16005keras.callbacks.BackupAndRestore docs by @lgeiger in https://github.com/keras-team/keras/pull/16018apply(), get_updates_for(), get_losses_for(), and remove the inputs argument in the add_loss() method. by @copybara-service in https://github.com/keras-team/keras/pull/16046DenseFeatures by @gadagashwini in https://github.com/keras-team/keras/pull/16165Full Changelog: https://github.com/keras-team/keras/compare/v2.8.0-rc0...v2.9.0-rc0
Source code(tar.gz)Please see the release history at https://github.com/tensorflow/tensorflow/releases/tag/v2.8.0 for more details.
Source code(tar.gz)Full Changelog: https://github.com/keras-team/keras/compare/v2.8.0-rc0...v2.8.0-rc1
Source code(tar.gz)Please see https://github.com/tensorflow/tensorflow/blob/r2.8/RELEASE.md for Keras release notes.
tf.keras:
tf.keras.layers.experimental.preprocessing.HashedCrossing
layer which applies the hashing trick to the concatenation of crossed
scalar inputs. This provides a stateless way to try adding feature crosses
of integer or string data to a model.keras.layers.experimental.preprocessing.CategoryCrossing. Users
should migrate to the HashedCrossing layer or use
tf.sparse.cross/tf.ragged.cross directly.standardize and split modes to TextVectorization.
standardize="lower" will lowercase inputs.standardize="string_punctuation" will remove all puncuation.split="character" will split on every unicode character.output_mode argument to the Discretization and Hashing
layers with the same semantics as other preprocessing layers. All
categorical preprocessing layers now support output_mode.tf.keras.mixed_precision.Policy, unless constructed with
output_mode="int" in which case output will be tf.int64.
The output type of any preprocessing layer can be controlled individually
by passing a dtype argument to the layer.tf.random.Generator for keras initializers and all RNG code.
tf.random.Generator in keras backend, which will be the new backend for
all the RNG in Keras. We plan to switch on the new code path by default in
tf 2.8, and the behavior change will likely to cause some breakage on user
side (eg if the test is checking against some golden nubmer). These 3 APIs
will allow user to disable and switch back to legacy behavior if they
prefer. In future (eg tf 2.10), we expect to totally remove the legacy
code path (stateful random Ops), and these 3 APIs will be removed as well.tf.keras.callbacks.experimental.BackupAndRestore is now available as
tf.keras.callbacks.BackupAndRestore. The experimental endpoint is
deprecated and will be removed in a future release.tf.keras.experimental.SidecarEvaluator is now available as
tf.keras.utils.SidecarEvaluator. The experimental endpoint is
deprecated and will be removed in a future release.Model.train_step() is now
customizable via overriding Model.compute_metrics().Model.train_step() is now
customizable via overriding Model.compute_loss().jit_compile added to Model.compile() on an opt-in basis to compile the
model's training step with XLA. Note that
jit_compile=True may not necessarily work for all models..tar.gz by @copybara-service in https://github.com/keras-team/keras/pull/14777int given for float args by @SamuelMarks in https://github.com/keras-team/keras/pull/14900MultiHeadAttention layer call argument return_attention_scores. by @guillesanbri in https://github.com/keras-team/keras/pull/14920optimizer to optimizers by @MohamedAliRashad in https://github.com/keras-team/keras/pull/15227stacklevel for warnings.warn to make it easier to identify which lines throw warnings by @harupy in https://github.com/keras-team/keras/pull/15209plot_model, model_to_dot and model.summary() by @krishrustagi in https://github.com/keras-team/keras/pull/15318expand_nested bug fix and changing model.summary style by @krishrustagi in https://github.com/keras-team/keras/pull/15355timeseries_dataset_from_array by @europeanplaice in https://github.com/keras-team/keras/pull/15646Full Changelog: https://github.com/keras-team/keras/compare/v2.7.0-rc0...v2.8.0-rc0
Source code(tar.gz)Please see the release history at https://github.com/tensorflow/tensorflow/releases/tag/v2.7.0 for more details.
Source code(tar.gz)Full Changelog: https://github.com/keras-team/keras/compare/v2.7.0-rc1...v2.7.0-rc2
Source code(tar.gz)Cherrypicked the documentation update for functional model slicing.
Source code(tar.gz)Keras 2.6.0 is the first release of TensorFlow implementation of Keras in the present repo.
The code under tensorflow/python/keras is considered legacy and will be removed in future releases (tf 2.7 or later). For any user who import tensorflow.python.keras, please update your code to public tf.keras instead.
The API endpoints for tf.keras stay unchanged, but are now backed by the keras PIP package. All Keras-related PRs and issues should now be directed to the GitHub repository keras-team/keras.
For the detailed release notes about tf.keras behavior changes, please take a look for tensorflow release notes.
Source code(tar.gz)Keras Release 2.6.0 RC3 fix a security issue for loading keras models via yaml, which could allow arbitrary code execution.
Source code(tar.gz)Keras 2.6.0 RC2 is a minor bug-fix release.
Keras 2.6.0 RC1 is a minor bug-fix release
Keras 2.6.0 is the first release of TensorFlow implementation of Keras in the present repo.
The code under tensorflow/python/keras is considered legacy and will be removed in future releases (tf 2.7 or later). For any user who import tensorflow.python.keras, please update your code to public tf.keras instead.
The API endpoints for tf.keras stay unchanged, but are now backed by the keras PIP package. All Keras-related PRs and issues should now be directed to the GitHub repository keras-team/keras.
For the detailed release notes about tf.keras behavior changes, please take a look for tensorflow release notes.
As previously announced, we have discontinued multi-backend Keras to refocus exclusively on the TensorFlow implementation of Keras.
In the future, we will develop the TensorFlow implementation of Keras in the present repo, at keras-team/keras. For the time being, it is being developed in tensorflow/tensorflow and distributed as tensorflow.keras. In this future, the keras package on PyPI will be the same as tf.keras.
This release (2.4.0) simply redirects all APIs in the standalone keras package to point to tf.keras. This helps address user confusion regarding differences and incompatibilities between tf.keras and the standalone keras package. There is now only one Keras: tf.keras.
from tensorflow import keras, rather than import keras, for the time being.Keras 2.3.1 is a minor bug-fix release. In particular, it fixes an issue with using Keras models across multiple threads.
Keras 2.3.0 is the first release of multi-backend Keras that supports TensorFlow 2.0. It maintains compatibility with TensorFlow 1.14, 1.13, as well as Theano and CNTK.
This release brings the API in sync with the tf.keras API as of TensorFlow 2.0. However note that it does not support most TensorFlow 2.0 features, in particular eager execution. If you need these features, use tf.keras.
This is also the last major release of multi-backend Keras. Going forward, we recommend that users consider switching their Keras code to tf.keras in TensorFlow 2.0. It implements the same Keras 2.3.0 API (so switching should be as easy as changing the Keras import statements), but it has many advantages for TensorFlow users, such as support for eager execution, distribution, TPU training, and generally far better integration between low-level TensorFlow and high-level concepts like Layer and Model. It is also better maintained.
Development will focus on tf.keras going forward. We will keep maintaining multi-backend Keras over the next 6 months, but we will only be merging bug fixes. API changes will not be ported.
size(x) to backend API.add_metric method added to Layer / Model (used in a similar way as add_loss, but for metrics), as well as the metrics property.layer.weights (including layer.trainable_weights or layer.non_trainable_weights as appropriate).Loss base class). This enables losses to be parameterized via constructor arguments. Loss classes added:
MeanSquaredErrorMeanAbsoluteErrorMeanAbsolutePercentageErrorMeanSquaredLogarithmicErrorBinaryCrossentropyCategoricalCrossentropySparseCategoricalCrossentropyHingeSquaredHingeCategoricalHingePoissonLogCoshKLDivergenceHuberMetric base class). This enables metrics to be stateful (e.g. required for supported AUC) and to be parameterized via constructor arguments. Metric classes added:
AccuracyMeanSquaredErrorHingeCategoricalHingeSquaredHingeFalsePositivesTruePositivesFalseNegativesTrueNegativesBinaryAccuracyCategoricalAccuracyTopKCategoricalAccuracyLogCoshErrorPoissonKLDivergenceCosineSimilarityMeanAbsoluteErrorMeanAbsolutePercentageErrorMeanSquaredErrorMeanSquaredLogarithmicErrorRootMeanSquaredErrorBinaryCrossentropyCategoricalCrossentropyPrecisionRecallAUCSparseCategoricalAccuracySparseTopKCategoricalAccuracySparseCategoricalCrossentropyreset_metrics argument to train_on_batch and test_on_batch. Set this to True to maintain metric state across different batches when writing lower-level training/evaluation loops. If False, the metric value reported as output of the method call will be the value for the current batch only.model.reset_metrics() method to Model. Use this at the start of an epoch to clear metric state when writing lower-level training/evaluation loops.lr to learning_rate for all optimizers.decay for all optimizers. For learning rate decay, use LearningRateSchedule objects in tf.keras.batch_size argument is deprecated (ignored) when used with TF 2.0write_grads is deprecated (ignored) when used with TF 2.0embeddings_freq, embeddings_layer_names, embeddings_metadata, embeddings_data are deprecated (ignored) when used with TF 2.0metrics=['acc'], your metric will be reported under the string "acc", not "accuracy", and inversely metrics=['accuracy'] will be reported under the string "accuracy".sigmoid (from hard_sigmoid) in all RNN layers.Keras 2.2.5 is the last release of Keras that implements the 2.2.* API. It is the last release to only support TensorFlow 1 (as well as Theano and CNTK).
The next release will be 2.3.0, which makes significant API changes and add support for TensorFlow 2.0. The 2.3.0 release will be the last major release of multi-backend Keras. Multi-backend Keras is superseded by tf.keras.
At this time, we recommend that Keras users who use multi-backend Keras with the TensorFlow backend switch to tf.keras in TensorFlow 2.0. tf.keras is better maintained and has better integration with TensorFlow features.
ResNet101, ResNet152, ResNet50V2, ResNet101V2, ResNet152V2.evaluate and predict.
callbacks argument (list of callback instances) in evaluate and predict.on_train_batch_begin, on_train_batch_end, on_test_batch_begin, on_test_batch_end, on_predict_batch_begin, on_predict_batch_end, as well as on_test_begin, on_test_end, on_predict_begin, on_predict_end. Methods on_batch_begin and on_batch_end are now aliases for on_train_batch_begin and on_train_batch_end.save_model and load_model (in place of the filepath)name argument in Sequential constructorvalidation_freq argument in fit, controlling the frequency of validation (e.g. setting validation_freq=3 would run validation every 3 epochs)fit, evaluate, and predict, instead of having to use *_generator methods.
max_queue_size, workers, use_multiprocessing to these methods.dilation_rate argument in layer DepthwiseConv2D.m to max_value.dtype argument in base layer (default dtype for layer's weights).Tokenizer class.H5Dict and model_to_dot to utils.expand_nested, dpi to plot_model.update_sub, stack, cumsum, cumprod, foldl, foldr to CNTK backendmerge_repeated argument to ctc_decode in TensorFlow backendThanks to the 89 committers who contributed code to this release!
Source code(tar.gz)This is a bugfix release, addressing two issues:
Sequential model.See here for the changelog since 2.2.2.
Source code(tar.gz)ThresholdedReLU and LeakyReLU into the ReLU layer.ReLU layer now takes new arguments negative_slope and threshold, and the relu function in the backend takes a new threshold argument.update_freq argument in TensorBoard callback, controlling how often to write TensorBoard logs.exponential function to keras.activations.data_format argument in all 4 Pooling1D layers.interpolation argument in UpSampling2D layer and in resize_images backend function, supporting modes "nearest" (previous behavior, and new default) and "bilinear" (new).dilation_rate argument in Conv2DTranspose layer and in conv2d_transpose backend function.LearningRateScheduler now receives the lr key as part of the logs argument in on_epoch_end (current value of the learning rate).GlobalAveragePooling1D layer support masking.filepath argument save_model and model.save() can now be a h5py.Group instance.restore_best_weights to EarlyStopping callback (optionally reverts to the weights that obtained the highest monitored score value).dtype argument to keras.utils.to_categorical.run_options and run_metadata as optional session arguments in model.compile() for the TensorFlow backend.Sequential.get_config(). Previously, the return value was a list of the config dictionaries of the layers of the model. Now, the return value is a dictionary with keys layers, name, and an optional key build_input_shape. The old config is equivalent to new_config['layers']. This makes the output of get_config consistent across all model classes.Thanks to our 38 contributors whose commits are featured in this release:
@BertrandDechoux, @ChrisGll, @Dref360, @JamesHinshelwood, @MarcoAndreaBuchmann, @ageron, @alfasst, @blue-atom, @chasebrignac, @cshubhamrao, @danFromTelAviv, @datumbox, @farizrahman4u, @fchollet, @fuzzythecat, @gabrieldemarmiesse, @hadifar, @heytitle, @hsgkim, @jankrepl, @joelthchao, @knightXun, @kouml, @linjinjin123, @lvapeab, @nikoladze, @ozabluda, @qlzh727, @roywei, @rvinas, @sriyogesh94, @tacaswell, @taehoonlee, @tedyu, @xuhdev, @yanboliang, @yongzx, @yuanxiaosc
Source code(tar.gz)This is a bugfix release, fixing a significant bug in multi_gpu_model.
For changes since version 2.2.0, see release notes for Keras 2.2.1.
Source code(tar.gz)output_padding argument in Conv2DTranspose (to override default padding behavior).No breaking changes recorded.
Thanks to our 33 contributors whose commits are featured in this release:
@Ajk4, @Anner-deJong, @Atcold, @Dref360, @EyeBool, @ageron, @briannemsick, @cclauss, @davidtvs, @dstine, @eTomate, @ebatuhankaynak, @eliberis, @farizrahman4u, @fchollet, @fuzzythecat, @gabrieldemarmiesse, @jlopezpena, @kamil-kaczmarek, @kbattocchi, @kmader, @kvechera, @maxpumperla, @mkaze, @pavithrasv, @rvinas, @sachinruk, @seriousmac, @soumyac1999, @taehoonlee, @yanboliang, @yongzx, @yuyang-huang
Source code(tar.gz)Reproducible Experiment Platform (REP) REP is ipython-based environment for conducting data-driven research in a consistent and reproducible way. Main
HyperTag HyperTag helps humans intuitively express how they think about their files using tags and machine learning.
Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos This repository is the official tensorflow python implementation
gensim – Topic Modelling in Python Gensim is a Python library for topic modelling, document indexing and similarity retrieval with large corpora. Targ
Human Attention for Text Classification Re-implementation of the paper Human Attention Maps for Text Classification: Do Humans and Neural Networks Foc
SURREACT: Synthetic Humans for Action Recognition from Unseen Viewpoints Gül Varol, Ivan Laptev and Cordelia Schmid, Andrew Zisserman, Synthetic Human
Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition The official code of ABINet (CVPR 2021, Oral).
SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements (CVPR 2021) This repository contains the official PyTorch implementa
Reconstructing 3D Human Pose by Watching Humans in the Mirror Qi Fang*, Qing Shuai*, Junting Dong, Hujun Bao, Xiaowei Zhou CVPR 2021 Oral The videos a
The Power of Points for Modeling Humans in Clothing (ICCV 2021) This repository contains the official PyTorch implementation of the ICCV 2021 paper: T
The PASS dataset: pretrained models and how to get the data - PASS: Pictures without humAns for Self-Supervised Pretraining
AI_Personal_Voice_Assistant_Using_Python A project to build an AI voice assistant using Python . The Voice assistant interacts with the humans to perf
ICON: Implicit Clothed humans Obtained from Normals Yuliang Xiu · Jinlong Yang · Dimitrios Tzionas · Michael J. Black CVPR 2022 News ?? [2022/04/26] H
Ivy is a templated deep learning framework which maximizes the portability of deep learning codebases. Ivy wraps the functional APIs of existing frameworks. Framework-agnostic functions, libraries and layers can then be written using Ivy, with simultaneous support for all frameworks. Ivy currently supports Jax, TensorFlow, PyTorch, MXNet and Numpy. Check out the docs for more info!
Deep-rPPG: Camera-based pulse estimation using deep learning tools Deep learning (neural network) based remote photoplethysmography: how to extract pu
deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.
Stock Price Prediction Using Deep Learning Univariate Time Series Predicting stock price using historical data of a company using Neural networks for
CANDIY-spectrum Human analyis of chemical spectra such as Mass Spectra (MS), Inf
Deep Learning: Architectures & Methods Project: Deep Learning for Audio Super-Resolution Figure: Example visualization of the method and baseline as a
662 Nov 20, 2022
165 Nov 4, 2022
287 Jan 6, 2023
13.8k Jan 3, 2023
15 Dec 13, 2021
59 Dec 14, 2022
334 Dec 31, 2022
133 Jan 5, 2023
178 Dec 13, 2022
249 Dec 22, 2022
1 Oct 30, 2021
1.1k Jan 4, 2023
8.2k Jan 2, 2023
138 Dec 17, 2022
63 Oct 17, 2022
7 Nov 27, 2022
1 Jan 3, 2022
16 Dec 23, 2022