全局指针统一处理嵌套与非嵌套NER

Overview

GlobalPointer

全局指针统一处理嵌套与非嵌套NER。

介绍

效果

人民日报NER

验证集F1 测试集F1 训练速度 预测速度
CRF 96.39% 95.46% 1x 1x
GlobalPointer (w/o RoPE) 54.35% 62.59% 1.61x 1.13x
GlobalPointer (w/ RoPE) 96.25% 95.51% 1.56x 1.11x

CLUENER

验证集F1 测试集F1 训练速度 预测速度
CRF 79.51% 78.70% 1x 1x
GlobalPointer 80.03% 79.44% 1.22x 1x

CMeEE

验证集F1 测试集F1 训练速度 预测速度
CRF 63.81% 64.39% 1x 1x
GlobalPointer 64.84% 65.98% 1.52x 1.13x

环境

需要bert4keras >= 0.10.6。个人实验环境是tensorflow 1.14 + keras 2.3.1 + bert4keras 0.10.6。

交流

QQ交流群:808623966,微信群请加机器人微信号spaces_ac_cn

You might also like...
Comments
  • AttributeError: 'tuple' object has no attribute 'layer'

    AttributeError: 'tuple' object has no attribute 'layer'

    请问 大大这是什么错误呀 环境: python3.8 tensorflow 1.5.1 keras2.3.1 bert4keras-0.10.6

    出错过程: 2021-07-21 23:23:30.850241: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10 2021-07-21 23:23:30.850272: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10 2021-07-21 23:23:30.850304: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11 2021-07-21 23:23:30.850334: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11 2021-07-21 23:23:30.850365: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8 2021-07-21 23:23:30.855468: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1794] Adding visible gpu devices: 0 2021-07-21 23:23:30.855547: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 2021-07-21 23:23:31.146373: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1206] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-07-21 23:23:31.146412: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] 0 2021-07-21 23:23:31.146418: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1225] 0: N 2021-07-21 23:23:31.147960: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1351] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21829 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3090, pci bus id: 0000:d5:00.0, compute capability: 8.6) WARNING:tensorflow:From /home/xhw205/.environments/venv/lib/python3.8/site-packages/bert4keras/layers.py:1199: The name tf.matrix_band_part is deprecated. Please use tf.linalg.band_part instead.

    WARNING:tensorflow:From /home/xhw205/.environments/venv/lib/python3.8/site-packages/bert4keras/layers.py:1199: The name tf.matrix_band_part is deprecated. Please use tf.linalg.band_part instead.

    Traceback (most recent call last): File "glp.py", line 103, in model = Model(model.input, output) File "/home/xhw205/.environments/venv/lib/python3.8/site-packages/tensorflow_core/python/keras/engine/training.py", line 147, in init super(Model, self).init(*args, **kwargs) File "/home/xhw205/.environments/venv/lib/python3.8/site-packages/tensorflow_core/python/keras/engine/network.py", line 164, in init self._init_graph_network(*args, **kwargs) File "/home/xhw205/.environments/venv/lib/python3.8/site-packages/tensorflow_core/python/training/tracking/base.py", line 457, in _method_wrapper result = method(self, *args, **kwargs) File "/home/xhw205/.environments/venv/lib/python3.8/site-packages/tensorflow_core/python/keras/engine/network.py", line 270, in _init_graph_network self._validate_graph_inputs_and_outputs() File "/home/xhw205/.environments/venv/lib/python3.8/site-packages/tensorflow_core/python/keras/engine/network.py", line 1482, in _validate_graph_inputs_and_outputs layer = x._keras_history.layer AttributeError: 'tuple' object has no attribute 'layer'

    opened by xhw205 1
  • 使用tf2.4 保存pb报错

    使用tf2.4 保存pb报错

    加载预训练模型

    model = build_transformer_model( config_path=config_path, checkpoint_path=checkpoint_path, return_keras_model=False )

    output = GlobalPointer(len(categories), 64)(model.output) model = keras.models.Model(model.input, output) model.summary()

    model.compile( loss=global_pointer_crossentropy, optimizer=Adam(learning_rate), metrics=[global_pointer_f1_score] )

    evaluator = Evaluator() train_generator = data_generator(train_data, batch_size) model.fit( train_generator.forfit(), steps_per_epoch=len(train_generator), epochs=epochs, callbacks=[evaluator] ) export_path = 'model' version = "1"
    model.save(export_path + version, save_format="tf")

    AttributeError: 'Dropout' object has no attribute '_saved_model_inputs_spec'

    opened by liuliuNLP 1
Owner
苏剑林(Jianlin Su)
科学爱好者
苏剑林(Jianlin Su)