This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

You might also like...
Script for automatic dump and brute-force passwords using Volatility Framework
Script for automatic dump and brute-force passwords using Volatility Framework

Volatility-auto-hashdump Script for automatic dump and brute-force passwords using Volatility Framework

 Subdomain enumeration,Web scraping and finding usernames automation script written in python
Subdomain enumeration,Web scraping and finding usernames automation script written in python

Subdomain enumeration,Web scraping and finding usernames automation script written in python

Providing DevOps and security teams script to identify cloud workloads that may be vulnerable to the Log4j vulnerability(CVE-2021-44228) in their AWS account.
Providing DevOps and security teams script to identify cloud workloads that may be vulnerable to the Log4j vulnerability(CVE-2021-44228) in their AWS account.

We are providing DevOps and security teams script to identify cloud workloads that may be vulnerable to the Log4j vulnerability(CVE-2021-44228) in their AWS account. The script enables security teams to identify external-facing AWS assets by running the exploit on them, and thus be able to map them and quickly patch them

Provides script to download and format public IP lists related to the Log4j exploit.

Provides script to download and format public IP lists related to the Log4j exploit. Current format includes: plain list, Cisco ASA Network Group.

This python script will automate the testing for the Log4J vulnerability for HTTP and HTTPS connections.

Log4J-Huntress-Automate-Script This python script will automate the testing for the Log4J vulnerability for HTTP and HTTPS connections. Pre-Requisits

A script to search, scrape and scan for Apache Log4j CVE-2021-44228 affected files using Google dorks

Log4j dork scanner This is an auto script to search, scrape and scan for Apache Log4j CVE-2021-44228 affected files using Google dorks. Installation:

Yesitsme - Simple OSINT script to find Instagram profiles by name and e-mail/phone
Yesitsme - Simple OSINT script to find Instagram profiles by name and e-mail/phone

Simple OSINT script to find Instagram profiles by name and e-mail/phone

Script to calculate Active Directory Kerberos keys (AES256 and AES128) for an account, using its plaintext password

Script to calculate Active Directory Kerberos keys (AES256 and AES128) for an account, using its plaintext password

An experimental script to perform bulk parsing of arbitrary file features with YARA and console logging.

RonnieColemanYARAParser This script is named after Ronnie Coleman, and peforms bulk lifts on arbitary file features using YARA console logging. Requir

Owner
신재욱
신재욱
Unauthenticated Sqlinjection that leads to dump data base but this one impersonated Admin and drops a interactive shell

Unauthenticated Sqlinjection that leads to dump database but this one impersonated Admin and drops a interactive shell

sam 16 Nov 9, 2022
Generate MIPS reverse shell shellcodes easily !

MIPS-Reverse MIPS-Reverse is a tool that can generate shellcodes for the MIPS architecture that launches a reverse shell where you can specify the IP

null 29 Jul 27, 2021
Meterpreter Reverse shell over TOR network using hidden services

Poiana Reverse shell over TOR network using hidden services Features -> Create a hidden service -> Generate non-staged payload (python/meterpreter_rev

calfcrusher 80 Dec 21, 2022
An advanced multi-threaded, multi-client python reverse shell for hacking linux systems

PwnLnX An advanced multi-threaded, multi-client python reverse shell for hacking linux systems. There's still more work to do so feel free to help out

0xTRAW 212 Dec 24, 2022
Vulnerability Exploitation Code Collection Repository

Introduction expbox is an exploit code collection repository List CVE-2021-41349 Exchange XSS PoC <= Exchange 2013 update 23 <= Exchange 2016 update 2

0x0021h 263 Feb 14, 2022
Dome - Subdomain Enumeration Tool. Fast and reliable python script that makes active and/or passive scan to obtain subdomains and search for open ports.

DOME - A subdomain enumeration tool Check the Spanish Version Dome is a fast and reliable python script that makes active and/or passive scan to obtai

Vadi 329 Jan 1, 2023
A python script to turn Ubuntu Desktop in a one stop security platform. The InfoSec Fortress installs the packages,tools, and resources to make Ubuntu 20.04 capable of both offensive and defensive security work.

infosec-fortress A python script to turn Ubuntu Desktop into a strong DFIR/RE System with some teeth (Purple Team Ops)! This is intended to create a s

James 41 Dec 30, 2022
This is python script that will extract the functions call in all used DLL in an executable and then provide a mapping of those functions to the attack classes defined and curated malapi.io.

F2Amapper This is python script that will extract the functions call in all used DLL in an executable and then provide a mapping of those functions to

Ajit Kumar 3 Sep 3, 2022
Downloads SEP, Baseband and BuildManifest automatically for signed iOS version's for connected iDevice

FutureHelper Supports macOS and Windows Downloads SEP, Baseband and BuildManifest automatically for signed iOS version's (including beta firmwares) fo

Kasim Hussain 7 Jan 5, 2023
Scans all drives for log4j jar files and gets their version from the manifest

log4shell_scanner Scans all drives for log4j jar files and gets their version from the manifest. Windows and Windows Server only.

Zdeněk Loučka 1 Dec 29, 2021