Manas206/anlp-assignment1-models

ANLP Assignment 1 — C1-C5 Checkpoints

This repository contains the best-validation checkpoints for five custom encoder-decoder Transformer configurations trained for encrypted binary sequence to plaintext reconstruction. The Transformer components were implemented from basic PyTorch operations rather than nn.Transformer or nn.MultiheadAttention.

Repository contents

├── C1/
│   ├── model.pt
│   └── config.json
├── C2/
│   ├── model.pt
│   └── config.json
├── C3/
│   ├── model.pt
│   └── config.json
├── C4/
│   ├── model.pt
│   └── config.json
├── C5/
│   ├── model.pt
│   └── config.json
└── tokenizer/
    └── merge_rules.json

Each config.json contains the resolved hyperparameters, dataset statistics, best checkpoint epoch, and final test metrics. All sequence-level test metrics were computed using greedy decoding from the best-validation checkpoint.

Configurations

Config	Positional encoding	Attention	Normalization	Input processing
C1	Sinusoidal	MHA	LayerNorm	Learned binary BPE
C2	RoPE	MHA	LayerNorm	Learned binary BPE
C3	Sinusoidal	GQA	LayerNorm	Learned binary BPE
C4	Sinusoidal	MHA	RMSNorm	Learned binary BPE
C5	Sinusoidal	MHA	LayerNorm	Raw-byte BLT

C2 is the corrected model in which RoPE is applied independently to Q and K in encoder self-attention, decoder self-attention, and decoder cross-attention.

Shared training setup

• Data split: 4,000 train / 500 validation / 500 test, seed 42
• Global model dimension: 256
• Encoder/decoder layers: 4/4
• Global attention heads: 8
• FFN dimension: 1,024
• Dropout: 0.1
• Batch size: 2
• Epochs: 20
• Optimizer: AdamW
• Learning rate: linear 5% warmup to 3e-4, then cosine decay to 1e-5
• Weight decay: 0.01
• Gradient clipping: 1.0
• Precision: FP16 mixed precision
• Maximum sequence/decode length: 4,096
• Decoding: greedy

Final test results

Config	Bit accuracy	Exact sequence accuracy	Mean Levenshtein	BLEU	ROUGE-1/2/L F1
C1	66.9379%	0.20%	228.092	0.247765	0.577761 / 0.358195 / 0.566949
C2	68.8562%	0.40%	143.894	0.413740	0.677726 / 0.497954 / 0.673646
C3	61.8402%	0.20%	315.952	0.127038	0.481853 / 0.230759 / 0.463845
C4	64.9490%	0.20%	245.870	0.226044	0.565474 / 0.339020 / 0.554313
C5	91.6518%	0.00%	103.882	N/A	N/A

BLEU and ROUGE apply only to the learned-tokenizer configurations C1-C4.

Tokenizers

C1-C4 use the same frozen binary BPE tokenizer stored in tokenizer/merge_rules.json. It starts from individual binary symbols and uses 1,022 learned variable-length merges. C5 does not use this tokenizer; it uses raw byte IDs with local BLT patch modules.

Loading

These are checkpoints for a custom PyTorch architecture, not Transformers AutoModel packages. Instantiate the matching configuration using the assignment source code, then load the state dictionary:

import torch

checkpoint = torch.load("C2/model.pt", map_location="cpu", weights_only=False)
config = checkpoint["config"]

# Construct the corresponding custom model using config, then:
model.load_state_dict(checkpoint["model_state_dict"])
model.eval()

The checkpoint also includes the optimizer state, selected epoch, validation metrics, and resolved configuration.

Limitations

• Results are from one random seed.
• Exact reconstruction is difficult because examples can contain thousands of output positions and one error invalidates the complete sequence.
• C5 loss and throughput are not directly comparable to C1-C4 because it uses a different vocabulary and raw-byte processing.
• These models are intended for coursework and research reproduction, not production or security use.