Instructions to use yoinked/simai-codegen with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use yoinked/simai-codegen with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-0.5B")
model = PeftModel.from_pretrained(base_model, "yoinked/simai-codegen")

Transformers

How to use yoinked/simai-codegen with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="yoinked/simai-codegen")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("yoinked/simai-codegen", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use yoinked/simai-codegen with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "yoinked/simai-codegen"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yoinked/simai-codegen",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/yoinked/simai-codegen

SGLang

How to use yoinked/simai-codegen with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "yoinked/simai-codegen" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yoinked/simai-codegen",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "yoinked/simai-codegen" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yoinked/simai-codegen",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use yoinked/simai-codegen with Docker Model Runner:
```
docker model run hf.co/yoinked/simai-codegen
```

yoinked commited on 8 days ago

Commit

2eaebaf

verified ·

1 Parent(s): fcd7824

Upload folder using huggingface_hub

Browse files

Files changed (12) hide show

.gitattributes +1 -0
README.md +207 -0
adapter_config.json +48 -0
adapter_model.safetensors +3 -0
chat_template.jinja +54 -0
optimizer.pt +3 -0
rng_state.pth +3 -0
scheduler.pt +3 -0
tokenizer.json +3 -0
tokenizer_config.json +30 -0
trainer_state.json +3534 -0
training_args.bin +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,207 @@

+---
+base_model: Qwen/Qwen2.5-Coder-0.5B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-Coder-0.5B
+- lora
+- transformers
+---
+# Model Card for Model ID
+<!-- Provide a quick summary of what the model is/does. -->
+## Model Details
+### Model Description
+<!-- Provide a longer summary of what this model is. -->
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+### Model Sources [optional]
+<!-- Provide the basic links for the model. -->
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+## Uses
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+### Direct Use
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+[More Information Needed]
+### Downstream Use [optional]
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+[More Information Needed]
+### Out-of-Scope Use
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+[More Information Needed]
+## Bias, Risks, and Limitations
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+[More Information Needed]
+### Recommendations
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+## How to Get Started with the Model
+Use the code below to get started with the model.
+[More Information Needed]
+## Training Details
+### Training Data
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+[More Information Needed]
+### Training Procedure
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+#### Preprocessing [optional]
+[More Information Needed]
+#### Training Hyperparameters
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+#### Speeds, Sizes, Times [optional]
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+[More Information Needed]
+## Evaluation
+<!-- This section describes the evaluation protocols and provides the results. -->
+### Testing Data, Factors & Metrics
+#### Testing Data
+<!-- This should link to a Dataset Card if possible. -->
+[More Information Needed]
+#### Factors
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+[More Information Needed]
+#### Metrics
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+[More Information Needed]
+### Results
+[More Information Needed]
+#### Summary
+## Model Examination [optional]
+<!-- Relevant interpretability work for the model goes here -->
+[More Information Needed]
+## Environmental Impact
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+## Technical Specifications [optional]
+### Model Architecture and Objective
+[More Information Needed]
+### Compute Infrastructure
+[More Information Needed]
+#### Hardware
+[More Information Needed]
+#### Software
+[More Information Needed]
+## Citation [optional]
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+**BibTeX:**
+[More Information Needed]
+**APA:**
+[More Information Needed]
+## Glossary [optional]
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+[More Information Needed]
+## More Information [optional]
+[More Information Needed]
+## Model Card Authors [optional]
+[More Information Needed]
+## Model Card Contact
+[More Information Needed]
+### Framework versions
+- PEFT 0.19.1

adapter_config.json ADDED Viewed

	@@ -0,0 +1,48 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-0.5B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "up_proj",
+    "down_proj",
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0bbe448cab95ecd5e787731e8c572b45e345a5bba89ed27545fed98d2fbfe36a
+size 35237104

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,54 @@

+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1c1abffec956e5aab47c1d4bfe109ee15b0c2bbf3c4e26af1ba84a8bbc7ee102
+size 18814149

rng_state.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:119fd597cd06bb2f75575fc3256638615a02f3423922db01b0bbe232b1d350cc
+size 14645

scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a31b48198ee26b8bc786d51eddcb10a78a4b877d58cbc8e25fd81f93215d43f7
+size 1465

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2f55e63353d3d978b390d346bae531be8b83bc9532c0be500d62b7253aa4c595
+size 11421991

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,30 @@

+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

trainer_state.json ADDED Viewed

	@@ -0,0 +1,3534 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.9541984732824428,
+  "eval_steps": 500,
+  "global_step": 500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0019083969465648854,
+      "grad_norm": 1.0319172143936157,
+      "learning_rate": 0.0002,
+      "loss": 1.1888583898544312,
+      "step": 1
+    },
+    {
+      "epoch": 0.003816793893129771,
+      "grad_norm": 2.558675527572632,
+      "learning_rate": 0.00019999980030622737,
+      "loss": 1.2132498025894165,
+      "step": 2
+    },
+    {
+      "epoch": 0.0057251908396946565,
+      "grad_norm": 1.5326608419418335,
+      "learning_rate": 0.00019999920122570704,
+      "loss": 1.149939775466919,
+      "step": 3
+    },
+    {
+      "epoch": 0.007633587786259542,
+      "grad_norm": 0.9747706651687622,
+      "learning_rate": 0.00019999820276083168,
+      "loss": 1.1339305639266968,
+      "step": 4
+    },
+    {
+      "epoch": 0.009541984732824428,
+      "grad_norm": 0.8459669351577759,
+      "learning_rate": 0.00019999680491558902,
+      "loss": 1.1253714561462402,
+      "step": 5
+    },
+    {
+      "epoch": 0.011450381679389313,
+      "grad_norm": 1.008865237236023,
+      "learning_rate": 0.00019999500769556186,
+      "loss": 1.2100791931152344,
+      "step": 6
+    },
+    {
+      "epoch": 0.013358778625954198,
+      "grad_norm": 0.8221519589424133,
+      "learning_rate": 0.00019999281110792807,
+      "loss": 1.2723747491836548,
+      "step": 7
+    },
+    {
+      "epoch": 0.015267175572519083,
+      "grad_norm": 0.8101316094398499,
+      "learning_rate": 0.0001999902151614606,
+      "loss": 1.1219682693481445,
+      "step": 8
+    },
+    {
+      "epoch": 0.01717557251908397,
+      "grad_norm": 0.9121878743171692,
+      "learning_rate": 0.0001999872198665273,
+      "loss": 1.1113958358764648,
+      "step": 9
+    },
+    {
+      "epoch": 0.019083969465648856,
+      "grad_norm": 1.0775856971740723,
+      "learning_rate": 0.000199983825235091,
+      "loss": 1.323795199394226,
+      "step": 10
+    },
+    {
+      "epoch": 0.02099236641221374,
+      "grad_norm": 0.8592789769172668,
+      "learning_rate": 0.00019998003128070946,
+      "loss": 1.1396949291229248,
+      "step": 11
+    },
+    {
+      "epoch": 0.022900763358778626,
+      "grad_norm": 0.7191622257232666,
+      "learning_rate": 0.0001999758380185352,
+      "loss": 0.872940719127655,
+      "step": 12
+    },
+    {
+      "epoch": 0.02480916030534351,
+      "grad_norm": 0.8468532562255859,
+      "learning_rate": 0.0001999712454653157,
+      "loss": 1.0025124549865723,
+      "step": 13
+    },
+    {
+      "epoch": 0.026717557251908396,
+      "grad_norm": 0.8349865674972534,
+      "learning_rate": 0.00019996625363939295,
+      "loss": 1.1482839584350586,
+      "step": 14
+    },
+    {
+      "epoch": 0.02862595419847328,
+      "grad_norm": 0.9650427103042603,
+      "learning_rate": 0.0001999608625607037,
+      "loss": 1.0744962692260742,
+      "step": 15
+    },
+    {
+      "epoch": 0.030534351145038167,
+      "grad_norm": 0.766567051410675,
+      "learning_rate": 0.00019995507225077926,
+      "loss": 1.132323980331421,
+      "step": 16
+    },
+    {
+      "epoch": 0.03244274809160305,
+      "grad_norm": 0.8034893870353699,
+      "learning_rate": 0.0001999488827327454,
+      "loss": 0.9454637765884399,
+      "step": 17
+    },
+    {
+      "epoch": 0.03435114503816794,
+      "grad_norm": 0.9051380157470703,
+      "learning_rate": 0.00019994229403132228,
+      "loss": 0.8953108191490173,
+      "step": 18
+    },
+    {
+      "epoch": 0.03625954198473282,
+      "grad_norm": 0.8768414855003357,
+      "learning_rate": 0.00019993530617282436,
+      "loss": 1.0621753931045532,
+      "step": 19
+    },
+    {
+      "epoch": 0.03816793893129771,
+      "grad_norm": 0.898906409740448,
+      "learning_rate": 0.00019992791918516033,
+      "loss": 1.0037579536437988,
+      "step": 20
+    },
+    {
+      "epoch": 0.04007633587786259,
+      "grad_norm": 0.7816566228866577,
+      "learning_rate": 0.00019992013309783277,
+      "loss": 0.8949717879295349,
+      "step": 21
+    },
+    {
+      "epoch": 0.04198473282442748,
+      "grad_norm": 0.7184618711471558,
+      "learning_rate": 0.00019991194794193845,
+      "loss": 1.0673213005065918,
+      "step": 22
+    },
+    {
+      "epoch": 0.04389312977099236,
+      "grad_norm": 0.7685574293136597,
+      "learning_rate": 0.00019990336375016784,
+      "loss": 1.0974361896514893,
+      "step": 23
+    },
+    {
+      "epoch": 0.04580152671755725,
+      "grad_norm": 0.7797583341598511,
+      "learning_rate": 0.0001998943805568051,
+      "loss": 0.9971912503242493,
+      "step": 24
+    },
+    {
+      "epoch": 0.04770992366412214,
+      "grad_norm": 0.6793952584266663,
+      "learning_rate": 0.00019988499839772804,
+      "loss": 0.9434319138526917,
+      "step": 25
+    },
+    {
+      "epoch": 0.04961832061068702,
+      "grad_norm": 0.8823168277740479,
+      "learning_rate": 0.00019987521731040778,
+      "loss": 1.1719576120376587,
+      "step": 26
+    },
+    {
+      "epoch": 0.05152671755725191,
+      "grad_norm": 0.8843436241149902,
+      "learning_rate": 0.00019986503733390883,
+      "loss": 1.0748037099838257,
+      "step": 27
+    },
+    {
+      "epoch": 0.05343511450381679,
+      "grad_norm": 0.6977912187576294,
+      "learning_rate": 0.00019985445850888867,
+      "loss": 1.0246281623840332,
+      "step": 28
+    },
+    {
+      "epoch": 0.05534351145038168,
+      "grad_norm": 0.756574273109436,
+      "learning_rate": 0.00019984348087759784,
+      "loss": 0.9375837445259094,
+      "step": 29
+    },
+    {
+      "epoch": 0.05725190839694656,
+      "grad_norm": 0.8058716654777527,
+      "learning_rate": 0.00019983210448387966,
+      "loss": 0.9935528039932251,
+      "step": 30
+    },
+    {
+      "epoch": 0.05916030534351145,
+      "grad_norm": 0.9702297449111938,
+      "learning_rate": 0.00019982032937316998,
+      "loss": 1.1681523323059082,
+      "step": 31
+    },
+    {
+      "epoch": 0.061068702290076333,
+      "grad_norm": 0.8501767516136169,
+      "learning_rate": 0.0001998081555924972,
+      "loss": 0.9978009462356567,
+      "step": 32
+    },
+    {
+      "epoch": 0.06297709923664122,
+      "grad_norm": 0.7746052742004395,
+      "learning_rate": 0.0001997955831904818,
+      "loss": 1.040431022644043,
+      "step": 33
+    },
+    {
+      "epoch": 0.0648854961832061,
+      "grad_norm": 0.8523740768432617,
+      "learning_rate": 0.00019978261221733642,
+      "loss": 1.0792338848114014,
+      "step": 34
+    },
+    {
+      "epoch": 0.06679389312977099,
+      "grad_norm": 0.808559775352478,
+      "learning_rate": 0.00019976924272486555,
+      "loss": 0.9610211253166199,
+      "step": 35
+    },
+    {
+      "epoch": 0.06870229007633588,
+      "grad_norm": 0.7221457958221436,
+      "learning_rate": 0.00019975547476646524,
+      "loss": 0.9176299571990967,
+      "step": 36
+    },
+    {
+      "epoch": 0.07061068702290077,
+      "grad_norm": 0.680614173412323,
+      "learning_rate": 0.000199741308397123,
+      "loss": 0.9780462384223938,
+      "step": 37
+    },
+    {
+      "epoch": 0.07251908396946564,
+      "grad_norm": 1.0235838890075684,
+      "learning_rate": 0.00019972674367341756,
+      "loss": 1.0874963998794556,
+      "step": 38
+    },
+    {
+      "epoch": 0.07442748091603053,
+      "grad_norm": 0.8324326276779175,
+      "learning_rate": 0.00019971178065351857,
+      "loss": 0.9447541236877441,
+      "step": 39
+    },
+    {
+      "epoch": 0.07633587786259542,
+      "grad_norm": 0.7472124695777893,
+      "learning_rate": 0.00019969641939718652,
+      "loss": 1.0423003435134888,
+      "step": 40
+    },
+    {
+      "epoch": 0.07824427480916031,
+      "grad_norm": 0.6840877532958984,
+      "learning_rate": 0.00019968065996577234,
+      "loss": 1.0449249744415283,
+      "step": 41
+    },
+    {
+      "epoch": 0.08015267175572519,
+      "grad_norm": 0.8184124827384949,
+      "learning_rate": 0.00019966450242221722,
+      "loss": 0.9014325141906738,
+      "step": 42
+    },
+    {
+      "epoch": 0.08206106870229007,
+      "grad_norm": 0.7632555961608887,
+      "learning_rate": 0.0001996479468310524,
+      "loss": 0.8362308144569397,
+      "step": 43
+    },
+    {
+      "epoch": 0.08396946564885496,
+      "grad_norm": 0.7152660489082336,
+      "learning_rate": 0.00019963099325839883,
+      "loss": 0.93896484375,
+      "step": 44
+    },
+    {
+      "epoch": 0.08587786259541985,
+      "grad_norm": 0.6840109825134277,
+      "learning_rate": 0.00019961364177196696,
+      "loss": 0.9620580673217773,
+      "step": 45
+    },
+    {
+      "epoch": 0.08778625954198473,
+      "grad_norm": 0.8080661296844482,
+      "learning_rate": 0.0001995958924410565,
+      "loss": 0.8377565145492554,
+      "step": 46
+    },
+    {
+      "epoch": 0.08969465648854962,
+      "grad_norm": 0.7813903093338013,
+      "learning_rate": 0.00019957774533655605,
+      "loss": 0.8326072096824646,
+      "step": 47
+    },
+    {
+      "epoch": 0.0916030534351145,
+      "grad_norm": 0.8098936080932617,
+      "learning_rate": 0.00019955920053094288,
+      "loss": 0.9860023856163025,
+      "step": 48
+    },
+    {
+      "epoch": 0.09351145038167939,
+      "grad_norm": 0.9210098385810852,
+      "learning_rate": 0.00019954025809828266,
+      "loss": 0.9747365713119507,
+      "step": 49
+    },
+    {
+      "epoch": 0.09541984732824428,
+      "grad_norm": 0.7985463738441467,
+      "learning_rate": 0.00019952091811422905,
+      "loss": 0.9031082987785339,
+      "step": 50
+    },
+    {
+      "epoch": 0.09732824427480916,
+      "grad_norm": 0.8715912699699402,
+      "learning_rate": 0.0001995011806560236,
+      "loss": 0.9334609508514404,
+      "step": 51
+    },
+    {
+      "epoch": 0.09923664122137404,
+      "grad_norm": 0.7367807030677795,
+      "learning_rate": 0.0001994810458024952,
+      "loss": 0.9366903901100159,
+      "step": 52
+    },
+    {
+      "epoch": 0.10114503816793893,
+      "grad_norm": 0.8102525472640991,
+      "learning_rate": 0.00019946051363405997,
+      "loss": 0.9092800617218018,
+      "step": 53
+    },
+    {
+      "epoch": 0.10305343511450382,
+      "grad_norm": 0.8269142508506775,
+      "learning_rate": 0.0001994395842327209,
+      "loss": 0.9335564374923706,
+      "step": 54
+    },
+    {
+      "epoch": 0.1049618320610687,
+      "grad_norm": 0.7004595398902893,
+      "learning_rate": 0.0001994182576820673,
+      "loss": 0.9463485479354858,
+      "step": 55
+    },
+    {
+      "epoch": 0.10687022900763359,
+      "grad_norm": 0.9793404936790466,
+      "learning_rate": 0.00019939653406727482,
+      "loss": 0.9135723114013672,
+      "step": 56
+    },
+    {
+      "epoch": 0.10877862595419847,
+      "grad_norm": 0.7052667140960693,
+      "learning_rate": 0.0001993744134751049,
+      "loss": 0.8296093940734863,
+      "step": 57
+    },
+    {
+      "epoch": 0.11068702290076336,
+      "grad_norm": 0.6771461367607117,
+      "learning_rate": 0.00019935189599390435,
+      "loss": 0.7838313579559326,
+      "step": 58
+    },
+    {
+      "epoch": 0.11259541984732824,
+      "grad_norm": 0.7421724200248718,
+      "learning_rate": 0.00019932898171360526,
+      "loss": 0.9136338233947754,
+      "step": 59
+    },
+    {
+      "epoch": 0.11450381679389313,
+      "grad_norm": 0.820115327835083,
+      "learning_rate": 0.00019930567072572438,
+      "loss": 0.9014999866485596,
+      "step": 60
+    },
+    {
+      "epoch": 0.11641221374045801,
+      "grad_norm": 0.7351306676864624,
+      "learning_rate": 0.00019928196312336285,
+      "loss": 0.9661044478416443,
+      "step": 61
+    },
+    {
+      "epoch": 0.1183206106870229,
+      "grad_norm": 0.7182132005691528,
+      "learning_rate": 0.00019925785900120596,
+      "loss": 0.8359523415565491,
+      "step": 62
+    },
+    {
+      "epoch": 0.12022900763358779,
+      "grad_norm": 0.630822479724884,
+      "learning_rate": 0.0001992333584555225,
+      "loss": 0.8223984241485596,
+      "step": 63
+    },
+    {
+      "epoch": 0.12213740458015267,
+      "grad_norm": 0.8691913485527039,
+      "learning_rate": 0.0001992084615841647,
+      "loss": 0.9800937175750732,
+      "step": 64
+    },
+    {
+      "epoch": 0.12404580152671756,
+      "grad_norm": 0.7840291261672974,
+      "learning_rate": 0.00019918316848656744,
+      "loss": 0.9353207349777222,
+      "step": 65
+    },
+    {
+      "epoch": 0.12595419847328243,
+      "grad_norm": 0.7878021001815796,
+      "learning_rate": 0.00019915747926374828,
+      "loss": 0.9968975186347961,
+      "step": 66
+    },
+    {
+      "epoch": 0.12786259541984732,
+      "grad_norm": 0.7407828569412231,
+      "learning_rate": 0.00019913139401830674,
+      "loss": 0.947467565536499,
+      "step": 67
+    },
+    {
+      "epoch": 0.1297709923664122,
+      "grad_norm": 0.7297830581665039,
+      "learning_rate": 0.00019910491285442407,
+      "loss": 0.9409926533699036,
+      "step": 68
+    },
+    {
+      "epoch": 0.1316793893129771,
+      "grad_norm": 0.7517293095588684,
+      "learning_rate": 0.00019907803587786274,
+      "loss": 0.7961177825927734,
+      "step": 69
+    },
+    {
+      "epoch": 0.13358778625954199,
+      "grad_norm": 0.6893132328987122,
+      "learning_rate": 0.000199050763195966,
+      "loss": 0.9285712242126465,
+      "step": 70
+    },
+    {
+      "epoch": 0.13549618320610687,
+      "grad_norm": 0.6858025789260864,
+      "learning_rate": 0.0001990230949176576,
+      "loss": 0.8860894441604614,
+      "step": 71
+    },
+    {
+      "epoch": 0.13740458015267176,
+      "grad_norm": 0.9707567691802979,
+      "learning_rate": 0.00019899503115344117,
+      "loss": 0.8908587694168091,
+      "step": 72
+    },
+    {
+      "epoch": 0.13931297709923665,
+      "grad_norm": 0.7052493691444397,
+      "learning_rate": 0.0001989665720153999,
+      "loss": 0.8562502264976501,
+      "step": 73
+    },
+    {
+      "epoch": 0.14122137404580154,
+      "grad_norm": 0.6578022837638855,
+      "learning_rate": 0.00019893771761719605,
+      "loss": 0.8883586525917053,
+      "step": 74
+    },
+    {
+      "epoch": 0.1431297709923664,
+      "grad_norm": 0.7066544890403748,
+      "learning_rate": 0.00019890846807407044,
+      "loss": 0.976402997970581,
+      "step": 75
+    },
+    {
+      "epoch": 0.1450381679389313,
+      "grad_norm": 0.9122876524925232,
+      "learning_rate": 0.00019887882350284216,
+      "loss": 0.9926351308822632,
+      "step": 76
+    },
+    {
+      "epoch": 0.14694656488549618,
+      "grad_norm": 0.7439401745796204,
+      "learning_rate": 0.00019884878402190795,
+      "loss": 0.829197108745575,
+      "step": 77
+    },
+    {
+      "epoch": 0.14885496183206107,
+      "grad_norm": 0.7522874474525452,
+      "learning_rate": 0.0001988183497512417,
+      "loss": 0.863883912563324,
+      "step": 78
+    },
+    {
+      "epoch": 0.15076335877862596,
+      "grad_norm": 1.0040572881698608,
+      "learning_rate": 0.0001987875208123941,
+      "loss": 1.069295883178711,
+      "step": 79
+    },
+    {
+      "epoch": 0.15267175572519084,
+      "grad_norm": 0.713905930519104,
+      "learning_rate": 0.00019875629732849214,
+      "loss": 0.8455244898796082,
+      "step": 80
+    },
+    {
+      "epoch": 0.15458015267175573,
+      "grad_norm": 0.8433374166488647,
+      "learning_rate": 0.00019872467942423853,
+      "loss": 0.8311648368835449,
+      "step": 81
+    },
+    {
+      "epoch": 0.15648854961832062,
+      "grad_norm": 0.7732320427894592,
+      "learning_rate": 0.00019869266722591115,
+      "loss": 0.9065580368041992,
+      "step": 82
+    },
+    {
+      "epoch": 0.15839694656488548,
+      "grad_norm": 0.7662688493728638,
+      "learning_rate": 0.0001986602608613628,
+      "loss": 0.9555010795593262,
+      "step": 83
+    },
+    {
+      "epoch": 0.16030534351145037,
+      "grad_norm": 0.9863274693489075,
+      "learning_rate": 0.00019862746046002048,
+      "loss": 0.9811774492263794,
+      "step": 84
+    },
+    {
+      "epoch": 0.16221374045801526,
+      "grad_norm": 0.673652708530426,
+      "learning_rate": 0.00019859426615288488,
+      "loss": 0.926813006401062,
+      "step": 85
+    },
+    {
+      "epoch": 0.16412213740458015,
+      "grad_norm": 0.8610010147094727,
+      "learning_rate": 0.00019856067807252992,
+      "loss": 0.7706037759780884,
+      "step": 86
+    },
+    {
+      "epoch": 0.16603053435114504,
+      "grad_norm": 0.869065523147583,
+      "learning_rate": 0.00019852669635310222,
+      "loss": 0.9335610270500183,
+      "step": 87
+    },
+    {
+      "epoch": 0.16793893129770993,
+      "grad_norm": 0.7144750356674194,
+      "learning_rate": 0.00019849232113032053,
+      "loss": 0.8826386332511902,
+      "step": 88
+    },
+    {
+      "epoch": 0.16984732824427481,
+      "grad_norm": 0.6938415765762329,
+      "learning_rate": 0.00019845755254147525,
+      "loss": 0.8689931631088257,
+      "step": 89
+    },
+    {
+      "epoch": 0.1717557251908397,
+      "grad_norm": 0.6958433389663696,
+      "learning_rate": 0.00019842239072542775,
+      "loss": 0.8018342852592468,
+      "step": 90
+    },
+    {
+      "epoch": 0.1736641221374046,
+      "grad_norm": 0.7545017600059509,
+      "learning_rate": 0.00019838683582260993,
+      "loss": 0.9315579533576965,
+      "step": 91
+    },
+    {
+      "epoch": 0.17557251908396945,
+      "grad_norm": 0.8562167882919312,
+      "learning_rate": 0.00019835088797502367,
+      "loss": 0.950774610042572,
+      "step": 92
+    },
+    {
+      "epoch": 0.17748091603053434,
+      "grad_norm": 0.7419416904449463,
+      "learning_rate": 0.0001983145473262402,
+      "loss": 1.0040173530578613,
+      "step": 93
+    },
+    {
+      "epoch": 0.17938931297709923,
+      "grad_norm": 0.7331036925315857,
+      "learning_rate": 0.0001982778140213996,
+      "loss": 0.703330934047699,
+      "step": 94
+    },
+    {
+      "epoch": 0.18129770992366412,
+      "grad_norm": 0.783308207988739,
+      "learning_rate": 0.00019824068820720997,
+      "loss": 0.9145213961601257,
+      "step": 95
+    },
+    {
+      "epoch": 0.183206106870229,
+      "grad_norm": 0.7295645475387573,
+      "learning_rate": 0.00019820317003194728,
+      "loss": 0.8577327728271484,
+      "step": 96
+    },
+    {
+      "epoch": 0.1851145038167939,
+      "grad_norm": 0.8237174153327942,
+      "learning_rate": 0.00019816525964545448,
+      "loss": 0.8371306657791138,
+      "step": 97
+    },
+    {
+      "epoch": 0.18702290076335878,
+      "grad_norm": 0.638324499130249,
+      "learning_rate": 0.00019812695719914085,
+      "loss": 0.7818671464920044,
+      "step": 98
+    },
+    {
+      "epoch": 0.18893129770992367,
+      "grad_norm": 0.7308298349380493,
+      "learning_rate": 0.00019808826284598165,
+      "loss": 0.8242241144180298,
+      "step": 99
+    },
+    {
+      "epoch": 0.19083969465648856,
+      "grad_norm": 0.8194659948348999,
+      "learning_rate": 0.0001980491767405173,
+      "loss": 1.0477763414382935,
+      "step": 100
+    },
+    {
+      "epoch": 0.19274809160305342,
+      "grad_norm": 0.7348308563232422,
+      "learning_rate": 0.0001980096990388528,
+      "loss": 0.8968179821968079,
+      "step": 101
+    },
+    {
+      "epoch": 0.1946564885496183,
+      "grad_norm": 0.8615684509277344,
+      "learning_rate": 0.00019796982989865722,
+      "loss": 0.9265606999397278,
+      "step": 102
+    },
+    {
+      "epoch": 0.1965648854961832,
+      "grad_norm": 0.9783602952957153,
+      "learning_rate": 0.00019792956947916292,
+      "loss": 1.0029759407043457,
+      "step": 103
+    },
+    {
+      "epoch": 0.1984732824427481,
+      "grad_norm": 0.8230413794517517,
+      "learning_rate": 0.000197888917941165,
+      "loss": 0.9710904955863953,
+      "step": 104
+    },
+    {
+      "epoch": 0.20038167938931298,
+      "grad_norm": 0.715769350528717,
+      "learning_rate": 0.00019784787544702064,
+      "loss": 1.0048372745513916,
+      "step": 105
+    },
+    {
+      "epoch": 0.20229007633587787,
+      "grad_norm": 0.657476544380188,
+      "learning_rate": 0.00019780644216064845,
+      "loss": 0.9512552618980408,
+      "step": 106
+    },
+    {
+      "epoch": 0.20419847328244276,
+      "grad_norm": 0.8867478370666504,
+      "learning_rate": 0.0001977646182475278,
+      "loss": 0.8841535449028015,
+      "step": 107
+    },
+    {
+      "epoch": 0.20610687022900764,
+      "grad_norm": 0.6844662427902222,
+      "learning_rate": 0.00019772240387469823,
+      "loss": 0.8210901618003845,
+      "step": 108
+    },
+    {
+      "epoch": 0.20801526717557253,
+      "grad_norm": 0.8655828833580017,
+      "learning_rate": 0.00019767979921075866,
+      "loss": 1.0782965421676636,
+      "step": 109
+    },
+    {
+      "epoch": 0.2099236641221374,
+      "grad_norm": 0.7254316210746765,
+      "learning_rate": 0.00019763680442586682,
+      "loss": 0.7998471260070801,
+      "step": 110
+    },
+    {
+      "epoch": 0.21183206106870228,
+      "grad_norm": 0.7396829724311829,
+      "learning_rate": 0.00019759341969173853,
+      "loss": 0.8773504495620728,
+      "step": 111
+    },
+    {
+      "epoch": 0.21374045801526717,
+      "grad_norm": 0.6355534791946411,
+      "learning_rate": 0.00019754964518164697,
+      "loss": 0.8282820582389832,
+      "step": 112
+    },
+    {
+      "epoch": 0.21564885496183206,
+      "grad_norm": 0.7496469020843506,
+      "learning_rate": 0.00019750548107042213,
+      "loss": 0.8907659649848938,
+      "step": 113
+    },
+    {
+      "epoch": 0.21755725190839695,
+      "grad_norm": 0.9808054566383362,
+      "learning_rate": 0.00019746092753444999,
+      "loss": 0.8398576974868774,
+      "step": 114
+    },
+    {
+      "epoch": 0.21946564885496184,
+      "grad_norm": 0.8305889964103699,
+      "learning_rate": 0.00019741598475167175,
+      "loss": 0.9800796508789062,
+      "step": 115
+    },
+    {
+      "epoch": 0.22137404580152673,
+      "grad_norm": 0.8272994160652161,
+      "learning_rate": 0.00019737065290158335,
+      "loss": 1.0569555759429932,
+      "step": 116
+    },
+    {
+      "epoch": 0.22328244274809161,
+      "grad_norm": 0.6429803371429443,
+      "learning_rate": 0.00019732493216523448,
+      "loss": 0.8179414868354797,
+      "step": 117
+    },
+    {
+      "epoch": 0.22519083969465647,
+      "grad_norm": 0.7551900148391724,
+      "learning_rate": 0.00019727882272522817,
+      "loss": 1.0400655269622803,
+      "step": 118
+    },
+    {
+      "epoch": 0.22709923664122136,
+      "grad_norm": 0.668594479560852,
+      "learning_rate": 0.00019723232476571973,
+      "loss": 0.7926125526428223,
+      "step": 119
+    },
+    {
+      "epoch": 0.22900763358778625,
+      "grad_norm": 0.7308197617530823,
+      "learning_rate": 0.0001971854384724162,
+      "loss": 0.7763074040412903,
+      "step": 120
+    },
+    {
+      "epoch": 0.23091603053435114,
+      "grad_norm": 0.6899462342262268,
+      "learning_rate": 0.0001971381640325756,
+      "loss": 0.8050013184547424,
+      "step": 121
+    },
+    {
+      "epoch": 0.23282442748091603,
+      "grad_norm": 0.6338201761245728,
+      "learning_rate": 0.0001970905016350062,
+      "loss": 0.7711692452430725,
+      "step": 122
+    },
+    {
+      "epoch": 0.23473282442748092,
+      "grad_norm": 0.818661093711853,
+      "learning_rate": 0.00019704245147006563,
+      "loss": 0.9701591730117798,
+      "step": 123
+    },
+    {
+      "epoch": 0.2366412213740458,
+      "grad_norm": 0.8936249613761902,
+      "learning_rate": 0.0001969940137296603,
+      "loss": 0.932243287563324,
+      "step": 124
+    },
+    {
+      "epoch": 0.2385496183206107,
+      "grad_norm": 0.6683034896850586,
+      "learning_rate": 0.00019694518860724445,
+      "loss": 0.8161548376083374,
+      "step": 125
+    },
+    {
+      "epoch": 0.24045801526717558,
+      "grad_norm": 0.8327929973602295,
+      "learning_rate": 0.00019689597629781962,
+      "loss": 0.8570858240127563,
+      "step": 126
+    },
+    {
+      "epoch": 0.24236641221374045,
+      "grad_norm": 0.8678820729255676,
+      "learning_rate": 0.00019684637699793358,
+      "loss": 0.849490761756897,
+      "step": 127
+    },
+    {
+      "epoch": 0.24427480916030533,
+      "grad_norm": 0.8075437545776367,
+      "learning_rate": 0.00019679639090567982,
+      "loss": 0.7586680054664612,
+      "step": 128
+    },
+    {
+      "epoch": 0.24618320610687022,
+      "grad_norm": 0.920081377029419,
+      "learning_rate": 0.00019674601822069647,
+      "loss": 0.9018468260765076,
+      "step": 129
+    },
+    {
+      "epoch": 0.2480916030534351,
+      "grad_norm": 0.809676468372345,
+      "learning_rate": 0.00019669525914416586,
+      "loss": 0.9085022807121277,
+      "step": 130
+    },
+    {
+      "epoch": 0.25,
+      "grad_norm": 0.6830601096153259,
+      "learning_rate": 0.00019664411387881336,
+      "loss": 0.8158378005027771,
+      "step": 131
+    },
+    {
+      "epoch": 0.25190839694656486,
+      "grad_norm": 0.8688009977340698,
+      "learning_rate": 0.00019659258262890683,
+      "loss": 0.8939949870109558,
+      "step": 132
+    },
+    {
+      "epoch": 0.2538167938931298,
+      "grad_norm": 0.8238283395767212,
+      "learning_rate": 0.00019654066560025567,
+      "loss": 0.8714411854743958,
+      "step": 133
+    },
+    {
+      "epoch": 0.25572519083969464,
+      "grad_norm": 0.8402740955352783,
+      "learning_rate": 0.00019648836300020995,
+      "loss": 0.8553993105888367,
+      "step": 134
+    },
+    {
+      "epoch": 0.25763358778625955,
+      "grad_norm": 0.8889774680137634,
+      "learning_rate": 0.0001964356750376598,
+      "loss": 0.9362643361091614,
+      "step": 135
+    },
+    {
+      "epoch": 0.2595419847328244,
+      "grad_norm": 0.7945830225944519,
+      "learning_rate": 0.00019638260192303438,
+      "loss": 0.915956437587738,
+      "step": 136
+    },
+    {
+      "epoch": 0.26145038167938933,
+      "grad_norm": 0.6740367412567139,
+      "learning_rate": 0.00019632914386830109,
+      "loss": 0.9191530346870422,
+      "step": 137
+    },
+    {
+      "epoch": 0.2633587786259542,
+      "grad_norm": 0.8426705002784729,
+      "learning_rate": 0.00019627530108696473,
+      "loss": 0.868916392326355,
+      "step": 138
+    },
+    {
+      "epoch": 0.2652671755725191,
+      "grad_norm": 0.8657882213592529,
+      "learning_rate": 0.00019622107379406667,
+      "loss": 0.872461199760437,
+      "step": 139
+    },
+    {
+      "epoch": 0.26717557251908397,
+      "grad_norm": 0.7300332188606262,
+      "learning_rate": 0.00019616646220618398,
+      "loss": 0.8058186173439026,
+      "step": 140
+    },
+    {
+      "epoch": 0.26908396946564883,
+      "grad_norm": 0.767820417881012,
+      "learning_rate": 0.00019611146654142854,
+      "loss": 0.8053113222122192,
+      "step": 141
+    },
+    {
+      "epoch": 0.27099236641221375,
+      "grad_norm": 0.9955086708068848,
+      "learning_rate": 0.00019605608701944615,
+      "loss": 0.8747786283493042,
+      "step": 142
+    },
+    {
+      "epoch": 0.2729007633587786,
+      "grad_norm": 0.8415116667747498,
+      "learning_rate": 0.0001960003238614158,
+      "loss": 0.7746783494949341,
+      "step": 143
+    },
+    {
+      "epoch": 0.2748091603053435,
+      "grad_norm": 0.7966413497924805,
+      "learning_rate": 0.00019594417729004845,
+      "loss": 0.9076138734817505,
+      "step": 144
+    },
+    {
+      "epoch": 0.2767175572519084,
+      "grad_norm": 0.7794169187545776,
+      "learning_rate": 0.00019588764752958668,
+      "loss": 0.8441069722175598,
+      "step": 145
+    },
+    {
+      "epoch": 0.2786259541984733,
+      "grad_norm": 0.8923323750495911,
+      "learning_rate": 0.00019583073480580323,
+      "loss": 0.9479147791862488,
+      "step": 146
+    },
+    {
+      "epoch": 0.28053435114503816,
+      "grad_norm": 0.7435654401779175,
+      "learning_rate": 0.00019577343934600044,
+      "loss": 0.7812725901603699,
+      "step": 147
+    },
+    {
+      "epoch": 0.2824427480916031,
+      "grad_norm": 0.6814793944358826,
+      "learning_rate": 0.00019571576137900927,
+      "loss": 0.7864100337028503,
+      "step": 148
+    },
+    {
+      "epoch": 0.28435114503816794,
+      "grad_norm": 0.7200238704681396,
+      "learning_rate": 0.00019565770113518825,
+      "loss": 0.9560705423355103,
+      "step": 149
+    },
+    {
+      "epoch": 0.2862595419847328,
+      "grad_norm": 0.6995819807052612,
+      "learning_rate": 0.00019559925884642286,
+      "loss": 0.8137438297271729,
+      "step": 150
+    },
+    {
+      "epoch": 0.2881679389312977,
+      "grad_norm": 0.5875421762466431,
+      "learning_rate": 0.0001955404347461243,
+      "loss": 0.7266908288002014,
+      "step": 151
+    },
+    {
+      "epoch": 0.2900763358778626,
+      "grad_norm": 0.7169772386550903,
+      "learning_rate": 0.00019548122906922865,
+      "loss": 0.8415418863296509,
+      "step": 152
+    },
+    {
+      "epoch": 0.2919847328244275,
+      "grad_norm": 0.638949990272522,
+      "learning_rate": 0.00019542164205219606,
+      "loss": 0.8274745941162109,
+      "step": 153
+    },
+    {
+      "epoch": 0.29389312977099236,
+      "grad_norm": 0.7600482106208801,
+      "learning_rate": 0.00019536167393300965,
+      "loss": 1.0291582345962524,
+      "step": 154
+    },
+    {
+      "epoch": 0.2958015267175573,
+      "grad_norm": 0.7919864058494568,
+      "learning_rate": 0.0001953013249511746,
+      "loss": 0.8221685290336609,
+      "step": 155
+    },
+    {
+      "epoch": 0.29770992366412213,
+      "grad_norm": 0.7551295757293701,
+      "learning_rate": 0.0001952405953477172,
+      "loss": 0.9822176098823547,
+      "step": 156
+    },
+    {
+      "epoch": 0.29961832061068705,
+      "grad_norm": 0.6301788091659546,
+      "learning_rate": 0.000195179485365184,
+      "loss": 0.6514051556587219,
+      "step": 157
+    },
+    {
+      "epoch": 0.3015267175572519,
+      "grad_norm": 0.6198784112930298,
+      "learning_rate": 0.00019511799524764062,
+      "loss": 0.8909632563591003,
+      "step": 158
+    },
+    {
+      "epoch": 0.30343511450381677,
+      "grad_norm": 0.6850892901420593,
+      "learning_rate": 0.0001950561252406709,
+      "loss": 0.8530802726745605,
+      "step": 159
+    },
+    {
+      "epoch": 0.3053435114503817,
+      "grad_norm": 0.6627747416496277,
+      "learning_rate": 0.00019499387559137596,
+      "loss": 0.8950958251953125,
+      "step": 160
+    },
+    {
+      "epoch": 0.30725190839694655,
+      "grad_norm": 0.6648580431938171,
+      "learning_rate": 0.00019493124654837318,
+      "loss": 0.8956643342971802,
+      "step": 161
+    },
+    {
+      "epoch": 0.30916030534351147,
+      "grad_norm": 0.8410931825637817,
+      "learning_rate": 0.00019486823836179512,
+      "loss": 0.8377202153205872,
+      "step": 162
+    },
+    {
+      "epoch": 0.3110687022900763,
+      "grad_norm": 0.7048870921134949,
+      "learning_rate": 0.00019480485128328868,
+      "loss": 0.8189906477928162,
+      "step": 163
+    },
+    {
+      "epoch": 0.31297709923664124,
+      "grad_norm": 0.7690779566764832,
+      "learning_rate": 0.0001947410855660139,
+      "loss": 0.7249743938446045,
+      "step": 164
+    },
+    {
+      "epoch": 0.3148854961832061,
+      "grad_norm": 0.9097875356674194,
+      "learning_rate": 0.0001946769414646431,
+      "loss": 0.9130119681358337,
+      "step": 165
+    },
+    {
+      "epoch": 0.31679389312977096,
+      "grad_norm": 0.798412024974823,
+      "learning_rate": 0.0001946124192353599,
+      "loss": 0.960759699344635,
+      "step": 166
+    },
+    {
+      "epoch": 0.3187022900763359,
+      "grad_norm": 0.6766208410263062,
+      "learning_rate": 0.000194547519135858,
+      "loss": 0.7708785533905029,
+      "step": 167
+    },
+    {
+      "epoch": 0.32061068702290074,
+      "grad_norm": 0.7984035015106201,
+      "learning_rate": 0.00019448224142534033,
+      "loss": 0.9001744985580444,
+      "step": 168
+    },
+    {
+      "epoch": 0.32251908396946566,
+      "grad_norm": 0.7736956477165222,
+      "learning_rate": 0.00019441658636451794,
+      "loss": 0.7067661285400391,
+      "step": 169
+    },
+    {
+      "epoch": 0.3244274809160305,
+      "grad_norm": 0.701378583908081,
+      "learning_rate": 0.00019435055421560896,
+      "loss": 0.8072033524513245,
+      "step": 170
+    },
+    {
+      "epoch": 0.32633587786259544,
+      "grad_norm": 0.7019128203392029,
+      "learning_rate": 0.00019428414524233752,
+      "loss": 0.8859261274337769,
+      "step": 171
+    },
+    {
+      "epoch": 0.3282442748091603,
+      "grad_norm": 0.6405873894691467,
+      "learning_rate": 0.0001942173597099329,
+      "loss": 0.7549964189529419,
+      "step": 172
+    },
+    {
+      "epoch": 0.3301526717557252,
+      "grad_norm": 0.6906740069389343,
+      "learning_rate": 0.0001941501978851281,
+      "loss": 0.7923817038536072,
+      "step": 173
+    },
+    {
+      "epoch": 0.3320610687022901,
+      "grad_norm": 0.6853337287902832,
+      "learning_rate": 0.00019408266003615913,
+      "loss": 0.7819351553916931,
+      "step": 174
+    },
+    {
+      "epoch": 0.33396946564885494,
+      "grad_norm": 0.6918132305145264,
+      "learning_rate": 0.0001940147464327637,
+      "loss": 0.6980127096176147,
+      "step": 175
+    },
+    {
+      "epoch": 0.33587786259541985,
+      "grad_norm": 0.8270894885063171,
+      "learning_rate": 0.00019394645734618037,
+      "loss": 0.9518585205078125,
+      "step": 176
+    },
+    {
+      "epoch": 0.3377862595419847,
+      "grad_norm": 0.6764659881591797,
+      "learning_rate": 0.00019387779304914715,
+      "loss": 0.8641853332519531,
+      "step": 177
+    },
+    {
+      "epoch": 0.33969465648854963,
+      "grad_norm": 0.9292405247688293,
+      "learning_rate": 0.00019380875381590074,
+      "loss": 0.9833829402923584,
+      "step": 178
+    },
+    {
+      "epoch": 0.3416030534351145,
+      "grad_norm": 0.6896494030952454,
+      "learning_rate": 0.00019373933992217524,
+      "loss": 0.8466549515724182,
+      "step": 179
+    },
+    {
+      "epoch": 0.3435114503816794,
+      "grad_norm": 0.6645339131355286,
+      "learning_rate": 0.0001936695516452011,
+      "loss": 0.7963259816169739,
+      "step": 180
+    },
+    {
+      "epoch": 0.34541984732824427,
+      "grad_norm": 0.7422249913215637,
+      "learning_rate": 0.000193599389263704,
+      "loss": 0.9273393154144287,
+      "step": 181
+    },
+    {
+      "epoch": 0.3473282442748092,
+      "grad_norm": 0.7360007166862488,
+      "learning_rate": 0.00019352885305790374,
+      "loss": 0.7015874981880188,
+      "step": 182
+    },
+    {
+      "epoch": 0.34923664122137404,
+      "grad_norm": 0.7029959559440613,
+      "learning_rate": 0.00019345794330951314,
+      "loss": 0.9108408093452454,
+      "step": 183
+    },
+    {
+      "epoch": 0.3511450381679389,
+      "grad_norm": 0.6600919365882874,
+      "learning_rate": 0.00019338666030173693,
+      "loss": 0.7598098516464233,
+      "step": 184
+    },
+    {
+      "epoch": 0.3530534351145038,
+      "grad_norm": 0.7309081554412842,
+      "learning_rate": 0.00019331500431927056,
+      "loss": 0.7844942808151245,
+      "step": 185
+    },
+    {
+      "epoch": 0.3549618320610687,
+      "grad_norm": 0.7310585975646973,
+      "learning_rate": 0.00019324297564829904,
+      "loss": 0.9316179752349854,
+      "step": 186
+    },
+    {
+      "epoch": 0.3568702290076336,
+      "grad_norm": 0.8466243743896484,
+      "learning_rate": 0.000193170574576496,
+      "loss": 0.8222724199295044,
+      "step": 187
+    },
+    {
+      "epoch": 0.35877862595419846,
+      "grad_norm": 0.7071732878684998,
+      "learning_rate": 0.00019309780139302224,
+      "loss": 0.8935707807540894,
+      "step": 188
+    },
+    {
+      "epoch": 0.3606870229007634,
+      "grad_norm": 0.7483112812042236,
+      "learning_rate": 0.00019302465638852483,
+      "loss": 0.8233731985092163,
+      "step": 189
+    },
+    {
+      "epoch": 0.36259541984732824,
+      "grad_norm": 0.6827342510223389,
+      "learning_rate": 0.0001929511398551358,
+      "loss": 0.9243869185447693,
+      "step": 190
+    },
+    {
+      "epoch": 0.36450381679389315,
+      "grad_norm": 0.7744309306144714,
+      "learning_rate": 0.00019287725208647096,
+      "loss": 0.8004241585731506,
+      "step": 191
+    },
+    {
+      "epoch": 0.366412213740458,
+      "grad_norm": 0.9104498624801636,
+      "learning_rate": 0.00019280299337762896,
+      "loss": 0.823358416557312,
+      "step": 192
+    },
+    {
+      "epoch": 0.3683206106870229,
+      "grad_norm": 0.7632629871368408,
+      "learning_rate": 0.0001927283640251898,
+      "loss": 0.9017253518104553,
+      "step": 193
+    },
+    {
+      "epoch": 0.3702290076335878,
+      "grad_norm": 0.6975040435791016,
+      "learning_rate": 0.00019265336432721378,
+      "loss": 0.8129168152809143,
+      "step": 194
+    },
+    {
+      "epoch": 0.37213740458015265,
+      "grad_norm": 0.6849466562271118,
+      "learning_rate": 0.0001925779945832404,
+      "loss": 0.6700172424316406,
+      "step": 195
+    },
+    {
+      "epoch": 0.37404580152671757,
+      "grad_norm": 0.707890510559082,
+      "learning_rate": 0.000192502255094287,
+      "loss": 0.828960120677948,
+      "step": 196
+    },
+    {
+      "epoch": 0.37595419847328243,
+      "grad_norm": 0.6629129648208618,
+      "learning_rate": 0.00019242614616284768,
+      "loss": 0.8441582322120667,
+      "step": 197
+    },
+    {
+      "epoch": 0.37786259541984735,
+      "grad_norm": 0.8846833109855652,
+      "learning_rate": 0.00019234966809289201,
+      "loss": 0.815739095211029,
+      "step": 198
+    },
+    {
+      "epoch": 0.3797709923664122,
+      "grad_norm": 0.7524842619895935,
+      "learning_rate": 0.00019227282118986394,
+      "loss": 0.9191688299179077,
+      "step": 199
+    },
+    {
+      "epoch": 0.3816793893129771,
+      "grad_norm": 0.6800813674926758,
+      "learning_rate": 0.00019219560576068034,
+      "loss": 0.893654465675354,
+      "step": 200
+    },
+    {
+      "epoch": 0.383587786259542,
+      "grad_norm": 0.7200490236282349,
+      "learning_rate": 0.00019211802211373006,
+      "loss": 0.8543277382850647,
+      "step": 201
+    },
+    {
+      "epoch": 0.38549618320610685,
+      "grad_norm": 0.6713653206825256,
+      "learning_rate": 0.00019204007055887256,
+      "loss": 0.7321712970733643,
+      "step": 202
+    },
+    {
+      "epoch": 0.38740458015267176,
+      "grad_norm": 0.761995792388916,
+      "learning_rate": 0.00019196175140743658,
+      "loss": 0.8762828707695007,
+      "step": 203
+    },
+    {
+      "epoch": 0.3893129770992366,
+      "grad_norm": 0.7371771335601807,
+      "learning_rate": 0.0001918830649722191,
+      "loss": 0.8501207828521729,
+      "step": 204
+    },
+    {
+      "epoch": 0.39122137404580154,
+      "grad_norm": 0.6583154797554016,
+      "learning_rate": 0.00019180401156748396,
+      "loss": 0.778701663017273,
+      "step": 205
+    },
+    {
+      "epoch": 0.3931297709923664,
+      "grad_norm": 0.6544883251190186,
+      "learning_rate": 0.00019172459150896051,
+      "loss": 0.7464669942855835,
+      "step": 206
+    },
+    {
+      "epoch": 0.3950381679389313,
+      "grad_norm": 0.7078415155410767,
+      "learning_rate": 0.00019164480511384268,
+      "loss": 0.8787112832069397,
+      "step": 207
+    },
+    {
+      "epoch": 0.3969465648854962,
+      "grad_norm": 0.7920764684677124,
+      "learning_rate": 0.00019156465270078735,
+      "loss": 0.8315320014953613,
+      "step": 208
+    },
+    {
+      "epoch": 0.3988549618320611,
+      "grad_norm": 0.6305553317070007,
+      "learning_rate": 0.00019148413458991326,
+      "loss": 0.7282600402832031,
+      "step": 209
+    },
+    {
+      "epoch": 0.40076335877862596,
+      "grad_norm": 0.6990242600440979,
+      "learning_rate": 0.00019140325110279975,
+      "loss": 0.837559163570404,
+      "step": 210
+    },
+    {
+      "epoch": 0.4026717557251908,
+      "grad_norm": 0.8446074724197388,
+      "learning_rate": 0.0001913220025624854,
+      "loss": 0.7799749970436096,
+      "step": 211
+    },
+    {
+      "epoch": 0.40458015267175573,
+      "grad_norm": 0.8433075547218323,
+      "learning_rate": 0.0001912403892934667,
+      "loss": 0.8550182580947876,
+      "step": 212
+    },
+    {
+      "epoch": 0.4064885496183206,
+      "grad_norm": 0.7501733303070068,
+      "learning_rate": 0.00019115841162169693,
+      "loss": 0.9382178783416748,
+      "step": 213
+    },
+    {
+      "epoch": 0.4083969465648855,
+      "grad_norm": 0.726523220539093,
+      "learning_rate": 0.00019107606987458473,
+      "loss": 0.8986192345619202,
+      "step": 214
+    },
+    {
+      "epoch": 0.41030534351145037,
+      "grad_norm": 0.6420486569404602,
+      "learning_rate": 0.00019099336438099272,
+      "loss": 0.688328742980957,
+      "step": 215
+    },
+    {
+      "epoch": 0.4122137404580153,
+      "grad_norm": 0.6891405582427979,
+      "learning_rate": 0.00019091029547123637,
+      "loss": 0.6529335379600525,
+      "step": 216
+    },
+    {
+      "epoch": 0.41412213740458015,
+      "grad_norm": 0.6893248558044434,
+      "learning_rate": 0.00019082686347708254,
+      "loss": 0.8435322046279907,
+      "step": 217
+    },
+    {
+      "epoch": 0.41603053435114506,
+      "grad_norm": 0.6454209089279175,
+      "learning_rate": 0.00019074306873174827,
+      "loss": 0.7318831086158752,
+      "step": 218
+    },
+    {
+      "epoch": 0.4179389312977099,
+      "grad_norm": 0.6378938555717468,
+      "learning_rate": 0.0001906589115698993,
+      "loss": 0.7579640746116638,
+      "step": 219
+    },
+    {
+      "epoch": 0.4198473282442748,
+      "grad_norm": 0.7132017612457275,
+      "learning_rate": 0.00019057439232764885,
+      "loss": 0.8506414294242859,
+      "step": 220
+    },
+    {
+      "epoch": 0.4217557251908397,
+      "grad_norm": 0.8259621858596802,
+      "learning_rate": 0.00019048951134255625,
+      "loss": 0.7576637268066406,
+      "step": 221
+    },
+    {
+      "epoch": 0.42366412213740456,
+      "grad_norm": 0.8124897480010986,
+      "learning_rate": 0.0001904042689536256,
+      "loss": 0.8990870118141174,
+      "step": 222
+    },
+    {
+      "epoch": 0.4255725190839695,
+      "grad_norm": 0.9131141304969788,
+      "learning_rate": 0.00019031866550130438,
+      "loss": 0.8226636052131653,
+      "step": 223
+    },
+    {
+      "epoch": 0.42748091603053434,
+      "grad_norm": 0.6879171133041382,
+      "learning_rate": 0.00019023270132748207,
+      "loss": 0.8552612662315369,
+      "step": 224
+    },
+    {
+      "epoch": 0.42938931297709926,
+      "grad_norm": 0.6828365921974182,
+      "learning_rate": 0.00019014637677548893,
+      "loss": 0.9507154226303101,
+      "step": 225
+    },
+    {
+      "epoch": 0.4312977099236641,
+      "grad_norm": 0.7905396223068237,
+      "learning_rate": 0.00019005969219009449,
+      "loss": 0.9569607377052307,
+      "step": 226
+    },
+    {
+      "epoch": 0.43320610687022904,
+      "grad_norm": 0.7431300282478333,
+      "learning_rate": 0.00018997264791750614,
+      "loss": 0.9772092700004578,
+      "step": 227
+    },
+    {
+      "epoch": 0.4351145038167939,
+      "grad_norm": 0.6802636981010437,
+      "learning_rate": 0.00018988524430536784,
+      "loss": 0.7958412766456604,
+      "step": 228
+    },
+    {
+      "epoch": 0.43702290076335876,
+      "grad_norm": 0.6833978295326233,
+      "learning_rate": 0.0001897974817027588,
+      "loss": 0.6717628240585327,
+      "step": 229
+    },
+    {
+      "epoch": 0.4389312977099237,
+      "grad_norm": 0.6823106408119202,
+      "learning_rate": 0.00018970936046019184,
+      "loss": 0.730979323387146,
+      "step": 230
+    },
+    {
+      "epoch": 0.44083969465648853,
+      "grad_norm": 0.6977585554122925,
+      "learning_rate": 0.00018962088092961231,
+      "loss": 0.7230042219161987,
+      "step": 231
+    },
+    {
+      "epoch": 0.44274809160305345,
+      "grad_norm": 0.6336137652397156,
+      "learning_rate": 0.00018953204346439643,
+      "loss": 0.78371262550354,
+      "step": 232
+    },
+    {
+      "epoch": 0.4446564885496183,
+      "grad_norm": 0.7715194225311279,
+      "learning_rate": 0.00018944284841934992,
+      "loss": 0.8616683483123779,
+      "step": 233
+    },
+    {
+      "epoch": 0.44656488549618323,
+      "grad_norm": 0.7614982724189758,
+      "learning_rate": 0.0001893532961507067,
+      "loss": 0.8027196526527405,
+      "step": 234
+    },
+    {
+      "epoch": 0.4484732824427481,
+      "grad_norm": 0.7270587682723999,
+      "learning_rate": 0.00018926338701612738,
+      "loss": 0.7089850306510925,
+      "step": 235
+    },
+    {
+      "epoch": 0.45038167938931295,
+      "grad_norm": 0.8073937296867371,
+      "learning_rate": 0.00018917312137469784,
+      "loss": 0.8359235525131226,
+      "step": 236
+    },
+    {
+      "epoch": 0.45229007633587787,
+      "grad_norm": 0.6497123837471008,
+      "learning_rate": 0.00018908249958692782,
+      "loss": 0.7364542484283447,
+      "step": 237
+    },
+    {
+      "epoch": 0.4541984732824427,
+      "grad_norm": 0.6599416732788086,
+      "learning_rate": 0.00018899152201474947,
+      "loss": 0.96146160364151,
+      "step": 238
+    },
+    {
+      "epoch": 0.45610687022900764,
+      "grad_norm": 0.7983773946762085,
+      "learning_rate": 0.00018890018902151584,
+      "loss": 0.8446069359779358,
+      "step": 239
+    },
+    {
+      "epoch": 0.4580152671755725,
+      "grad_norm": 0.7493962645530701,
+      "learning_rate": 0.0001888085009719996,
+      "loss": 0.8703193664550781,
+      "step": 240
+    },
+    {
+      "epoch": 0.4599236641221374,
+      "grad_norm": 0.8169968128204346,
+      "learning_rate": 0.00018871645823239128,
+      "loss": 0.720066487789154,
+      "step": 241
+    },
+    {
+      "epoch": 0.4618320610687023,
+      "grad_norm": 0.8485615253448486,
+      "learning_rate": 0.00018862406117029824,
+      "loss": 0.6776174306869507,
+      "step": 242
+    },
+    {
+      "epoch": 0.4637404580152672,
+      "grad_norm": 0.7612597346305847,
+      "learning_rate": 0.00018853131015474276,
+      "loss": 0.8039662837982178,
+      "step": 243
+    },
+    {
+      "epoch": 0.46564885496183206,
+      "grad_norm": 0.7841982841491699,
+      "learning_rate": 0.0001884382055561609,
+      "loss": 0.7466532588005066,
+      "step": 244
+    },
+    {
+      "epoch": 0.4675572519083969,
+      "grad_norm": 0.8019384145736694,
+      "learning_rate": 0.00018834474774640078,
+      "loss": 0.9646111130714417,
+      "step": 245
+    },
+    {
+      "epoch": 0.46946564885496184,
+      "grad_norm": 0.7281209230422974,
+      "learning_rate": 0.00018825093709872132,
+      "loss": 0.7897037863731384,
+      "step": 246
+    },
+    {
+      "epoch": 0.4713740458015267,
+      "grad_norm": 0.797062337398529,
+      "learning_rate": 0.00018815677398779048,
+      "loss": 0.7747257351875305,
+      "step": 247
+    },
+    {
+      "epoch": 0.4732824427480916,
+      "grad_norm": 0.9131742119789124,
+      "learning_rate": 0.00018806225878968403,
+      "loss": 0.7692083716392517,
+      "step": 248
+    },
+    {
+      "epoch": 0.4751908396946565,
+      "grad_norm": 0.6530922651290894,
+      "learning_rate": 0.00018796739188188394,
+      "loss": 0.8715366721153259,
+      "step": 249
+    },
+    {
+      "epoch": 0.4770992366412214,
+      "grad_norm": 0.9586912393569946,
+      "learning_rate": 0.0001878721736432768,
+      "loss": 0.9756409525871277,
+      "step": 250
+    },
+    {
+      "epoch": 0.47900763358778625,
+      "grad_norm": 0.935222864151001,
+      "learning_rate": 0.00018777660445415235,
+      "loss": 0.7077156901359558,
+      "step": 251
+    },
+    {
+      "epoch": 0.48091603053435117,
+      "grad_norm": 0.6541524529457092,
+      "learning_rate": 0.00018768068469620207,
+      "loss": 0.6768779754638672,
+      "step": 252
+    },
+    {
+      "epoch": 0.48282442748091603,
+      "grad_norm": 0.8140842914581299,
+      "learning_rate": 0.00018758441475251754,
+      "loss": 0.8803874254226685,
+      "step": 253
+    },
+    {
+      "epoch": 0.4847328244274809,
+      "grad_norm": 0.7510896325111389,
+      "learning_rate": 0.00018748779500758888,
+      "loss": 0.8937657475471497,
+      "step": 254
+    },
+    {
+      "epoch": 0.4866412213740458,
+      "grad_norm": 0.8417599201202393,
+      "learning_rate": 0.0001873908258473034,
+      "loss": 0.7629004120826721,
+      "step": 255
+    },
+    {
+      "epoch": 0.48854961832061067,
+      "grad_norm": 0.6690160036087036,
+      "learning_rate": 0.00018729350765894372,
+      "loss": 0.7300665974617004,
+      "step": 256
+    },
+    {
+      "epoch": 0.4904580152671756,
+      "grad_norm": 0.7725889682769775,
+      "learning_rate": 0.00018719584083118666,
+      "loss": 0.827294111251831,
+      "step": 257
+    },
+    {
+      "epoch": 0.49236641221374045,
+      "grad_norm": 0.7742236256599426,
+      "learning_rate": 0.00018709782575410136,
+      "loss": 0.8049525022506714,
+      "step": 258
+    },
+    {
+      "epoch": 0.49427480916030536,
+      "grad_norm": 0.6990798711776733,
+      "learning_rate": 0.0001869994628191478,
+      "loss": 0.8596548438072205,
+      "step": 259
+    },
+    {
+      "epoch": 0.4961832061068702,
+      "grad_norm": 0.7861542701721191,
+      "learning_rate": 0.0001869007524191753,
+      "loss": 0.9379395246505737,
+      "step": 260
+    },
+    {
+      "epoch": 0.49809160305343514,
+      "grad_norm": 0.6927436590194702,
+      "learning_rate": 0.00018680169494842094,
+      "loss": 0.7861412763595581,
+      "step": 261
+    },
+    {
+      "epoch": 0.5,
+      "grad_norm": 0.8491238951683044,
+      "learning_rate": 0.00018670229080250788,
+      "loss": 0.7700883746147156,
+      "step": 262
+    },
+    {
+      "epoch": 0.5019083969465649,
+      "grad_norm": 0.8999499082565308,
+      "learning_rate": 0.00018660254037844388,
+      "loss": 0.8536050915718079,
+      "step": 263
+    },
+    {
+      "epoch": 0.5038167938931297,
+      "grad_norm": 0.7316823601722717,
+      "learning_rate": 0.00018650244407461975,
+      "loss": 0.7228575944900513,
+      "step": 264
+    },
+    {
+      "epoch": 0.5057251908396947,
+      "grad_norm": 0.7099804878234863,
+      "learning_rate": 0.00018640200229080763,
+      "loss": 0.8247818350791931,
+      "step": 265
+    },
+    {
+      "epoch": 0.5076335877862596,
+      "grad_norm": 0.6847162842750549,
+      "learning_rate": 0.00018630121542815954,
+      "loss": 0.6822383403778076,
+      "step": 266
+    },
+    {
+      "epoch": 0.5095419847328244,
+      "grad_norm": 0.696711540222168,
+      "learning_rate": 0.0001862000838892056,
+      "loss": 0.7580799460411072,
+      "step": 267
+    },
+    {
+      "epoch": 0.5114503816793893,
+      "grad_norm": 0.6475000381469727,
+      "learning_rate": 0.00018609860807785263,
+      "loss": 0.7244993448257446,
+      "step": 268
+    },
+    {
+      "epoch": 0.5133587786259542,
+      "grad_norm": 0.835381269454956,
+      "learning_rate": 0.00018599678839938236,
+      "loss": 0.8199276924133301,
+      "step": 269
+    },
+    {
+      "epoch": 0.5152671755725191,
+      "grad_norm": 0.667424201965332,
+      "learning_rate": 0.00018589462526044992,
+      "loss": 0.7150349020957947,
+      "step": 270
+    },
+    {
+      "epoch": 0.517175572519084,
+      "grad_norm": 0.9183174967765808,
+      "learning_rate": 0.00018579211906908215,
+      "loss": 0.8441946506500244,
+      "step": 271
+    },
+    {
+      "epoch": 0.5190839694656488,
+      "grad_norm": 0.7149916291236877,
+      "learning_rate": 0.00018568927023467598,
+      "loss": 0.8440540432929993,
+      "step": 272
+    },
+    {
+      "epoch": 0.5209923664122137,
+      "grad_norm": 0.7080127596855164,
+      "learning_rate": 0.0001855860791679969,
+      "loss": 0.7530430555343628,
+      "step": 273
+    },
+    {
+      "epoch": 0.5229007633587787,
+      "grad_norm": 0.8946796655654907,
+      "learning_rate": 0.00018548254628117714,
+      "loss": 0.7249343991279602,
+      "step": 274
+    },
+    {
+      "epoch": 0.5248091603053435,
+      "grad_norm": 0.826902449131012,
+      "learning_rate": 0.00018537867198771416,
+      "loss": 0.9324170351028442,
+      "step": 275
+    },
+    {
+      "epoch": 0.5267175572519084,
+      "grad_norm": 0.7095419764518738,
+      "learning_rate": 0.000185274456702469,
+      "loss": 0.841179370880127,
+      "step": 276
+    },
+    {
+      "epoch": 0.5286259541984732,
+      "grad_norm": 0.7262468338012695,
+      "learning_rate": 0.00018516990084166442,
+      "loss": 0.8151962757110596,
+      "step": 277
+    },
+    {
+      "epoch": 0.5305343511450382,
+      "grad_norm": 0.7610070705413818,
+      "learning_rate": 0.00018506500482288365,
+      "loss": 0.8434594869613647,
+      "step": 278
+    },
+    {
+      "epoch": 0.5324427480916031,
+      "grad_norm": 0.6930210590362549,
+      "learning_rate": 0.00018495976906506822,
+      "loss": 0.8486746549606323,
+      "step": 279
+    },
+    {
+      "epoch": 0.5343511450381679,
+      "grad_norm": 0.6274219751358032,
+      "learning_rate": 0.00018485419398851666,
+      "loss": 0.7165244817733765,
+      "step": 280
+    },
+    {
+      "epoch": 0.5362595419847328,
+      "grad_norm": 0.7024329304695129,
+      "learning_rate": 0.00018474828001488266,
+      "loss": 0.8439553380012512,
+      "step": 281
+    },
+    {
+      "epoch": 0.5381679389312977,
+      "grad_norm": 0.6275829672813416,
+      "learning_rate": 0.0001846420275671735,
+      "loss": 0.7700610756874084,
+      "step": 282
+    },
+    {
+      "epoch": 0.5400763358778626,
+      "grad_norm": 0.8504899144172668,
+      "learning_rate": 0.0001845354370697482,
+      "loss": 0.9510678052902222,
+      "step": 283
+    },
+    {
+      "epoch": 0.5419847328244275,
+      "grad_norm": 0.9339554905891418,
+      "learning_rate": 0.00018442850894831588,
+      "loss": 0.8770922422409058,
+      "step": 284
+    },
+    {
+      "epoch": 0.5438931297709924,
+      "grad_norm": 0.7716355919837952,
+      "learning_rate": 0.00018432124362993425,
+      "loss": 0.7500790953636169,
+      "step": 285
+    },
+    {
+      "epoch": 0.5458015267175572,
+      "grad_norm": 0.6428256034851074,
+      "learning_rate": 0.00018421364154300749,
+      "loss": 0.7144466638565063,
+      "step": 286
+    },
+    {
+      "epoch": 0.5477099236641222,
+      "grad_norm": 0.6565148830413818,
+      "learning_rate": 0.00018410570311728504,
+      "loss": 0.7853836417198181,
+      "step": 287
+    },
+    {
+      "epoch": 0.549618320610687,
+      "grad_norm": 0.7436274290084839,
+      "learning_rate": 0.00018399742878385947,
+      "loss": 0.7885539531707764,
+      "step": 288
+    },
+    {
+      "epoch": 0.5515267175572519,
+      "grad_norm": 0.7915308475494385,
+      "learning_rate": 0.000183888818975165,
+      "loss": 0.8914458751678467,
+      "step": 289
+    },
+    {
+      "epoch": 0.5534351145038168,
+      "grad_norm": 0.6639530658721924,
+      "learning_rate": 0.0001837798741249757,
+      "loss": 0.8313987851142883,
+      "step": 290
+    },
+    {
+      "epoch": 0.5553435114503816,
+      "grad_norm": 0.7638710737228394,
+      "learning_rate": 0.00018367059466840372,
+      "loss": 0.7018760442733765,
+      "step": 291
+    },
+    {
+      "epoch": 0.5572519083969466,
+      "grad_norm": 0.6375514268875122,
+      "learning_rate": 0.0001835609810418976,
+      "loss": 0.6819204688072205,
+      "step": 292
+    },
+    {
+      "epoch": 0.5591603053435115,
+      "grad_norm": 0.7897312045097351,
+      "learning_rate": 0.0001834510336832405,
+      "loss": 0.8072556257247925,
+      "step": 293
+    },
+    {
+      "epoch": 0.5610687022900763,
+      "grad_norm": 0.6562504172325134,
+      "learning_rate": 0.00018334075303154848,
+      "loss": 0.6301808953285217,
+      "step": 294
+    },
+    {
+      "epoch": 0.5629770992366412,
+      "grad_norm": 0.6480065584182739,
+      "learning_rate": 0.00018323013952726875,
+      "loss": 0.8817023038864136,
+      "step": 295
+    },
+    {
+      "epoch": 0.5648854961832062,
+      "grad_norm": 0.7154799103736877,
+      "learning_rate": 0.00018311919361217784,
+      "loss": 0.873283863067627,
+      "step": 296
+    },
+    {
+      "epoch": 0.566793893129771,
+      "grad_norm": 0.6573131084442139,
+      "learning_rate": 0.00018300791572937994,
+      "loss": 0.7678433060646057,
+      "step": 297
+    },
+    {
+      "epoch": 0.5687022900763359,
+      "grad_norm": 0.6776493191719055,
+      "learning_rate": 0.00018289630632330504,
+      "loss": 0.8190892934799194,
+      "step": 298
+    },
+    {
+      "epoch": 0.5706106870229007,
+      "grad_norm": 0.7599857449531555,
+      "learning_rate": 0.0001827843658397072,
+      "loss": 0.8244090676307678,
+      "step": 299
+    },
+    {
+      "epoch": 0.5725190839694656,
+      "grad_norm": 0.6910902857780457,
+      "learning_rate": 0.00018267209472566277,
+      "loss": 0.7675949335098267,
+      "step": 300
+    },
+    {
+      "epoch": 0.5744274809160306,
+      "grad_norm": 0.6786538362503052,
+      "learning_rate": 0.00018255949342956863,
+      "loss": 0.7707676887512207,
+      "step": 301
+    },
+    {
+      "epoch": 0.5763358778625954,
+      "grad_norm": 0.7985155582427979,
+      "learning_rate": 0.00018244656240114031,
+      "loss": 0.8402585983276367,
+      "step": 302
+    },
+    {
+      "epoch": 0.5782442748091603,
+      "grad_norm": 0.802747905254364,
+      "learning_rate": 0.00018233330209141025,
+      "loss": 0.8247464299201965,
+      "step": 303
+    },
+    {
+      "epoch": 0.5801526717557252,
+      "grad_norm": 0.7794269919395447,
+      "learning_rate": 0.00018221971295272606,
+      "loss": 0.813611626625061,
+      "step": 304
+    },
+    {
+      "epoch": 0.5820610687022901,
+      "grad_norm": 0.6826204657554626,
+      "learning_rate": 0.0001821057954387486,
+      "loss": 0.8085711002349854,
+      "step": 305
+    },
+    {
+      "epoch": 0.583969465648855,
+      "grad_norm": 0.8226385712623596,
+      "learning_rate": 0.00018199155000445024,
+      "loss": 0.920879065990448,
+      "step": 306
+    },
+    {
+      "epoch": 0.5858778625954199,
+      "grad_norm": 0.6477347612380981,
+      "learning_rate": 0.00018187697710611298,
+      "loss": 0.6817933917045593,
+      "step": 307
+    },
+    {
+      "epoch": 0.5877862595419847,
+      "grad_norm": 0.5807804465293884,
+      "learning_rate": 0.0001817620772013267,
+      "loss": 0.6386286020278931,
+      "step": 308
+    },
+    {
+      "epoch": 0.5896946564885496,
+      "grad_norm": 0.8978777527809143,
+      "learning_rate": 0.0001816468507489874,
+      "loss": 0.9605103135108948,
+      "step": 309
+    },
+    {
+      "epoch": 0.5916030534351145,
+      "grad_norm": 0.778271496295929,
+      "learning_rate": 0.00018153129820929506,
+      "loss": 0.8793523907661438,
+      "step": 310
+    },
+    {
+      "epoch": 0.5935114503816794,
+      "grad_norm": 0.6236903667449951,
+      "learning_rate": 0.00018141542004375223,
+      "loss": 0.6926164627075195,
+      "step": 311
+    },
+    {
+      "epoch": 0.5954198473282443,
+      "grad_norm": 0.6846082806587219,
+      "learning_rate": 0.00018129921671516182,
+      "loss": 0.9161442518234253,
+      "step": 312
+    },
+    {
+      "epoch": 0.5973282442748091,
+      "grad_norm": 0.6092160940170288,
+      "learning_rate": 0.00018118268868762546,
+      "loss": 0.7681725025177002,
+      "step": 313
+    },
+    {
+      "epoch": 0.5992366412213741,
+      "grad_norm": 0.6718221306800842,
+      "learning_rate": 0.00018106583642654158,
+      "loss": 0.8049030900001526,
+      "step": 314
+    },
+    {
+      "epoch": 0.601145038167939,
+      "grad_norm": 0.6517919898033142,
+      "learning_rate": 0.00018094866039860356,
+      "loss": 0.672369658946991,
+      "step": 315
+    },
+    {
+      "epoch": 0.6030534351145038,
+      "grad_norm": 0.7806390523910522,
+      "learning_rate": 0.00018083116107179788,
+      "loss": 0.940979540348053,
+      "step": 316
+    },
+    {
+      "epoch": 0.6049618320610687,
+      "grad_norm": 0.7215765118598938,
+      "learning_rate": 0.00018071333891540213,
+      "loss": 0.7792754769325256,
+      "step": 317
+    },
+    {
+      "epoch": 0.6068702290076335,
+      "grad_norm": 0.7212367653846741,
+      "learning_rate": 0.00018059519439998347,
+      "loss": 0.7840749621391296,
+      "step": 318
+    },
+    {
+      "epoch": 0.6087786259541985,
+      "grad_norm": 0.6932458877563477,
+      "learning_rate": 0.00018047672799739628,
+      "loss": 0.8413757681846619,
+      "step": 319
+    },
+    {
+      "epoch": 0.6106870229007634,
+      "grad_norm": 0.7376019358634949,
+      "learning_rate": 0.00018035794018078064,
+      "loss": 0.7670861482620239,
+      "step": 320
+    },
+    {
+      "epoch": 0.6125954198473282,
+      "grad_norm": 0.7866705060005188,
+      "learning_rate": 0.0001802388314245603,
+      "loss": 0.8528102040290833,
+      "step": 321
+    },
+    {
+      "epoch": 0.6145038167938931,
+      "grad_norm": 0.7400007247924805,
+      "learning_rate": 0.0001801194022044408,
+      "loss": 0.7491836547851562,
+      "step": 322
+    },
+    {
+      "epoch": 0.6164122137404581,
+      "grad_norm": 0.7856490015983582,
+      "learning_rate": 0.00017999965299740753,
+      "loss": 0.8039355278015137,
+      "step": 323
+    },
+    {
+      "epoch": 0.6183206106870229,
+      "grad_norm": 0.7606260776519775,
+      "learning_rate": 0.00017987958428172397,
+      "loss": 0.6929895877838135,
+      "step": 324
+    },
+    {
+      "epoch": 0.6202290076335878,
+      "grad_norm": 0.7794435024261475,
+      "learning_rate": 0.0001797591965369296,
+      "loss": 0.718076765537262,
+      "step": 325
+    },
+    {
+      "epoch": 0.6221374045801527,
+      "grad_norm": 0.7260003089904785,
+      "learning_rate": 0.00017963849024383804,
+      "loss": 0.7952104210853577,
+      "step": 326
+    },
+    {
+      "epoch": 0.6240458015267175,
+      "grad_norm": 0.7415996193885803,
+      "learning_rate": 0.00017951746588453524,
+      "loss": 0.7466655373573303,
+      "step": 327
+    },
+    {
+      "epoch": 0.6259541984732825,
+      "grad_norm": 0.7453495860099792,
+      "learning_rate": 0.0001793961239423774,
+      "loss": 0.8698800802230835,
+      "step": 328
+    },
+    {
+      "epoch": 0.6278625954198473,
+      "grad_norm": 0.7051957249641418,
+      "learning_rate": 0.0001792744649019891,
+      "loss": 0.804383397102356,
+      "step": 329
+    },
+    {
+      "epoch": 0.6297709923664122,
+      "grad_norm": 0.8170496225357056,
+      "learning_rate": 0.00017915248924926143,
+      "loss": 0.7736644744873047,
+      "step": 330
+    },
+    {
+      "epoch": 0.6316793893129771,
+      "grad_norm": 0.7568200826644897,
+      "learning_rate": 0.00017903019747134998,
+      "loss": 0.7610517144203186,
+      "step": 331
+    },
+    {
+      "epoch": 0.6335877862595419,
+      "grad_norm": 0.6409168243408203,
+      "learning_rate": 0.00017890759005667278,
+      "loss": 0.6060128808021545,
+      "step": 332
+    },
+    {
+      "epoch": 0.6354961832061069,
+      "grad_norm": 0.7364526987075806,
+      "learning_rate": 0.00017878466749490866,
+      "loss": 0.9377261400222778,
+      "step": 333
+    },
+    {
+      "epoch": 0.6374045801526718,
+      "grad_norm": 0.8193963766098022,
+      "learning_rate": 0.000178661430276995,
+      "loss": 0.7472018003463745,
+      "step": 334
+    },
+    {
+      "epoch": 0.6393129770992366,
+      "grad_norm": 0.7738126516342163,
+      "learning_rate": 0.00017853787889512588,
+      "loss": 0.7138077020645142,
+      "step": 335
+    },
+    {
+      "epoch": 0.6412213740458015,
+      "grad_norm": 0.7044742703437805,
+      "learning_rate": 0.0001784140138427502,
+      "loss": 0.9724718332290649,
+      "step": 336
+    },
+    {
+      "epoch": 0.6431297709923665,
+      "grad_norm": 0.9464431405067444,
+      "learning_rate": 0.00017828983561456941,
+      "loss": 0.7842336297035217,
+      "step": 337
+    },
+    {
+      "epoch": 0.6450381679389313,
+      "grad_norm": 0.8200290203094482,
+      "learning_rate": 0.00017816534470653606,
+      "loss": 0.7992740869522095,
+      "step": 338
+    },
+    {
+      "epoch": 0.6469465648854962,
+      "grad_norm": 0.712479829788208,
+      "learning_rate": 0.00017804054161585123,
+      "loss": 0.6871699690818787,
+      "step": 339
+    },
+    {
+      "epoch": 0.648854961832061,
+      "grad_norm": 0.6953346133232117,
+      "learning_rate": 0.0001779154268409629,
+      "loss": 0.8726075291633606,
+      "step": 340
+    },
+    {
+      "epoch": 0.6507633587786259,
+      "grad_norm": 0.7262577414512634,
+      "learning_rate": 0.00017779000088156396,
+      "loss": 0.8324016332626343,
+      "step": 341
+    },
+    {
+      "epoch": 0.6526717557251909,
+      "grad_norm": 0.7458943128585815,
+      "learning_rate": 0.00017766426423859006,
+      "loss": 0.8070228099822998,
+      "step": 342
+    },
+    {
+      "epoch": 0.6545801526717557,
+      "grad_norm": 0.6674166917800903,
+      "learning_rate": 0.00017753821741421769,
+      "loss": 0.7008100748062134,
+      "step": 343
+    },
+    {
+      "epoch": 0.6564885496183206,
+      "grad_norm": 0.7021476626396179,
+      "learning_rate": 0.0001774118609118621,
+      "loss": 0.7791208028793335,
+      "step": 344
+    },
+    {
+      "epoch": 0.6583969465648855,
+      "grad_norm": 0.5728323459625244,
+      "learning_rate": 0.00017728519523617554,
+      "loss": 0.7050975561141968,
+      "step": 345
+    },
+    {
+      "epoch": 0.6603053435114504,
+      "grad_norm": 0.6441164612770081,
+      "learning_rate": 0.00017715822089304486,
+      "loss": 0.7866590619087219,
+      "step": 346
+    },
+    {
+      "epoch": 0.6622137404580153,
+      "grad_norm": 0.7046230435371399,
+      "learning_rate": 0.0001770309383895898,
+      "loss": 0.8273329138755798,
+      "step": 347
+    },
+    {
+      "epoch": 0.6641221374045801,
+      "grad_norm": 0.6595011353492737,
+      "learning_rate": 0.00017690334823416084,
+      "loss": 0.6689213514328003,
+      "step": 348
+    },
+    {
+      "epoch": 0.666030534351145,
+      "grad_norm": 0.5818625688552856,
+      "learning_rate": 0.00017677545093633713,
+      "loss": 0.6251814961433411,
+      "step": 349
+    },
+    {
+      "epoch": 0.6679389312977099,
+      "grad_norm": 0.6281395554542542,
+      "learning_rate": 0.00017664724700692454,
+      "loss": 0.665907621383667,
+      "step": 350
+    },
+    {
+      "epoch": 0.6698473282442748,
+      "grad_norm": 0.5682083368301392,
+      "learning_rate": 0.00017651873695795367,
+      "loss": 0.7040910720825195,
+      "step": 351
+    },
+    {
+      "epoch": 0.6717557251908397,
+      "grad_norm": 0.8523306846618652,
+      "learning_rate": 0.00017638992130267756,
+      "loss": 0.9397965669631958,
+      "step": 352
+    },
+    {
+      "epoch": 0.6736641221374046,
+      "grad_norm": 0.8172243237495422,
+      "learning_rate": 0.00017626080055557,
+      "loss": 0.7563872337341309,
+      "step": 353
+    },
+    {
+      "epoch": 0.6755725190839694,
+      "grad_norm": 0.7371250987052917,
+      "learning_rate": 0.00017613137523232308,
+      "loss": 0.7437839508056641,
+      "step": 354
+    },
+    {
+      "epoch": 0.6774809160305344,
+      "grad_norm": 0.6615129709243774,
+      "learning_rate": 0.00017600164584984546,
+      "loss": 0.8135066628456116,
+      "step": 355
+    },
+    {
+      "epoch": 0.6793893129770993,
+      "grad_norm": 0.695019543170929,
+      "learning_rate": 0.00017587161292626012,
+      "loss": 0.8044134378433228,
+      "step": 356
+    },
+    {
+      "epoch": 0.6812977099236641,
+      "grad_norm": 0.6614181399345398,
+      "learning_rate": 0.00017574127698090236,
+      "loss": 0.8590314984321594,
+      "step": 357
+    },
+    {
+      "epoch": 0.683206106870229,
+      "grad_norm": 0.6806448698043823,
+      "learning_rate": 0.00017561063853431774,
+      "loss": 0.7957973480224609,
+      "step": 358
+    },
+    {
+      "epoch": 0.6851145038167938,
+      "grad_norm": 0.6237355470657349,
+      "learning_rate": 0.00017547969810825995,
+      "loss": 0.7711411118507385,
+      "step": 359
+    },
+    {
+      "epoch": 0.6870229007633588,
+      "grad_norm": 0.7326170206069946,
+      "learning_rate": 0.00017534845622568869,
+      "loss": 0.7691990733146667,
+      "step": 360
+    },
+    {
+      "epoch": 0.6889312977099237,
+      "grad_norm": 0.6910527348518372,
+      "learning_rate": 0.00017521691341076774,
+      "loss": 0.6117557287216187,
+      "step": 361
+    },
+    {
+      "epoch": 0.6908396946564885,
+      "grad_norm": 0.7196635007858276,
+      "learning_rate": 0.00017508507018886268,
+      "loss": 0.8305698037147522,
+      "step": 362
+    },
+    {
+      "epoch": 0.6927480916030534,
+      "grad_norm": 0.8049364686012268,
+      "learning_rate": 0.00017495292708653897,
+      "loss": 0.8799786567687988,
+      "step": 363
+    },
+    {
+      "epoch": 0.6946564885496184,
+      "grad_norm": 0.6581491231918335,
+      "learning_rate": 0.00017482048463155967,
+      "loss": 0.7862188220024109,
+      "step": 364
+    },
+    {
+      "epoch": 0.6965648854961832,
+      "grad_norm": 0.7264696955680847,
+      "learning_rate": 0.00017468774335288343,
+      "loss": 0.7697022557258606,
+      "step": 365
+    },
+    {
+      "epoch": 0.6984732824427481,
+      "grad_norm": 0.7711309790611267,
+      "learning_rate": 0.00017455470378066243,
+      "loss": 0.711321234703064,
+      "step": 366
+    },
+    {
+      "epoch": 0.700381679389313,
+      "grad_norm": 0.7543802857398987,
+      "learning_rate": 0.00017442136644624015,
+      "loss": 0.7811844944953918,
+      "step": 367
+    },
+    {
+      "epoch": 0.7022900763358778,
+      "grad_norm": 0.7283676862716675,
+      "learning_rate": 0.00017428773188214925,
+      "loss": 0.8818699717521667,
+      "step": 368
+    },
+    {
+      "epoch": 0.7041984732824428,
+      "grad_norm": 0.6169099807739258,
+      "learning_rate": 0.00017415380062210954,
+      "loss": 0.6504080891609192,
+      "step": 369
+    },
+    {
+      "epoch": 0.7061068702290076,
+      "grad_norm": 0.6292209625244141,
+      "learning_rate": 0.00017401957320102587,
+      "loss": 0.7281547784805298,
+      "step": 370
+    },
+    {
+      "epoch": 0.7080152671755725,
+      "grad_norm": 0.8087360858917236,
+      "learning_rate": 0.00017388505015498575,
+      "loss": 0.911018967628479,
+      "step": 371
+    },
+    {
+      "epoch": 0.7099236641221374,
+      "grad_norm": 0.6205286979675293,
+      "learning_rate": 0.00017375023202125753,
+      "loss": 0.6708084940910339,
+      "step": 372
+    },
+    {
+      "epoch": 0.7118320610687023,
+      "grad_norm": 0.651057243347168,
+      "learning_rate": 0.00017361511933828801,
+      "loss": 0.8169975280761719,
+      "step": 373
+    },
+    {
+      "epoch": 0.7137404580152672,
+      "grad_norm": 0.6071662902832031,
+      "learning_rate": 0.0001734797126457004,
+      "loss": 0.8124607801437378,
+      "step": 374
+    },
+    {
+      "epoch": 0.7156488549618321,
+      "grad_norm": 0.6285260915756226,
+      "learning_rate": 0.00017334401248429227,
+      "loss": 0.8034093976020813,
+      "step": 375
+    },
+    {
+      "epoch": 0.7175572519083969,
+      "grad_norm": 0.6304917931556702,
+      "learning_rate": 0.00017320801939603304,
+      "loss": 0.6551451683044434,
+      "step": 376
+    },
+    {
+      "epoch": 0.7194656488549618,
+      "grad_norm": 0.6045227646827698,
+      "learning_rate": 0.00017307173392406221,
+      "loss": 0.6780649423599243,
+      "step": 377
+    },
+    {
+      "epoch": 0.7213740458015268,
+      "grad_norm": 0.7062244415283203,
+      "learning_rate": 0.000172935156612687,
+      "loss": 0.8283180594444275,
+      "step": 378
+    },
+    {
+      "epoch": 0.7232824427480916,
+      "grad_norm": 0.6424290537834167,
+      "learning_rate": 0.00017279828800738017,
+      "loss": 0.6931131482124329,
+      "step": 379
+    },
+    {
+      "epoch": 0.7251908396946565,
+      "grad_norm": 0.5736993551254272,
+      "learning_rate": 0.0001726611286547779,
+      "loss": 0.6850195527076721,
+      "step": 380
+    },
+    {
+      "epoch": 0.7270992366412213,
+      "grad_norm": 0.583165168762207,
+      "learning_rate": 0.0001725236791026775,
+      "loss": 0.6734647750854492,
+      "step": 381
+    },
+    {
+      "epoch": 0.7290076335877863,
+      "grad_norm": 0.6753141283988953,
+      "learning_rate": 0.00017238593990003543,
+      "loss": 0.6214074492454529,
+      "step": 382
+    },
+    {
+      "epoch": 0.7309160305343512,
+      "grad_norm": 0.6378006935119629,
+      "learning_rate": 0.0001722479115969649,
+      "loss": 0.6616349220275879,
+      "step": 383
+    },
+    {
+      "epoch": 0.732824427480916,
+      "grad_norm": 0.6691845059394836,
+      "learning_rate": 0.00017210959474473373,
+      "loss": 0.7897334694862366,
+      "step": 384
+    },
+    {
+      "epoch": 0.7347328244274809,
+      "grad_norm": 0.7121267318725586,
+      "learning_rate": 0.00017197098989576222,
+      "loss": 0.7752635478973389,
+      "step": 385
+    },
+    {
+      "epoch": 0.7366412213740458,
+      "grad_norm": 0.7109902501106262,
+      "learning_rate": 0.00017183209760362087,
+      "loss": 0.7637916803359985,
+      "step": 386
+    },
+    {
+      "epoch": 0.7385496183206107,
+      "grad_norm": 0.6235750913619995,
+      "learning_rate": 0.0001716929184230282,
+      "loss": 0.7449964284896851,
+      "step": 387
+    },
+    {
+      "epoch": 0.7404580152671756,
+      "grad_norm": 0.6020548939704895,
+      "learning_rate": 0.00017155345290984853,
+      "loss": 0.7526451945304871,
+      "step": 388
+    },
+    {
+      "epoch": 0.7423664122137404,
+      "grad_norm": 0.7183219790458679,
+      "learning_rate": 0.0001714137016210897,
+      "loss": 0.8206169009208679,
+      "step": 389
+    },
+    {
+      "epoch": 0.7442748091603053,
+      "grad_norm": 0.7202697396278381,
+      "learning_rate": 0.00017127366511490103,
+      "loss": 0.9569057822227478,
+      "step": 390
+    },
+    {
+      "epoch": 0.7461832061068703,
+      "grad_norm": 0.5628166794776917,
+      "learning_rate": 0.00017113334395057087,
+      "loss": 0.7293851971626282,
+      "step": 391
+    },
+    {
+      "epoch": 0.7480916030534351,
+      "grad_norm": 0.5517001152038574,
+      "learning_rate": 0.0001709927386885244,
+      "loss": 0.6535385847091675,
+      "step": 392
+    },
+    {
+      "epoch": 0.75,
+      "grad_norm": 0.6714818477630615,
+      "learning_rate": 0.00017085184989032158,
+      "loss": 0.9788048267364502,
+      "step": 393
+    },
+    {
+      "epoch": 0.7519083969465649,
+      "grad_norm": 0.6490929126739502,
+      "learning_rate": 0.00017071067811865476,
+      "loss": 0.7962111234664917,
+      "step": 394
+    },
+    {
+      "epoch": 0.7538167938931297,
+      "grad_norm": 0.6007545590400696,
+      "learning_rate": 0.00017056922393734637,
+      "loss": 0.6212320923805237,
+      "step": 395
+    },
+    {
+      "epoch": 0.7557251908396947,
+      "grad_norm": 0.6076050400733948,
+      "learning_rate": 0.00017042748791134673,
+      "loss": 0.742911696434021,
+      "step": 396
+    },
+    {
+      "epoch": 0.7576335877862596,
+      "grad_norm": 0.692491888999939,
+      "learning_rate": 0.000170285470606732,
+      "loss": 0.6534058451652527,
+      "step": 397
+    },
+    {
+      "epoch": 0.7595419847328244,
+      "grad_norm": 0.6946771740913391,
+      "learning_rate": 0.00017014317259070148,
+      "loss": 0.7185120582580566,
+      "step": 398
+    },
+    {
+      "epoch": 0.7614503816793893,
+      "grad_norm": 0.7877113819122314,
+      "learning_rate": 0.00017000059443157586,
+      "loss": 0.8319492936134338,
+      "step": 399
+    },
+    {
+      "epoch": 0.7633587786259542,
+      "grad_norm": 0.7517495155334473,
+      "learning_rate": 0.00016985773669879443,
+      "loss": 0.9311786890029907,
+      "step": 400
+    },
+    {
+      "epoch": 0.7652671755725191,
+      "grad_norm": 0.6660169959068298,
+      "learning_rate": 0.0001697145999629133,
+      "loss": 0.7865347862243652,
+      "step": 401
+    },
+    {
+      "epoch": 0.767175572519084,
+      "grad_norm": 0.6578369736671448,
+      "learning_rate": 0.0001695711847956026,
+      "loss": 0.7376701831817627,
+      "step": 402
+    },
+    {
+      "epoch": 0.7690839694656488,
+      "grad_norm": 0.8675081133842468,
+      "learning_rate": 0.0001694274917696448,
+      "loss": 0.7598669528961182,
+      "step": 403
+    },
+    {
+      "epoch": 0.7709923664122137,
+      "grad_norm": 0.6933845281600952,
+      "learning_rate": 0.0001692835214589319,
+      "loss": 0.8819526433944702,
+      "step": 404
+    },
+    {
+      "epoch": 0.7729007633587787,
+      "grad_norm": 0.600820779800415,
+      "learning_rate": 0.0001691392744384633,
+      "loss": 0.6052135229110718,
+      "step": 405
+    },
+    {
+      "epoch": 0.7748091603053435,
+      "grad_norm": 0.6492821574211121,
+      "learning_rate": 0.00016899475128434382,
+      "loss": 0.7452248334884644,
+      "step": 406
+    },
+    {
+      "epoch": 0.7767175572519084,
+      "grad_norm": 0.6682466268539429,
+      "learning_rate": 0.00016884995257378078,
+      "loss": 0.7919533848762512,
+      "step": 407
+    },
+    {
+      "epoch": 0.7786259541984732,
+      "grad_norm": 0.6175159811973572,
+      "learning_rate": 0.00016870487888508224,
+      "loss": 0.75071120262146,
+      "step": 408
+    },
+    {
+      "epoch": 0.7805343511450382,
+      "grad_norm": 0.6515896320343018,
+      "learning_rate": 0.00016855953079765448,
+      "loss": 0.8758217692375183,
+      "step": 409
+    },
+    {
+      "epoch": 0.7824427480916031,
+      "grad_norm": 0.7739993929862976,
+      "learning_rate": 0.00016841390889199963,
+      "loss": 0.700404942035675,
+      "step": 410
+    },
+    {
+      "epoch": 0.7843511450381679,
+      "grad_norm": 0.6714352965354919,
+      "learning_rate": 0.00016826801374971343,
+      "loss": 0.6998648643493652,
+      "step": 411
+    },
+    {
+      "epoch": 0.7862595419847328,
+      "grad_norm": 0.5435221195220947,
+      "learning_rate": 0.0001681218459534829,
+      "loss": 0.6169798970222473,
+      "step": 412
+    },
+    {
+      "epoch": 0.7881679389312977,
+      "grad_norm": 0.5985938310623169,
+      "learning_rate": 0.00016797540608708407,
+      "loss": 0.5850165486335754,
+      "step": 413
+    },
+    {
+      "epoch": 0.7900763358778626,
+      "grad_norm": 0.7272723317146301,
+      "learning_rate": 0.0001678286947353795,
+      "loss": 0.7439934611320496,
+      "step": 414
+    },
+    {
+      "epoch": 0.7919847328244275,
+      "grad_norm": 0.7125585079193115,
+      "learning_rate": 0.00016768171248431602,
+      "loss": 0.6568199396133423,
+      "step": 415
+    },
+    {
+      "epoch": 0.7938931297709924,
+      "grad_norm": 0.6785078644752502,
+      "learning_rate": 0.0001675344599209225,
+      "loss": 0.7679774761199951,
+      "step": 416
+    },
+    {
+      "epoch": 0.7958015267175572,
+      "grad_norm": 0.680644154548645,
+      "learning_rate": 0.00016738693763330726,
+      "loss": 0.7425507307052612,
+      "step": 417
+    },
+    {
+      "epoch": 0.7977099236641222,
+      "grad_norm": 0.5993022918701172,
+      "learning_rate": 0.00016723914621065598,
+      "loss": 0.7285944819450378,
+      "step": 418
+    },
+    {
+      "epoch": 0.799618320610687,
+      "grad_norm": 0.6360685229301453,
+      "learning_rate": 0.00016709108624322924,
+      "loss": 0.8232290744781494,
+      "step": 419
+    },
+    {
+      "epoch": 0.8015267175572519,
+      "grad_norm": 0.6330887079238892,
+      "learning_rate": 0.0001669427583223601,
+      "loss": 0.6810738444328308,
+      "step": 420
+    },
+    {
+      "epoch": 0.8034351145038168,
+      "grad_norm": 0.7288772463798523,
+      "learning_rate": 0.0001667941630404517,
+      "loss": 0.8008633852005005,
+      "step": 421
+    },
+    {
+      "epoch": 0.8053435114503816,
+      "grad_norm": 0.684376060962677,
+      "learning_rate": 0.00016664530099097521,
+      "loss": 0.7976124286651611,
+      "step": 422
+    },
+    {
+      "epoch": 0.8072519083969466,
+      "grad_norm": 0.674179196357727,
+      "learning_rate": 0.00016649617276846704,
+      "loss": 0.947729229927063,
+      "step": 423
+    },
+    {
+      "epoch": 0.8091603053435115,
+      "grad_norm": 0.6153618693351746,
+      "learning_rate": 0.00016634677896852678,
+      "loss": 0.6947682499885559,
+      "step": 424
+    },
+    {
+      "epoch": 0.8110687022900763,
+      "grad_norm": 0.7484484314918518,
+      "learning_rate": 0.00016619712018781462,
+      "loss": 0.8903341293334961,
+      "step": 425
+    },
+    {
+      "epoch": 0.8129770992366412,
+      "grad_norm": 0.6236786842346191,
+      "learning_rate": 0.00016604719702404915,
+      "loss": 0.6754611730575562,
+      "step": 426
+    },
+    {
+      "epoch": 0.8148854961832062,
+      "grad_norm": 0.5537739992141724,
+      "learning_rate": 0.00016589701007600476,
+      "loss": 0.7181618213653564,
+      "step": 427
+    },
+    {
+      "epoch": 0.816793893129771,
+      "grad_norm": 0.7256785035133362,
+      "learning_rate": 0.00016574655994350943,
+      "loss": 0.8675359487533569,
+      "step": 428
+    },
+    {
+      "epoch": 0.8187022900763359,
+      "grad_norm": 0.6986706852912903,
+      "learning_rate": 0.00016559584722744227,
+      "loss": 0.8101359009742737,
+      "step": 429
+    },
+    {
+      "epoch": 0.8206106870229007,
+      "grad_norm": 0.684287428855896,
+      "learning_rate": 0.00016544487252973105,
+      "loss": 0.8069151639938354,
+      "step": 430
+    },
+    {
+      "epoch": 0.8225190839694656,
+      "grad_norm": 0.6052039861679077,
+      "learning_rate": 0.00016529363645335,
+      "loss": 0.751659095287323,
+      "step": 431
+    },
+    {
+      "epoch": 0.8244274809160306,
+      "grad_norm": 0.6484826803207397,
+      "learning_rate": 0.00016514213960231702,
+      "loss": 0.6957074403762817,
+      "step": 432
+    },
+    {
+      "epoch": 0.8263358778625954,
+      "grad_norm": 0.5899217128753662,
+      "learning_rate": 0.0001649903825816918,
+      "loss": 0.7061124444007874,
+      "step": 433
+    },
+    {
+      "epoch": 0.8282442748091603,
+      "grad_norm": 0.5950580835342407,
+      "learning_rate": 0.00016483836599757296,
+      "loss": 0.6376693248748779,
+      "step": 434
+    },
+    {
+      "epoch": 0.8301526717557252,
+      "grad_norm": 0.6089735627174377,
+      "learning_rate": 0.0001646860904570958,
+      "loss": 0.653281033039093,
+      "step": 435
+    },
+    {
+      "epoch": 0.8320610687022901,
+      "grad_norm": 0.7903490662574768,
+      "learning_rate": 0.0001645335565684298,
+      "loss": 0.9486740231513977,
+      "step": 436
+    },
+    {
+      "epoch": 0.833969465648855,
+      "grad_norm": 0.6438021659851074,
+      "learning_rate": 0.00016438076494077638,
+      "loss": 0.7429736852645874,
+      "step": 437
+    },
+    {
+      "epoch": 0.8358778625954199,
+      "grad_norm": 0.7143797874450684,
+      "learning_rate": 0.00016422771618436623,
+      "loss": 0.7504128813743591,
+      "step": 438
+    },
+    {
+      "epoch": 0.8377862595419847,
+      "grad_norm": 0.6937310695648193,
+      "learning_rate": 0.00016407441091045706,
+      "loss": 0.7668763995170593,
+      "step": 439
+    },
+    {
+      "epoch": 0.8396946564885496,
+      "grad_norm": 0.6838023662567139,
+      "learning_rate": 0.00016392084973133097,
+      "loss": 0.744976282119751,
+      "step": 440
+    },
+    {
+      "epoch": 0.8416030534351145,
+      "grad_norm": 0.7726618647575378,
+      "learning_rate": 0.0001637670332602923,
+      "loss": 0.775231659412384,
+      "step": 441
+    },
+    {
+      "epoch": 0.8435114503816794,
+      "grad_norm": 0.5843496322631836,
+      "learning_rate": 0.0001636129621116648,
+      "loss": 0.7458910346031189,
+      "step": 442
+    },
+    {
+      "epoch": 0.8454198473282443,
+      "grad_norm": 0.6611420512199402,
+      "learning_rate": 0.0001634586369007894,
+      "loss": 0.7827946543693542,
+      "step": 443
+    },
+    {
+      "epoch": 0.8473282442748091,
+      "grad_norm": 0.6660049557685852,
+      "learning_rate": 0.0001633040582440219,
+      "loss": 0.7378859519958496,
+      "step": 444
+    },
+    {
+      "epoch": 0.8492366412213741,
+      "grad_norm": 0.6203767657279968,
+      "learning_rate": 0.0001631492267587301,
+      "loss": 0.7874734997749329,
+      "step": 445
+    },
+    {
+      "epoch": 0.851145038167939,
+      "grad_norm": 0.5838927030563354,
+      "learning_rate": 0.0001629941430632917,
+      "loss": 0.7828131914138794,
+      "step": 446
+    },
+    {
+      "epoch": 0.8530534351145038,
+      "grad_norm": 0.7298924922943115,
+      "learning_rate": 0.00016283880777709172,
+      "loss": 0.8850565552711487,
+      "step": 447
+    },
+    {
+      "epoch": 0.8549618320610687,
+      "grad_norm": 0.5728611946105957,
+      "learning_rate": 0.00016268322152051984,
+      "loss": 0.7501472234725952,
+      "step": 448
+    },
+    {
+      "epoch": 0.8568702290076335,
+      "grad_norm": 0.5815840363502502,
+      "learning_rate": 0.00016252738491496824,
+      "loss": 0.7860700488090515,
+      "step": 449
+    },
+    {
+      "epoch": 0.8587786259541985,
+      "grad_norm": 0.6321979761123657,
+      "learning_rate": 0.00016237129858282895,
+      "loss": 0.8447892069816589,
+      "step": 450
+    },
+    {
+      "epoch": 0.8606870229007634,
+      "grad_norm": 0.5760623216629028,
+      "learning_rate": 0.0001622149631474913,
+      "loss": 0.8055865168571472,
+      "step": 451
+    },
+    {
+      "epoch": 0.8625954198473282,
+      "grad_norm": 0.7377736568450928,
+      "learning_rate": 0.00016205837923333953,
+      "loss": 0.9598544239997864,
+      "step": 452
+    },
+    {
+      "epoch": 0.8645038167938931,
+      "grad_norm": 0.6470854878425598,
+      "learning_rate": 0.00016190154746575033,
+      "loss": 0.792332112789154,
+      "step": 453
+    },
+    {
+      "epoch": 0.8664122137404581,
+      "grad_norm": 0.6200245022773743,
+      "learning_rate": 0.00016174446847109025,
+      "loss": 0.7063107490539551,
+      "step": 454
+    },
+    {
+      "epoch": 0.8683206106870229,
+      "grad_norm": 0.6435256004333496,
+      "learning_rate": 0.00016158714287671318,
+      "loss": 0.7804868817329407,
+      "step": 455
+    },
+    {
+      "epoch": 0.8702290076335878,
+      "grad_norm": 0.7654232382774353,
+      "learning_rate": 0.000161429571310958,
+      "loss": 0.7346771359443665,
+      "step": 456
+    },
+    {
+      "epoch": 0.8721374045801527,
+      "grad_norm": 0.6881013512611389,
+      "learning_rate": 0.00016127175440314596,
+      "loss": 0.7772030234336853,
+      "step": 457
+    },
+    {
+      "epoch": 0.8740458015267175,
+      "grad_norm": 0.7205625176429749,
+      "learning_rate": 0.000161113692783578,
+      "loss": 0.8162299990653992,
+      "step": 458
+    },
+    {
+      "epoch": 0.8759541984732825,
+      "grad_norm": 0.6343117952346802,
+      "learning_rate": 0.00016095538708353266,
+      "loss": 0.6849362850189209,
+      "step": 459
+    },
+    {
+      "epoch": 0.8778625954198473,
+      "grad_norm": 0.7836147546768188,
+      "learning_rate": 0.0001607968379352631,
+      "loss": 0.7396897673606873,
+      "step": 460
+    },
+    {
+      "epoch": 0.8797709923664122,
+      "grad_norm": 0.6314235925674438,
+      "learning_rate": 0.00016063804597199499,
+      "loss": 0.7888175249099731,
+      "step": 461
+    },
+    {
+      "epoch": 0.8816793893129771,
+      "grad_norm": 0.6601391434669495,
+      "learning_rate": 0.00016047901182792353,
+      "loss": 0.6901624202728271,
+      "step": 462
+    },
+    {
+      "epoch": 0.8835877862595419,
+      "grad_norm": 0.5762699842453003,
+      "learning_rate": 0.0001603197361382114,
+      "loss": 0.6644793748855591,
+      "step": 463
+    },
+    {
+      "epoch": 0.8854961832061069,
+      "grad_norm": 0.6763225197792053,
+      "learning_rate": 0.00016016021953898572,
+      "loss": 0.7466002702713013,
+      "step": 464
+    },
+    {
+      "epoch": 0.8874045801526718,
+      "grad_norm": 0.7207778692245483,
+      "learning_rate": 0.00016000046266733607,
+      "loss": 0.8403519988059998,
+      "step": 465
+    },
+    {
+      "epoch": 0.8893129770992366,
+      "grad_norm": 0.7191675305366516,
+      "learning_rate": 0.0001598404661613114,
+      "loss": 0.8248481750488281,
+      "step": 466
+    },
+    {
+      "epoch": 0.8912213740458015,
+      "grad_norm": 0.6308473348617554,
+      "learning_rate": 0.00015968023065991783,
+      "loss": 0.6770390868186951,
+      "step": 467
+    },
+    {
+      "epoch": 0.8931297709923665,
+      "grad_norm": 0.7143163681030273,
+      "learning_rate": 0.0001595197568031161,
+      "loss": 0.774874746799469,
+      "step": 468
+    },
+    {
+      "epoch": 0.8950381679389313,
+      "grad_norm": 0.7183948755264282,
+      "learning_rate": 0.0001593590452318187,
+      "loss": 0.7654241919517517,
+      "step": 469
+    },
+    {
+      "epoch": 0.8969465648854962,
+      "grad_norm": 0.6145818829536438,
+      "learning_rate": 0.00015919809658788763,
+      "loss": 0.7091991305351257,
+      "step": 470
+    },
+    {
+      "epoch": 0.898854961832061,
+      "grad_norm": 0.7230892777442932,
+      "learning_rate": 0.00015903691151413182,
+      "loss": 0.8808704614639282,
+      "step": 471
+    },
+    {
+      "epoch": 0.9007633587786259,
+      "grad_norm": 0.7757624983787537,
+      "learning_rate": 0.0001588754906543043,
+      "loss": 0.7996405959129333,
+      "step": 472
+    },
+    {
+      "epoch": 0.9026717557251909,
+      "grad_norm": 0.8004796504974365,
+      "learning_rate": 0.00015871383465309991,
+      "loss": 0.7810606360435486,
+      "step": 473
+    },
+    {
+      "epoch": 0.9045801526717557,
+      "grad_norm": 0.7248153686523438,
+      "learning_rate": 0.00015855194415615256,
+      "loss": 0.7624359130859375,
+      "step": 474
+    },
+    {
+      "epoch": 0.9064885496183206,
+      "grad_norm": 0.6396761536598206,
+      "learning_rate": 0.00015838981981003273,
+      "loss": 0.8043227195739746,
+      "step": 475
+    },
+    {
+      "epoch": 0.9083969465648855,
+      "grad_norm": 0.6938030123710632,
+      "learning_rate": 0.0001582274622622449,
+      "loss": 0.7283197045326233,
+      "step": 476
+    },
+    {
+      "epoch": 0.9103053435114504,
+      "grad_norm": 0.7794911861419678,
+      "learning_rate": 0.0001580648721612249,
+      "loss": 0.7841243147850037,
+      "step": 477
+    },
+    {
+      "epoch": 0.9122137404580153,
+      "grad_norm": 0.6454060077667236,
+      "learning_rate": 0.00015790205015633733,
+      "loss": 0.7931807041168213,
+      "step": 478
+    },
+    {
+      "epoch": 0.9141221374045801,
+      "grad_norm": 0.6589789390563965,
+      "learning_rate": 0.00015773899689787294,
+      "loss": 0.7430353164672852,
+      "step": 479
+    },
+    {
+      "epoch": 0.916030534351145,
+      "grad_norm": 0.6935415863990784,
+      "learning_rate": 0.0001575757130370462,
+      "loss": 0.8219285607337952,
+      "step": 480
+    },
+    {
+      "epoch": 0.9179389312977099,
+      "grad_norm": 0.6725053191184998,
+      "learning_rate": 0.00015741219922599253,
+      "loss": 0.7946504354476929,
+      "step": 481
+    },
+    {
+      "epoch": 0.9198473282442748,
+      "grad_norm": 0.6859068274497986,
+      "learning_rate": 0.00015724845611776565,
+      "loss": 0.8827815055847168,
+      "step": 482
+    },
+    {
+      "epoch": 0.9217557251908397,
+      "grad_norm": 0.6601479053497314,
+      "learning_rate": 0.00015708448436633522,
+      "loss": 0.8341709971427917,
+      "step": 483
+    },
+    {
+      "epoch": 0.9236641221374046,
+      "grad_norm": 0.6317791938781738,
+      "learning_rate": 0.000156920284626584,
+      "loss": 0.6237555742263794,
+      "step": 484
+    },
+    {
+      "epoch": 0.9255725190839694,
+      "grad_norm": 0.6455380320549011,
+      "learning_rate": 0.0001567558575543052,
+      "loss": 0.8918779492378235,
+      "step": 485
+    },
+    {
+      "epoch": 0.9274809160305344,
+      "grad_norm": 0.7169380187988281,
+      "learning_rate": 0.0001565912038062001,
+      "loss": 0.7054572105407715,
+      "step": 486
+    },
+    {
+      "epoch": 0.9293893129770993,
+      "grad_norm": 0.6330932378768921,
+      "learning_rate": 0.00015642632403987535,
+      "loss": 0.8734369874000549,
+      "step": 487
+    },
+    {
+      "epoch": 0.9312977099236641,
+      "grad_norm": 0.6074062585830688,
+      "learning_rate": 0.0001562612189138401,
+      "loss": 0.6660835146903992,
+      "step": 488
+    },
+    {
+      "epoch": 0.933206106870229,
+      "grad_norm": 0.6423531174659729,
+      "learning_rate": 0.00015609588908750376,
+      "loss": 0.769476592540741,
+      "step": 489
+    },
+    {
+      "epoch": 0.9351145038167938,
+      "grad_norm": 0.646690309047699,
+      "learning_rate": 0.00015593033522117298,
+      "loss": 0.7003253698348999,
+      "step": 490
+    },
+    {
+      "epoch": 0.9370229007633588,
+      "grad_norm": 0.6849831938743591,
+      "learning_rate": 0.00015576455797604928,
+      "loss": 0.7950581908226013,
+      "step": 491
+    },
+    {
+      "epoch": 0.9389312977099237,
+      "grad_norm": 0.7034041285514832,
+      "learning_rate": 0.0001555985580142264,
+      "loss": 0.7060476541519165,
+      "step": 492
+    },
+    {
+      "epoch": 0.9408396946564885,
+      "grad_norm": 0.7383472323417664,
+      "learning_rate": 0.00015543233599868742,
+      "loss": 0.7794291973114014,
+      "step": 493
+    },
+    {
+      "epoch": 0.9427480916030534,
+      "grad_norm": 0.6245297193527222,
+      "learning_rate": 0.00015526589259330246,
+      "loss": 0.6570280194282532,
+      "step": 494
+    },
+    {
+      "epoch": 0.9446564885496184,
+      "grad_norm": 0.6037810444831848,
+      "learning_rate": 0.00015509922846282568,
+      "loss": 0.775126576423645,
+      "step": 495
+    },
+    {
+      "epoch": 0.9465648854961832,
+      "grad_norm": 0.68060302734375,
+      "learning_rate": 0.00015493234427289295,
+      "loss": 0.8163896203041077,
+      "step": 496
+    },
+    {
+      "epoch": 0.9484732824427481,
+      "grad_norm": 0.6397544145584106,
+      "learning_rate": 0.00015476524069001882,
+      "loss": 0.7781999111175537,
+      "step": 497
+    },
+    {
+      "epoch": 0.950381679389313,
+      "grad_norm": 0.6421068906784058,
+      "learning_rate": 0.00015459791838159424,
+      "loss": 0.6983038783073425,
+      "step": 498
+    },
+    {
+      "epoch": 0.9522900763358778,
+      "grad_norm": 0.6688042879104614,
+      "learning_rate": 0.0001544303780158837,
+      "loss": 0.654350221157074,
+      "step": 499
+    },
+    {
+      "epoch": 0.9541984732824428,
+      "grad_norm": 0.5361288189888,
+      "learning_rate": 0.00015426262026202255,
+      "loss": 0.7276004552841187,
+      "step": 500
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 1572,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4505964380160000.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:63000f8630f600463bb3de9bc3cdf2f4886e159693a607e12dc0c1d4bd91e3ae
+size 5201