Sugato Ray sugatoray

🎯

Focusing

I am a Physicist turned Data Scientist + ML Practitioner. Research Interests: Data Science, ML, DL, Statistics, Math, Computing, LLMs.

sugatoray / train.py

Created February 7, 2025 14:01 — forked from ddh0/train.py

Janky pretraining script for small llama models using HF fineweb - modify according to your needs

	import os
	import torch
	import psutil
	import datasets
	import glob

	from transformers import (
	AutoTokenizer, LlamaConfig, LlamaForCausalLM, Trainer, TrainingArguments,
	DataCollatorForLanguageModeling
	)

sugatoray / sft_data_mlx.py

Created February 2, 2025 14:13 — forked from davidberenstein1957/sft_data_mlx.py

	# /// script
	# requires-python = ">=3.11,<3.12"
	# dependencies = [
	# "distilabel[mlx]",
	# ]
	# ///
	from distilabel.models import MlxLLM
	from distilabel.pipeline import InstructionResponsePipeline

	llm = MlxLLM(

sugatoray / synthetic_data_deepseekr1_qwen_distill.py

Created January 27, 2025 22:48 — forked from davidberenstein1957/synthetic_data_deepseekr1_qwen_distill.py

	# /// script
	# requires-python = ">=3.11,<3.12"
	# dependencies = [
	# "distilabel[hf-transformers, hf-inference-endpoints]",
	# ]
	# ///
	from distilabel.models import InferenceEndpointsLLM
	from distilabel.pipeline import InstructionResponsePipeline

	repo_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"

sugatoray / export_locally.py

Created October 15, 2024 14:32 — forked from tomaarsen/export_locally.py

Export Sentence Transformer models to ONNX (+ optimization, quantization) & OpenVINO

	# requires sentence_transformers>=3.2.0
	from sentence_transformers import SentenceTransformer, export_optimized_onnx_model, export_dynamic_quantized_onnx_model

	# The model to export to ONNX (+ optimize, quantize), OpenVINO
	model_id = "mixedbread-ai/mxbai-embed-large-v1"
	# Where to save the exported models locally
	output_dir = model_id.replace("/", "-")

	onnx_model = SentenceTransformer(model_id, backend="onnx", model_kwargs={"export": True})
	onnx_model.save_pretrained(output_dir)

sugatoray / prompt.txt

Created October 6, 2024 14:53 — forked from philschmid/prompt.txt

	Begin by enclosing all thoughts within <thinking> tags, exploring multiple angles and approaches.
	Break down the solution into clear steps within <step> tags. Start with a 20-step budget, requesting more for complex problems if needed.
	Use <count> tags after each step to show the remaining budget. Stop when reaching 0.
	Continuously adjust your reasoning based on intermediate results and reflections, adapting your strategy as you progress.
	Regularly evaluate progress using <reflection> tags. Be critical and honest about your reasoning process.
	Assign a quality score between 0.0 and 1.0 using <reward> tags after each reflection. Use this to guide your approach:

	0.8+: Continue current approach
	0.5-0.7: Consider minor adjustments
	Below 0.5: Seriously consider backtracking and trying a different approach

sugatoray / pipeline_parallel.py

Created October 2, 2024 17:20 — forked from 3outeille/pipeline_parallel.py

Self contained example of how pipeline parallel works (AFAB and 1F1B) in 200 LOC

	#VERBOSE=0 torchrun --nproc_per_node 3 self_contained_pp_LOC.py
	import os, random, numpy as np, torch, torch.nn as nn, torch.distributed as dist, torch.nn.functional as F
	from torch.optim import AdamW
	from torch.utils.data import DataLoader, DistributedSampler
	from datasets import load_dataset
	from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer

	STEP, local_rank, world_size, verbose = 0, int(os.environ["LOCAL_RANK"]), int(os.environ["WORLD_SIZE"]), os.environ.get("VERBOSE", "0") == "1"

	def set_all_seed(seed):

sugatoray / l3min.py

Created August 16, 2024 05:18 — forked from awni/l3min.py

	"""
	A minimal, fast example generating text with Llama 3.1 in MLX.

	To run, install the requirements:

	pip install -U mlx transformers fire

	Then generate text with:

	python l3min.py "How tall is K2?"

sugatoray / mlx_lm_openai.md

Created August 16, 2024 05:14 — forked from awni/mlx_lm_openai.md

MLX LM with the OpenAI Python Package

Install MLX LM and openai:

pip install mlx-lm openai

sugatoray / mlx_lm_openai.md

Created August 16, 2024 05:13 — forked from awni/mlx_lm_openai.md

Install MLX LM and openai:

pip install mlx-lm openai

sugatoray / mflux_steps.md

Created August 16, 2024 03:37 — forked from awni/mflux_steps.md

This can take a little while:

# Make sure you have git-lfs installed (https://git-lfs.com)
git lfs install

git clone https://huggingface.co/black-forest-labs/FLUX.1-schnell