Q Project docs

Q Project is an independent lab training language and agent models from scratch. These docs cover how to run our open models today — and what is coming next.

Everything published under the q-project organization is open-weight and Apache 2.0. Our current release is Q-50M-Base, a compact base language model you can run on a laptop.

Base, not chat. Q-50M-Base continues text; it does not follow instructions out of the box. See Fine-tuning to teach it a task.

Quickstart

Q models use the standard Hugging Face Transformers API. Install the dependencies:

shell

# Python 3.9+
pip install transformers torch

Load and generate

Pull the checkpoint from the Hub and continue a prompt:

quickstart.py

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "q-project/Q-50M-Base"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16)

prompt = "The theory of relativity states that"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=64, do_sample=True, temperature=0.8)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Keep prompts within the 2,048-token context window. Lower the temperature for steadier text; raise it for more variety.

Model reference

Q-50M-Base is a decoder-only causal language model pretrained from scratch on ~5B tokens of FineWeb-Edu.

Field	Value
parameters	50,878,208 (50.9M)
hidden_size	512
num_layers	10
attention	Grouped-query — 8 query heads, 2 KV heads
head_dim	64, with per-head QK-Norm
mlp	SwiGLU / SiLU, intermediate 1,792
norm	RMSNorm (eps 1e-5)
positional	RoPE, all layers
vocab_size	32,768 (byte-level BPE)
context	2,048 tokens
val_perplexity	24.47
license	Apache 2.0

The full training recipe — optimizer, schedule, and evaluation — lives on the model card.

Fine-tuning

Because it is a base model, Q-50M-Base is a good starting point for supervised fine-tuning on your own task — classification, domain text, or a simple instruction format.

Format your data as plain text sequences under 2,048 tokens.
Start from q-project/Q-50M-Base and train with a small learning rate (2e-5 to 5e-5).
The model is small enough to fine-tune on a single consumer GPU.

Sharing back. If you release a fine-tune, we'd love to see it — tag q-project or open a thread on the model card.

Roadmap

Where Q Project is heading. Dates are intentions, not promises.

Q-50M-Base — shipped. Open weights, Apache 2.0.
Q · Next — in training. A larger model built for perception and agentic tool use. Details under wraps until release.
Hosted API — managed endpoints so you can call Q models without your own hardware.
Private models — closed models and custom training for teams.

Hosted API

A managed API for Q models is on the way. It will offer OpenAI-compatible endpoints, so most existing clients work with only a base-URL change.

Coming soon. Want early access? Join the waitlist and we'll reach out when it opens.

Back to topIntroduction NextAccess & pricing