Unique Research · Interview · November 23, 2023 · Guest: Wang Tiezhen (王铁震), Engineer, Hugging Face · From AWS × Unique Capital “AI Globalization Accelerator” Shanghai offline
Series context: Babel CEO Zhang Hailong on complex LLM apps; China vs. U.S. AI commercialization ecosystems; Opportunities and challenges for AI apps going global.
Hugging Face is among the world’s leading open LLM communities and most influential AI companies — valuation above $4.5B, the highest-valued AI unicorn outside OpenAI — with investors including Google, Amazon, NVIDIA, Intel, and Salesforce. Wang walks the full pipeline through a code-generation model case.
Hugging Face hosts models, datasets, and large demos. Model count more than doubled last year. Code generation is a strong commercial example: name a function, add a docstring, and the model implements it — a large productivity lift. Studies cite ~20% increase in engineer creative value after adopting code gen. Product design stresses user-friendliness so enterprises will pay; many engineers report coding becomes enjoyable again. As models strengthen, users may drive tool evolution — describing needs clearly and generating maintainable code on demand rather than fearing duplication.
Wang’s advice: code gen is worth investing in now and later — but don’t only chase market trends; think as creators about how the tech helps your own team.
Open vs. closed for code
First soul-searching: open or closed source? For software companies, code is sensitive — security and competitors matter. Open models can run in controlled environments, easing those worries, and can be adjusted for company-specific needs. Closed APIs often cannot meet special process requirements. For solving company problems, open models offer more flexible control over logic, internal process, and customization.
Four-stage pipeline
Dataset → Training → Evaluation → Deployment.
Dataset. Built a public dataset of 6.4 TB, much generated on-platform. Even generated code can “infect” via interaction — quality matters. Covers 358 programming languages plus data-check tools and exit mechanisms so users can opt out of training use. Started from 220 million GitHub repo names → ~102 TB raw → filter by suffix to 69 TB base → remove unlicensed / low-value → 6.4 TB → deduplicate (critical: duplicate accounts wreck performance) → 2.9 TB of usable data. Only a fraction of cleaned data may actually train. Extra work: trained StarEncoder to label and replace names, emails, passports, IPs — beating regex for PII. Opt-out: request removal and the next iteration drops related code.
Tokenization & training. Tokenizers convert text to machine-readable tokens; much of the code path is Rust (far faster than pure Python). For code gen they chose a latency-friendly, capable 15B code-optimized model with Multi-Query Attention for efficiency/cost, long context up to 8,192 tokens, and Fill-in-the-Middle (so mid-file edits see both before and after the cursor — unlike next-token-only training). Training: 512 GPUs × ~24 days — expensive but worth it for base capability. Domain adaptation need not match that cost: PEFT (Parameter-Efficient Fine-Tuning) can train a defer/adapter at ~0.1%–1% cost that understands an internal codebase.
Evaluation. Given inputs such as function definitions, generated completions must pass unit tests — at least one successful pass counts.
Deployment. Hugging Face Inference Endpoints can deploy a selected model onto AWS with minimal ops. For self-hosted production, Text Generation Inference (TGI) offers Tensor Parallelism (slice tensors across GPUs), Token streaming (lower time-to-first-token), Quantization (e.g. bitsandbytes), and Optimizations such as Flash Attention.
Closing: training LLMs is not “just have GPUs” — data work and infra are required before you have a product.
— END —