---
title: "Hugging Face: Open-Source ML — From Pretrained Models to Production LLMs"
author: "Unique Research"
sourcePublication: "Unique Research (FFCAP English library)"
originalPublishedAt: "2023-11-23T18:00:00+08:00"
canonical: "https://ffcap.cn/en/research/20231123a07cxz00"
source: "https://ffcap.cn/en/research/20231123a07cxz00"
language: "en"
cnSourceUrl: "https://view.inews.qq.com/a/20231123A07CXZ00"
---

# Hugging Face: Open-Source ML — From Pretrained Models to Production LLMs

_Unique Research · Interview · November 23, 2023 · Guest: Wang Tiezhen (王铁震), Engineer, Hugging Face · From AWS × Unique Capital “AI Globalization Accelerator” Shanghai offline_

Series context: Babel CEO Zhang Hailong on complex LLM apps; China vs. U.S. AI commercialization ecosystems; Opportunities and challenges for AI apps going global.

Hugging Face is among the world’s leading open LLM communities and most influential AI companies — valuation above **$4.5B**, the highest-valued AI unicorn outside OpenAI — with investors including Google, Amazon, NVIDIA, Intel, and Salesforce. Wang walks the full pipeline through a code-generation model case.

Hugging Face hosts models, datasets, and large demos. Model count more than doubled last year. Code generation is a strong commercial example: name a function, add a docstring, and the model implements it — a large productivity lift. Studies cite ~**20%** increase in engineer creative value after adopting code gen. Product design stresses user-friendliness so enterprises will pay; many engineers report coding becomes enjoyable again. As models strengthen, users may drive tool evolution — describing needs clearly and generating maintainable code on demand rather than fearing duplication.

Wang’s advice: code gen is worth investing in now and later — but don’t only chase market trends; think as creators about how the tech helps your own team.

## Open vs. closed for code

First soul-searching: open or closed source? For software companies, code is sensitive — security and competitors matter. Open models can run in controlled environments, easing those worries, and can be adjusted for company-specific needs. Closed APIs often cannot meet special process requirements. For solving company problems, open models offer more flexible control over logic, internal process, and customization.

## Four-stage pipeline

Dataset → Training → Evaluation → Deployment.

**Dataset.** Built a public dataset of **6.4 TB**, much generated on-platform. Even generated code can “infect” via interaction — quality matters. Covers **358** programming languages plus data-check tools and exit mechanisms so users can opt out of training use. Started from **220 million** GitHub repo names → ~**102 TB** raw → filter by suffix to **69 TB** base → remove unlicensed / low-value → **6.4 TB** → deduplicate (critical: duplicate accounts wreck performance) → **2.9 TB** of usable data. Only a fraction of cleaned data may actually train. Extra work: trained **StarEncoder** to label and replace names, emails, passports, IPs — beating regex for PII. Opt-out: request removal and the next iteration drops related code.

**Tokenization & training.** Tokenizers convert text to machine-readable tokens; much of the code path is Rust (far faster than pure Python). For code gen they chose a latency-friendly, capable **15B** code-optimized model with Multi-Query Attention for efficiency/cost, long context up to **8,192** tokens, and Fill-in-the-Middle (so mid-file edits see both before and after the cursor — unlike next-token-only training). Training: **512 GPUs × ~24 days** — expensive but worth it for base capability. Domain adaptation need not match that cost: PEFT (Parameter-Efficient Fine-Tuning) can train a defer/adapter at ~0.1%–1% cost that understands an internal codebase.

**Evaluation.** Given inputs such as function definitions, generated completions must pass unit tests — at least one successful pass counts.

**Deployment.** Hugging Face Inference Endpoints can deploy a selected model onto AWS with minimal ops. For self-hosted production, **Text Generation Inference (TGI)** offers Tensor Parallelism (slice tensors across GPUs), Token streaming (lower time-to-first-token), Quantization (e.g. bitsandbytes), and Optimizations such as Flash Attention.

Closing: training LLMs is not “just have GPUs” — data work and infra are required before you have a product.

— END —

---

Editorial English edition published on UniqueCapital: https://ffcap.cn/en/research/20231123a07cxz00
On-site reading page: https://ffcap.cn/en/research/20231123a07cxz00