Unique Research · Interview / Roundtable · November 20, 2023 · Guests: Zhang Weixian (张伟贤), Founder & CEO, Lifewood; Qin Xuye (秦续业), CEO, Xorbits (未来速度); Zhang Yingfeng (张颖峰), CEO, InfiniFlow (英飞流) · From AWS × Unique Capital “AI Globalization Accelerator” Shanghai offline
Q1: How should enterprises land generative AI / LLMs commercially — contrarian takes?
Zhang Yingfeng (InfiniFlow): Two anti-consensus views. (1) Obsessing over context token counts (including well-funded / open projects like Stream LLM) is overrated vs. hallucination — the bigger ToB pain. Tokens aren’t the top bottleneck because RAG can bound context; even with accurate RAG results of only a few thousand tokens (<10k), many models still hallucinate from that given context and are hard to control. (2) Vector DBs as “LLM memory” are incomplete: in ToB, vectors are one recall path — you need at least dense (vector) and sparse (classic full-text) recall; vector weight may even be lower than full-text today given precision gaps.
Qin Xuye (Xorbits): Echoes Gates — short-term overestimate, long-term underestimate. Near term, uncontrollability and hallucination dominate; without foundation-model lifts, tricks (including data) only go so far. Hard parts of RAG: extracting information from models/data, then summarizing it well; techniques like “lost in the middle” are still required.
Zhang Weixian (Lifewood): Prefer interpretability over raw complexity for trust and compliance; data diversity/quality over mere volume for accuracy and generality; design user–AI interaction for acceptance.
Q2: How has genAI changed data labeling?
Zhang Weixian: Synthetic data instances as powerful supplements when real samples are scarce/expensive; genAI for QC — auto-detect/fix label errors at scale; demand shifts toward verifying synthetic data realism/usefulness and supplying high-quality training data for genAI; new businesses: custom datasets and genAI model tuning/debug. Ethics/compliance matter when synthetic data is hard to distinguish. Pre-genAI AI-assisted labeling existed; genAI still expands automation of industrial data production, quality, efficiency, and new lines.
Q3: Why does data quality so strongly affect LLM training?
Analogy: data = ingredients, algorithms = recipes, compute = kitchen gear, model = dish design. Without ingredients, best gear and skill still yield nothing. Quality, diversity, timeliness, and domain specificity all shape outcomes — data is foundation, innovation, and efficiency.
Q4: Xorbits open-source practice?
Qin: Democratize open LLMs for developers — one-command start, hide model differences. China pain: Hugging Face access/review friction for downloads — partnering domestic platforms for local download paths. Performance/cost: distributed systems background (ex-Alibaba) plus optimizations for inference price/performance; local training vs. their platform claims 15–25× performance. New: speculative sampling — put a smaller aligned model (e.g. 7B) in front of a large one (e.g. 270B) to draft tokens the large model accepts/rejects, cutting large-model inference steps ~2×.
Q5: Database/data value points when building LLM apps?
Zhang Yingfeng: Recent experience: RAG often lets you skip fine-tuning first — not that FT is useless, but RAG gets ToB to a passing ~60 before FT lifts 60→70. Reaching 60 needs more than wiring LangChain/LlamaIndex: chunking sensitivity (too fine loses global semantics; too coarse loses precise local semantics); PDF parsing for Chinese enterprise docs (scrambled blocks in papers/financial reports); tables need auto extract + correct row/col + cell meaning or RAG fails. Enterprises often over-idealize “omniscient accurate Q&A”; vectors are fuzzy semantic recall only. Precision needs full-text + industry dictionaries (finance/legal/medical) — hybrid ranking may weight full-text higher. Internal codegen: Code Llama alone isn’t enough — RAG over internal docs/code, plus graph DB mapping requirements to implementations. Finance: keep existing knowledge graphs in graph DBs for risk control, combine with vectors; models emit multi-path queries (vector + graph + full-text) and unified query statements for permissioned multi-table join/agg — not a single vector.
Q6: Key factors for LLM/AI product commercialization?
Zhang Yingfeng: “Digital transformation” no longer sells — talk intelligentization. Humans still in the loop to tune retrieval/chunking from feedback for good RAG delivery. Sell end-to-end so clients use LLMs for data management and faster decisions. Emphasize: customized solutions; data truthfulness/accuracy (quantified results, not empty promises); cost efficiency; human+automation balance.
Q7: Open source impact — must AI products go global?
Zhang Yingfeng: Open source is global — host on GitHub. High ROI GTM especially for AI/data-infra startups via developer standards. Profit may sit in (not necessarily open) cloud/SaaS — and SaaS implies global.
Qin: Enterprise AI globalization can be politically/market-hard; consumer caps may limit pure global. Global may be inevitable long-term but execute with company type, positioning, and users in mind — pure PLG hard for B; middleware/online inference for developers has lower cost but fierce U.S. competition and price wars — differentiate or choose markets carefully.
Q8: Lifewood advice on truly going global?
Zhang Weixian: Historic shift even seasoned managers find unprecedented. Chinese founders carry global China-dream ambition — hard. Vision leadership can stall in 2–3 years despite strong domestic share (e.g. Beijing) as global competition intensifies. Globalization is a delicate balance; many firms fragment. Singapore-style neutrality offers a lens; LLM apps sit in a neither-fully-neutral nor-overtly-aligned bind. Be cautious on shiny investment opportunities; set clear short/mid/long goals; open source as pressure relief; survive year 3, grow to 5, aim for 7. Globalization is cultural/strategic adaptation — respect each market’s uniqueness.
Q9: Future of AI data services — Lifewood’s long-term value?
Zhang Weixian: Automation/intelligence will mature (DL/NLP) for auto/semi-auto labeling — higher efficiency, fewer human errors — but complex/high-precision work elevates human value. Combat data bias via diverse collection, fair labeling, and anti-bias training for annotators. Demand for volume and quality rises with AI adoption. Privacy/compliance (GDPR, CCPA) raise the bar. Stay customer-centric with personalized labeling solutions.
Audience Q1: InfiniFlow and graph neural nets?
Zhang Yingfeng: Not GNNs — graph databases. Internal codegen: requirement docs as hierarchical graphs map flexibly in graph DBs while code uses vector semantic recall. Finance: don’t abandon expensive existing knowledge graphs when adding LLMs — store KG in graph DB for precise retrieval + vectors for semantics.
Audience Q2: Vector–graph relation? Delivery cycles and profitability?
Zhang Yingfeng: Graph + vector = precise retrieval + semantic recall for requirement mapping — but custom and non-universal (multiple vectors per function mapping). Critical for project success; ToB in China still needs person-day custom delivery unlike the U.S. Push technical generalization (industry-standard RAG, AI-native DB) for scalable profit, still plus custom workload fees for specific B clients.
— END —