Unique Research · Interview / Talk · November 13, 2024
At Unique Capital’s AIGC summit, Google Cloud senior ML architect Dr. Shun Wang shared “A New Journey on Cloud: Latest Generative AI Progress on Google Cloud.” Edited highlights:
Vertex AI demand surge
Vertex AI for train/deploy ML and custom LLMs. Past year: Gemini API call volume on Vertex ×36; Imagen 3 drove Imagen API volume ×5. Among Google Cloud Top 10 customers, two are Chinese (notably outbound).
Snapchat × Gemini: smart assistant, translation/generation, summarization, multimodal understanding.
PUMA × Imagen: customized product imagery for local markets — India CTR +10%.
Gemini family & 002 upgrades
Gemini (DeepMind, Dec 6, 2023): text/image/audio/video/code. Gemini 1.5 → 1.5 Pro (Aug) with up to 2M-token context; 1.5 Flash for speed/low latency; then 1.5-Pro-002 & 1.5-Flash-002:
- Cost: Pro-002 price down >50% for prompts with <128K in/out tokens.
- Quality: ~+20% on MATH/HiddenMath; +2–7% vision & code.
- Rate limits: Flash 1000→2000 RPM; Pro 360→1000 RPM.
- Output ~2× faster, latency ~3× lower; responses 5–20% shorter; fewer refusals.
- 2M context multimodal (e.g. 1000-page PDF / long video).
- Default safety filters no longer auto-applied — developer-configurable.
Core capabilities
- YouTube analysis: URL → multimodal insights (Studio UI or API; Batch for volume/cost).
- Multimodal streaming API: stream audio/video for conversational UX (Flash 002; VoiceLM responses).
- Model Garden: Flash & Pro 002 quality; 2M context differentiation; YouTube API; streaming; dynamic shared quota (no more quota-increase tickets); Chirp v2 streaming speech + language detect/translate.
Tuning & customization paths
- Prompt Optimizer — Monte Carlo / AlphaGo-like seed search for best prompts on a test set.
- Supervised fine-tuning — Gemini 1.5 Pro/Flash; hundreds of labeled samples; text/image/audio; pay for train compute.
- Distillation — transfer large → smaller (incl. Gemma 2 → 2B/9B/27B); no huge labeled set; self-host for cost/latency.
- Model Evaluation — GenAI Eval Service across Gemini versions and third-party models on your test set.
- Controlled Generation (1.5) — structured/machine-readable outputs (enums, lists, nullable/required fields).
Imagen 3
Multiple aspect ratios (3:4, 4:3, 9:16… not only 1:1); DeepMind SynthID-style watermarking (CN: CCID) for commercial provenance; inpainting add/remove; background edit via prompt; few-shot customization from 1–few product images for brand consistency. Multimodal tools also support function calling (e.g. Golden Gate photo → location → weather API).
— END —