vi
new
0
Avatar
Truong Phung @truong396
800 34 114
Đã đăng vào Thứ Ba, 8:47 SA 18 phút đọc
42

🐹 Golang for AI Developers 🤖 — From 0 to Pro ⚡

One file, one path: from package main to shipping a concurrent, observable Go service that fronts your models and never falls over.

Every example is drawn from what AI engineers actually build in Go — streaming proxies, tool dispatchers, rate limiters, worker pools, context-cancelled model calls. No foo/bar filler.

Companion reads: 🐍 Python for AI Developers (the sibling to this guide), 📘 The Complete Guide to LLMs and AI Agents 🤖 to understand modern AI deeply, ⚠️ Common Issues 🪲 with LLMs & AI Agents — and How to Fix Them 🛠️, 🏗️ Building High-Quality AI Agents 🤖 for the agent architecture on top of this foundation, 🔄 The Agentic Loop Guide for the control loop itself, 🏢 Enterprise-Ready AI Agents, and 🛠️ The Senior Software Engineer Playbook 📖.

📖 How to read this guide

You are… Start at Skip
New to Go Part 1 → read straight through Parts 12–13 on first pass
Coming from Python Part 1 (the phrasebook), then Part 5 and Part 6 —
Coming from Java/C# Part 4, Part 5 — inheritance and exceptions are gone Part 2 (skim)
Building AI services Part 6, Part 7, Part 9 —
Reviewing code Part 14, Part 15 everything else

Convention: // ✅ = do this, // ❌ = don't. Snippets target Go 1.22+, with newer-version wins called out inline.

📋 Table of Contents

1. 🧠 The Go Mental Model

1.1 What Go optimizes for

Go was designed for large teams maintaining network services over years. Every trade-off follows from that:

Go chose Instead of Consequence for you
A tiny spec (25 keywords) Rich features You can read any Go file after a week
Compile to one static binary Runtime + deps FROM scratch images, 10 ms cold start
Explicit errors as values Exceptions Failure paths are visible in the code
Composition + interfaces Inheritance No class hierarchies to reverse-engineer
Goroutines + channels Callbacks / async colouring Blocking code that scales to 100k connections
One formatter, one toolchain Ecosystem choice Zero config debates; go test, go fmt, pprof are built in

Go is boring on purpose. The payoff is that a service written by someone who left two years ago still compiles, still reads clearly, and still runs.

1.2 Compiled and statically typed — what that buys you

[your .go files] → [compiler: types, escape analysis, inlining] → [one native binary]
                                                                   ↑ includes the runtime
                                                                     (scheduler + GC)
  • Errors caught at compile time: type mismatches, unused variables, unused imports, missing returns. A whole class of Python 3 a.m. incidents simply cannot happen.
  • No interpreter, no venv, no site-packages at runtime. Deploy is COPY binary /.
  • Predictable performance: no JIT warmup, no GIL, real parallelism across cores.

The cost: more ceremony up front, no REPL, and a smaller ML ecosystem.

1.3 Go vs Python — pick per service, not per company

Dimension Go Python
Execution Native binary + embedded runtime Bytecode on the CPython VM
Typing Static, enforced by the compiler Dynamic; static only via mypy in CI
Parallelism Real: goroutines across all cores GIL-limited; processes or C extensions
Concurrency cost ~2 KB per goroutine ~KB per coroutine, ~MB per thread
p99 latency Stable (GC pauses < 1 ms) Noisier
Deploy artifact 15–40 MB static binary Interpreter + wheels + lockfile
Startup ~5 ms 100–500 ms (imports)
ML/AI libraries Thin (inference clients, ONNX, tokenizers) Everything
Best at API gateways, streaming proxies, orchestrators, high-fan-out workers Model training, data science, ML inference glue

The production shape that wins — and the one in this repo's CLAUDE.md — is both: Go as the BFF that owns HTTP, auth, tenancy, streaming and fan-out; Python as the ML service it calls for heavy computation. Use Go where request volume and connection count live; use Python where the models live.

1.4 A Python → Go phrasebook

Python Go Note
x = 5 x := 5 := declares + infers, inside functions only
list[int] []int Slice — dynamic array
dict[str, int] map[string]int Iteration order is randomized
tuple struct, or multiple return values No tuple type
None nil (pointers, slices, maps, interfaces, funcs, chans) Value types have zero values instead
Optional[T] *T, or (T, bool), or (T, error) Pointers are the "maybe" of Go
raise ValueError(...) return fmt.Errorf("...: %w", err) Errors are returned, not thrown
try/except if err != nil { … } Explicit at every call
with open(...) as f: f, err := os.Open(...); defer f.Close() defer is the context manager
@decorator Higher-order function / middleware Wrap the function or the handler
class A: def m(self) type A struct{} + func (a A) M() Methods live outside the type
Protocol (structural) interface Go interfaces are structural too — no implements
async def / await just call it, in a go routine No function colouring
asyncio.gather errgroup.Group Bounded with SetLimit
asyncio.Semaphore(8) buffered channel or SetLimit(8)
f"{x:.2f}" fmt.Sprintf("%.2f", x)
pytest go test ./... Testing is in the stdlib
venv + pyproject.toml go.mod Modules, no activation

1.5 Hello, service

package main

import (
	"fmt"
	"log/slog"
	"net/http"
	"os"
)

func main() {
	logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
	mux := http.NewServeMux()
	mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintln(w, "ok")
	})
	logger.Info("listening", "addr", ":8080")
	if err := http.ListenAndServe(":8080", mux); err != nil {
		logger.Error("server failed", "err", err)
		os.Exit(1)
	}
}

Three things a Python developer should notice: no framework, no decorators, and errors returned rather than raised. ("GET /healthz" method-and-pattern routing is Go 1.22+.)

🎯 Actionable rules

  1. Choose Go for the request path and the fan-out; keep Python where the models are.
  2. Let the compiler carry the weight you spend mypy effort on in Python.
  3. Learn error, interface, defer, and context — everything else is syntax.

2. 🧱 Core Types & Syntax

2.1 Declarations and zero values

var name string          // "" — declared variables are ALWAYS initialized
var count int            // 0
var ratio float64        // 0
var ok bool              // false
var tools []string       // nil (usable: len 0, append works)
var index map[string]int // nil (readable, but WRITING panics)
var client *http.Client  // nil

model := "claude-opus-5"           // := infers the type; functions only
timeout, retries := 30, 3          // multiple assignment
_, err := doThing()                // _ discards a value you must accept

Zero values are Go's answer to None. There is no uninitialized memory, so a struct is useful the moment it exists. Design your types so the zero value works (sync.Mutex, bytes.Buffer, and http.Client all do).

⚠️ var m map[string]int is nil: reads return the zero value, writes panic. Always m := make(map[string]int) or m := map[string]int{}.

2.2 The type set

int, int8/16/32/64, uint…    // int is 64-bit on modern platforms; use it by default
float32, float64             // float64 unless you're storing millions of embeddings
string                       // immutable, UTF-8 bytes
byte  = uint8                // a raw byte
rune  = int32                // one Unicode code point
bool
[]T, map[K]V, chan T, *T, func(...) ..., interface{ … }, struct{ … }

Go has no implicit conversion, not even int → int64:

var i int = 42
var f float64 = float64(i)          // explicit, always
var u uint8 = uint8(300)            // ⚠️ silently wraps to 44 — check ranges yourself
n, err := strconv.Atoi("42")        // string → int (returns an error!)
s := strconv.Itoa(42)               // int → string
f, err := strconv.ParseFloat("0.7", 64)
b, err := strconv.ParseBool("true")

⚠️ string(65) gives "A", not "65" — it converts a code point. Use strconv. (go vet flags this.)

2.3 Constants and iota

const MaxHistoryTurns = 20                    // untyped: adapts to context
const ToolTimeout = 30 * time.Second          // typed by inference

type Role string
const (
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleSystem    Role = "system"
)

type Status int
const (
	StatusOK Status = iota   // 0 — iota counts from 0 within a const block
	StatusRetry              // 1
	StatusFailed             // 2
)

func (s Status) String() string {              // makes it print nicely everywhere
	switch s {
	case StatusOK:     return "ok"
	case StatusRetry:  return "retry"
	case StatusFailed: return "failed"
	default:           return fmt.Sprintf("Status(%d)", int(s))
	}
}

A named string type (type Role string) is Go's enum: the compiler rejects a raw "usr" typo where a Role is expected, while JSON marshalling still just works.

2.4 Strings, bytes, runes

Strings are immutable byte slices holding UTF-8. Indexing gives bytes; ranging gives runes.

s := "café"
len(s)                       // 5 — BYTES, not characters
s[0]                         // 99 (byte 'c')
for i, r := range s {        // i = byte offset, r = rune
	fmt.Printf("%d:%c ", i, r)   // 0:c 1:a 2:f 3:é
}
utf8.RuneCountInString(s)    // 4 — actual character count
[]rune(s)[3]                 // 'é' — index by character (allocates)
[]byte(s)                    // copy to a mutable byte slice

The strings package covers what Python puts on str:

strings.TrimSpace("  hi \n")            // "hi"
strings.ToLower("Calculate 2+2")
strings.Split("a,b,c", ",")             // []string{"a","b","c"}
strings.SplitN("calculate 10*5", "calculate", 2)[1]   // " 10*5"  (maxsplit)
strings.Join([]string{"a", "b"}, ", ")  // "a, b"
strings.HasPrefix(name, "tool:")        // also HasSuffix, Contains, EqualFold
strings.ReplaceAll(s, "ok", "done")
strings.Fields("  a  b ")               // ["a","b"] — split on any whitespace
strings.TrimPrefix(path, "docs/")       // prefix-safe (not Trim, which is a char set)
strings.Cut("key=value", "=")           // "key", "value", true — the modern splitter

Building strings: += in a loop is O(n²) and allocates every time. Use a builder:

var b strings.Builder
b.Grow(len(history) * 64)                  // one allocation if you can estimate
for _, m := range history {
	fmt.Fprintf(&b, "%s: %s\n", m.Role, m.Content)
}
prompt := b.String()

2.5 fmt verbs you'll actually use

fmt.Sprintf("%s scored %.2f", name, score)   // string, 2-decimal float
fmt.Sprintf("%d/%d tokens", used, limit)     // int
fmt.Sprintf("%q", name)                      // "calculator" — quoted, like Python's !r
fmt.Sprintf("%v", cfg)                       // default format
fmt.Sprintf("%+v", cfg)                      // {Name:agent Model:claude-opus-5} ← field names
fmt.Sprintf("%#v", cfg)                      // Go syntax — best for debugging
fmt.Sprintf("%T", v)                         // the dynamic type: *main.Agent
fmt.Errorf("run tool %q: %w", name, err)     // %w WRAPS an error (see §5)

%q is your !r: it makes "" and " " visible in logs. %+v on a struct is the fastest debugging tool in the language.

2.6 Slices — the type you must actually understand

A slice is a 3-word header: pointer to a backing array, length, capacity. That header is copied on assignment; the array is not.

xs := []string{"a", "b"}          // literal
ys := make([]string, 0, 100)      // len 0, cap 100 — preallocate when you know the size
ys = append(ys, "x")              // append RETURNS a new header; always reassign
len(xs); cap(xs)
xs = append(xs, ys...)            // ... spreads a slice (like Python's *)
copy(dst, src)                    // copies min(len(dst), len(src))
last10 := history[max(0, len(history)-10):]   // sliding window (min/max builtins: Go 1.21+)

⚠️ The aliasing trap — slicing shares the backing array:

all := []int{1, 2, 3, 4, 5}
head := all[:3]
head = append(head, 99)      // cap allows it → OVERWRITES all[3]
fmt.Println(all)             // [1 2 3 99 5]

Fixes: three-index slicing to cap it (all[:3:3] forces append to copy), or slices.Clone(head).

⚠️ Never keep a small slice of a huge one — the whole backing array stays alive:

snippet := slices.Clone(bigDoc[:100])   // ✅ 100 bytes retained, not 50 MB

The slices package (Go 1.21+) replaces most hand-written loops:

slices.Contains(tools, "bash")
slices.Sort(scores)
slices.SortFunc(docs, func(a, b Doc) int { return cmp.Compare(b.Score, a.Score) })  // desc
slices.Index(names, "calculator")
slices.Clone(xs); slices.Reverse(xs); slices.Max(scores)

2.7 Maps

scores := map[string]float64{"calculator": 0.94}
v := scores["missing"]                 // 0 — no error, zero value
v, ok := scores["missing"]             // ✅ the comma-ok idiom: v=0, ok=false
delete(scores, "calculator")
len(scores)
clear(scores)                          // Go 1.21+

for k, v := range scores { … }         // ⚠️ ORDER IS RANDOMIZED, deliberately
keys := slices.Sorted(maps.Keys(scores))   // Go 1.23+ — deterministic iteration
  • The comma-ok form is how you distinguish "absent" from "present and zero" — Go's answer to dict.get vs [].
  • Maps are not safe for concurrent use. Concurrent read+write panics with a fatal error the race detector can't recover from. Guard with sync.RWMutex or use sync.Map (only for its two specific patterns — see §6.6).
  • Preallocate when you know the size: make(map[string]int, 1000).

2.8 Structs and pointers

type AgentConfig struct {
	Name        string   `json:"name"`
	Model       string   `json:"model"`
	Temperature float64  `json:"temperature,omitempty"`
	Tools       []string `json:"tools,omitempty"`
	apiKey      string   `json:"-"`     // lowercase = unexported; "-" = never marshalled
}

cfg := AgentConfig{Name: "researcher", Model: "claude-opus-5"}   // ✅ field names, always
p := &cfg                       // pointer
p.Temperature = 0.2             // auto-dereference — no -> in Go
fmt.Printf("%+v\n", cfg)

Exported = capitalized. Name is visible outside the package; apiKey is not. That single rule replaces public/private.

Struct tags are metadata read by reflection — the JSON, DB, and validation layers all use them.

Value or pointer?

Use a value Use a pointer
Small, immutable-ish (time.Time, Point) The method mutates the receiver
You want a copy (concurrency safety) The struct is large (copying costs)
Zero value is meaningful Nil must be distinguishable from empty

Go is always pass-by-value — passing a struct copies it; passing a pointer copies the pointer. Slices, maps, and channels contain internal pointers, so copying the header still shares the data.

2.9 Control flow

if err := run(ctx); err != nil {          // ✅ init statement scopes err to the if
	return fmt.Errorf("run: %w", err)
}

switch {                                   // no condition = cleaner if/else-if chain
case score > 0.9:  label = "high"
case score > 0.5:  label = "medium"
default:           label = "low"
}

switch status {                            // no fallthrough by default (unlike C)
case StatusOK, StatusRetry:                // multiple values per case
	continue
}

for i := 0; i < n; i++ { }                 // classic
for i, msg := range history { }            // range: index+value
for _, msg := range history { }            // value only
for k := range scores { }                  // map: keys only
for range 5 { }                            // Go 1.22+: repeat N times
for { break }                              // infinite loop — the only `while`

for msg := range ch { }                    // range over a channel until it's closed
for tok := range stream.Tokens() { }       // Go 1.23+: range over an iterator function

There is no while, no ternary, and no do/while. That's not an oversight — it's the "one obvious way" principle.

⚠️ range copies each element: for _, d := range docs { d.Score = 0 } mutates a copy. Use for i := range docs { docs[i].Score = 0 }.

✅ Since Go 1.22, loop variables are per-iteration, so the classic "all goroutines see the last value" bug is gone. On older versions you needed i := i inside the loop.

2.10 Labels, goto, and other things you won't need

goto exists; you will not use it. Labeled break/continue are occasionally right for breaking out of nested loops:

outer:
for _, doc := range docs {
	for _, chunk := range doc.Chunks {
		if chunk.Match(q) { break outer }
	}
}

🎯 Actionable rules

  1. Design types so the zero value is useful; never return a nil map you expect callers to write to.
  2. Always reassign the result of append, and slices.Clone anything you retain from a big slice.
  3. Use comma-ok on map reads whenever "absent" and "zero" differ.
  4. %+v and %q in every debug print; %w in every wrapped error.

3. 🔧 Functions, Closures, defer

3.1 Signatures and multiple returns

// Summarize returns a summary of text capped at maxWords words.
//
// It collapses whitespace and never splits a word. maxWords must be > 0.
func Summarize(text string, maxWords int) (string, error) {
	if maxWords <= 0 {
		return "", fmt.Errorf("maxWords must be positive, got %d", maxWords)
	}
	words := strings.Fields(text)
	if len(words) > maxWords {
		words = words[:maxWords]
	}
	return strings.Join(words, " "), nil
}

summary, err := Summarize(doc, 50)
if err != nil { … }

(T, error) is the signature of Go. The error is the last return value, always. There is no Optional, no exception, no hidden control flow.

Doc comments start with the identifier's name and are the package's documentation (go doc, pkg.go.dev). Exported identifiers without a comment are flagged by linters — and the comment is what an LLM reads when your function becomes a tool.

func splitHostPort(s string) (host string, port int, err error) {   // named returns
	// … named results are pre-declared and zero-valued; a bare `return` returns them
	return host, port, nil                     // ✅ still return explicitly for clarity
}

Use named returns for documentation and for defer-based error wrapping (§3.4) — not as an excuse for naked returns in long functions.

3.2 Variadic functions and function values

func RunTool(name string, args ...any) (string, error) { … }
RunTool("calculator", "2+2")
RunTool("search", queryArgs...)                 // spread a slice

type ToolFunc func(ctx context.Context, args json.RawMessage) (string, error)

var registry = map[string]ToolFunc{}            // string → behaviour, the Go way

func Register(name string, fn ToolFunc) { registry[name] = fn }

Functions are values: assign them, store them in maps, pass them, return them. That covers most of what Python decorators do.

3.3 Closures

func makeRetrier(attempts int, base time.Duration) func(context.Context, func() error) error {
	return func(ctx context.Context, op func() error) error {
		var err error
		for i := range attempts {
			if err = op(); err == nil {
				return nil
			}
			select {
			case <-time.After(base << i):          // exponential backoff
			case <-ctx.Done():
				return ctx.Err()
			}
		}
		return fmt.Errorf("after %d attempts: %w", attempts, err)
	}
}

retry := makeRetrier(3, 100*time.Millisecond)

Closures capture variables by reference, so a closure can outlive the function that made it — the compiler moves those variables to the heap (see escape analysis, §7.4).

3.4 defer in practice

defer schedules a call to run when the surrounding function returns — on any path, including panic. It is Go's with/finally.

func fetchDoc(ctx context.Context, url string) ([]byte, error) {
	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
	if err != nil {
		return nil, fmt.Errorf("fetchDoc: build request: %w", err)
	}
	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		return nil, fmt.Errorf("fetchDoc: %w", err)
	}
	defer resp.Body.Close()          // ✅ immediately after the error check, every time
	…
}

Four rules that cover every defer bug:

  1. LIFO order. Multiple defers run in reverse.
  2. Arguments are evaluated at defer time, the call happens later:
    start := time.Now()
defer log.Printf("took %s", time.Since(start))   // ❌ Since() runs NOW → always ~0
defer func() { log.Printf("took %s", time.Since(start)) }()   // ✅ closure defers the read
  3. It's function-scoped, not block-scoped. Deferring inside a loop accumulates until the function ends:
    for _, p := range paths {
    f, _ := os.Open(p)
    defer f.Close()        // ❌ 10 000 open files, all closed at the very end
}
for _, p := range paths {  // ✅ give each iteration its own function
    func() {
        f, _ := os.Open(p); defer f.Close(); process(f)
    }()
}
  4. A deferred closure can modify named return values — the idiomatic way to wrap every error exit at once:
    func (s *Store) Save(ctx context.Context, d Doc) (err error) {
    tx, err := s.db.BeginTx(ctx, nil)
    if err != nil { return err }
    defer func() {
        if err != nil { _ = tx.Rollback(); return }
        err = tx.Commit()
    }()
    …
}

⚠️ Deferred Close() on a writer can silently drop errors. For files you write, close explicitly and check, or capture it: defer func() { err = errors.Join(err, f.Close()) }().

3.5 init() and package-level state

func init() { … }        // runs once, after package vars, before main

Use it almost never: it hides work, runs on import, and makes tests order-dependent. Prefer an explicit constructor called from main. The one defensible use is registering a driver or a codec.

🎯 Actionable rules

  1. Return (T, error); handle or wrap the error at the very next line.
  2. defer the cleanup on the line after the error check that acquired the resource.
  3. No defer inside loops — wrap the body in a function.
  4. Doc-comment every exported identifier, starting with its name.

4. 🧬 Structs, Methods, Interfaces, Generics

4.1 Methods and receivers

type Agent struct {
	cfg      AgentConfig
	llm      LLMClient
	history  []Message
	mu       sync.Mutex
}

// NewAgent constructs an Agent. Constructor functions are Go's __init__.
func NewAgent(cfg AgentConfig, llm LLMClient) (*Agent, error) {
	if cfg.Name == "" {
		return nil, errors.New("agent: name is required")
	}
	return &Agent{cfg: cfg, llm: llm}, nil
}

func (a *Agent) AddMessage(role Role, content string) {   // pointer receiver: mutates
	a.mu.Lock()
	defer a.mu.Unlock()
	a.history = append(a.history, Message{Role: role, Content: content})
}

func (a *Agent) Len() int { return len(a.history) }        // pointer for consistency

func (c AgentConfig) Describe() string {                   // value receiver: read-only, small
	return fmt.Sprintf("%s/%s@%.1f", c.Name, c.Model, c.Temperature)
}

Receiver rules:

  • Use a pointer receiver if the method mutates, if the struct is large, or if it contains a sync.Mutex (copying a mutex is a bug go vet catches).
  • Be consistent: if any method needs a pointer receiver, give them all pointer receivers.
  • Only *T satisfies an interface when methods have pointer receivers — a plain T value won't compile. This is the #1 "why doesn't my type implement this interface" error.

4.2 Embedding — composition instead of inheritance

type BaseTool struct {
	Name        string
	Description string
}

func (b BaseTool) Schema() string { … }

type CalculatorTool struct {
	BaseTool           // embedded: no field name
	Precision int
}

calc := CalculatorTool{BaseTool: BaseTool{Name: "calculator"}, Precision: 4}
calc.Name          // promoted field
calc.Schema()      // promoted method

Embedding promotes fields and methods — it looks like inheritance but it's delegation: there is no virtual dispatch and no super. Embedding an interface is the standard way to build decorators and partial fakes:

type loggingStore struct {
	Store                     // embedded interface: unimplemented methods pass through
	log *slog.Logger
}
func (s loggingStore) Get(ctx context.Context, id string) (Doc, error) {
	s.log.Info("get", "id", id)
	return s.Store.Get(ctx, id)
}

4.3 Interfaces — small, implicit, defined by the consumer

There is no implements keyword. If the method set matches, the type satisfies the interface.

// Defined in the package that USES it, not the one that implements it.
type LLMClient interface {
	Complete(ctx context.Context, prompt string) (string, error)
}

type AnthropicClient struct{ … }
func (c *AnthropicClient) Complete(ctx context.Context, p string) (string, error) { … }
// *AnthropicClient now satisfies LLMClient. No import of your package required.

agent, _ := NewAgent(cfg, &AnthropicClient{})     // prod
agent, _ := NewAgent(cfg, &fakeLLM{reply: "42"})  // test — no mocking library needed

The three rules that make Go interfaces work:

  1. "Accept interfaces, return structs." Take the narrowest interface you need as a parameter; return concrete types so callers keep every method.
  2. Define the interface where it's consumed. This inverts the dependency without a DI framework.
  3. Keep them tiny. io.Reader has one method. A 12-method interface is a class in disguise; nobody can fake it in a test.
var _ LLMClient = (*AnthropicClient)(nil)    // compile-time assertion that it satisfies

4.4 any, type assertions, and type switches

var v any = payload                    // any == interface{} (Go 1.18+ alias)

s, ok := v.(string)                    // ✅ comma-ok: never panics
s := v.(string)                        // ❌ panics if v isn't a string

switch x := v.(type) {                 // type switch
case string:
	return x
case map[string]any:
	return fmt.Sprintf("%d keys", len(x))
case nil:
	return "null"
default:
	return fmt.Sprintf("unsupported %T", x)
}

any throws away the compiler's help — use it only at the JSON/reflection boundary and convert into a real type immediately (the same discipline as Python's Any).

⚠️ The typed-nil trap — an interface holding a nil pointer is not nil:

func newClient() *AnthropicClient { return nil }
var c LLMClient = newClient()
c == nil        // false! the interface has a type (*AnthropicClient) and a nil value

Fix: return the interface type as a literal nil, never a typed nil pointer. Most commonly this bites with error — never declare var err *MyError and return it as error.

4.5 Generics

Type parameters (Go 1.18+) exist to remove copy-paste, not to build hierarchies.

func Map[T, U any](xs []T, f func(T) U) []U {
	out := make([]U, 0, len(xs))
	for _, x := range xs {
		out = append(out, f(x))
	}
	return out
}
names := Map(tools, func(t Tool) string { return t.Name() })

func Keys[K comparable, V any](m map[K]V) []K { … }   // comparable = usable as a map key

type Number interface{ ~int | ~int64 | ~float64 }      // ~ = "any type whose underlying type is"
func Sum[T Number](xs []T) T { var s T; for _, x := range xs { s += x }; return s }

// A generic, type-safe cache — the common real-world use.
type Cache[K comparable, V any] struct {
	mu sync.RWMutex
	m  map[K]V
}
func NewCache[K comparable, V any]() *Cache[K, V] {
	return &Cache[K, V]{m: make(map[K]V)}
}
func (c *Cache[K, V]) Get(k K) (V, bool) {
	c.mu.RLock(); defer c.mu.RUnlock()
	v, ok := c.m[k]
	return v, ok
}

When not to use generics: if an interface expresses it, use the interface. Generics can't have methods with their own type parameters, they inflate compile times, and Map/Filter chains read worse in Go than a plain for loop. The slices, maps, and cmp packages already cover 90% of what you'd write.

4.6 Interfaces worth knowing by heart

Interface Method Why it matters
error Error() string Every failure (§5)
fmt.Stringer String() string Custom formatting in every %v
io.Reader / io.Writer Read/Write Files, sockets, buffers, HTTP bodies — all compose
io.Closer Close() error Pairs with defer
json.Marshaler / Unmarshaler Custom JSON Enums, time formats, LLM payload quirks
context.Context Done, Err, Value, Deadline Cancellation everywhere (§6.5)
http.Handler ServeHTTP Every middleware in Go
sort.Interface Len/Less/Swap Mostly superseded by slices.SortFunc

io.Reader/io.Writer are the reason Go plumbing composes so well: an HTTP body, a gzip stream, a file, and a bytes.Buffer are interchangeable.

🎯 Actionable rules

  1. Constructors return (*T, error); validate there, so an existing value is always valid.
  2. Define small interfaces in the consuming package; accept interfaces, return structs.
  3. var _ Iface = (*T)(nil) to assert satisfaction at compile time.
  4. Reach for generics only after you've written the same function twice.

5. 💥 Errors Are Values

5.1 The whole mechanism

type error interface {
	Error() string
}

That's it. An error is any value with an Error() string method. There is no stack unwinding, no exception hierarchy, no invisible control flow — which is why Go code has if err != nil everywhere and why you can always see the failure path.

errors.New("agent: name is required")                       // static message
fmt.Errorf("embed batch %d: %w", i, err)                    // wrap with context
fmt.Errorf("parse config: %v", err)                         // %v = context WITHOUT wrapping
errors.Join(err1, err2)                                     // multiple failures (Go 1.20+)

%w vs %v: %w keeps the original error reachable by errors.Is/errors.As; %v flattens it to text. Wrap by default; use %v deliberately when you don't want callers coupling to an internal error type.

5.2 The wrapping convention

Follow one convention across the codebase — this repo's (CLAUDE.md) is fmt.Errorf("packagename.FuncName: %w", err):

func (r *Repo) GetDoc(ctx context.Context, id string) (Doc, error) {
	var d Doc
	if err := r.db.GetContext(ctx, &d, qGetDoc, id); err != nil {
		return Doc{}, fmt.Errorf("repo.GetDoc: %w", err)
	}
	return d, nil
}

Read top-to-bottom, the final message becomes a trace: handler.Query: service.Answer: repo.GetDoc: sql: no rows in result set

Rules: add context, not restatement (never "error: %w"); don't capitalize or end with punctuation; never log and return the same error — pick one, and log at the boundary that handles it.

5.3 Sentinels, custom types, Is, As

// Sentinel: a comparable, exported value callers can test for.
var (
	ErrNotFound   = errors.New("not found")
	ErrRateLimit  = errors.New("rate limited")
)

// Custom type: when the caller needs structured detail.
type ToolError struct {
	Tool string
	Code int
	Err  error
}

func (e *ToolError) Error() string { return fmt.Sprintf("tool %s: %v", e.Tool, e.Err) }
func (e *ToolError) Unwrap() error { return e.Err }        // makes errors.Is see through it

// Callers:
if errors.Is(err, ErrNotFound) {                            // ✅ works through any wrapping
	return http.StatusNotFound, nil
}

var toolErr *ToolError
if errors.As(err, &toolErr) {                               // ✅ extract the typed error
	metrics.ToolFailures.WithLabelValues(toolErr.Tool).Inc()
}

if err == ErrNotFound { }                                   // ❌ breaks the moment someone wraps

errors.Is for identity, errors.As for structure. Never compare error strings.

5.4 Handling patterns that keep code readable

// ✅ Handle immediately; the happy path stays at the left margin.
resp, err := c.Complete(ctx, prompt)
if err != nil {
	return fmt.Errorf("agent.Run: %w", err)
}
use(resp)

// ✅ Retry only what's retryable.
for attempt := range maxAttempts {
	out, err = call(ctx)
	if err == nil { break }
	if !errors.Is(err, ErrRateLimit) && !isTransient(err) {
		return fmt.Errorf("agent.call: %w", err)     // permanent → stop immediately
	}
	select {
	case <-time.After(backoff(attempt)):
	case <-ctx.Done():
		return ctx.Err()
	}
}

// ✅ Deliberately ignoring an error is written, not implied.
_ = resp.Body.Close()
defer func() { _ = tx.Rollback() }()   // rollback after a commit is a no-op

// ✅ Collect failures across a batch instead of stopping at the first.
var errs []error
for _, chunk := range chunks {
	if err := index(ctx, chunk); err != nil {
		errs = append(errs, fmt.Errorf("chunk %s: %w", chunk.ID, err))
	}
}
return errors.Join(errs...)     // nil if the slice is empty

5.5 Panic and recover — and when they're legitimate

panic unwinds the goroutine and crashes the process unless recovered. It is not an exception system.

Panic only when the program cannot sensibly continue: an impossible invariant, a programming bug, or failed initialization at startup (regexp.MustCompile, template.Must — the Must prefix is the convention).

Recover only at a process boundary — one bad request must not kill the server:

func Recoverer(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		defer func() {
			if rec := recover(); rec != nil {
				slog.Error("panic in handler",
					"err", rec, "path", r.URL.Path, "stack", string(debug.Stack()))
				http.Error(w, "internal error", http.StatusInternalServerError)
			}
		}()
		next.ServeHTTP(w, r)
	})
}

⚠️ recover only works in the same goroutine. A panic inside go func(){…}() kills the whole process no matter what your HTTP middleware does — every goroutine you spawn needs its own recover, or must be provably panic-free.

5.6 Python ↔ Go error mapping

Python Go
raise ValueError("bad temp") return fmt.Errorf("bad temperature %v", t)
except ValueError: if errors.Is(err, ErrBadTemp)
except SomeError as e: e.field var e *SomeError; errors.As(err, &e)
raise X from err fmt.Errorf("context: %w", err)
finally: defer
except Exception: pass _ = f() (and a comment saying why)
Traceback The wrap chain you built by hand
sys.exit(1) on fatal config log.Fatal / panic in main only

🎯 Actionable rules

  1. Wrap with %w and a pkg.Func: prefix at every layer; log once, at the top.
  2. errors.Is for sentinels, errors.As for typed detail — never string comparison.
  3. Panic only for programmer bugs and startup failures; recover only at boundaries.
  4. Every goroutine you start needs its own panic protection.

6. 🌀 Concurrency: Goroutines, Channels, Context

Go's headline feature. It is also where every serious Go bug lives.

6.1 Goroutines

go doWork()                      // that's the entire syntax
go func(id string) { … }(docID)  // pass arguments explicitly

A goroutine is a user-space thread multiplexed onto OS threads by the Go runtime: ~2 KB of initial stack (grown on demand), microsecond creation. A hundred thousand of them in one process is normal; a hundred thousand OS threads is not.

The rule that prevents most production incidents: never start a goroutine without knowing how it stops. Every goroutine needs an exit condition — a closed channel, a cancelled context, or a finite loop. A goroutine blocked forever on a channel nobody writes to is a leak: its stack, its captured variables, and everything they reference stay alive until the process dies.

// ❌ leaks one goroutine per request, forever, if nobody reads results
go func() { results <- expensive() }()

// ✅ it can always exit
go func() {
	select {
	case results <- expensive():
	case <-ctx.Done():
	}
}()

6.2 Channels

A channel is a typed, concurrency-safe queue. Unbuffered channels are a rendezvous: the sender blocks until a receiver takes the value.

ch := make(chan Token)             // unbuffered: synchronous handoff
buf := make(chan Job, 100)         // buffered: sender proceeds until full
ch <- tok                          // send
tok := <-ch                        // receive
tok, ok := <-ch                    // ok == false when the channel is closed AND drained
close(ch)                          // only the SENDER closes, and only once
for tok := range ch { … }          // receives until closed

Directional types document intent and are checked by the compiler:

func produce(out chan<- Token)  { … }   // send-only
func consume(in  <-chan Token)  { … }   // receive-only
Operation On a nil channel On a closed channel
Send blocks forever panics
Receive blocks forever returns zero value immediately, ok=false
Close panics panics

Consequences: only ever close from the single owning sender; closing signals "no more values", not "stop". To stop a consumer, cancel its context.

6.3 select

select {
case tok := <-tokens:
	emit(tok)
case err := <-errs:
	return err
case <-ctx.Done():                       // cancellation, always include it
	return ctx.Err()
case <-time.After(5 * time.Second):      // per-iteration timeout
	return errors.New("stream stalled")
default:                                 // non-blocking: runs if nothing else is ready
	metrics.Idle.Inc()
}

select blocks until one case is ready, choosing randomly among ready cases. With default it never blocks. ⚠️ time.After allocates a timer per call — inside a hot loop use a reusable time.NewTimer/Ticker and stop it.

6.4 The three concurrency shapes you'll actually build

1. Bounded worker pool — N workers over a job channel. The default for embedding, indexing, or crawling:

func EmbedAll(ctx context.Context, chunks []string, workers int) ([][]float32, error) {
	type result struct {
		i   int
		vec []float32
		err error
	}
	jobs := make(chan int)
	out := make(chan result, len(chunks))

	var wg sync.WaitGroup
	for range workers {                     // fixed number of goroutines
		wg.Add(1)
		go func() {
			defer wg.Done()
			for i := range jobs {           // exits when jobs is closed
				v, err := embed(ctx, chunks[i])
				out <- result{i, v, err}
			}
		}()
	}

	go func() {                             // feed, then close so workers exit
		defer close(jobs)
		for i := range chunks {
			select {
			case jobs <- i:
			case <-ctx.Done():
				return
			}
		}
	}()

	wg.Wait()
	close(out)

	vecs := make([][]float32, len(chunks))
	for r := range out {
		if r.err != nil {
			return nil, fmt.Errorf("embed chunk %d: %w", r.i, r.err)
		}
		vecs[r.i] = r.vec                   // index carries the order back
	}
	return vecs, nil
}

2. errgroup — the concise version when you just need "run these, stop on first error":

import "golang.org/x/sync/errgroup"

g, ctx := errgroup.WithContext(ctx)         // ctx is cancelled as soon as one task fails
g.SetLimit(8)                               // ← bounded concurrency, one line

results := make([]Doc, len(ids))
for i, id := range ids {
	g.Go(func() error {                     // Go 1.22+: no `i := i` needed
		d, err := fetch(ctx, id)
		if err != nil {
			return fmt.Errorf("fetch %s: %w", id, err)
		}
		results[i] = d                      // ✅ distinct indices — no mutex required
		return nil
	})
}
if err := g.Wait(); err != nil {
	return nil, err
}

This is Go's asyncio.gather + Semaphore, with cancellation included.

3. Pipeline / fan-in — merge several streams into one, the shape behind multi-model or multi-tool streaming:

func merge[T any](ctx context.Context, chans ...<-chan T) <-chan T {
	out := make(chan T)
	var wg sync.WaitGroup
	for _, c := range chans {
		wg.Add(1)
		go func(c <-chan T) {
			defer wg.Done()
			for v := range c {
				select {
				case out <- v:
				case <-ctx.Done():
					return
				}
			}
		}(c)
	}
	go func() { wg.Wait(); close(out) }()    // close exactly once, after all senders finish
	return out
}

6.5 context: cancellation that actually propagates

context.Context carries a deadline, a cancellation signal, and request-scoped values down the call tree. Every function that does I/O takes one as its first parameter.

ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
defer cancel()                            // ✅ ALWAYS defer cancel — otherwise the timer leaks

resp, err := agent.Run(ctx, prompt)
switch {
case errors.Is(err, context.DeadlineExceeded):
	http.Error(w, "upstream timeout", http.StatusGatewayTimeout)
case errors.Is(err, context.Canceled):
	return                                // client hung up; nothing to write
}

Why it matters for AI services: when a user closes the browser mid-stream, r.Context() is cancelled, and that cancellation flows into your model call, your DB query, and every worker goroutine — so you stop paying for tokens nobody will read.

// Values: request-scoped metadata only, with an unexported key type.
type ctxKey struct{}
var tenantKey ctxKey

ctx = context.WithValue(ctx, tenantKey, tenant)
tenant, ok := ctx.Value(tenantKey).(string)

Rules: ctx is the first parameter, never stored in a struct; context.Background() only in main/tests; never pass nil; values are for tracing/tenancy, never for optional arguments.

6.6 sync: when channels are overkill

"Don't communicate by sharing memory; share memory by communicating." …but a mutex around a cache is simpler than a channel, and simpler wins.

type Cache struct {
	mu sync.RWMutex                     // zero value is ready — no initialization
	m  map[string][]float32
}
func (c *Cache) Get(k string) ([]float32, bool) {
	c.mu.RLock()                        // many concurrent readers
	defer c.mu.RUnlock()
	v, ok := c.m[k]
	return v, ok
}
func (c *Cache) Put(k string, v []float32) {
	c.mu.Lock()                         // one writer, excludes readers
	defer c.mu.Unlock()
	c.m[k] = v
}

var once sync.Once
once.Do(func() { tokenizer = loadTokenizer() })      // exactly-once init

var wg sync.WaitGroup                    // wg.Add before `go`, wg.Done in a defer
var inflight atomic.Int64                // lock-free counters
inflight.Add(1); defer inflight.Add(-1)

Use sync.Map only for its two documented patterns (write-once/read-many, or disjoint key sets per goroutine); otherwise a plain map with an RWMutex is faster and clearer. Put the mutex next to the data it protects, and document what it guards.

6.7 The race detector is not optional

go test -race ./...
go run -race ./cmd/api

It catches unsynchronized concurrent access at runtime (~10× slower, more memory — fine for CI). A data race in Go is undefined behaviour, not just a wrong number: a torn map write crashes the process.

6.8 Concurrency bug checklist

Symptom Cause Fix
Memory grows forever Goroutine leak — blocked send/receive Add <-ctx.Done() to every select; close channels
all goroutines are asleep - deadlock! Unbuffered send with no receiver; wg.Wait() before Done Check ownership; wg.Add before go
send on closed channel panic Multiple senders, or closing to signal "stop" Only the sole sender closes; cancel via context
Results in the wrong order Concurrency doesn't preserve order Carry an index, or write into a preallocated slice
Rare corrupt data Data race -race, then a mutex or channel
429s / OOM under load Unbounded fan-out g.SetLimit(n) or a worker pool
context deadline exceeded everywhere One deadline shared by N sequential calls Give each call its own budget

🎯 Actionable rules

  1. Every goroutine has a known exit path; every blocking select has <-ctx.Done().
  2. Bound concurrency explicitly — errgroup.SetLimit or a fixed worker pool. Never go in an unbounded loop.
  3. ctx first parameter, defer cancel() always.
  4. Run -race in CI, permanently.

7. ⚡ The Runtime: Scheduler, GC, Memory

You don't have to know this to write Go. You do have to know it to explain a p99 latency spike.

7.1 The scheduler (G-M-P)

G = goroutine   M = OS thread   P = processor (a scheduling context, GOMAXPROCS of them)

   [P0]──local run queue──> G G G        each P owns a queue of runnable Gs
   [P1]──local run queue──> G            an idle P steals work from a busy one
     ↑ bound to an M (thread) while running
   [global run queue] ── overflow ──
  • GOMAXPROCS = how many goroutines execute Go code simultaneously. It defaults to the number of CPUs — and since Go 1.25 it respects the container's CPU limit. On older versions inside Kubernetes, set it from the cgroup quota (go.uber.org/automaxprocs) or your 500m-CPU pod will spawn 64 Ps and thrash.
  • When a goroutine makes a blocking syscall, the runtime detaches its M and hands the P to another thread — so blocking I/O doesn't stall your other goroutines. This is why Go needs no async/await colouring.
  • Since Go 1.14 the scheduler preempts asynchronously, so a tight CPU loop can't starve everyone else.
  • Channel operations, mutex contention, and network I/O park a goroutine cheaply (the netpoller integrates with epoll/kqueue).

Versus Python: asyncio gives you one thread cooperatively multiplexing coroutines, and any blocking call freezes all of them. Go gives you preemptive scheduling across every core with no code-colour distinction. That's the core reason a Go gateway holds 50k streaming connections on hardware where a Python one needs process fan-out.

7.2 Garbage collection

Go's GC is a concurrent, tri-colour mark-and-sweep collector, non-generational and non-compacting. It's tuned for latency, not throughput: sub-millisecond stop-the-world pauses, at the cost of some CPU and headroom.

GOGC=100      # default: collect when the heap doubles since the last GC
GOGC=200      # collect half as often — more RAM, less CPU
GOMEMLIMIT=6GiB   # soft memory ceiling (Go 1.19+) — the setting for containers
GODEBUG=gctrace=1 ./api    # one line per GC cycle: heap size, pause, CPU share

In containers, set GOMEMLIMIT to ~80% of the pod's memory limit. Without it, Go sizes the heap from GOGC alone, happily grows past the cgroup limit, and gets OOM-killed with no Go-level error. With it, the GC works harder as you approach the ceiling instead of dying.

Pointer-heavy structures make GC scan more. Fewer, larger allocations of pointer-free data ([]float32 for embeddings, not []*float32) is the single biggest GC win in AI workloads.

7.3 Memory model in one paragraph

A write in one goroutine is only guaranteed visible to another if they synchronize — via a channel operation, a mutex, sync/atomic, sync.Once, or WaitGroup. Without that, the compiler and CPU may reorder freely, and the race detector will (eventually) tell you. There is no "volatile"; there is sync/atomic.

7.4 Escape analysis and allocation

The compiler puts values on the stack (free, no GC) unless they can outlive the function, in which case they escape to the heap.

go build -gcflags='-m' ./...      # prints "escapes to heap" / "does not escape"

Common causes of escape: returning a pointer to a local, storing in an interface, closing over a variable, sending on a channel, fmt.Sprintf.

Allocation-reduction techniques, in order of payoff:

out := make([]Doc, 0, len(ids))         // 1. preallocate with capacity — avoids log(n) regrowths
m := make(map[string]int, 1000)

var b strings.Builder                    // 2. builders instead of += concatenation
b.Grow(estimate)

var bufPool = sync.Pool{                 // 3. pool big, short-lived buffers on hot paths
	New: func() any { return new(bytes.Buffer) },
}
buf := bufPool.Get().(*bytes.Buffer)
defer func() { buf.Reset(); bufPool.Put(buf) }()

func (s *Scanner) Fill(dst []byte) int   // 4. let the caller own the buffer

Do these where a profile says they matter (§12), not everywhere. sync.Pool used carelessly is a memory leak with extra steps.

7.5 When Go beats Python — and when it doesn't

Workload Winner Why
20k concurrent SSE streams Go, decisively 2 KB goroutines vs event-loop + process fan-out
Fan-out to 50 tools/APIs per request Go errgroup + real parallelism
JSON/protobuf transformation at volume Go Compiled, GC-friendly, no interpreter overhead
Token/rate accounting, queues, schedulers Go Predictable latency, cheap primitives
Embedding, training, fine-tuning Python torch/numpy/CUDA live there
Data science, notebooks, evaluation Python The ecosystem is the product
Model-specific pre/post-processing Python Tokenizers and libraries exist already

🎯 Actionable rules

  1. In containers: set GOMEMLIMIT (~80% of the limit) and make GOMAXPROCS cgroup-aware.
  2. Preallocate slices and maps whose size you know.
  3. Prefer pointer-free bulk data ([]float32) to reduce GC scan time.
  4. Optimize allocations only where a pprof profile points.

8. 📦 The Standard Library & AI Toolkit

Go's stdlib is unusually complete: an HTTP/2 server, JSON, TLS, templating, profiling, and testing all ship with the compiler. The list below is what an AI service actually uses.

8.1 net/http — the server

mux := http.NewServeMux()
mux.HandleFunc("POST /v1/query", h.Query)          // Go 1.22+: method + wildcards
mux.HandleFunc("GET /v1/jobs/{id}", h.GetJob)      // r.PathValue("id")

srv := &http.Server{
	Addr:              ":8080",
	Handler:           Recoverer(RequestID(Logging(mux))),   // middleware = wrapped handlers
	ReadHeaderTimeout: 5 * time.Second,     // ✅ blocks Slowloris; the one people forget
	ReadTimeout:       30 * time.Second,
	WriteTimeout:      0,                   // 0 for SSE/streaming endpoints; set it otherwise
	IdleTimeout:       120 * time.Second,
	MaxHeaderBytes:    1 << 20,
}

// Graceful shutdown: stop accepting, let in-flight requests finish.
go func() {
	if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
		slog.Error("listen", "err", err); os.Exit(1)
	}
}()

ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
<-ctx.Done()
shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
_ = srv.Shutdown(shutdownCtx)

chi adds routers, groups, and middleware chains on top of http.Handler without inventing a new handler type — which is why it composes with everything (and why this repo uses it).

8.2 net/http — the client

var client = &http.Client{                 // ✅ ONE client for the process, reused
	Timeout: 60 * time.Second,             // total budget, including body read
	Transport: &http.Transport{
		MaxIdleConns:        200,
		MaxIdleConnsPerHost: 100,          // default is 2 — far too low for an LLM proxy
		IdleConnTimeout:     90 * time.Second,
	},
}

req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
if err != nil { return fmt.Errorf("llm.Complete: %w", err) }
req.Header.Set("Content-Type", "application/json")

resp, err := client.Do(req)
if err != nil { return fmt.Errorf("llm.Complete: %w", err) }
defer resp.Body.Close()                    // ✅ ALWAYS — otherwise the connection leaks
if resp.StatusCode != http.StatusOK {
	b, _ := io.ReadAll(io.LimitReader(resp.Body, 4<<10))    // cap what you read on errors
	return fmt.Errorf("llm.Complete: status %d: %s", resp.StatusCode, b)
}

Three non-negotiables: reuse the client, always close the body, always pass a context. Creating an http.Client per request disables connection pooling and exhausts sockets under load.

8.3 encoding/json

type QueryIn struct {
	Query       string   `json:"query"`
	Temperature float64  `json:"temperature,omitempty"`   // omit when zero
	Tools       []string `json:"tools,omitempty"`
	internal    string   `json:"-"`                       // never marshalled
}

b, err := json.Marshal(v)
err = json.Unmarshal(b, &v)                               // note the pointer

dec := json.NewDecoder(r.Body)                            // ✅ stream, don't ReadAll
dec.DisallowUnknownFields()                               // ✅ typo'd client fields become errors
if err := dec.Decode(&in); err != nil {
	http.Error(w, "invalid body", http.StatusBadRequest); return
}

var raw json.RawMessage                                    // defer parsing tool args
enc := json.NewEncoder(w); enc.Encode(out)                 // stream the response out

⚠️ Only exported fields are marshalled. ⚠️ Unmarshalling into map[string]any turns every number into float64 — decode into a struct whenever you can. For hot paths, json.Decoder on the body avoids materializing the whole payload.

Custom marshalling for domain types:

func (r Role) MarshalJSON() ([]byte, error) { return json.Marshal(string(r)) }

8.4 log/slog — structured logging (Go 1.21+)

logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo}))
slog.SetDefault(logger)

slog.Info("tool completed", "tool", name, "ms", elapsed.Milliseconds(), "tokens", n)
slog.Error("model call failed", "err", err, "model", cfg.Model, "attempt", i)

reqLog := logger.With("request_id", rid, "tenant", tenant)   // bind once, reuse per request
reqLog.Info("received")

Structured key-value output is what makes logs queryable in Loki/Datadog. Never log prompts, keys, or full request bodies — log ids, counts, durations, and truncated previews.

8.5 time

time.Now(); time.Since(start)                    // monotonic for durations
30 * time.Second; 500 * time.Millisecond         // Durations are typed ints — no unit bugs
t.Format(time.RFC3339); time.Parse(time.RFC3339, s)
time.Now().UTC()                                 // store UTC, convert at the edge

tick := time.NewTicker(10 * time.Second)
defer tick.Stop()                                // ✅ tickers leak if not stopped
select {
case <-tick.C: flushMetrics()
case <-ctx.Done(): return
}

8.6 io and bufio — the composable plumbing

io.Copy(dst, src)                                  // stream, constant memory
io.ReadAll(io.LimitReader(r, 10<<20))              // ✅ always cap untrusted input
io.MultiWriter(w, &buf)                            // tee the response into a buffer

sc := bufio.NewScanner(resp.Body)                  // line-by-line: perfect for SSE
sc.Buffer(make([]byte, 0, 64*1024), 1<<20)         // ✅ raise the 64 KB line limit
for sc.Scan() {
	line := sc.Text()
	…
}
if err := sc.Err(); err != nil { … }               // ✅ Scan() returning false isn't always EOF

8.7 The rest, in one breath

Package Use it for
context Cancellation and deadlines (§6.5)
sync / sync/atomic Mutexes, WaitGroup, Once, counters (§6.6)
errors Is, As, Join, Unwrap (§5)
strconv / strings / bytes Conversion and text handling (§2.4)
regexp RE2 — linear time, no catastrophic backtracking; MustCompile at package level
os / os/signal Env, files, SIGTERM handling
flag Small CLIs; use cobra for a command tree
embed //go:embed prompts/*.md — bake prompts and migrations into the binary
text/template Prompt templating with named fields
database/sql (+ sqlx, pgx) SQL; always QueryContext, always defer rows.Close(), always check rows.Err()
encoding/base64, crypto/* Tokens, signatures, crypto/rand for secrets
net/http/httptest In-process HTTP tests (§10)
runtime/pprof, net/http/pprof Profiling (§12)
testing Tests, benchmarks, fuzzing — all built in

Third-party worth adopting: golang.org/x/sync/errgroup and singleflight, go-chi/chi, jmoiron/sqlx, stretchr/testify/require, pressly/goose, golang.org/x/time/rate, and OpenTelemetry for traces. Go culture keeps dependency trees small — prefer the stdlib until it genuinely hurts.

🎯 Actionable rules

  1. One http.Client per process with a timeout and a tuned transport; defer resp.Body.Close() always.
  2. Explicit http.Server timeouts and graceful shutdown on SIGTERM.
  3. json.Decoder + DisallowUnknownFields on request bodies; io.LimitReader on anything untrusted.
  4. slog with key-value pairs from day one — retrofitting structure is miserable.

9. 🤖 AI Service Patterns in Go

What Go is actually for in an AI stack: the request path, the fan-out, and the streaming.

9.1 Consuming an SSE token stream

func (c *LLM) Stream(ctx context.Context, prompt string, out chan<- string) error {
	req, _ := http.NewRequestWithContext(ctx, http.MethodPost, c.url, encode(prompt))
	req.Header.Set("Accept", "text/event-stream")

	resp, err := c.http.Do(req)
	if err != nil {
		return fmt.Errorf("llm.Stream: %w", err)
	}
	defer resp.Body.Close()

	sc := bufio.NewScanner(resp.Body)
	sc.Buffer(make([]byte, 0, 64*1024), 1<<20)      // model chunks exceed the 64 KB default
	for sc.Scan() {
		line, ok := strings.CutPrefix(sc.Text(), "data: ")
		if !ok || line == "" {
			continue
		}
		if line == "[DONE]" {
			return nil
		}
		var ev struct {
			Delta struct{ Text string } `json:"delta"`
		}
		if err := json.Unmarshal([]byte(line), &ev); err != nil {
			return fmt.Errorf("llm.Stream: decode %q: %w", truncate(line, 80), err)
		}
		select {
		case out <- ev.Delta.Text:
		case <-ctx.Done():                          // client disconnected: stop paying for tokens
			return ctx.Err()
		}
	}
	return sc.Err()
}

9.2 Serving SSE to the browser

func (h *Handler) Stream(w http.ResponseWriter, r *http.Request) {
	rc := http.NewResponseController(w)             // Go 1.20+; replaces the http.Flusher cast
	w.Header().Set("Content-Type", "text/event-stream")
	w.Header().Set("Cache-Control", "no-cache")
	w.Header().Set("X-Accel-Buffering", "no")       // stop nginx from buffering your stream

	ctx := r.Context()                              // cancelled when the client goes away
	tokens := make(chan string, 16)
	errc := make(chan error, 1)
	go func() { errc <- h.llm.Stream(ctx, r.FormValue("q"), tokens); close(tokens) }()

	for {
		select {
		case tok, ok := <-tokens:
			if !ok {
				fmt.Fprint(w, "data: [DONE]\n\n")
				_ = rc.Flush()
				return
			}
			fmt.Fprintf(w, "data: %s\n\n", tok)
			_ = rc.Flush()                          // ✅ without Flush nothing reaches the client
		case <-ctx.Done():
			return
		case <-time.After(30 * time.Second):
			slog.Warn("stream stalled", "path", r.URL.Path)
			return
		}
	}
}

Remember to set WriteTimeout: 0 on the server for streaming routes (§8.1), or the connection dies mid-answer.

9.3 A tool registry with schemas

type Tool struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Schema      json.RawMessage `json:"input_schema"`     // sent verbatim to the model
	Run         func(ctx context.Context, args json.RawMessage) (string, error) `json:"-"`
}

type Registry struct {
	mu    sync.RWMutex
	tools map[string]Tool
}

func (r *Registry) Register(t Tool) error {
	r.mu.Lock(); defer r.mu.Unlock()
	if _, dup := r.tools[t.Name]; dup {
		return fmt.Errorf("registry.Register: duplicate tool %q", t.Name)
	}
	r.tools[t.Name] = t
	return nil
}

func (r *Registry) Dispatch(ctx context.Context, name string, args json.RawMessage) (string, error) {
	r.mu.RLock(); t, ok := r.tools[name]; r.mu.RUnlock()
	if !ok {
		return "", fmt.Errorf("registry.Dispatch: unknown tool %q", name)   // never trust the model
	}
	ctx, cancel := context.WithTimeout(ctx, 30*time.Second)                // ✅ per-tool budget
	defer cancel()
	return t.Run(ctx, args)
}

Two things the model must never control: which tools exist, and how long they may run.

9.4 Retries, rate limits, and backpressure

import "golang.org/x/time/rate"

type Client struct {
	http    *http.Client
	limiter *rate.Limiter          // rate.NewLimiter(rate.Limit(50), 100) → 50 rps, burst 100
	sem     chan struct{}          // concurrency cap: make(chan struct{}, 16)
}

func (c *Client) Complete(ctx context.Context, prompt string) (string, error) {
	if err := c.limiter.Wait(ctx); err != nil {          // blocks or returns on cancellation
		return "", fmt.Errorf("llm.Complete: rate wait: %w", err)
	}
	select {                                             // bound in-flight requests
	case c.sem <- struct{}{}:
		defer func() { <-c.sem }()
	case <-ctx.Done():
		return "", ctx.Err()
	}

	var lastErr error
	for attempt := range 4 {
		out, err := c.do(ctx, prompt)
		if err == nil {
			return out, nil
		}
		lastErr = err
		var re *RetryableError
		if !errors.As(err, &re) {
			return "", fmt.Errorf("llm.Complete: %w", err)          // permanent → stop
		}
		delay := re.RetryAfter                                       // honour the server's hint
		if delay == 0 {
			delay = time.Duration(1<<attempt) * 200 * time.Millisecond
		}
		jitter := time.Duration(rand.Int64N(int64(delay / 2)))       // math/rand/v2
		select {
		case <-time.After(delay + jitter):
		case <-ctx.Done():
			return "", ctx.Err()
		}
	}
	return "", fmt.Errorf("llm.Complete: exhausted retries: %w", lastErr)
}

9.5 Calling the Python ML service (the BFF shape)

// Go owns HTTP, auth, tenancy, and fan-out; Python owns the model work.
func (s *Service) Answer(ctx context.Context, tenant, q string) (Answer, error) {
	ctx, cancel := context.WithTimeout(ctx, 45*time.Second)
	defer cancel()

	g, gctx := errgroup.WithContext(ctx)
	var (
		docs []Doc
		vec  []float32
	)
	g.Go(func() (err error) { docs, err = s.repo.Search(gctx, tenant, q); return })
	g.Go(func() (err error) { vec, err = s.python.Embed(gctx, q); return })   // internal REST
	if err := g.Wait(); err != nil {
		return Answer{}, fmt.Errorf("service.Answer: %w", err)
	}
	…
}

Retrieval and embedding run in parallel; either failure cancels the other; the whole request shares one deadline. That is ~15 lines of Go for what needs careful orchestration elsewhere.

9.6 singleflight — collapse duplicate work

When 500 users ask the same question in the same second, do the expensive thing once:

import "golang.org/x/sync/singleflight"

var group singleflight.Group

func (c *Cache) Embed(ctx context.Context, text string) ([]float32, error) {
	key := hash(text)
	if v, ok := c.Get(key); ok {
		return v, nil
	}
	v, err, _ := group.Do(key, func() (any, error) {     // concurrent callers share one result
		return c.upstream.Embed(ctx, text)
	})
	if err != nil {
		return nil, fmt.Errorf("cache.Embed: %w", err)
	}
	return v.([]float32), nil
}

🎯 Actionable rules

  1. Propagate r.Context() into every model call so a disconnect stops the spend.
  2. Bound everything: rate limiter, concurrency semaphore, per-tool timeout, retry cap.
  3. Flush after every SSE write, and disable proxy buffering.
  4. Validate tool names against the registry — the model's output is untrusted input.

(...to be continued...) Read full version here https://dev.to/truongpx396/golang-for-ai-developers-from-0-to-pro-1enk

If you found this helpful, let me know by leaving a 👍 or a comment!, or if you think this post could help someone, feel free to share it! Thank you very much! 😃

Android iOS JavaScript ReactJS Ruby on Rails

All rights reserved

Mục lục


Viblo
Hãy đăng ký một tài khoản Viblo để nhận được nhiều bài viết thú vị hơn.
Đăng kí