Strings, bytes and runes in Go: best practices explained

Strings, bytes and runes in Go: best practices explained. Text handling done right.

Text handling in Go looks deceptively simple at first, but the relationship between strings, bytes, and runes trips up engineers who haven't internalized how Go represents text. Understanding these distinctions isn't academic—it directly affects whether your API handles Unicode correctly, performs efficiently, and avoids subtle bugs in production.

Strings are immutable byte sequences

In Go, a string is fundamentally a read-only slice of bytes. This immutability is a core design decision that has real implications for how we work with text. When you create a string literal like "hello", you're creating a sequence of bytes that cannot be modified after creation. Any operation that appears to change a string actually creates a new string.

This immutability gives us safety and predictability. In a concurrent system, we can pass strings between goroutines without synchronization overhead—there's no risk of one goroutine modifying a string another goroutine is reading. For APIs processing user input, this means we can safely store and reuse request bodies without worrying about accidental mutation.

However, immutability also means that building strings through repeated concatenation is expensive. Each concatenation operation allocates a new string in memory and copies all bytes from both operands. Consider this problematic pattern:

result := ""
for i := 0; i < 1000; i++ {
    result = result + fmt.Sprintf("line %d\n", i)
}

This creates roughly 1,000 intermediate strings, with each iteration copying all previously accumulated bytes plus new ones. For a loop of n iterations, we're looking at O(n²) complexity. The bytes themselves are immutable, but we're constantly creating new string objects that hold different byte sequences.

Understanding this property helps us make better architectural decisions. In an e-commerce API, if you're building response bodies by concatenating strings in a loop, you're leaving performance on the table. The bytes underneath are fixed and efficient; the inefficiency comes from how we're combining multiple immutable sequences.

Range over a string yields runes

This is where many Go engineers encounter their first surprise. When you iterate over a string using range, you don't get individual bytes—you get runes. A rune is Go's term for a Unicode code point, represented as a 32-bit integer (the int32 type). This distinction is crucial because Unicode characters often occupy multiple bytes.

The English word "hello" is five bytes and five runes—a one-to-one mapping. But the German word "Grüße" (hello in a more elaborate form) contains five visible characters, seven bytes, and five runes. The character ü is encoded as two bytes in UTF-8 but represents a single rune.

s := "Grüße"

// Iterating with range gives us runes
for i, r := range s {
    fmt.Printf("Position %d: %c (U+%04X) bytes: %d\n", i, r, r, len(string(r)))
}
// Output:
// Position 0: G (U+0047) bytes: 1
// Position 0: r (U+0072) bytes: 1
// Position 5: ü (U+00FC) bytes: 2
// Position 7: ß (U+00DF) bytes: 2
// Position 9: e (U+0065) bytes: 1

Notice that the byte position jumps from 5 to 7 when we encounter the ü character. This is fundamental: range on strings decodes UTF-8 on the fly and yields the decoded rune plus its byte offset in the original string.

In a real API scenario, imagine you're building a text search feature. If you iterate over a string with a traditional index loop, you're working with byte offsets. But if you iterate with range, you're working with logical character positions. This matters when you need to extract substrings or validate input. Mixing byte offsets and character positions leads to corrupted output when non-ASCII characters are involved.

strings.Builder for concatenation in loops

Now we've identified the problem: string concatenation in loops is inefficient because strings are immutable. The solution is strings.Builder, which provides a mutable buffer for building strings efficiently.

var sb strings.Builder
for i := 0; i < 1000; i++ {
    sb.WriteString(fmt.Sprintf("line %d\n", i))
}
result := sb.String()

strings.Builder maintains an internal byte buffer that grows as needed. Each write operation appends to this buffer without creating intermediate strings. When you call String() at the end, you get a single final string. This reduces complexity from O(n²) to O(n).

The builder has several useful methods beyond WriteString:

• WriteRune(r rune) — writes a single rune, properly encoding it in UTF-8
• WriteByte(b byte) — writes a single byte
• WriteRune(r rune) — writes a single rune and returns the number of bytes written
• Len() — returns the current length in bytes
• Cap() — returns the builder's capacity
• Reset() — clears the builder for reuse

In a CRM API building CSV export responses, strings.Builder shines. You might have thousands of contacts, and for each one you need to write multiple fields and a newline. Using Builder` prevents memory thrashing and keeps response times predictable.

One optimization: if you know roughly how many bytes you'll write, you can pre-allocate capacity using Grow():

var sb strings.Builder
sb.Grow(10000) // Reserve 10KB upfront
for i := 0; i < 1000; i++ {
    sb.WriteString(fmt.Sprintf("line %d\n", i))
}
result := sb.String()

This eliminates reallocations as the buffer grows, making the operation even faster in high-throughput scenarios.

Use utf8 package for character counts

Because range iteration and byte-level operations work differently with UTF-8, Go provides the utf8

s := "Grüße"
fmt.Println("Byte length:", len(s))           // 7 bytes
fmt.Println("Rune count:", utf8.RuneCountInString(s))  // 5 runes

In an API validating user input, this distinction matters. If you have a business rule that usernames must be between 3 and 20 characters, you probably mean runes, not bytes. A user with the name "José" has 4 runes but 5 bytes. Using len() would incorrectly reject valid names if they contain multi-byte characters.

The utf8

• ValidString(s string) bool — checks whether a string is valid UTF-8
• DecodeRune(p []byte) (r rune, size int) — decodes the first rune from a byte slice
• DecodeLastRune(p []byte) (r rune, size int) — decodes the last rune
• EncodeRune(p []byte, r rune) — encodes a rune into a byte slice

Validation is particularly important in trading or financial systems where we're parsing structured text data from external sources. Invalid UTF-8 in feed data can cause crashes or silent data corruption. A simple check early in the pipeline prevents cascading failures:

func processMarketData(raw string) error {
    if !utf8.ValidString(raw) {
        return fmt.Errorf("invalid UTF-8 in market data feed")
    }
    // Safe to process
    return nil
}

Common mistakes

Mistake 1: Using byte indexing for string slicing with non-ASCII text. If you slice a string containing multi-byte characters using byte indices calculated from visual inspection, you'll get corrupted output:

s := "café"
// Incorrect: café is 5 bytes but 4 runes
fmt.Println(s[:4])  // Output: "caf" followed by garbage byte

Always use range or the utf8 package when working with character boundaries in non-ASCII text.

Mistake 2: Concatenating strings in tight loops without Builder. We've covered this, but it bears repeating because it's so common in generated code or rapid prototyping that graduates to production:

// Slow—never do this at scale
for _, item := range items {
    response += item.Name + ", "
}

// Fast—use Builder
var sb strings.Builder
for _, item := range items {
    sb.WriteString(item.Name)
    sb.WriteString(", ")
}
response := sb.String()

Mistake 3: Assuming string length is character count. The len() function returns byte count, not character count. If you're building APIs that accept human-readable input (names, addresses, comments), always validate using utf8.RuneCountInString() for user-facing constraints.

Mistake 4: Not validating UTF-8 from external sources. If your API accepts strings from HTTP request bodies, files, or message queues, those bytes might not be valid UTF-8. Blindly converting them to strings and operating on them can cause panics or logical errors downstream. Use utf8.ValidString() on untrusted input.

Key takeaways

• Strings in Go are immutable byte sequences, making them safe for concurrent access but expensive to concatenate repeatedly.
• Iterating over a string with range yields runes (Unicode code points), not bytes, and handles UTF-8 decoding automatically.
• Use strings.Builder whenever you're concatenating strings in a loop; it's the standard efficient pattern for building strings in Go.
• Distinguish between byte length (len(s)) and character count (utf8.RuneCountInString(s)) based on your validation requirements.
• Validate that strings from external sources are valid UTF-8 before processing them, especially in APIs accepting user input or parsing structured data.
• When slicing strings with non-ASCII characters, use range iteration or the utf8 package rather than manual byte indexing.