Parsing and Building URLs Safely
You are building a redirect service. A user submits https://example.com/search?q=shoes&size=10. You need to change the host to your analytics domain, add a tracking parameter, and ensure the result is valid before sending it to the browser. String concatenation feels risky. One missing & breaks the query. One unescaped & in the value turns into a separator. You need structure.
The net/url package treats a URL like a structured form rather than a blob of text. Think of a URL as a mailing address. You wouldn't write "123 Main St Apt 4 Springfield IL 62704" as a single unbreakable string if you wanted to change just the apartment number. You fill out distinct fields. url.URL is that form. It breaks the string into Scheme, Host, Path, RawQuery, and Fragment. You manipulate the fields, then the package reassembles the string safely.
package main
import (
"fmt"
"net/url"
)
// ParseURL demonstrates basic parsing and field access.
func ParseURL() {
// url.Parse converts a raw string into a structured object.
// It returns an error if the syntax is invalid.
u, err := url.Parse("https://example.com/path?query=1")
if err != nil {
// Handle the error. In real code, return or log.
panic(err)
}
// Access fields directly. Scheme is the protocol.
fmt.Println("Scheme:", u.Scheme)
// Host includes the port if present.
fmt.Println("Host:", u.Host)
// Path is the hierarchy after the host.
fmt.Println("Path:", u.Path)
// RawQuery is the uninterpreted query string.
fmt.Println("RawQuery:", u.RawQuery)
}
When you call url.Parse, the function scans the string left to right. It looks for the scheme separator ://. If found, everything before goes to Scheme. Next, it grabs the host. If there's a port, it stays attached to the host in the Host field. Then comes the path. The question mark ? signals the start of the query string. The hash # signals the fragment. The parser fills the url.URL struct with these pieces. If the string violates RFC 3986 rules, Parse returns an error immediately. You never get a half-baked object.
Parse early. Validate often.
Modifying Query Parameters
Real code rarely just reads URLs. You need to add tracking IDs, override parameters, or strip sensitive data. The url.Values type handles this. It is a map of key-value pairs that manages encoding and separators automatically.
package main
import (
"fmt"
"net/url"
)
// AddTrackingParam appends a tracking ID to an existing URL.
// It preserves existing query parameters and handles encoding.
func AddTrackingParam(baseURL string, trackingID string) (string, error) {
// Parse the base URL to get a mutable struct.
u, err := url.Parse(baseURL)
if err != nil {
// Wrap the error to provide context.
return "", fmt.Errorf("invalid base URL: %w", err)
}
// Query() returns a url.Values map.
// It parses the RawQuery into a map of key-value pairs.
// This is a copy. Changes here do not affect u.RawQuery yet.
q := u.Query()
// Set adds or replaces the parameter.
// The value is automatically URL-encoded.
q.Set("tracking_id", trackingID)
// RawQuery must be updated with the encoded string.
// Query.Encode() produces the "key=val&key=val" format.
u.RawQuery = q.Encode()
// String() reassembles the URL from the struct fields.
return u.String(), nil
}
The type url.Values is an alias for map[string][]string. The slice allows multiple values for the same key, which matches HTML form behavior. Get returns the first value. Set replaces all values for that key. Add appends a new value to the slice. Knowing the underlying map prevents surprises when debugging.
The Query() method returns a copy of the parsed query. Modifying the returned url.Values does not update the url.URL struct. You must assign the encoded result back to u.RawQuery. Forgetting this step is a common bug. The URL remains unchanged, and the missing parameter causes silent failures downstream.
Query() returns a copy. Write it back.
Relative URLs and Path Resolution
API clients and web scrapers often deal with relative links. An API response might return "/users/123" instead of the full URL. You need to combine this with a base URL to make a request. ResolveReference handles the math.
package main
import (
"fmt"
"net/url"
)
// ResolveRelative demonstrates combining a base URL with a relative path.
// This is useful for resolving links found in HTML or API responses.
func ResolveRelative() {
// Parse the base URL.
// The trailing slash matters for path resolution.
base, err := url.Parse("https://example.com/api/v1/")
if err != nil {
panic(err)
}
// ResolveReference handles the path joining logic.
// It respects trailing slashes and relative segments like "..".
resolved, err := base.ResolveReference("/users/123")
if err != nil {
panic(err)
}
// Result: https://example.com/api/v1/users/123
fmt.Println(resolved.String())
}
ResolveReference follows the same rules as a web browser. If the reference starts with /, it replaces the path entirely relative to the host. If it starts with .., it walks up the directory tree. If the base URL lacks a trailing slash, the last segment is treated as a file, and relative paths resolve against the parent directory. This behavior matches HTTP standards exactly.
Use url.JoinPath when constructing a path from multiple segments in Go 1.19 and later. It normalizes separators and handles encoding, avoiding the pitfalls of string concatenation for paths.
ResolveReference handles the math. You provide the base.
Path Encoding and Raw Fields
The url.URL struct separates decoded data from raw encoded data. The Path field holds the decoded path. If the URL is /hello%20world, Path contains /hello world. Use EscapedPath() to retrieve the raw encoded string /hello%20world.
This distinction matters when you pass the path to a library that expects raw bytes. If you use Path directly in a network request, the library might double-encode spaces, turning %20 into %2520. Always use EscapedPath() when you need the exact bytes that appeared in the URL.
The RawPath field exists for edge cases. If RawPath is set, String() uses it instead of encoding Path. This allows you to preserve non-standard encoding. Modifying RawPath directly is dangerous. The parser does not validate RawPath against Path. Inconsistencies can produce malformed URLs. Stick to Path for modifications and let String() handle encoding.
If you modify Path directly, String() will re-encode it. This is usually safe. However, if the path contains characters that cannot be percent-encoded, String() may produce unexpected results. Trust the parser to manage encoding.
Path is for reading. EscapedPath is for transmitting.
Common Pitfalls and Compiler Errors
The net/url package is strict. It rejects invalid syntax at runtime. If you pass a malformed URL to Parse, you get an error like parse "http://example.com:abc": invalid port "abc" after host. The error message tells you exactly what went wrong. Check the error before proceeding.
Query parameters require careful handling. The RawQuery field is a string. If you modify it directly, you risk breaking encoding. Always use url.Values for manipulation. If you concatenate strings into RawQuery, you must escape values manually using url.QueryEscape. Forgetting to escape turns a value like a&b into two parameters.
The Host field includes the port. If you need just the hostname, use u.Hostname(). If you need the port, use u.Port(). Accessing Host directly gives you example.com:8080. Splitting this string manually is error-prone. Use the helper methods.
URLs can contain user info, like https://user:pass@example.com. The Userinfo field holds this data. Modern security practices discourage credentials in URLs. Browsers often strip them. If you encounter Userinfo, handle it carefully. Do not log the full URL, as it may expose secrets.
The compiler does not catch URL syntax errors. url.Parse runs at runtime. If you hardcode a bad URL, the program panics or fails when Parse returns an error. Validate configuration URLs at startup. Fail fast.
Validate input early. Trust the parser.
Decision Matrix
Use url.Parse when you receive a URL string and need to inspect or modify its components safely.
Use url.Values when you need to add, remove, or change query parameters without worrying about & separators or percent-encoding.
Use url.QueryEscape when you are building a query value manually and need to encode special characters like spaces or ampersands.
Use url.PathEscape when you are encoding a segment of the path, such as a filename with spaces, to ensure it survives the URL structure.
Use url.ResolveReference when you need to combine a base URL with a relative path, matching browser resolution rules.
Use url.JoinPath when constructing a path from multiple segments in Go 1.19 and later, to normalize separators and handle encoding automatically.
Use plain string concatenation only when constructing a static URL where every part is known and trusted at compile time.