Go Programming Experience

Novice

Other Languages Experience

No response

Related Idea

  • [ ] Has this idea, or one like it, been proposed before?
  • [x] Does this affect error handling?
  • [ ] Is this about generics?
  • [x] Is this change backward compatible? Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit

Has this idea, or one like it, been proposed before?

Yes, the concept of "non-nullable types" or "required fields" has been a recurring topic in the Go community. Discussions often arise in forums, issues, and proposals related to improving type safety, especially concerning nil values, data validation, and reducing boilerplate for common data structures (e.g., API payloads, database models). While specific syntax might differ, the underlying problem of ensuring data presence and validity is a common pain point.

Does this affect error handling?

Introducing a new keyword like required would likely break backward compatibility if required is already used as an identifier in existing Go code. To avoid this, this proposal suggests using a non-conflicting suffix symbol (e.g., !). A symbol like ! is more likely to be backward compatible, provided it doesn't conflict with existing syntax in that specific position.

Existing struct definitions would remain unchanged and would not implicitly gain "required" fields. Their behavior would be unaffected. New struct definitions using the proposed ! syntax would enforce new rules. The main compatibility challenge arises when new code using required fields interacts with old code that doesn't understand the concept, or when unmarshaling into structs with required fields. The proposal aims for an additive change that does not alter the behavior of existing code.

Is this about generics?

No!

Proposal

Example 1: API Request Body Validation

Consider a REST API endpoint for creating a new product. Certain fields like Name, Price, and SKU are mandatory.

Before (Current Go):

type CreateProductRequest struct {
    Name  string  `json:"name"`
    Price float64 `json:"price"`
    SKU   string  `json:"sku"`
    Desc  string  `json:"description,omitempty"`
}

func CreateProductHandler(w http.ResponseWriter, r *http.Request) {
    var req CreateProductRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        http.Error(w, "Invalid request body", http.StatusBadRequest)
        return
    }

    // Manual validation boilerplate
    if req.Name == "" {
        http.Error(w, "Name is required", http.StatusBadRequest)
        return
    }
    if req.Price <= 0 { // Assuming price must be positive
        http.Error(w, "Price must be positive", http.StatusBadRequest)
        return
    }
    if req.SKU == "" {
        http.Error(w, "SKU is required", http.StatusBadRequest)
        return
    }

    // ... business logic ...
}

After (With Required Fields):

type CreateProductRequest struct {
    Name  string!  `json:"name"`
    Price float64! `json:"price"` // Price must be explicitly provided, even if 0.0
    SKU   string!  `json:"sku"`
    Desc  string   `json:"description,omitempty"`
}

func CreateProductHandler(w http.ResponseWriter, r *http.Request) {
    var req CreateProductRequest
    err := json.NewDecoder(r.Body).Decode(&req)
    if err != nil {
        // If Name, Price, or SKU are missing from JSON, err will be non-nil
        // and indicate the missing required field.
        // Example: "json: missing required field 'name' for type main.CreateProductRequest"
        http.Error(w, fmt.Sprintf("Invalid request body: %v", err), http.StatusBadRequest)
        return
    }

    // Only additional business logic validation needed (e.g., Price > 0)
    if req.Price <= 0 { // This is a business rule, not just presence
        http.Error(w, "Price must be positive", http.StatusBadRequest)
        return
    }

    // ... business logic ...
}

This significantly reduces boilerplate and shifts the "missing field" check to the unmarshaler, making the code cleaner and less error-prone.

Example 2: Database Model Definition

Consider a Book struct where Title, AuthorID, and ISBN are always required.

Before (Current Go):

type Book struct {
    ID       int
    Title    string
    AuthorID int
    ISBN     string
    PublishedYear int
}

func CreateBook(db *sql.DB, book Book) error {
    // Manual validation
    if book.Title == "" || book.AuthorID == 0 || book.ISBN == "" {
        return fmt.Errorf("title, author ID, and ISBN are required")
    }

    // ... insert into DB ...
    return nil
}

func GetBookByID(db *sql.DB, id int) (*Book, error) {
    var book Book
    row := db.QueryRow("SELECT id, title, author_id, isbn, published_year FROM books WHERE id = ?", id)
    err := row.Scan(&book.ID, &book.Title, &book.AuthorID, &book.ISBN, &book.PublishedYear)
    if err != nil {
        if err == sql.ErrNoRows {
            return nil, fmt.Errorf("book not found")
        }
        return nil, fmt.Errorf("failed to scan book: %w", err)
    }
    // No explicit check for missing fields here, relies on DB schema NOT NULL constraints
    // and Go's zero values if a column is NULL.
    return &book, nil
}

After (With Required Fields):

type Book struct {
    ID       int! // Assuming ID is always generated/present
    Title!    string
    AuthorID! int
    ISBN!     string
    PublishedYear! int // Optional
}

func CreateBook(db *sql.DB, book Book) error {
    // Compile-time check ensures Title, AuthorID, ISBN are provided when 'book' is created
    // as a composite literal. If 'book' comes from an external source (e.g., JSON),
    // the unmarshaling would handle the presence check.
    // No need for explicit 'if book.Title == ""' checks here.

    // ... insert into DB ...
    return nil
}

func GetBookByID(db *sql.DB, id int) (*Book, error) {
    var book Book
    row := db.QueryRow("SELECT id, title, author_id, isbn, published_year FROM books WHERE id = ?", id)
    err := row.Scan(&book.ID, &book.Title, &book.AuthorID, &book.ISBN, &book.PublishedYear)
    if err != nil {
        // If a required column (e.g., title) is NULL in the DB, Scan would return an error
        // indicating a missing required field, similar to JSON unmarshaling.
        if err == sql.ErrNoRows {
            return nil, fmt.Errorf("book not found")
        }
        return nil, fmt.Errorf("failed to scan book: %w", err)
    }
    return &book, nil
}

This makes the struct definition itself a stronger contract, reducing the need for repetitive runtime checks and relying on the standard library's enhanced behavior.

Language Spec Changes

StructType = "struct" "{" { FieldDecl ";" } "}"
FieldDecl  = (IdentifierList RequiredMarker? Type | EmbeddedField) [ Tag ]

RequiredMarker = "!"

When a composite literal initializes a struct, fields are specified using field names.

For a field declared as a required field (i.e., FieldName! Type), it must be explicitly listed in the ElementList of the composite literal. Omitting a required field from a composite literal is a compile-time error.

For a field that is not declared as a required field, if it is omitted from the literal, it is initialized to its zero value.

Example:

type User struct {
    ID!       int
    Username! string
    Email!    string
    Age       int
}

u1 := User{ID: 1, Username: "alice", Email: "alice@example.com"} // Valid
u2 := User{ID: 2, Username: "bob"}                               // Compile-time error: missing required field 'Email'
u3 := User{ID: 3, Username: "charlie", Email: ""}               // Valid: Email explicitly provided as zero value

Informal Change

Alright class, let's talk about a really common problem we face when building applications in Go, especially when dealing with data coming from outside our program, like from a web API or a database.

Imagine you're defining a User struct for your application:

type User struct {
    ID       int
    Username string
    Email    string
    Age      int
}

Now, in your application's logic, you probably have some rules. For instance, an ID, Username, and Email are always required for a User to be considered valid. Age might be optional.

The Problem Today (Without Required Fields):

How do you enforce that ID, Username, and Email are always present?

  1. Manual Checks: You write if user.Username == "" { return errors.New("username required") } everywhere. This gets repetitive and you might forget a check.
  2. Constructor Functions: You create NewUser(id int, username, email string) (*User, error) functions, but this can be cumbersome for structs with many fields.
  3. Validation Libraries: You pull in a third-party library, which adds a dependency and its own learning curve.

This is a lot of boilerplate code just to say "this field must be here."

Introducing "Required Fields" (The Proposed Change):

What if Go itself could help us declare these "must-have" fields right in the struct definition? That's what this proposal is about!

The idea is simple: you add an exclamation mark ! right after the field's name to mark it as required.

So, our User struct would look like this:

type User struct {
    ID!       int     // ID is now a required field
    Username! string  // Username is now a required field
    Email!    string  // Email is now a required field
    Age       int     // Age remains optional
    Bio       string  // Bio remains optional
}

See that ! after ID, Username, and Email? That's the magic!

How Does This Work? (The "Under the Hood" Part):

This ! isn't just for show; it changes how Go treats these fields in two main scenarios:

  1. At Compile-Time (When You Write Your Code):

    • When you create a new User value using a composite literal (that's when you use {} to initialize a struct), the Go compiler will force you to provide values for all required fields.

    • Example: ```go u1 := User{ID: 1, Username: "alice", Email: "alice@example.com"} // ✅ This is perfectly fine! All required fields are there.

      u2 := User{ID: 2, Username: "bob"} // ❌ ERROR! The compiler will yell at you here. // It will say something like: "missing required field 'Email' in struct literal" `` * This is fantastic because it catches errors *before* your program even runs! You can't accidentally forget a required field when creating a new struct. * **Important Note:** For value types (likeint,string), "required" means "you must explicitly provide a value." It doesn't mean "it can't be its zero value." So,User{ID: 3, Username: "charlie", Email: ""}would still be valid if you explicitly setEmailto an empty string. The!just ensures you *thought* about it and provided *something*. For pointer or interface types, it would mean "must not benil`."

  2. At Runtime (When Your Program is Running):

    • This is where it gets really powerful for things like parsing JSON or reading from a database.
    • Go's reflect package (which libraries like encoding/json use to understand your structs) would be updated to know which fields have that ! marker.
    • Imagine json.Unmarshal: If you try to unmarshal a JSON payload into a User struct, and the JSON is missing a field like Email (which is marked Email!), json.Unmarshal would automatically return an error. You wouldn't need to write if user.Email == "" { ... } after unmarshaling! go jsonPayload := `{"ID": 1, "Username": "diana"}` // Missing "Email" var u User err := json.Unmarshal([]byte(jsonPayload), &u) if err != nil { fmt.Println("Error:", err) // This error would tell you "missing required field 'Email'" }
    • The same principle would apply to database/sql when scanning rows. If a required field corresponds to a NULL column in the database, Scan would return an error.

Why is This a Big Deal?

  • Less Boilerplate: You write less validation code. The language handles the "is it present?" check for you.
  • Fewer Bugs: Errors are caught earlier – either by the compiler or by standard library functions returning clear errors.
  • Clearer Intent: Your struct definition immediately tells anyone reading your code which fields are absolutely essential. It's self-documenting.
  • Go-Idiomatic: It integrates naturally with existing Go features like composite literals and the reflect package, making it feel like a natural extension of the language.

Important Caveats:

  • Not All Validation: This feature is about presence (or non-nil for pointers/interfaces). It doesn't replace all your validation. You'd still need to write checks for business rules like "Is Age greater than 0?" or "Is Email a valid email format?".
  • Post-Initialization Assignment: Once a struct is created, you can still assign a zero value or nil to a required field. The ! only enforces the initial creation/unmarshaling. go u := User{ID: 1, Username: "test", Email: "test@example.com"} // Valid u.Email = "" // This is allowed! The 'required' check was at creation. This is by design, giving you flexibility for internal logic where a struct might temporarily be in an "incomplete" state before final processing.

Backward Compatibility:

The best part? This change is designed to be backward compatible. If you have existing Go code with structs that don't use the ! marker, they will continue to work exactly as they do today. You only opt into this new behavior when you explicitly add the ! to your field names.

So, in essence, "required fields" give us a powerful, built-in way to define and enforce data contracts, making our Go code safer, cleaner, and more robust.

Is this change backward compatible?

Yes, this proposed change is designed to be backward compatible. The ! marker for required fields is an additive syntax feature, meaning it introduces new capabilities without altering the behavior of existing Go code. Any struct definitions currently in use that do not include the ! suffix will continue to compile and run exactly as they do today, with their fields behaving as optional. Only new or modified struct definitions that explicitly adopt the ! syntax will gain the compile-time enforcement for composite literals and the enhanced runtime validation behavior in standard library packages like encoding/json, ensuring that the Go 1 compatibility guarantee is upheld.

Orthogonality: How does this change interact or overlap with existing features?

The proposed "required fields" feature is designed to be largely orthogonal to most existing Go language features, meaning it introduces a new, independent dimension of expressiveness without fundamentally altering or overlapping with the core mechanics of other constructs.

For instance, it is entirely orthogonal to generics, as it operates solely on concrete struct types and their field definitions, having no bearing on type parameters, constraints, or generic programming. While it interacts with the reflect package by adding a new IsRequired field to reflect.StructField, this is an orthogonal extension of reflection's metadata capabilities, providing more information about a field's properties without changing how reflection fundamentally works or how types are represented. Similarly, its impact on error handling is orthogonal; it doesn't change Go's error propagation mechanism (if err != nil) but rather standardizes and shifts the source of certain errors (missing data) to earlier stages (compile-time for composite literals, or during standard library unmarshaling), thereby reducing boilerplate without altering the error interface or the fundamental approach to error handling. Even its interaction with zero values is orthogonal, as it doesn't eliminate them but rather dictates that a required field's zero value must be explicitly provided during initialization, maintaining the concept of zero values while adding a presence constraint. In essence, it adds a new, self-contained property to struct fields that complements existing features without creating complex interdependencies or breaking established paradigms.

Would this change make Go easier or harder to learn, and why?

Honestly, this change isn't going to make Go significantly harder to learn. Think of it less as adding a whole new complex system and more like adding a tiny, helpful little flag. You're just putting a ! next to a field name to say, "Hey, this one's important, don't forget it!" It's a very visual and intuitive way to express something developers already have to deal with constantly – making sure essential data is present. If anything, it makes the intent of your code clearer right from the start, which can actually make it easier to understand what a struct needs at a glance, rather than having to hunt through pages of validation logic. It's a small, practical addition that solves a common pain point, not a new paradigm to master.

Cost Description

Compiler Implementation: Lexer/Parser Changes: The Go compiler's lexer and parser will need to be updated to correctly recognize the Identifier! sequence within a FieldDecl without introducing ambiguities with existing grammar rules. This is a non-trivial but manageable change to the language's syntax parsing.

Type Checker Logic: The type checker will need new logic to enforce the "all required fields must be present" rule for composite literals. This involves iterating through the struct's fields and verifying that each !-marked field has a corresponding entry in the literal's ElementList. Reflection Metadata: The compiler's runtime component will need to populate the new IsRequired field within reflect.StructField for each struct type it compiles. This involves adding a new flag to the type information generated by the compiler.

Standard Library Adaptation: encoding/json: The json.Unmarshal function (and potentially json.Decoder) will need significant modifications to inspect the reflect.IsRequired flag for each field and return a specific error if a required field is missing from the JSON payload. This requires careful error handling design to provide clear, actionable messages. database/sql: The sql.Rows.Scan method would similarly need to be updated to check for NULL values in columns corresponding to required fields, returning an error in such cases. Other encoding packages (e.g., encoding/xml, encoding/gob) might also warrant similar updates for consistency, adding to the implementation burden.

Performance Impact (Minor): Compilation Time: The additional type checking logic and reflection metadata generation will add a very small, likely negligible, amount of overhead to compilation times. Runtime Performance: The runtime checks within json.Unmarshal or sql.Rows.Scan for required fields will introduce a tiny amount of overhead during deserialization. However, this is typically dwarfed by the I/O and parsing costs already involved in these operations. Maintenance Burden: The Go team will incur the ongoing cost of maintaining this new language feature, including documentation, bug fixes, and ensuring its continued compatibility and correct behavior with future language changes.

Changes to Go ToolChain

9

Performance Costs

CompileTime: Parser Overhead. Run Time: Changes to Unmarshalling!

Prototype

Here's how the ! implementation would work in short steps through the compiler:

  1. Lexing: The lexer identifies FieldName as an identifier, then ! as a distinct "bang" token, followed by Type.
  2. Parsing: The parser, upon seeing Identifier BANG Type in a struct field declaration, creates an Abstract Syntax Tree (AST) node for that field and sets an internal IsRequired flag on it to true.
  3. Type Checking:
    • When processing the struct definition, the type checker stores this IsRequired flag in its internal representation of the struct type.
    • When processing a composite literal for that struct, the type checker iterates through all fields marked IsRequired in its internal type definition. If any of these fields are missing from the literal's provided elements, it emits a compile-time error.
  4. Code Generation / Reflection: The code generator embeds the IsRequired flag from the AST directly into the runtime type information for that struct field, making it accessible via reflect.StructField.IsRequired at runtime.

Comment From: gabyhelp

Related Issues

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

Comment From: ianlancetaylor

Thanks for the extensive proposal. Besides the fact that has been proposed before, Go in general has a lot of ways to create the zero values of types. That is incompatible with anything like required fields. For example, see the discussion in #32076.

In the future consider discussing ideas on golang-nuts or some other forum (see https://go.dev/wiki/Questions) before diving into writing a lengthy proposal.