Internals: Overview

AST Parsing of Go Source

Gohandlers uses Go’s AST packages (go/parser and go/ast) to load your project’s Go source files. It parses an entire directory of Go files into an AST, treating your existing handler functions and data structs as the “spec” for the API. In other words, your Go code (handlers and types) is the single source of truth – there’s no separate schema file to maintain. The parser walks through all top-level declarations in the package, picking out struct type definitions and function definitions that look like HTTP handlers.

• Binding Structs: First, it identifies struct types that represent request or response payloads (often by naming convention – e.g. CreatePetRequest or CreatePetResponse). For each struct, it records its name and fields. Struct field tags are parsed here: for example, a field tagged route:"id" is recognized as a path parameter, query:"q" as a query param, json:"name" as part of the JSON body, etc. These tag annotations tell gohandlers where each field’s value comes from. In the AST, the tool reads the literal tag strings on each ast.Field and categorizes them accordingly (route, query, form, JSON). This information is stored in an internal descriptor for the struct (often called a BindingTypeInfo). The descriptor holds lists of fields per category (e.g. all Route fields vs. Query fields) along with their names and types. This way, gohandlers knows which fields should be pulled from URL path segments, which from r.URL.Query(), which from form data, and which from a JSON request body (gohandlers/docs/commands/bindings.md at dev · ufukty/gohandlers · GitHub).

• Handler Functions: Next, it inspects function declarations (including methods) to find HTTP handlers. Gohandlers doesn’t require you to call any framework – it infers handlers by pattern. Typically, any function whose name matches a binding struct’s prefix is treated as a handler for that endpoint. For example, if it finds a CreatePetRequest struct, a method or function named CreatePet is assumed to be the handler using that request type. It also handles both function handlers and methods (receiver functions) seamlessly (a recent fix ensured it works whether your handler is a function or a method on a struct). For each handler ast.FuncDecl, gohandlers pulls documentation comments and any special directives.

  Doc Comments as Directives: Documentation above handlers can provide extra hints like HTTP method or explicit route path. For instance, a comment like // GET /pets/{id} would be parsed so that Method="GET" and Path="/pets/{id}" are attached to that handler’s info. Internally, gohandlers’ parser splits the doc comment into words and checks for known patterns: if the first word is an HTTP method (GET, POST, etc.), it assigns that as the handler’s method (pkg/inspects: changes the gh:X directives in doc comments to be pro… · ufukty/gohandlers@4a46ecd · GitHub). If the second token is a path starting with /, it records that as the route path. Gohandlers also supports special flags in comments, prefixed with gh:. For example, // gh:ignore on a handler will mark it to be skipped during generation (pkg/inspects: changes the gh:X directives in doc comments to be pro… · ufukty/gohandlers@4a46ecd · GitHub). These “mode” directives (like gh:ignore, and others such as gh:list for list endpoints) are parsed at the start of comments and saved in a Doc struct along with any method/path info. This Doc metadata is later used to adjust code generation – e.g. skipping an endpoint, or treating a “list” endpoint slightly differently.

After this parsing phase, the tool builds an internal model of the API:

• It knows all binding structs (request/response types) and the purpose of each field (via tags).
• It knows all handlers, their names, and (if provided or inferable) their HTTP method and route path. If the comment didn’t specify an HTTP method, gohandlers can infer one by naming conventions (e.g. prefix “Get” implies GET, “Create” implies POST, etc., as an automatic heuristic) (History for pkg - ufukty/gohandlers · GitHub) (History for pkg - ufukty/gohandlers · GitHub). If no path is given in comments, it can derive a default path: it will kebab-case the handler name or use the binding struct name to form a URL path, and include placeholders for any route parameters. All these decisions (method and path determination) are encoded in the handler’s descriptor.

The discovered handlers are then paired with their corresponding request/response structs by name. Gohandlers enforces a naming convention for this pairing: for a handler named X, it expects an XRequest and XResponse struct (defined in the code) to exist. The internal descriptor for a handler will link to the descriptors of its request and response binding types. (There’s logic to handle edge cases – e.g. if two handlers share a name in one file, or a handler with no request body – to ensure correct matching.)

Building Data Descriptors

Once the AST is processed, gohandlers constructs higher-level descriptor objects that drive code generation. Key components of this internal model include:

• Receiver (optional): If your handlers are methods of a struct (say a service or controller struct), the tool notes the receiver type. It groups handlers by receiver so it can, for example, allow generating code only for a specific service (via a CLI flag). The receiver info isn’t directly part of generation logic except for scoping, but it’s tracked in the descriptor mapping.

• BindingTypeInfo: For each binding struct (request/response), this descriptor holds the structured info about its fields. For example, BindingTypeInfo.Params.Route might be a slice of all route param fields (with their names and types), similarly Params.Query, Params.Form, and Params.JSON for other categories. This makes it easy to iterate over “all query params” vs “all JSON fields” when writing the template logic. The descriptor may also include flags like whether a JSON body is present, etc. Basic type info (like whether a field’s type is a custom bool wrapper or an int, etc.) is recorded to know what conversion is needed. (Gohandlers even provides some custom types in pkg/types/basics – e.g. types.Boolean, types.Int – which are helpers to parse and validate primitives from strings. These can be used in your binding structs for extra type safety. If used, the generation will call their helper methods (like .Validate()) instead of raw strconv on every field.)

• Handler Info: For each handler function, the tool creates a descriptor (let’s call it HandlerInfo) capturing the endpoint’s key data. This includes the handler name, the HTTP method, the URL path template, and references to its request/response BindingTypeInfo. The HandlerInfo is populated using both code analysis and doc directives: e.g. if a doc comment specified a custom path or a mode (like ignore), that’s reflected in this object. If not, defaults from naming conventions and the content of the BindingTypeInfo (like presence of body fields) are used to decide things like HTTP method. Any route placeholders in the path are cross-checked against the Route fields in the BindingTypeInfo to ensure they match.

All these descriptors are stored in data structures (maps/slices) within the inspects package. For example, after inspection, you might have a map of receiver -> handlers, where each handler entry contains its HandlerInfo (with links to binding info). This structured representation of “what endpoints exist and what data they expect/produce” is now ready to feed into code generation.

Tag Extraction and Validation

A crucial part of building the descriptors is extracting struct tags and validating that they make sense. Gohandlers goes field-by-field in each binding struct and parses the tag string (the content inside backticks). It looks for known keys like route, query, json, and form. If a field has a json:"..." tag, it’s treated as part of the JSON payload; if it has query:"...", it’s marked as coming from the URL query parameters, and so on. This multi-source handling is done without runtime reflection – it’s all determined at compile time and hard-coded into the generated code. For instance, if XRequest has ID string \route:“id”` and Name string \query:“name”` defined, the generator knows that on an incoming request, ID must be pulled from the URL path and Name from the query string. The documentation confirms this: “Fields tagged route:"id" will be extracted from the URL path. Fields tagged query:"q" come from r.URL.Query(), and json:"field" from the JSON body, etc.” (gohandlers/docs/commands/bindings.md at dev · ufukty/gohandlers · GitHub). The tool ensures each field is accounted for in one of these sources. (If a field is tagged in a way gohandlers doesn’t expect, or if required tags are missing for a given context, it will likely skip or raise an error via the CLI validate command – but generally the convention is that every field in a binding struct should have a recognized tag indicating where to get/set its value.)

Struct tags vs. Content-Type: Based on the tags present, gohandlers also deduces what content types an endpoint uses. For example, if a request struct has any json:"..." fields (and no form:"..."), the request body is assumed to be JSON. Conversely, if it has form:"..." fields, it expects application/x-www-form-urlencoded form data (the tool added support for parsing form fields in addition to JSON). It will generate different code paths depending on this – e.g. calling r.ParseForm() for form data versus decoding a JSON body for JSON fields. It does not mix JSON and form in one request; one struct is generally one content type for the body. Route and query tags can coexist with either, of course. This analysis is done up front so that the code generator knows whether to include form-parsing logic, JSON unmarshaling, both, or neither for each binding. (Notably, earlier versions even had experimental support for multipart/form-data with file uploads, with tags like part and file, but that was removed as it added complexity with minimal benefit (History for pkg - ufukty/gohandlers · GitHub) (History for pkg - ufukty/gohandlers · GitHub).)

Before generation, gohandlers effectively has a complete picture of each binding struct (the “shape” of the data and where it comes from/goes) and each handler (the endpoint details and linked data types). Now it’s ready to produce code.

Template Application & Code Generation

No Reflection – Static Code: Gohandlers emphasizes type-safe, static code generation – it produces plain Go code that directly marshals/unmarshals data, instead of using reflection at runtime. This means the generation step is essentially writing out boilerplate code (like calls to strconv.Atoi, JSON encoder/decoder, etc.) tailored to your types. The generated code is placed into new Go files (by default with a .gh.go suffix). For example, running the bindings command creates a file (e.g. bindings.gh.go) in your project. This file is marked as codegen output (often you’ll see a comment at the top indicating “do not edit”).

Under the hood, gohandlers uses the collected descriptors to either fill in templates or programmatically construct Go AST nodes for the new code. In fact, much of the code generation is done by building an AST of the output file and then formatting it to source code. This approach helps manage imports and code formatting automatically. For instance, the generator will create an ast.File for the new code, set the package name to match your package, and then populate it with declarations: functions for each Parse/Build/Write method, as well as any necessary import specs (like "net/http", "encoding/json", "strconv", etc.). Using the AST, it ensures that an import is added only if needed (e.g. it adds the "net/url" import if and only if some binding has query or form fields that require URL encoding). Finally, it renders the AST to a .go file – effectively compiling the template into real Go source. The result is deterministic, properly formatted (via gofmt under the hood), and ready to compile.

Generating Parse Methods: For every binding struct, gohandlers generates a Parse method that knows how to populate that struct from an HTTP message. Request structs get a Parse(*http.Request) method, and response structs get a Parse(*http.Response) method (gohandlers/docs/commands/bindings.md at dev · ufukty/gohandlers · GitHub). These methods are essentially reading from the raw http.Request/Response and filling in the struct’s fields:

• For each route:"..." field, the generated code will extract the path variable from the URL. Typically, it uses the known path template to locate the segment. (If your path is /pets/{id}, it will split the URL path and assign the id segment to the struct field, performing type conversion if the field is not a string.)
• For each query:"..." field, it calls r.URL.Query().Get("...") to get the query parameter string, then converts it to the field’s type. For basic types, code uses strconv (e.g. strconv.Atoi for ints, etc.) and handles errors if the conversion fails. If the field is a custom type (like the provided types.Int), it might call a helper like types.Int.Validate() to check the string format. Missing required query params or invalid values result in an error being returned by Parse.
• For form:"..." fields, the code first invokes r.ParseForm() (to parse URL-encoded form body), then accesses r.Form or r.PostForm to get the values. Each form field is pulled out and converted similarly to query params.
• For json:"..." fields, the code uses the standard JSON library. Typically it will do something like: decoder := json.NewDecoder(r.Body) and decode into the struct (or into just the JSON-targeted portion). Because the struct itself might contain non-JSON fields (route/query), a common pattern is to decode into the same struct – JSON will populate the JSON-tagged fields and ignore others (or the generator may embed an anonymous struct for the JSON part). In any case, it ensures the JSON body is read and parsed. Errors in JSON decoding (malformed JSON) are handled by returning an error.

All these operations are assembled into the Parse function body. The end result is that calling, say, err := req.Parse(r) on an incoming *http.Request will fill your CreatePetRequest struct from r completely – reading path, query, form, and JSON parts as needed (gohandlers/docs/commands/bindings.md at dev · ufukty/gohandlers · GitHub). If anything is missing or invalid, the Parse method returns an error describing the problem (e.g. "invalid id parameter").

Generating Build and Write: Gohandlers also creates output methods: a Build method for request structs (to turn a struct into an outgoing *http.Request), and a Write method for response structs (to write the struct’s data into an http.ResponseWriter in a handler). These are essentially the inverse of Parse:

• Build(host string) (*http.Request, error) is generated for each request binding. It knows the HTTP method and path for the corresponding handler (recorded in the HandlerInfo). Inside, it will construct the full URL: it takes the provided host (which could be a base URL or host name) and appends the endpoint’s path. If the path had {id} placeholders, it replaces them with the actual values from the struct’s route fields. Query fields are added to the URL query string (using net/url to ensure proper encoding). For the body, if the struct has JSON fields, the Build method will marshal the struct (or a portion of it) to JSON (using json.Marshal or NewEncoder). It then creates an http.Request with the correct method and body (setting the appropriate Content-Type header, e.g. application/json). If the struct has form fields instead, it will URL-encode those into the request body (using url.Values.Encode()), set Content-Type: application/x-www-form-urlencoded, and so on. The method returns the constructed *http.Request ready to send. This means your client code can simply do req := CreatePetRequest{...}; httpReq, err := req.Build(baseURL) and get a fully prepared request.
• Write(w http.ResponseWriter) error is generated for each response binding. This takes the data in the struct and writes it out to the HTTP response. The code will set headers (e.g. content type) and encode the body. In most cases, response bindings are meant to be JSON – the generated Write will set Content-Type: application/json and do json.NewEncoder(w).Encode(resStruct) (or manual marshaling) (gohandlers/docs/commands/bindings.md at dev · ufukty/gohandlers · GitHub). If there are form tags in the response (less common for APIs), it could similarly form-encode the response. The Write method does not typically set the status code (that is left to the handler, or assumed to be 200 OK by default); it focuses on the body. After calling res.Write(w), the HTTP response will contain the JSON (or form) representation of your response struct, with all fields serialized appropriately.

Additionally, for client-side usage, the tool generates a Parse method on response types as mentioned (to read *http.Response). That method would check the HTTP status and content type, then parse the body accordingly (e.g. decode JSON into the struct). This pairs with the Build method to simplify writing client libraries.

Internally, these methods are created using templates that iterate over the collected field info. Gohandlers ensures that the correct conversion or encoding logic is used for each field. Because this code is generated ahead of time, it’s fully type-checked by the compiler. There’s no runtime reflection – for example, instead of using reflect.Value to set a field, the generated code simply does req.ID = id (with id coming from strconv.Atoi(r.URL.Query().Get("id")) for instance). This approach yields zero reflection overhead and catches type errors at compile time. The maintainers describe this as “type safety, no reflection” in contrast to frameworks that might use generic maps or interface{} for handlers (GitHub - ufukty/gohandlers: Skip the boilerplate. Generate Go handler binding type parser and writer methods with type safety and zero reflection. Make sure each build registers all handlers to router. Always keep the client code of all microservices up-to-date with latest handler parameters. Go framework-less).

Finally, once all the method code is generated, gohandlers writes out the new Go file. The output file will include the package declaration, necessary import statements, and the definitions of each generated method. If you open the generated bindings.gh.go, you’ll see functions like:

func (req *CreatePetRequest) Parse(r *http.Request) error { /* ... */ }
func (req CreatePetRequest) Build(host string) (*http.Request, error) { /* ... */ }
func (res *CreatePetResponse) Parse(resp *http.Response) error { /* ... */ }
func (res CreatePetResponse) Write(w http.ResponseWriter) error { /* ... */ }

along with any helper code needed. Thanks to the structured generation, even error messages are consistent (the tool may include formatted error strings indicating which step failed, e.g., an error wrapping "ParseForm: %w" to pinpoint form parsing issues in the request pipeline).

Throughout this pipeline, key packages and types keep the process organized. The pkg/inspects package is responsible for AST parsing and building the handler/type info (types like inspects.Info for handler metadata and inspects.BindingTypeInfo for struct field data). The command implementations (e.g. cmd/gohandlers/commands/bindings.go) call into inspects to get the model, then use it to generate code (often by constructing new AST nodes for functions, statements, etc.). The result is a set of Go source files that glue your handlers and data together. In summary, gohandlers goes from parsing your source, to modeling your API’s endpoints and fields, to emitting new Go code – all in one run. It automates the boilerplate of request parsing and response writing by leveraging your code’s structure and a robust AST-driven template engine. The generated code uses your struct tags as directives for where to get data, ensuring that if you update a tag or add a new field, regenerating will update all the parsing logic accordingly (gohandlers/docs/commands/bindings.md at dev · ufukty/gohandlers · GitHub). This architecture allows your server code to truly serve as the spec for the API, with gohandlers doing the heavy lifting to keep everything in sync and type-safe.

Sources: The gohandlers README and docs provide insight into these internals and confirm the behavior of generated methods and tag usage (gohandlers/docs/commands/bindings.md at dev · ufukty/gohandlers · GitHub) (gohandlers/docs/commands/bindings.md at dev · ufukty/gohandlers · GitHub) (gohandlers/docs/commands/bindings.md at dev · ufukty/gohandlers · GitHub). The implementation uses AST parsing of Go code and comment directives (pkg/inspects: changes the gh:X directives in doc comments to be pro… · ufukty/gohandlers@4a46ecd · GitHub) to build its internal model, and then generates code accordingly, as described above. This design achieves the goal stated by the project: up-to-date handler bindings and client code with zero manual coding of boilerplate and no runtime reflection (GitHub - ufukty/gohandlers: Skip the boilerplate. Generate Go handler binding type parser and writer methods with type safety and zero reflection. Make sure each build registers all handlers to router. Always keep the client code of all microservices up-to-date with latest handler parameters. Go framework-less).