bhttp

package module
v0.0.80 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 11, 2025 License: Apache-2.0 Imports: 16 Imported by: 0

README

BHTTP - Binary HTTP Message Format

This package implements the Binary HTTP (BHTTP) message format as specified in RFC 9292. BHTTP is a simple binary format for representing HTTP requests and responses outside of the HTTP protocol so that they can be transformed or stored.

BHTTP is designed to convey the semantics of HTTP in an efficient way, it doesn't capture all the technical details of HTTP messages.

This package encoded/decodes *http.Request and *http.Response types to/from BHTTP.

These types don't always encode all the information required to construct a valid HTTP message. Normally a http.Server or http.Transport does some processing of these type before writing the actual HTTP messages to the wire.

For example, the *http.Request type has slightly different rules depending on it being used in a client-side or a server-side environment.

Similarly, this package needs to do some processing of these types before they can be encoded to a BHTTP message. The default behavior is similar to what you're used from net/http. See details below.

Features

  • Uses familiar *http.Request and *http.Response types.
  • User-provided mapping functions for full control of the encoding/decoding behavior.
  • Full implementation of RFC 9292 Binary HTTP message format including:
    • Known-Length and Indeterminate-length (streaming) messages.
    • Trailers: HTTP Headers after the body.
    • Padding: Messages can be padded so that their length is always a multiple of a specific nr.
    • Informational (1xx) responses: Not mapped by default, but can be extracted/included using custom mapping functions.

Installation

go get github.com/openpcc/bhttp

Usage

Encoding HTTP Requests
import (
    "bytes"
    "github.com/openpcc/bhttp"
    "net/http"
)

// Create a simple HTTP request
request, _ := http.NewRequest(http.MethodGet, "https://example.com/hello.txt", nil)
request.Header.Set("User-Agent", "custom-client/1.0")


// Encode the request to binary format
encoder := &RequestEncoder{}
msg, err := encoder.EncodeRequest(request)
if err != nil {
    // Handle error
}

// msg is an io.Reader with some additional methods to get info about the framing indicator
b, err := io.ReadAll(msg)
if err != nil {
    // Handle error
}

// b now contains all the bytes of the encoded request, but you probably want to
// write the msg itself wherever to not block on the request being finished.
Decoding HTTP Requests
import (
    "bytes"
    "github.com/openpcc/bhttp"
    "net/http"
)

ctx := context.Background()

// Assuming you have an io.Reader from somewhere
var encodedReq io.Reader

// or if you don't, get a reader for your bytes
var encBytes []byte
encodedReq = bytes.NewReader(encBytes)

decoder := &RequestDecoder{}
decodedReq, err := decoder.DecodeRequest(ctx, encodedReq)
if err != nil {
    // Handle error
}

// decodedReq now contains the decoded HTTP request
// once you have read all of decodedReq.Body, you know the whole req has come through
HTTP Responses

Encoding and decoding http.Response types work much the same

import (
    "bytes"
    "github.com/openpcc/bhttp"
    "net/http"
)

// Create a response
response := &http.Response{
    Status:     "OK",
    StatusCode: http.StatusOK,
    Header:     make(http.Header),
}
response.Header.Set("Content-Type", "text/plain")

// Encode the response
encoder := &ResponseEncoder{}
msg, err := encoder.EncodeResponse(res)
if err != nil {
    // Handle error
}

// The encoded message is a Reader so can be easily passed through to a decoder
decoder := &ResponseDecoder{}
ctx := context.Background()
decodedResp, err := decoder.DecodeResponse(ctx, msg)
if err != nil {
    // Handle error
}

// decoded Resp now contains an http.Response, with a Body ready for reading.

Default encoding/decoding behavior

  • When encoding from a *http.Request to BHTTP, it is interpreted as happening in a HTTP/1.1 client-side environment.
  • When decoding from BHTTP to a *http.Request, it is interpreted as happing in a HTTP1/1 server-side environment.
  • *http.Response carries no client-side or server-side distinction, it's always interpreted as a HTTP/1.1 environment.
  • *http.Request or *http.Response values that would normally result in HTTP messages with Transfer-Encoding: chunked will now result in indeterminate-length BHTTP messages. Note that these messages won't included any transfer-encoding, as this isn't supported by BHTTP.

Message Format Details

The binary format follows RFC 9292 specifications and includes:

  1. Framing indicator (indicates message type and length encoding)
  2. Control data (method, scheme, authority, path for requests; status code for responses)
  3. Header section
  4. Content
  5. Trailer section
  6. Optional padding
Frame Indicators
  • 0: Known-length request
  • 1: Known-length response
  • 2: Indeterminate-length request
  • 3: Indeterminate-length response

Important Notes

  1. Header Field Names: All header field names are automatically canonicalized according to HTTP/2 rules.

  2. Prohibited Fields: The following pseudo-header fields are not allowed:

    • :method
    • :scheme
    • :authority
    • :path
    • :status
  3. Empty Values: The package properly handles empty header values and trailers.

  4. Connection Headers: Headers related to connection management should not be included in binary messages.

  5. Maximum Payload Size: The default maximum payload size is 64MB. This can be modified by changing the MaxPayloadSize constant.

BHTTP Limitations

  • Does not support chunk extensions from HTTP/1.1
  • Does not preserve HTTP/1.1 reason phrases
  • Does not support header compression (HPACK/QPACK)
  • CONNECT and upgrade requests, while representable, serve no practical purpose in this format

License

This project is licensed under the Apache License 2.0.

Contributing

For guidelines on contributing to this project, please see CONTRIBUTING.md.

Development

Run tests with go test ./...

References

Documentation

Index

Constants

View Source
const DefaultEncodingBufferLen = 4096

DefaultEncodingBufferLen is the default buffer length

Variables

View Source
var (
	ErrTooMuchData = errors.New("too much data")
)

Functions

func DefaultRequestToHTTP

func DefaultRequestToHTTP(ctx context.Context, br *Request) (*http.Request, error)

DefaultRequestToHTTP is the default request mapping function used by the request decoder. It interprets BHTTP requests as unproxied HTTP/1.1 server side requests.

func MapToHTTP1Response

func MapToHTTP1Response(br *Response) (*http.Response, error)

MapToHTTP1Response maps a bhttp Response to a net/http Response

func RequestToHTTP1

func RequestToHTTP1(ctx context.Context, br *Request, usingProxy, serverSide bool) (*http.Request, error)

RequestToHTTP1 maps a bhttp Request to a net/http request. The request context will be set to the ctx parameter.

Types

type InformationalResponse

type InformationalResponse struct {
	StatusCode int
	// Header contains the headers as FieldLines for the BHTTP encoded message. This
	// usually differs from what a net/http response contains, as Go 	moves some headers
	// to/from fields. For example, the Host and Content-Length headers.
	Header http.Header
}

InformationalResponse contains control data and header of an informational response response.

type InvalidMessageError

type InvalidMessageError struct {
	Err error
}

func (InvalidMessageError) Error

func (e InvalidMessageError) Error() string

func (InvalidMessageError) Unwrap

func (e InvalidMessageError) Unwrap() error

type Message

type Message struct {
	// contains filtered or unexported fields
}

func (*Message) IsIndeterminateLength

func (m *Message) IsIndeterminateLength() bool

func (*Message) IsKnownLength

func (m *Message) IsKnownLength() bool

func (*Message) IsRequest

func (m *Message) IsRequest() bool

func (*Message) IsResponse

func (m *Message) IsResponse() bool

func (*Message) Read

func (m *Message) Read(p []byte) (int, error)

type Request

type Request struct {
	// KnownLength indicates whether the request should/is encoded as a known or indeterminate
	// length message.
	KnownLength bool
	// ContentLength is the Body length of a known length request. Ignored for indeterminate length messages.
	ContentLength int64
	// ControlData contains the RequestControlData. This data is passed as-is to the encoded/decoded messages,
	// mapping functions should take extra care to validate this data.
	ControlData RequestControlData
	// Header contains the headers that will be encoded to the BHTTP message.
	//
	// This might differ from what the original net/http request contains as net/http manages a few headers
	// behind the scenes.
	//
	// For example, while net/http takes the Content-Length header from the .ContentLength field on the original
	// request, the Content-Length header should be set in this map (if it's used).
	Header http.Header
	// Body usually reads the body of the original net/http request.
	Body io.Reader
	// Trailer works similar to the http.Request.Trailer field. It is up the MapFunc to:
	// - Initialize the keys of the Trailer map.
	// - Ensure that an appropriate Trailer header is added to the Header field.
	// - The values of the Trailer map are set when Body returns io.EOF.
	Trailer http.Header
}

Request is the BHTTP representation of a net/http request that the encoders and decoders work with.

func DefaultRequestFromHTTP

func DefaultRequestFromHTTP(hr *http.Request) (*Request, error)

DefaultRequestFromHTTP is the default request mapping function used by the request encoder. It interprets net/http requests as unproxied HTTP/1.1 client side requests.

func RequestFromHTTP1

func RequestFromHTTP1(hr *http.Request, usingProxy, serverSide bool) (*Request, error)

RequestFromHTTP1 interprets the provided net/http request as a HTTP/1.1 request and maps it to a bhttp Request.

Just like net/http this method will automatically handle a few headers when appropriate: - Host: always set based on .Host or .URL.Host. Matches :authority request control data. - Content-Length: for known length requests.

Note: Unless the caller explicitly sets .TransferEncoding, we don't add it.

type RequestControlData

type RequestControlData struct {
	// Method contains the :method pseudo-header according to RFC9113 Section 8.1.2.3.
	Method []byte
	// Scheme contains the :scheme pseudo-header according to RFC9113 Section 8.1.2.3.
	// Note: BHTTP RFC makes this required, so we can't omit it like in HTTP 2.
	Scheme []byte
	// Authority contains the :authority pseudo-header according to RFC9113 Section 8.1.2.3.
	// Note: BHTTP RFC requires us to provide an empty encoding when omitted.
	Authority []byte
	// Path contains the :scheme pseudo-header according to RFC9113 Section 8.1.2.3.
	// Note: BHTTP RFC makes this required, so we can't omit it like in HTTP 2.
	Path []byte
}

RequestControlData contains the control data for a request.

RFC9292 (BHTTP): The values of these fields follow the rules in HTTP/2 (Section 8.3.1 of HTTP/2) that apply to the ":method", ":scheme", ":authority", and ":path" pseudo-header fields, respectively. However, where the ":authority" pseudo-header field might be omitted in HTTP/2, a zero-length value is encoded instead.

Note: BHTTP RFC only specifies to encode a zero-length value when the authority field is omitted. However, in HTTP/2 :scheme and :path can also be omitted. We'll encode zero length values for these cases as well.

RFC9113 8.1.2.3: https://www.rfc-editor.org/rfc/rfc9113#name-request-pseudo-header-field

type RequestDecoder

type RequestDecoder struct {
	// MaxHeaderBytes is the maximum number of header bytes that can be read. Will default to 16KB.
	MaxHeaderBytes int64

	// MapFunc determines how a BHTTP request is interpreted. If this field is nil, the decoder will default
	// to interpreting the BHTTP request as a an unproxied HTTP/1.1 server side request.
	MapFunc RequestToHTTP
}

RequestDecoder decodes a BHTTP message to a net/http request.

An empty decoder is safe to use and is the recommended way to construct a new request decoder. An empty request decoder will: - Interpret the incoming BHTTP messages as unproxied HTTP/1.1 server side requests. - Allow for header sections of up to 16KB.

func (*RequestDecoder) DecodeRequest

func (d *RequestDecoder) DecodeRequest(ctx context.Context, r io.Reader) (*http.Request, error)

DecodeRequest decodes a request from the provided reader. The context of the request will be set to ctx.

type RequestEncoder

type RequestEncoder struct {
	// MapFunc maps a net/http Request to a BHTTP response. If this field is nil, [DefaultRequestFromHTTP] will be used
	// which interprets net/http requests as unproxied HTTP/1.1 client side requests.
	MapFunc RequestFromHTTP

	// PadToMultipleOf pads the message with zeroes until it reaches a multiple of this number. 0 will add no padding.
	PadToMultipleOf uint64

	// MaxEncodedChunkLen is the maximum length of indeterminate length content chunks (including their length prefix). MaxEncodedChunkLen
	// should be at least 2 bytes so that it will always fit a quicencoded integer with some data. If this field is 0, it
	// will default to 4096.
	MaxEncodedChunkLen int
	// contains filtered or unexported fields
}

RequestEncoder encodes net/http requests to bhttp messages.

An empty encoder is safe to use and is the recommended way to construct a new request encoder. An empty request encoder will:

  • Interpret the net/http requests as unproxied HTTP/1.1 client side requests.
  • Encode an indeterminate-length message where net/http would use chunked transfer encoding. BHTTP Message chunks will at most be 4096 bytes in length.
  • Not including padding in the message.

If you need different encoding logic, use NewKnownLengthRequestEncoder, NewIndeterminateLengthRequestEncoder or create a custom RequestEncoder by setting the fields below.

func NewIndeterminateLengthRequestEncoder

func NewIndeterminateLengthRequestEncoder() *RequestEncoder

NewIndeterminateLengthRequestEncoder returns an encoder that will encode all requests as indeterminate-length BHTTP messages, regardless of what the net/http request looks like.

Even requests where the Content-Length is known will be encoded as an indeterminate-length BHTTP message.

func NewKnownLengthRequestEncoder

func NewKnownLengthRequestEncoder() *RequestEncoder

NewKnownLengthRequestEncoder returns an encoder that will encode all requests as known-length BHTTP messages, regardless of what the the net/http request looks like. Even requests that would normally use Transfer-Encoding: chunked will be encoded as known-length BHTTP messages.

Note: this encoder might read the full body of the request into memory to determine its exact length.

func (*RequestEncoder) EncodeRequest

func (e *RequestEncoder) EncodeRequest(hr *http.Request) (*Message, error)

EncodeRequests encodes the provided net/http request as an bhttp message. The exact interpretation of the request depends on the MapFunc of this RequestEncoder, see that type for more details.

If this request has a body, the encoder will encode the body until EOF is encountered. It is the responsibility of the caller to close the body.

type RequestFromHTTP

type RequestFromHTTP func(r *http.Request) (*Request, error)

RequestFromHTTP maps a net/http request to a Request.

type RequestToHTTP

type RequestToHTTP func(ctx context.Context, r *Request) (*http.Request, error)

RequestToHTTP maps a Request to a net/http request.

type Response

type Response struct {
	// KnownLength indicates whether the response should/is encoded as a known or indeterminate
	// length message.
	KnownLength bool
	// ContentLength is the Body length of a known length request. Ignored for indeterminate length messages.
	ContentLength int64
	// Informational contains an array of InformationalResponses (1xx status codes and attendant headers)
	// They do not map to go Response objects, so would need to be manually encoded.
	Informational []InformationalResponse
	// FinalStatusCode contains the 2xx-5xx status code of the final response.
	FinalStatusCode int
	// FinalHeader contains the headers for the final response.
	//
	// This might differ from what the original net/http response contains as net/http manages a few headers
	// behind the scenes.
	//
	// For example, while net/http takes the Content-Length header from the .ContentLength field on the original response, the
	// Content-Length header should be set in this map (if it's used).
	FinalHeader http.Header
	// Body usually reads the body of the original net/http response.
	Body io.Reader
	// Trailer works similar to the http.Response.Trailer field. It is up the MapFunc to:
	// - Initialize the keys of the Trailer map.
	// - Ensure that an appropriate Trailer header is added to the Header field.
	// - The values of the Trailer map are set when Body returns io.EOF.
	Trailer http.Header
}

Response is the bhttp representation of a net/http response that the encoders/decoders work with.

A BHTTP encoded response represents a "resolved response", so it actually represents 0 or more informational 1xx responses followed by a final 2xx-5xx response.

func DefaultResponseFromHTTP

func DefaultResponseFromHTTP(hr *http.Response) (*Response, error)

DefaultResponseFromHTTP interpret the net/http response as an unproxied HTTP/1.1 client side responses.

func MapFromHTTP1Response

func MapFromHTTP1Response(hr *http.Response) (*Response, error)

MapFromHTTP1Response interprets the provided net/http response as a HTTP/1.1 response and maps it to a bhttp Response.

It will never set informational responses on the BHTTP response, as this information is not available in a net/http response.

Note: DetermineResponseContentLength might consume the first byte of the reader on http.Response to check if the body actually contains data. If you don't want hr to be modified, be sure to pass in a clone.

type ResponseDecoder

type ResponseDecoder struct {
	// MaxHeaderBytes is the maximum number of header bytes that can be read. Will default to 16KB.
	MaxHeaderBytes int64

	// MapFunc maps a [*Response] to a net/http Response. If this field is nil, [*Response] will be mapped to a
	// server-side, unproxied response.
	MapFunc ResponseToHTTP
}

func (*ResponseDecoder) DecodeResponse

func (d *ResponseDecoder) DecodeResponse(ctx context.Context, r io.Reader) (*http.Response, error)

DecodeResponse decodes a response from the provided reader. The context of the response will be set to ctx.

type ResponseEncoder

type ResponseEncoder struct {
	// MapFunc maps a net/http response . If this field is nil, DefaultResponseMapFunc will be used.
	MapFunc ResponseFromHTTP

	// PadToMultipleOf pads the message with zeroes until it reaches a multiple of this number. 0 will add no padding.
	PadToMultipleOf uint64

	// MaxEncodedChunkLen is the maximum length of indeterminate length content chunks (including their length prefix). MaxEncodedChunkLen
	// should be at least 2 bytes so that it will always fit a quicencoded integer with some data. If this field is 0, it
	// will default to 4096.
	MaxEncodedChunkLen int
	// contains filtered or unexported fields
}

ResponseEncoder encodes net/http responses to bhttp messages.

An empty encoder is safe to use and is the recommended way to construct a new response encoder. An empty request response will:

  • Interpret the net/http responses as unproxied HTTP/1.1 client side responses.
  • Encode an indeterminate-length message where net/http would use chunked transfer encoding. BHTTP Message chunks will at most be 4096 bytes in length.
  • Not including padding in the message.

If you need different encoding logic, use NewKnownLengthResponseEncoder, NewIndeterminateLengthResponseEncoder or create a custom ResponseEncoder by setting the fields below.

func NewIndeterminateLengthResponseEncoder

func NewIndeterminateLengthResponseEncoder() *ResponseEncoder

NewIndeterminateLengthResponseEncoder returns an encoder that will encode all requests as indeterminate-length BHTTP messages, regardless of what the net/http request looks like.

Even requests where the Content-Length is known will be encoded as an indeterminate-length BHTTP message.

func NewKnownLengthResponseEncoder

func NewKnownLengthResponseEncoder() *ResponseEncoder

NewKnownLengthResponseEncoder returns an encoder that will encode all responses as known-length BHTTP messages, regardless of what the the net/http response looks like. Even responses that would normally use Transfer-Encoding: chunked will be encoded as known-length BHTTP messages.

Note: this encoder might read the full body of the response into memory to determine its exact length.

func (*ResponseEncoder) EncodeResponse

func (e *ResponseEncoder) EncodeResponse(hr *http.Response) (*Message, error)

EncodeResponse encodes the provided net/http response as an bhttp message. The exact interpretation of the request depends on the MapFunc of this [ResponseEncoding], see that type for more details.

If this response has a body, the encoder will encode the body until EOF is encountered. It is the responsibility of the caller to close the body.

type ResponseFromHTTP

type ResponseFromHTTP func(r *http.Response) (*Response, error)

ResponseFromHTTP maps a net/http response to a Response.

type ResponseToHTTP

type ResponseToHTTP func(ctx context.Context, r *Response) (*http.Response, error)

ResponseToHTTP maps a Response to a net/http response.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL