weatherfeeder/internal/normalizers/doc.go

// Package normalizers defines weatherfeeder’s **prescriptive** conventions for
// writing feedkit normalizers and provides the recommended project layout.
//
// Summary
// -------
// weatherfeeder ingests multiple upstream providers whose payloads differ.
// Sources should focus on polling/fetching. Normalizers should focus on
// transforming provider-specific raw payloads into canonical internal models.
//
// This package is domain code (weatherfeeder). feedkit’s normalize package is
// infrastructure (normalizer contracts + processor).
//
// Directory layout (required)
// ---------------------------
// Normalizers are organized by provider:
//
//	internal/normalizers/<provider>/
//
// Example:
//
//	internal/normalizers/nws/observation.go
//	internal/normalizers/nws/types.go
//	internal/normalizers/nws/wmo_map.go
//	internal/normalizers/openweather/observation.go
//	internal/normalizers/openmeteo/observation.go
//	internal/normalizers/common/units.go
//
// Rules:
//
//  1. One normalizer per file.
//     Each file contains exactly one Normalizer implementation (one type that
//     satisfies feedkit/processors/normalize.Normalizer).
//     Helper files are encouraged (types.go, common.go, mapping.go, etc.) as long
//     as they do not define additional Normalizer types.
//
//  2. Provider-level shared helpers live under the provider directory:
//     internal/providers/<provider>/
//
//     Use this for provider-specific quirks that should be shared by BOTH sources
//     and normalizers (time parsing, URL/unit invariants, ID normalization, etc.).
//     Keep these helpers pure (no I/O) and easy to test.
//
//     You may use multiple helper files (recommended) when it improves clarity:
//     - types.go        (provider JSON structs)
//     - common.go       (provider-shared helpers)
//     - mapping.go      (provider mapping logic)
//
//  3. Cross-provider helpers live in:
//     internal/normalizers/common/
//
//     Prefer extracting small, pure helpers here when they are reused by ≥2 providers.
//     Keep these helpers:
//     - deterministic (no I/O)
//     - side-effect free
//     - easy to read (avoid clever abstractions)
//
//  4. Matching is standardized on Event.Schema.
//     (Do not match on event.Source or event.Kind in weatherfeeder normalizers.)
//
// Schema conventions (required)
// -----------------------------
// Sources emit RAW events with provider-specific schemas.
// Normalizers convert RAW -> CANONICAL schemas.
//
// Raw schemas:
//
//	raw.<provider>.<thing>.vN
//
// Canonical schemas:
//
//	weather.<kind>.vN
//
// weatherfeeder centralizes schema strings in standards/schema.go.
// Always use those constants (do not inline schema strings).
//
// Example mappings:
//
//	standards.SchemaRawOpenWeatherCurrentV1 -> standards.SchemaWeatherObservationV1
//	standards.SchemaRawOpenMeteoCurrentV1   -> standards.SchemaWeatherObservationV1
//	standards.SchemaRawNWSObservationV1     -> standards.SchemaWeatherObservationV1
//
// Normalizer structure (required template)
// ----------------------------------------
// Each normalizer file must follow this structure (with helpful comments):
//
//	type OpenWeatherObservationNormalizer struct{}
//
//	func (OpenWeatherObservationNormalizer) Match(e event.Event) bool { ... }
//
//	func (OpenWeatherObservationNormalizer) Normalize(ctx context.Context, in event.Event) (*event.Event, error) {
//	    // 1) Decode raw payload (recommended: json.RawMessage)
//	    // 2) Parse into provider structs
//	    // 3) Map provider -> canonical internal/model types
//	    // 4) Build output event (copy input, modify intentionally)
//	    // 5) Set EffectiveAt if applicable
//	    // 6) Validate out.Validate()
//	}
//
// Required doc comment content
// ----------------------------
// Every normalizer type must have a doc comment that states:
//
//   - what it converts (e.g., “OpenWeather current -> WeatherObservation”)
//   - which raw schema it matches (constant identifier from standards)
//   - which canonical schema it produces (constant identifier from standards)
//   - any special caveats (units, day/night inference, missing fields, etc.)
//
// Including literal schema string values is optional,
// but the constant identifiers are required.
//
// Event field handling (strong defaults)
// --------------------------------------
// Normalizers should treat the incoming event envelope as stable identity and
// should only change fields intentionally.
//
// Default behavior:
//
//   - Keep: ID, Kind, Source, EmittedAt
//   - Set:  Schema to the canonical schema
//   - Set:  Payload to the canonical payload (internal/model/*)
//   - Optional: EffectiveAt (often derived from observation timestamp)
//   - Avoid changing Kind unless you have a clear “raw kind vs canonical kind” design.
//
// Always validate the output event:
//
//	if err := out.Validate(); err != nil { ... }
//
// Payload representation for RAW events
// -------------------------------------
// weatherfeeder recommends RAW payloads be stored as json.RawMessage for JSON APIs.
// This keeps sources small and defers schema-specific decoding to normalizers.
//
// If a source already decodes into typed provider structs, it can still emit the
// raw event; it should simply re-marshal to json.RawMessage (or better: decode
// once in the normalizer instead to keep “fetch” separate from “transform”).
//
// Registration pattern
// --------------------
// feedkit normalization uses an ordered normalizer list ("first match wins").
//
// Provider subpackages should expose:
//
//	func Register(in []normalize.Normalizer) []normalize.Normalizer
//
// And internal/normalizers/builtins.go should provide one entrypoint:
//
//	func RegisterBuiltins(in []normalize.Normalizer) []normalize.Normalizer
//
// which appends each provider's normalizers in a stable order and is then passed
// to normalize.NewProcessor(...).
//
// Normalizer ordering
// -----------------------------
// feedkit normalization is "first match wins" by list order.
// Therefore order matters:
//
//   - Register more specific normalizers before more general ones.
//   - Avoid “catch-all” Match() implementations.
//   - Keep Match() cheap and deterministic (Schema equality checks are ideal).
//
// Reuse guidance (strong recommendation)
// --------------------------------------
// Before adding provider-specific logic, check internal/normalizers/common for an
// existing helper (payload extraction, unit conversions, text fallbacks, etc.).
// If you discover logic that could potentially apply to another provider, prefer extracting
// it into internal/normalizers/common as appropriate.
//
// Testing guidance (recommended)
// ------------------------------
// Add a unit test per normalizer:
//
//	internal/normalizers/openweather/observation_test.go
//
// Tests should:
//
//   - build a RAW event with Schema=standards.SchemaRaw... and Payload=json.RawMessage
//   - run the normalizer
//   - assert canonical Schema + key payload fields + EffectiveAt
//   - assert out.Validate() passes
package normalizers