Semantic Matter

Federated Data Layer (FDL)

Data FederationOne way to query across distributed sources

Query and govern distributed systems through a single interface, without centralizing all raw data. FDL provides federation-first access, metadata and semantic workflows, and policy checks at the gateway for cross-domain use.

Request walkthroughRead whitepaper

Differentiator

Federation-first approach

Register nodes, expose a unified API, and run queries across sources in parallel—without relying on copy-first pipelines.

Standards

SensorThings, OData query semantics, and FAIR metadata

Use standard query patterns and metadata practices so integrations stay portable.

Design principle

Keep ownership with the source teams

Share access and governance without taking over how each source is operated.

Why teams move to federation

Access is spread across many source systems

Governance differs from team to team

New integrations take too long to build and maintain

Trust is low without lineage and quality signals

Federation vs replication

Federation

One access layer, sources stay owned by their teams

  • Query across nodes without moving all raw data into a central store
  • Keep ownership and operations with the teams running each source
  • Reduce copy-heavy ETL when you need cross-domain access

Replication

Copy-first approach

  • Large data movement and lag between source and copy
  • Lineage and governance get duplicated across copies
  • Each new dataset adds integration work and maintenance

What’s built vs what’s next

A snapshot of what’s implemented in the UI today, and what’s still in progress.

Node registry UI

Implemented

Node cards, health states, protocol fields, and inspection views.

API contract docs

Implemented

REST endpoints, query options, and examples rendered in the docs UI.

Metadata + FAIR wizard

Implemented

Multi-step flow, readiness checks, and scoring are in place.

Parallel query orchestration

Illustrated

Shown in the architecture flow; the planner service is not in this repo.

Policy checks at the gateway

Illustrated

Described in the product flow; no runtime evaluator is included here.

Production connectors

Partial

Adapter contracts exist; several concrete connectors are still stubbed.

Architecture overview

User experience

  • Node operations and health status
  • Federated explorer and API guidance
  • Metadata, semantics, and workflow UI

Federation management

  • Node registry and connector setup
  • Metadata catalog and semantic mappings
  • Policies and workflow orchestration

Query runtime

  • Single API for federated queries
  • Plans queries across registered nodes
  • Normalizes and merges results before returning

Live federation views

Demonstrational screenshots of a UI application implemented on the federated data interfaces for showcasing connected knowledge, and FAIR alignment workflows.

Dataset discovery

Dataset discovery

Browse distributed datasets through a federated catalog view.

How a query runs

  1. 1Client sends a query with filters, selected fields, and paging.
  2. 2Gateway verifies authentication and checks policy scope.
  3. 3Planner selects eligible nodes from the registry.
  4. 4Subqueries run in parallel with pushdown where possible.
  5. 5Results are normalized, deduplicated, and merged.
  6. 6Response returns unified results plus lineage context.

Connectors and adapters

  • `CatalogAdapter` and `WorkflowsAdapter` define clean integration boundaries.
  • Adapter contracts map endpoints and translate payloads.
  • Mock and production modes support a staged rollout.
  • Add or upgrade connectors without changing the UI.

API examples

Examples that follow the FDL contract model and standard OData-style query parameters.

Federated list query

curl -H "Authorization: Bearer <token>" \
  "https://<fdl-host>/api/v1/Entity?$filter=status eq 'active'&$select=id,name,updatedAt&$top=100&$skip=0"

Publish observation

curl -X POST -H "Authorization: Bearer <token>" -H "Content-Type: application/json" \
  "https://<fdl-host>/api/v1/Observations" \
  -d '{"phenomenonTime":"2026-02-18T10:00:00Z","result":12.5,"Datastream":{"@iot.id":1}}'

Governance, security, and operations

Capture metadata with FAIR guidance and readiness scoring.

Track node health with clear degraded states.

Workflow runs follow a simple lifecycle: queued, running, failed, completed.

Operational targets: 99.9% gateway uptime, P95 query latency under 2s, freshness lag under 5 minutes.

Roadmap priorities

1.Build the production planner/executor with pushdown optimization.
2.Replace adapter stubs with contract-tested connectors.
3.Add a policy runtime for RBAC/ABAC with decision logging.
4.Add tracing, metrics, and SLO-based alerting.
5.Store wizard outputs in a versioned catalog service.
6.Add tenant-scoped registries, credentials, and policy namespaces.
7.Publish machine-readable API contracts (OpenAPI and SensorThings profiles).
8.Harden security with OIDC and encrypted connector secrets.

Move from prototype to production federation

The UI already covers node management, documentation, and standards-based governance. The next step is the production runtime: planner, policy checks, durable metadata services, and connector execution.

Start integrationExplore Blobfish