Building a Provider

Adding a health data source to Mirobody means writing one Python class — a BasePullProvider subclass — in a directory the engine scans at startup. There is nothing to compile, no registry to edit, and no core file to touch: the loader finds your class, calls its factory, and registers whatever comes back.

Note

This page is the how-to. The contract behind it — the platform/provider split, the base class, link types, and the discovery rules — is explained in Pulse Provider System. For a finished implementation read end to end, see Garmin Provider Example.

Prerequisites

A running source checkout

A virtual environment with pip install -e ., plus Postgres and Redis from docker compose up -d pg redis. Redis is not optional if your source uses OAuth — the temporary authorization state lives there. See Development Setup.

The source's API reference

The base URL, the auth scheme, the endpoints for the data you want, and the response shapes. Everything you write is a transformation of those payloads, so capture a few real responses early — they become your test fixtures.

Somewhere to put credentials

Client ids, secrets and base URLs are read from the config through safe_read_cfg("YOUR_KEY"), never from the environment directly. Any key whose name contains _KEY, _PASSWORD, _PASS, _PWD, _SECRET, _SK or _TOKEN is encrypted at rest. See Configuration.

Lay out the directory

At startup the engine walks every entry of PROVIDER_DIRS, which ships as mirobody/pulse/providers plus the repository-root providers/. Put your own provider in providers/ so upgrades never touch it:

providers/ any directory listed in PROVIDER_DIRS

mirobody_acme/ the directory: mirobody_<slug>

__init__.py

provider_acme.py the module: provider_<something>.py

Four rules decide whether your provider exists at all: the loader globs mirobody_*/provider_*.py and then looks for a class that subclasses BasePullProvider (the class name only pre-filters on Provider):

Rule	Value
Directory	mirobody_<slug>/
Module	provider_<something>.py
Class	<Something>Provider(BasePullProvider)
Factory	create_provider(config) returning an instance, or None to stay disabled

Caution

Loading is best-effort: a provider that raises on import or in its factory is logged as a warning and skipped, and the server still starts. If your provider never appears, read the startup log before suspecting anything else.

The smallest complete provider in the tree is the PostgreSQL one — no vendor API, and it implements exactly the required surface. Read it as a template.

Only three methods are yours to implement: info, save_raw_data_to_db and is_data_already_processed. Everything else either has a working default on the base class or raises an error naming the method you left out.

The factory

create_provider(config) is a classmethod, and its only job is a feasibility check: whether this provider can work in the current environment. Returning None is the supported way to stay out of the registry — that is how an unconfigured integration disappears instead of failing at request time.

Both patterns apply, depending on the integration: gate on a feature flag when it is opt-in, and gate on the credentials themselves when it requires them. ENABLE_PGSQL_DEVICE is not present in the shipped config.yaml, which is precisely why the PostgreSQL provider is off until you add the key.

mirobody_acme/provider_acme.py

class AcmeProvider(BasePullProvider):
    @classmethod
    def create_provider(cls, config: dict) -> Optional["AcmeProvider"]:
        client_id = config.get("ACME_CLIENT_ID")
        if not client_id:          # not configured → this provider does not exist
            return None
        return cls(config)

Declare the metadata

info returns a ProviderInfo and must not touch the network — listing providers is supposed to be free. The two interesting fields are auth_type and connect_info_fields, and they travel together.

For LinkType.CUSTOMIZED, you describe a form and the frontend renders it; the values arrive back under credentials["connect_info"]. The PostgreSQL provider declares five fields — username, password, host, port, database — of which two are shown here.

An OAuth provider declares no fields at all — there is no form, only a redirect.

Tip

Prefix your slug with theta_. POST /api/v1/pulse/user/providers/link reads the prefix and rewrites the platform to theta on its own, so a slug without it forces callers to name the platform correctly by hand.

Validate credentials

For PASSWORD and CUSTOMIZED providers, BasePullProvider.link() calls _validate_credentials_v2(credentials) before it stores anything, and a raised exception is the whole failure protocol. Raise ValueError for “your input is wrong” and RuntimeError for “the source is unreachable” — the message reaches the caller.

Note

_validate_credentials(username, password) is the v1 interface and still exists. Override _validate_credentials_v2 instead — it receives the whole credentials dict, which is the only way to see connect_info.

OAuth authorization

OAuth providers take a different path: link() returns a link_web_url for the browser, and the connection is finished later by a callback route. Both OAuth versions are represented in the tree — Whoop and Oura are OAuth 2.0, Garmin is OAuth 1.0a — and neither of them stores state in process memory. The temporary handshake state lives in Redis, so the browser can come back to a different instance than the one that started the flow.

OAuth 2.0 with the shared client

Don’t hand-roll the flow: the engine ships a reusable OAuth 2.0 client, and a provider wires it in by composition rather than inheritance.

refresh_extra_params is the escape hatch for sources that deviate from RFC 6749 on the refresh grant — Whoop insists on scope being resent, so it goes there instead of into a forked copy of the client.

Stage one builds the authorization URL and parks the handshake state in Redis, with a TTL from OAUTH_TEMP_TTL_SECONDS (default 900 seconds).

The state carries the caller’s return_url so the browser can be sent home afterwards, and the redirect_uri is stored beside it because the token endpoint has to be given the same value it saw during authorization.

Stage two exchanges the code. exchange_code_for_tokens takes and clears that handshake state, exchanges the authorization_code grant, computes the expiry and stores the credentials encrypted. Your provider only wires the two ends together.

Refreshing is not your problem either. Call get_valid_access_token(user_id, provider_slug, db_service) whenever you need a token: it returns the stored one while more than five minutes of life remain, otherwise spends the refresh token, saves the new pair, and hands back the fresh access token. With no refresh token stored it returns None, which is the signal that the user must re-authorize.

OAuth 1.0a (Garmin)

OAuth 1.0a has an extra round trip and signs every request, so it does not reuse the client above. There is an implementation to copy from: the Garmin Provider Example. What you owe is still one callback method, taking (oauth_token, oauth_verifier).

Note

The request signing here is for outgoing requests. It is not webhook verification — see below.

The callback route

You do not add a route. One route serves every provider, and it dispatches on info.auth_type.

So your obligation is a callback method with the right arity: (code, state) for OAuth 2.0, (oauth_token, oauth_verifier) for OAuth 1.0a. The route is GET /api/v1/pulse/{platform}/{provider}/callback. When no return_url was supplied it answers with a small HTML page that notifies the opener and closes the popup. For the linking flow end to end, see Using Providers.

Configuration keys

Per-provider keys are read with safe_read_cfg at construction time, and the endpoint URLs have defaults so only the three secrets are mandatory. What config.yaml ships:

config.yaml

GARMIN_TOKEN_URL: https://connectapi.garmin.com/oauth-service/oauth/request_token
GARMIN_AUTH_URL: https://connect.garmin.com/oauthConfirm/
GARMIN_ACCESS_TOKEN_URL: https://connectapi.garmin.com/oauth-service/oauth/access_token
GARMIN_API_BASE_URL: https://apis.garmin.com/wellness-api/rest
OAUTH_TEMP_TTL_SECONDS: 900
GARMIN_CLIENT_ID: ""
GARMIN_CLIENT_SECRET: ""
GARMIN_REDIRECT_URL: ""

WHOOP_TOKEN_URL: https://api.prod.whoop.com/oauth/oauth2/token
WHOOP_AUTH_URL: https://api.prod.whoop.com/oauth/oauth2/auth
WHOOP_API_BASE_URL: https://api.prod.whoop.com/developer/v2
WHOOP_CLIENT_ID: ""
WHOOP_CLIENT_SECRET: ""
WHOOP_REDIRECT_URL: ""

Follow the same shape for your source: <SLUG>_CLIENT_ID, <SLUG>_CLIENT_SECRET, <SLUG>_REDIRECT_URL, and overridable <SLUG>_AUTH_URL / <SLUG>_TOKEN_URL / <SLUG>_API_BASE_URL. Because the name ends in _SECRET, the client secret is encrypted at rest automatically. Oura follows the convention (OURA_CLIENT_ID, OURA_CLIENT_SECRET, OURA_REDIRECT_URL) even though those three keys are not in the shipped template — reading a key that isn’t there simply returns empty, and the factory then declines to build the provider.

Webhooks

For push-based sources there is nothing to register either. Two routes accept payloads, and both dispatch to your provider:

# explicit provider — preferred
POST /api/v1/pulse/providers/theta_garmin/webhook

# universal: provider is sniffed out of the body's `source` field
POST /api/v1/pulse/providers/webhook

msg_id is taken from the Svix-Id request header, falling back to a formatted timestamp when the header is absent. It is what your is_data_already_processed and your storage table use to recognise a redelivery.

Caution

These endpoints perform no signature verification — there is no HMAC check anywhere on the Pulse webhook path. If your source signs its deliveries and you need that guarantee, the check has to be added, and until then the endpoint should be protected at the edge.

Scheduled pulls

If your source has to be polled instead, say so and implement one method. register_pull_task returning True (the base class default) gets you a scheduled task with a distributed lock; returning False opts out, which is what both the Garmin and PostgreSQL providers do — Garmin because its webhook does the work, PostgreSQL because it only validates a connection.

The base class drives the loop for you: it loads every linked user’s credentials, calls your pull_from_vendor_api per user, skips anything is_data_already_processed rejects, and pushes the rest through the write path. Return a list of self-describing packages, one per data kind, the way the Whoop provider does.

That data_type is the tag your format_data_v2 switches on later, so choose the names once and use them in both places. Two hooks exist for sources whose credentials aren’t a username and password: override pull_from_vendor_api with your own signature and override _pull_and_push_for_user(credentials) to call it — that is how Whoop passes an access token and a refresh token instead. Per-slug cadence and lock duration are listed on Pulse Provider System; a slug with no entry polls hourly with a 30-minute lock.

Data mapping

This is the part only you can write, and the only hard constraint in the whole provider contract: whatever shape your source speaks, format_data_v2 must return a StandardPulseData. Everything downstream — aggregation, indicator search, the agent’s health tools — reads that model and nothing else.

The input is a FormatDataInput, which is deliberately two separate things: a context the base class already resolved from the database (internal user id, vendor-side user id, timezone, message id) and the payload, untouched. Because identity and timezone arrive pre-resolved, format_data_v2 does no I/O at all; it is a pure function that can therefore be tested offline.

Each record’s type must be a registered indicator name, not your source’s field name — that is what makes a reading from an Oura ring comparable with one from a Garmin watch. So the real work is a table from vendor field to StandardIndicator, kept as data rather than scattered through if branches. Oura’s is the tersest form: a bare indicator when the source unit already matches the standard one, or a (indicator, source_unit) tuple when the write path should convert.

"total_sleep_duration"

(StandardIndicator.DAILY_TOTAL_SLEEP_TIME, "s")

"efficiency"

StandardIndicator.SLEEP_EFFICIENCY

"average_hrv"

StandardIndicator.HRV_RMSSD

"steps"

StandardIndicator.DAILY_STEPS

"spo2_percentage.average"

StandardIndicator.BLOOD_OXYGEN

Oura's mapping table: vendor field on the left, registered indicator (plus source unit where it differs) on the right.

Whoop’s table is the same idea with an explicit converter, because its payload speaks milliseconds and kilojoules: each entry is (indicator_name, converter, unit). Pick the indicator names out of the registry — see Health Indicators — and if nothing in the registry fits, extend the registry rather than inventing a type string here.

mirobody_acme/provider_acme.py

def format_data_v2(self, raw: dict, user_id: str) -> StandardPulseData:
    records = [
        StandardPulseRecord(
            source="acme",
            type="heartrate",          # a registered indicator name, not Acme's field name
            value=sample["bpm"],
            unit="/min",
            timestamp=sample["epoch_ms"],
        )
        for sample in raw["samples"]
    ]
    return StandardPulseData(userId=user_id, healthData=records)

Store the raw payload

Two abstract methods remain, and they are about durability rather than transformation. save_raw_data_to_db writes the payload untouched — so a formatting bug is a re-run, not lost data — and returns one entry per user found in it, which is how a batched webhook fans out into several format_data_v2 calls. is_data_already_processed is the idempotency gate before pushing.

The column names are not arbitrary. get_table_name() defaults to health_data_<slug minus "theta_">, get_user_id_column() to theta_user_id, and get_query_columns() to id, the user id column, external_user_id, msg_id, raw_data, create_at, update_at, is_del. Match that layout and the management console can page through and re-format your stored payloads for free; deviate and override those three methods.

Note

Where the vendor-side user id lives differs per source, so the base class exposes _extract_theta_user_id, _extract_external_user_id and _extract_msg_id as overridable hooks. Whoop overrides the middle one to read data[0]["user_id"]; the default reads a top-level user_id.

Load and verify

There is no build step — restart the process and read the log. Every successful load prints a line, and GET /api/v1/pulse/providers needs no token, so it is the fastest confirmation that your class was found:

/path/to/providers

mirobody serve
# → Loaded provider from /path/to/providers/mirobody_acme/provider_acme.py
# → ✅ Loaded provider: [theta_acme]

curl -s http://localhost:18080/api/v1/pulse/providers

If the slug isn’t in that response, the cause is one of four things and the log says which: the directory or module name didn’t match the glob, the class didn’t subclass BasePullProvider, the import raised, or create_provider returned None because a config key was missing.

Then verify the transformation properly — fixtures, snapshots, and the live routes — in Provider Testing.

Next steps

Provider Testing

Gate tests, fixtures, and the live Pulse routes

→

Garmin Provider Example

A complete implementation, read end to end

→

Data Flow

What happens to your records after they land

→

Using Providers

Configure credentials and connect an account

→

The four providers that ship are worth reading alongside this: mirobody/pulse/providers/.