North Data via Cormorant

Cormorant fetches, parses and caches North Data (northdata.com / northdata.de) company profiles for you: discovery of the profile URL from a company name, the parsed profile with its register data, history, related companies and officers, and North Data's network graph. You send JSON and get structured JSON back, with the provenance of every page and the exact bytes behind it.

This is the scraping path (the public pages); it does not use North Data's paid API.

Four endpoints#

Most clients only need profile: give it the company name, country and headquarters and it discovers the URL, fetches the page, parses it and merges the graph in one call.

curl -s https://api.cormorant.ch/v1/northdata/profile \
  -H "Authorization: Bearer $CORMORANT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"company_name": "Novartis Pharma AG", "country": "CH",
       "headquarters": "Basel, Switzerland", "session_key": "CH0012005267"}'

import httpx

client = httpx.Client(
    base_url="https://api.cormorant.ch",
    headers={"Authorization": f"Bearer {key}"},
    timeout=180,
)
r = client.post(
    "/v1/northdata/profile",
    json={
        "url": "https://www.northdata.com/Unilever+plc",
        "session_key": "GB00B10RZP78",
    },
)
body = r.json()
if body["error"]:
    print("no profile:", body["error"], body["retry_after_s"])
else:
    profile = body["profile"]
    print(profile["name"], profile["register_id"], profile["city"])
    print("page bytes:", body["fetch"]["body_url"])

Suggest, discover and graph work the same way:

curl -s https://api.cormorant.ch/v1/northdata/suggest \
  -H "Authorization: Bearer $CORMORANT_API_KEY" -H "Content-Type: application/json" \
  -d '{"query": "Novartis", "country": "CH"}'
# {"op": "suggest", "query": "Novartis", "country": "CH",
#  "items": [{"name": "Novartis Pharma AG", "url": "https://www.northdata.com/…", "type": "company"}, …],
#  "fetch": {…}, "error": null, …}

curl -s https://api.cormorant.ch/v1/northdata/discover \
  -H "Authorization: Bearer $CORMORANT_API_KEY" -H "Content-Type: application/json" \
  -d '{"company_name": "Novartis Pharma AG", "headquarters": "Basel", "country": "CH"}'

curl -s https://api.cormorant.ch/v1/northdata/graph \
  -H "Authorization: Bearer $CORMORANT_API_KEY" -H "Content-Type: application/json" \
  -d '{"entity_id": "1234567", "session_key": "CH0012005267"}'

session_key and country#

• session_key keeps one residential exit for the company: suggest, profile and graph requests with the same key go out through the same Massive exit for 15 minutes ("flex" mode: single errors do not rotate it). Use a stable identifier of the company you are researching, such as its ISIN. After a proxy error the request is retried once on a fresh exit ({session_key}-{8 hex}).
• country is the company's or research market's country: ISO code (CH) or an English or German name (Switzerland, Schweiz). It filters suggest results and geotargets the exit. Unknown names are ignored, never cut to two letters.

How Cormorant fetches North Data#

• Only through Massive, never from Cormorant's own addresses. The residential proxy goes first, with the headers of one consistent desktop Chrome (the same User-Agent, client hints and language on pages and on the site's JSON calls). Massive Web Render follows when the proxy was blocked (a challenge, 403/429/451/5xx, a refusal) or the page was thin. North Data's fraud prevention refuses most residential exits, so after a refusal Cormorant skips the proxy for an hour and goes straight to Web Render (about 1 in 10 requests still tries the proxy, and a success ends the skip). The network graph is requested exactly as North Data's own page requests it (a GET of data.json with the page's extra token), so Web Render can fetch it as well. There is no direct, browser or archive tier. Every request therefore uses a paid tier: allow_paid: false answers paid_tier_required, an exhausted budget budget_exhausted. The same policy applies when you fetch these hosts with POST /v1/fetch.
• One throttle for the whole site. .com and .de count as one site. Across the whole cluster, at most 2 requests are in flight and starts are at least 1 s apart, for suggest, profile and graph alike. A 429 or a challenge answered to Web Render slows the site down further for a while (a refused residential exit does not: that limit is per exit, not for the cluster). Synchronous calls wait up to 60 s for a slot; batch items go back to the queue instead.
• Bot protection (Cloudflare, Akamai, DataDome, PerimeterX, Imperva, AWS WAF, Sucuri, captcha and "unusual traffic" pages) is detected and never served or cached as a profile. Thin pages (under 200 characters of text) and pages that do not parse as a profile (parse_failed) are not cached either.
• Domain memory. When every tier failed for the site in 3 consecutive requests, the site is skipped for a while (blocked with retry_after_s). Cormorant never learns a cheaper start tier for North Data.

Caching#

cache_ttl_s sets the maximum age you accept (0 bypasses the cache). The cache is shared: a profile one project fetched costs the next project nothing. Cache hits answer cached: true and billing_category: northdata_cached.

robots.txt#

Cormorant checks North Data's robots.txt for CormorantBot (and *) before every profile page and before suggest.json and data.json. The file is read through the residential proxy, like everything else, and cached for 24 h. A disallowed URL answers robots_disallowed without fetching.

We made that call for the JSON endpoints on purpose. They are not documented APIs but the requests North Data's own pages make, and robots.txt is the site's statement about automated access to any path. So Cormorant follows it for them too, instead of exempting them as "API calls". At the time of writing, North Data's robots.txt blocks only a list of named crawlers (AI and archive bots) and none of the paths Cormorant uses.

Errors and retries#

Upstream failures are HTTP 200 with error set, never an HTTP 5xx. Errors worth retrying are rate_limited and blocked (both after retry_after_s), timeout, politeness_timeout and all_tiers_failed (later, with backoff). not_found, parse_failed and invalid_identifier are final for that input. paid_tier_required and budget_exhausted are about your request or project. A profile whose graph failed is still returned, with graph_error set.

Many companies: batch#

For more than a handful of companies, submit one job:

{
  "items": [
    {"op": "profile", "company_name": "Novartis Pharma AG", "country": "CH", "session_key": "CH0012005267"},
    {"op": "profile", "url": "https://www.northdata.com/Unilever+plc", "session_key": "GB00B10RZP78"}
  ],
  "idempotency_key": "research-run-2026-10-20-northdata",
  "callback_url": "https://example.org/hooks/cormorant"
}

POST /v1/batch/northdata returns a job_id. Poll GET /v1/jobs/{job_id} or wait for the callback, then page through ?include=items. Each item's result has the shape of the matching single endpoint. The site throttle sets the pace, about one profile per second across the cluster, so plan large runs accordingly.

Costs#

Every uncached North Data request goes through Massive: the residential proxy is billed by traffic, Web Render by request. A profile call can make up to three requests (suggest, page, graph). cost_usd (for projects without a customer account) and GET /v1/usage (the northdata block, the northdata / northdata_cached categories) show what it cost; see Costs & billing.

What a profile contains, and checking the match#

• Officers come from the page's management sections and its Persons timeline (full name, role, start and end date, e.g. Roland Busch, CEO since 2021-03-15), then from the network graph (role person). North Data's free pages show a selection of persons; complete lists (for example every authorised signatory) are part of its paid service and not available here.
• Related companies: the page's subsidiary/parent/shareholder lists, merger, acquisition and control partners from its M&A timeline (relation merger, acquisition, control), and the graph's companies (related).
• Network: network_graph is North Data's graph; when that cannot be fetched (graph_error), it is built from the page's persons and related companies and says source: "page".
• Is it the right company? Check status and identifiers before you use a discovered profile. A name match can be a branch or a foreign registration: "Novartis Pharma AG, Basel" on North Data is (also) the company's registration in Norway's Brønnøysund register (identifiers: "The Brønnøysund Register Centre 925942480", EUID NOFOR.925942480), terminated in 2024 (status: "Terminated"). Its register_id BR 925942480 is that Norwegian number, not a Swiss UID.

Reference#

Company profiles, discovery and network graphs from North Data (northdata.com / northdata.de). All four endpoints are synchronous (like POST /v1/fetch), authenticated, metered (kind northdata, see Billing categories) and answer HTTP 200 with error for upstream failures — never an HTTP 5xx. For many companies use POST /v1/batch/northdata.

Transport. North Data is fetched only through Massive: the residential proxy first (Chrome-consistent headers; a sticky exit per session_key, 15 minutes, retried once with a fresh exit after a proxy error), then Massive Web Render when the proxy was blocked. North Data's fraud prevention usually refuses residential exits (429, 403 for data.json): after a refusal the proxy is skipped for an hour (NORTHDATA_PROXY_SKIP_S; 10 % of requests still try it) and requests go straight to Web Render (attempt reason proxy_skipped_refused). The network graph is requested like North Data's own page does: GET /data.json?domain=company&id=…&type=graph&extra=<token>, so Web Render can serve it too. Never from Cormorant's own addresses: no direct, browser or archive tier and no alternative sources. This also applies to POST /v1/fetch of these hosts. allow_paid: false (or a key without paid fetches) answers error: paid_tier_required. A cluster-wide throttle covers the whole site (.com and .de together; suggest, profile and graph): by default at most 2 requests in flight and 1 s between starts. robots.txt is read through the proxy and applies to profile pages, suggest.json and data.json.

Cache. Profile pages: served for 24 h, then revalidated with a conditional request (fetch.revalidated), kept 7 days. Suggest and graph answers: 24 h. Challenge pages, thin pages (< 200 characters of text) and pages that do not parse are never kept. cache_ttl_s sets the maximum acceptable age (0 bypasses the cache).

Every request also takes cache_ttl_s, allow_paid (default true) and project_ref. Every response has op, error, retry_after_s, cost_usd (null for external customers), billing_category (northdata or northdata_cached) and request_id. country is an ISO code (CH) or an English/German country name (Switzerland); unknown names are ignored. It filters suggest (countries=) and geotargets the Massive exit.

{
  "op": "profile",
  "profile": {
    "name": "Novartis Pharma AG", "legal_form": "AG", "register_id": "BR 925942480",
    "court": null, "status": "Active", "address": "Lichtstrasse 35, 4056 Basel, Switzerland",
    "city": "4056 Basel", "country": "Switzerland", "path_country": "CH",
    "entity_id": "…", "subsidiaries": [{"name": "Novartis International AG",
      "relation": "subsidiaries", "url": "/Novartis%20International%20AG",
      "absolute_url": "https://www.northdata.com/Novartis%20International%20AG"}],
    "source_url": "https://www.northdata.com/Novartis%20Pharma%20AG,%20Basel,%20CH/BR%20925942480",
    "discovery_method": "suggest", "discovery_score": 1.0, "name_similarity": 100.0,
    "fetch_tier": "web_render", "parser_version": "4", "provider": "northdata"
  },
  "fetch": {
    "url": "https://www.northdata.com/…", "final_url": "https://www.northdata.com/…",
    "status": 200, "tier": "proxy", "body_url": "/v1/bodies/9f2c…", "sha256": "9f2c…",
    "body_kind": "origin", "retrieved_at": "2026-10-19T12:00:00+00:00", "cached": false,
    "revalidated": false, "attempts": [{"tier": "proxy", "status": 200, "reason": "ok", "ms": 2100}],
    "cost_usd": 0.0004, "parser_version": "4"
  },
  "discovery": {"url": "https://www.northdata.com/…", "query": "Novartis Pharma AG",
    "score": 1.0, "name_similarity": 100.0, "discovery_method": "suggest", "candidates": []},
  "graph_fetch": null, "graph_error": "no_entity_id",
  "error": null, "retry_after_s": null, "cost_usd": 0.0006,
  "billing_category": "northdata", "request_id": "req_…"
}

Profile fields (parser_version 4; parses HTML and the Markdown Web Render sometimes returns): name, legal_form (from the company's own name or legal-form block), register_id (check identifiers for which register it belongs to), court (German register court as shown, e.g. Amtsgericht München or District Court of Munich), status (Terminated / Liquidation when North Data marks it so, else Active for a register profile), identifiers[{label, source, value, current}] (the Identification block: register entries, LEI, EUID, …), address, city, country (free text, not normalised), path_country (ISO code in the URL), entity_id, lei, industry, former_names[], former_addresses[], corporate_purpose, additional_info, history[{category, date, description, text, url}], publications[{date, summary, company_name}], mentions[…], network_graph {company_connections, person_connections, other_connections, source} (source: data.json, or page when the graph could not be fetched and the network is built from the page's persons and merger/acquisition data), officers[{name, role, since, until, url, absolute_url}] (from the page's management sections and its Persons timeline, e.g. Roland Busch, CEO, since 2021-03-15, then graph persons), shareholders/subsidiaries/parent_companies/related (related also holds merger/acquisition/control partners of the page's M&A timeline) [{name, relation, url, absolute_url}], financials (always null), source_url, discovery_query, discovery_score (0..1), discovery_method, name_similarity (0..100), scraped_at, fetch_tier (proxy or web_render), parser_version, provider (northdata). Graph merge: graph companies not already listed become related (relation related); graph persons become officers with role person; parents and subsidiaries are never invented.

Discovery. Suggest first: company items only; the one with the highest name similarity (Jaro-Winkler + Levenshtein on names without legal suffixes, 0..100) wins when it reaches min_similarity; score = similarity / 100; no first-result fallback. Otherwise, with search_fallback, Cormorant's own search with up to four queries (northdata "{name}" {city cc}, site:northdata.com OR site:northdata.de "{name}" {city or cc}, northdata "{name}" {cc}, northdata {name}), 5 results each, profile URLs scored 0..1 (name tokens in the path and title, city, country, host, register in the path); it stops at ≥ 0.75 and rejects < 0.35. Suggest candidates carry the name similarity (0..100) as score, search candidates the URL score.

graph_error (profile only): why the graph is missing while the profile is returned — an error code from the table, or no_entity_id.

Batch: POST /v1/batch/northdata#

{
  "items": [
    {"op": "profile", "company_name": "Novartis Pharma AG", "country": "CH", "session_key": "CH0012005267"},
    {"op": "graph", "entity_id": "1234567"}
  ],
  "idempotency_key": "chipiron-run-2026-10-20-northdata-0001",
  "callback_url": "https://chipiron.example/hooks/cormorant"
}

op is suggest, discover, profile (default) or graph, plus the fields of that endpoint (422 when the field an operation needs is missing). Items run like the single endpoints and their results have the same shape; the site throttle sets the pace (a busy throttle puts an item back in the queue).

Coming from Chipiron's own scraper#

The profile follows Chipiron's NorthdataProfile, with these differences: