rinoxRinox
Log inTry free
Enrichment→SIEM

Shodan to Elastic Security Integration

Pull external attack-surface data from Shodan and index it in Elasticsearch so the SOC can monitor exposure changes alongside endpoint and identity telemetry. The integration runs Shodan host lookups or saved-search queries on a schedule and writes results to a dedicated index with a stable mapping.

Shodan returns rich nested objects (services, vulns, locations); the integration preserves them under a `shodan.*` namespace so existing Elastic dashboards can be extended without remapping.

What you get

A real excerpt from a generated Shodan-to-Elastic Security integration. The full script ships with logging, env-var loading, error handling, FIFO-capped dedup state, and a README.

shodan_to_elastic.py
# RINOX INTEGRATION: Shodan -> Elastic
def bulk_payload(hosts):
    out = []
    for h in hosts:
        doc_id = hashlib.sha256((h["ip_str"] + h["last_update"]).encode()).hexdigest()
        out.append(json.dumps({"index": {"_index": ES_INDEX, "_id": doc_id}}))
        out.append(json.dumps({
            "@timestamp": h["last_update"],
            "ip": h["ip_str"],
            "shodan": h,
        }))
    return "\n".join(out) + "\n"  # NDJSON, trailing newline required

Common pitfalls

The mistakes that turn this integration from "works once" into "loses data silently for three weeks."

  • 01Shodan paginates with `page=` and rate-limits aggressively. The free tier caps at 1 request/sec; honour it or you will get throttled to nothing.
  • 02Elasticsearch bulk API expects NDJSON: `{action}\n{doc}\n{action}\n{doc}\n` — pretty-printed JSON breaks bulk indexing silently.
  • 03Use `op_type: index` with deterministic `_id = sha256(ip + last_update)` for idempotent writes. Otherwise repeated runs duplicate the same host record.
  • 04Shodan `last_update` is the source-of-truth timestamp for the record, not the script execution time. Map it to `@timestamp` per the contextual fidelity rule.

Generate this for your environment.

Pre-fills the form with Shodan and Elastic Security. You write a sentence about your use case, we write the rest.

Generate Shodan → Elastic Security

Frequently asked

  • Can I monitor a corporate IP range?

    Yes — set `SHODAN_QUERY=net:198.51.100.0/24` and the script will paginate through every host in the range on each run.

  • How does the integration handle deletions?

    Shodan does not signal deletes. The integration writes a TTL-style `expires_at` field; Elastic ILM can age out documents older than `SHODAN_EXPIRES_DAYS`.

  • What about Shodan Monitor webhooks?

    If your Shodan plan supports Monitor, switch `SHODAN_TRANSPORT=webhook` and the script becomes an HTTP receiver instead of a poller. Same translation logic, different ingestion path.

Related integrations