Part four of the MITRE ATT&CK series. ATT&CK catalogues what attackers do; MITRE D3FEND catalogues what defenders do about it. The two are linked — D3FEND exposes a mapping from each defensive technique to the offensive techniques it counters — but the link lives behind a separate API and requires some careful plumbing to use at scale. This post wires D3FEND into the mapped dataset built in Part 2 and addresses the practical problems (API quirks, payload size, deduplication) that come up along the way.
Key Takeaways
- D3FEND exposes its mappings via a JSON API at
d3fend.mitre.org/api/. Two endpoints carry most of the value: offensive-to-defensive mapping per technique, and full defensive-technique details. - The API returns SPARQL-style "bindings" — slightly awkward to walk, but stable.
- Cache responses locally. The API is friendly but rate-limited and the data does not change minute to minute.
- The combined output bloats fast (~80 MB raw) because references and authors repeat across techniques. Deduplicate with a lookup table to cut size by 75%.
- The point of the mapping is decision support: gaps in D3FEND coverage for techniques attackers actually use are the highest-leverage defensive investments.
Environment
- Project structure from Part 1 and mapped data from Part 2.
- Python 3.10+ with
requests. - Internet access for the initial D3FEND fetch (cache locally afterwards).
The Problem
ATT&CK answers the question "what are attackers doing?" D3FEND answers "what could a defender do about it?" — but only if you can resolve the link between them. The D3FEND API surfaces that link, with some quirks: payloads are SPARQL-shaped, descriptions are deeply nested, and the same references repeat in dozens of techniques. The recipe below tames the data into something usable for downstream analysis and reporting.
The Solution
Step 1 — Fetch the D3FEND mapping for an ATT&CK technique
One endpoint per technique. Cache by technique ID:
import json, logging, requests
from pathlib import Path
logger = logging.getLogger(__name__)
D3F_CACHE = Path('cache/d3fend')
D3F_CACHE.mkdir(parents=True, exist_ok=True)
def fetch_d3fend_mapping(attack_id: str) -> dict | None:
cached = D3F_CACHE / f'{attack_id}.json'
if cached.exists():
return json.loads(cached.read_text())
try:
url = f'https://d3fend.mitre.org/api/offensive-technique/attack/{attack_id}.json'
resp = requests.get(url, timeout=30)
resp.raise_for_status()
data = resp.json()
cached.write_text(json.dumps(data))
return data
except requests.RequestException as exc:
logger.warning('D3FEND fetch failed for %s: %s', attack_id, exc)
return NoneStep 2 — Fetch defensive-technique details
Each defensive technique has its own endpoint with the long-form description, references, and synonyms:
def fetch_d3fend_detail(def_id: str) -> dict | None:
cached = D3F_CACHE / f'{def_id}_detail.json'
if cached.exists():
return json.loads(cached.read_text())
try:
url = f'https://d3fend.mitre.org/api/technique/d3f:{def_id}.json'
resp = requests.get(url, timeout=30)
resp.raise_for_status()
data = resp.json()
cached.write_text(json.dumps(data))
return data
except requests.RequestException as exc:
logger.warning('D3FEND detail failed for %s: %s', def_id, exc)
return NoneStep 3 — Resolve mapping into a flat list of defensive techniques
The mapping payload nests the actual links under off_to_def.results.bindings. Each binding has a defensive-technique URI; the ID is the fragment after #:
def defensive_techniques_for(attack_id: str) -> list[dict]:
raw = fetch_d3fend_mapping(attack_id)
if not raw or 'off_to_def' not in raw:
return []
out = []
for binding in raw['off_to_def']['results']['bindings']:
label = binding.get('def_tech_label', {}).get('value')
uri = binding.get('def_tech', {}).get('value')
if not (label and uri):
continue
def_id = uri.rsplit('#', 1)[-1]
detail = fetch_d3fend_detail(def_id) or {}
description = (
detail.get('description', {})
.get('@graph', [{}])[0]
.get('d3f:definition')
)
out.append({
'id': def_id,
'label': label,
'description': description,
'url': f'https://d3fend.mitre.org/technique/d3f:{def_id}',
})
return outStep 4 — Decorate the mapped ATT&CK data
Iterate the mapped techniques from Part 2 and add a d3fend field. The API is fast but caching is essential — the first run takes minutes, subsequent runs are seconds:
def add_d3fend(mapped: list[dict]) -> list[dict]:
for i, tech in enumerate(mapped, start=1):
tech['d3fend'] = defensive_techniques_for(tech['technique_id'])
if i % 50 == 0:
logger.info('D3FEND mapped %d/%d', i, len(mapped))
return mappedStep 5 — Deduplicate references to control file size
Naively decorated, the combined output bloats to ~80 MB on the current dataset. Most of the bulk is references and authors that repeat across techniques. A lookup table cuts it sharply:
def deduplicate_references(mapped: list[dict]) -> dict:
ref_table: dict[str, int] = {}
author_table: dict[str, int] = {}
for tech in mapped:
for d3f in tech.get('d3fend', []):
new_refs = []
for ref in d3f.get('references', []):
key = ref['url'] if isinstance(ref, dict) else ref
if key not in ref_table:
ref_table[key] = len(ref_table) + 1
new_refs.append(ref_table[key])
d3f['references'] = new_refs
new_authors = []
for author in d3f.get('authors', []):
if author not in author_table:
author_table[author] = len(author_table) + 1
new_authors.append(author_table[author])
d3f['authors'] = new_authors
return {
'techniques': mapped,
'metadata': {
'references': {v: k for k, v in ref_table.items()},
'authors': {v: k for k, v in author_table.items()},
},
}
The replacement integers are small; the canonical reference data lives once in metadata.references. Output drops from ~80 MB to ~20 MB.
Step 6 — Strip empty values
A small recursive cleanup removes null, empty strings, empty lists, and empty dicts before serialisation. Saves another couple of megabytes and makes the output noticeably easier to read:
def strip_empty(value):
if isinstance(value, dict):
return {k: strip_empty(v) for k, v in value.items()
if v not in (None, '', [], {})}
if isinstance(value, list):
return [strip_empty(v) for v in value if v not in (None, '', [], {})]
return valueStep 7 — Act on the gap analysis
Once the dataset is enriched, the most useful query is the inverse mapping: which ATT&CK techniques have no D3FEND coverage, weighted by group usage. Those are the high-leverage defensive investments:
uncovered = [
t for t in mapped
if not t.get('d3fend') and len(t.get('groups', [])) >= 5
]
uncovered.sort(key=lambda t: len(t['groups']), reverse=True)
for t in uncovered[:20]:
print(t['technique_id'], t['name'], 'groups:', len(t['groups']))The top of that list is where security investment maps directly onto reduction of real-world attacker capability. Take it to your next strategy review.
Frequently Asked Questions
Is D3FEND a replacement for ATT&CK's mitigations?
No — they overlap but are not interchangeable. ATT&CK mitigations are broad, often process-level recommendations. D3FEND is a structured ontology of defensive techniques with more granularity. Use ATT&CK mitigations for high-level mapping; use D3FEND for detailed control-design work.
Is the D3FEND API rate-limited?
Not in a documented way at the time of writing, but it is a free service hosted by MITRE. Cache locally, respect the service, and back off if you start seeing failures. The mapping changes slowly; weekly refresh is plenty.
Why are the API payloads SPARQL-shaped?
D3FEND is built on top of an OWL ontology, which is queried in SPARQL natively. The JSON wrapper is a thin serialisation of SPARQL result bindings. Once you know the shape, the data is straightforward; the shape itself is just legacy from the underlying technology.
Can I query D3FEND directly with SPARQL?
Yes — D3FEND publishes an OWL/RDF dataset you can load into a local triplestore (Apache Jena, Blazegraph, GraphDB). Useful for advanced analysts; not necessary for typical defender-side mapping. The JSON API covers the common cases.
How big is the final combined dataset?
After mapping, deduplicating references, and stripping empty values, the combined ATT&CK + D3FEND JSON sits at about 20 MB for the current enterprise dataset. Compress on disk if you check it into git.
Conclusion
ATT&CK without D3FEND is half the picture; pairing them gives you both halves and lets you do useful gap analysis on the defensive side. The API has rough edges — SPARQL-shaped responses, repeated references, deeply nested descriptions — but they are the kind of thing a small loader + deduplicator pipeline handles once and never bothers you about again. The output is something you can hand to a security architect and have a useful conversation about which controls to build next, which is what these frameworks are for.
Related Posts
- Getting Started with MITRE ATT&CK: Fetching and Processing Data — part 1 of the series.
- Mapping with MITRE ATT&CK: Connecting Techniques, Groups, and Mitigations — part 2, the data source decorated above.
- Visualizing with MITRE ATT&CK Navigator — part 3, render D3FEND coverage as a Navigator layer.
Authoritative references: MITRE D3FEND and D3FEND API documentation.
Editorial note: posts on this blog are drafted with AI assistance and then reviewed, edited, and tested against a real environment before publishing. Commands, output, and screenshots come from systems I actually ran the work on.
0 comments:
Post a Comment