Deep Dive: Knowledge Lifecycle & Memory

# Confidence = base từ source count × decay factor

def calculate_confidence(
    source_count: int,
    last_confirmed_days_ago: int,
    has_contradiction: bool = False,
    is_superseded: bool = False
) -> float:
    # Base confidence từ số sources
    base = min(0.95, 0.3 + (source_count - 1) * 0.2)
    # Tối đa: 1 source → 0.3, 2 sources → 0.5, 4 sources → 0.9

    # Decay: 2% per week (Ebbinghaus-inspired)
    weeks_old = last_confirmed_days_ago / 7
    decay = 0.02 * weeks_old
    decayed = max(0.1, base - decay)

    # Penalties
    if has_contradiction:
        decayed *= 0.6  # contradiction halves trust
    if is_superseded:
        decayed = 0.05  # superseded → near-zero

    return round(decayed, 2)

# Examples:
# 2 sources, 1 week old → confidence = 0.48
# 4 sources, 0 days old → confidence = 0.90
# 1 source, 3 months old → confidence = 0.04 (stale!)
# 3 sources, 2 weeks + contradiction → confidence = 0.28

# Redis Caching — Project X
**Confidence**: 0.85
**Sources**: [[raw/architecture-doc.md]], [[raw/infra-review.pdf]]
**Last Confirmed**: 2026-04-10
**Confidence History**: 0.30 (2025-10-01) → 0.50 (2026-01-15) → 0.85 (2026-04-10)

Project X uses Redis for session caching with a 24h TTL.

{`<!-- TRƯỚC (stale claim) -->
# Redis Caching — Project X
**Confidence**: 0.85

Project X uses Redis for session caching.

---

<!-- SAU (supersession applied) -->
# Redis Caching — Project X
**Confidence**: 0.05
**Status**: ⚠️ SUPERSEDED by [[memcached-migration]]
**Superseded on**: 2026-04-15
**Reason**: Infrastructure migration completed, see [[raw/infra-migration-q2.md]]

~~Project X uses Redis for session caching.~~

> **Update (2026-04-15):** Project X migrated to Memcached.
> See [[memcached-migration]] for current state.
`}

# Memcached Migration — Project X
**Confidence**: 0.90
**Sources**: [[raw/infra-migration-q2.md]], [[raw/arch-review-2026-q2.pdf]]
**Supersedes**: [[redis-caching]] (as of 2026-04-15)

Project X completed migration to Memcached in Q2 2026.
Reasons: lower memory footprint, better eviction policies for session data.

This source contains new information that may update existing wiki claims.

For each factual claim in the source:
1. Check if a similar claim exists in wiki/
2. If the new claim UPDATES or CONTRADICTS existing:
   a. Mark old page as superseded: add "⚠️ SUPERSEDED" badge, strikethrough old claim
   b. Create/update new page with current claim
   c. Set old page confidence to 0.05
   d. Link old → new and new → old
   e. Log: "[DATE] supersession | [old page] → [new page]"

import datetime

def calculate_retention(last_reinforced: datetime, decay_rate: float) -> float:
    """
    Ebbinghaus-inspired retention.
    decay_rate: fraction lost per week (0.02 = 2%/week for architecture)
    """
    weeks_elapsed = (datetime.now() - last_reinforced).days / 7
    retention = max(0.05, 1.0 * (1 - decay_rate) ** weeks_elapsed)
    return round(retention, 2)

def classify_retention_action(retention: float) -> str:
    if retention > 0.7: return "active"
    if retention > 0.4: return "review_soon"
    if retention > 0.2: return "flag_for_review"
    return "deprioritize"

# Weekly lint: find facts with retention < 0.3 → flag for review
# Annual cleanup: facts with retention < 0.1 and no inbound links → archive

Review today's session notes in wiki/_inbox/session-[DATE].md.

Extract and promote:
1. EPISODIC: Write wiki/episodes/[DATE].md with:
   - What we worked on
   - Key decisions made
   - New information discovered
   - Entities referenced (link to wiki pages)

2. SEMANTIC: For each fact mentioned 3+ times or from authoritative source:
   - Find or create entity page in wiki/entities/
   - Update confidence (+0.1 for each corroboration)
   - Add cross-references

3. PROCEDURAL: If we repeated a workflow pattern:
   - Check if wiki/procedures/ has this pattern
   - If not: propose adding it
   - If yes: update with new insight

Then archive session notes to wiki/_archive/sessions/[DATE].md

# Page Quality Checklist (0-10 score)

Structure (3 pts):
  [+1] Has Summary field
  [+1] Has Tags field
  [+1] Has Confidence + Last Updated

Content (4 pts):
  [+1] Summary is standalone (no "it"/"this" references)
  [+1] Has at least 1 specific, verifiable claim
  [+2] Cites sources with [[wiki-links]] or [[raw/]] links

Connections (3 pts):
  [+1] Has ≥1 inbound link (not orphan)
  [+1] Has ≥1 outbound [[wiki-link]]
  [+1] Relationships section filled with typed relations

Score < 5 → flag for rewrite
Score 5-7 → flag for improvement
Score ≥ 8 → active status

Run self-healing lint on wiki/.

For each issue found, FIX IT (don't just report):

AUTO-FIX (do immediately):
- Broken [[wiki-links]] → remove or redirect
- Missing Tags field → infer from content, add
- Orphan pages with obvious home → add backlink
- Duplicate pages (same concept, different names) → merge, redirect

FLAG FOR REVIEW (don't auto-fix):
- Contradictions (need human judgment)
- Confidence < 0.3 (need source verification)
- Missing Summary (need human knowledge)

After fixing, show summary:
"Fixed: N items | Flagged: M items | Wiki health: X/10"

Today's research session on [TOPIC] is complete.

Create a crystal page at wiki/crystals/[topic]-[date].md:

1. QUESTION: What were we trying to understand?
2. FINDINGS: What did we learn? (bullet list, each item linkable to a source/entity)
3. ENTITIES TOUCHED: [[list of entity pages we read/updated]]
4. LESSONS: What general principles emerged?
   (these become candidates for wiki/entities/ if not already there)
5. CONTRADICTIONS: Did we find anything that contradicts existing wiki?
6. OPEN QUESTIONS: What's still unclear?

After creating crystal page:
- Add to wiki/index.md under "Crystals" section
- For each lesson, check if it should strengthen existing entity confidence
- Log: "[DATE] crystal | [topic]"

import re

SENSITIVE_PATTERNS = [
    r'(?i)(api[_-]?key|token|secret|password|credential)["s:=]+S+',
    r'[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+.[A-Z|a-z]{2,}',  # email
    r'd{3}[-.s]?d{3}[-.s]?d{4}',  # phone
    r'sk-[a-zA-Z0-9]{48}',  # OpenAI API key pattern
    r'Bearer [a-zA-Z0-9._-]+',  # auth tokens
]

def sanitize_for_wiki(text: str) -> str:
    for pattern in SENSITIVE_PATTERNS:
        text = re.sub(pattern, '[REDACTED]', text)
    return text

# Run before every ingest — not optional

2026-04-20T14:23:01Z | INGEST | agent:claude | source:raw/paper1.pdf | pages_created:3 | pages_updated:5
2026-04-20T15:01:22Z | QUERY  | agent:claude | question:"What is attention?" | answer_filed:yes | page:wiki/comparisons/attention-types.md
2026-04-20T16:45:00Z | LINT   | agent:claude | issues_found:4 | auto_fixed:3 | human_flagged:1
2026-04-20T17:00:00Z | DELETE | agent:claude | page:wiki/old-architecture.md | reason:superseded | archived:yes