Weekly AI Digest v3 – Automated Content Curation Script

🤖 Weekly AI Digest v3 — Automated Content Curation Script

Reviewed: June 4, 2026

Python script that auto-curates arXiv papers, GitHub trending repos, and AI news RSS feeds into a ready-to-publish digest.

📋 What It Does

The weekly_digest_v3_curator.py script automatically gathers content from multiple sources and produces a formatted HTML digest ready to publish to WordPress via the dg/v1/publish endpoint.

Data Sources

• 📄 arXiv API — Fetches top papers from cs.AI, cs.CL, cs.LG, cs.RO, cs.NE categories, sorted by submission date. Selects Paper of the Week + 3 runners-up.
• ⭐ GitHub Search API — Finds trending AI/ML repos created in the past week with 50+ stars. Filters by topics: machine-learning, artificial-intelligence, deep-learning, llm, transformers.
• 📰 RSS Feeds — Aggregates from 6 sources: arXiv CL Blog, arXiv AI Blog, Hugging Face Blog, OpenAI Blog, Google AI Blog, MIT Technology Review. Deduplicates by title hash.

🏗️ Architecture

🚀 Usage

# Basic run — outputs JSON with embedded HTML
python3 weekly_digest_v3_curator.py

# Specify output file and also generate standalone HTML
python3 weekly_digest_v3_curator.py --output /tmp/digest_v3_latest.json --format html

# Set a custom week-ending date
python3 weekly_digest_v3_curator.py --week-ending "June 3, 2026"

📊 Output Format

The script outputs a JSON file with the following structure:

{
  "version": "3.0",
  "week_ending": "May 27, 2026",
  "generated_at": "2026-05-27T10:30:00+00:00",
  "stats": {
    "papers": 4,
    "repos": 5,
    "news_items": 6
  },
  "data": {
    "papers": [...],   // Array of paper objects
    "repos": [...],    // Array of repo objects
    "news": [...]      // Array of news item objects
  },
  "html": "<div class='dg-digest'>...</div>"  // Ready-to-publish HTML
}

✅ Live Test Results

Script tested successfully on May 27, 2026:

• ✅ arXiv: 4 papers fetched from cs.AI, cs.CL, cs.LG categories
• ✅ RSS News: 6 items aggregated from 6 feeds, deduplicated
• ⚠ GitHub: Rate-limited on free API (422). Falls back gracefully with empty repos list.
• ✅ HTML Generation: Full digest HTML produced with all sections

🔧 Integration with Digest Publishing

To publish a digest edition, the WordPress dg/v1/publish endpoint reads the html field from the JSON output and creates a new post:

import json, subprocess

with open('/tmp/digest_v3_latest.json') as f:
    data = json.load(f)

payload = {
    "posts": [{
        "title": f"Weekly AI Digest v3 — {data['week_ending']}",
        "content": data["html"],
        "status": "publish",
        "slug": f"weekly-ai-digest-v3-{data['week_ending'].lower().replace(' ', '-')[:20]}"
    }]
}

📌 Notes

• Requires only Python 3 stdlib — no pip install needed
• arXiv API has rate limits; script fetches max 8 papers to stay within bounds
• GitHub Search API may rate-limit unauthenticated requests (60/hour)
• RSS feeds that fail to load are skipped gracefully
• Output HTML is self-contained with inline styles — works in WordPress without external CSS