feat: enhance research skills with validation and English translation

- Add validate_json.py for field coverage validation
- Add hard constraints and one-shot examples to prompt templates
- Add parameter retrieval sections before prompts
- Translate all SKILL.md files to English
- Update report.md with technical requirements

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
jilinchen
2025-12-30 13:23:34 +08:00
co-authored by Claude Opus 4.5
parent a34ef20fd4
commit 5eaf859274
6 changed files with 513 additions and 128 deletions
+121 -36
View File
@@ -1,55 +1,140 @@
--- ---
allowed-tools: Read, Write, Glob, WebSearch, Task, AskUserQuestion allowed-tools: Read, Write, Glob, WebSearch, Task, AskUserQuestion
description: 对目标话题进行初步调研,生成调研outline。用于学术调研、benchmark调研、技术选型等场景。 description: Conduct preliminary research on a topic and generate a research outline. Use for academic research, benchmark research, technology selection, etc.
--- ---
# Research Skill - 初步调研 # Research Skill - Preliminary Research
## 触发方式 ## Trigger
`/research <topic>` `/research <topic>`
## 执行流程 ## Workflow
### Step 1: 模型内部知识生成初步框架 ### Step 1: Generate Initial Framework from Model Knowledge
基于topic,利用模型已有知识生成: Based on the topic, use model's existing knowledge to generate:
- 该领域的主要研究对象/items列表 - Main research objects/items list in this domain
- 建议的调研字段框架 - Suggested research field framework
### Step 2: Web Search补充 Output {step1_output}.
启动1个web-search-agent(后台),传入topic和当前日期备注,agent自行设计搜索策略补充最新items和字段建议。等待完成后获取搜集知识。
### Step 3: 询问用户已有字段 ### Step 2: Web Search Supplement
使用AskUserQuestion询问用户是否有已定义的字段文件,如有则读取并合并。 Use AskUserQuestion to ask for time range (e.g., last 6 months, since 2024, unlimited).
### Step 4: 生成Outline(分离文件) **Parameter Retrieval**:
合并所有信息,生成两个文件: - `{topic}`: User input research topic
- `{YYYY-MM-DD}`: Current date
- `{step1_output}`: Complete output from Step 1
- `{time_range}`: User specified time range
**outline.yaml**items + 配置): **Hard Constraint**: The following prompt must be strictly reproduced, only replacing variables in {xxx}, do not modify structure or wording.
- topic: 调研主题
- items: 调研对象列表 Launch 1 web-search-agent (background), **Prompt Template**:
```python
prompt = f"""## Task
Research topic: {topic}
Current date: {YYYY-MM-DD}
Based on the following initial framework, supplement latest items and recommended research fields.
## Existing Framework
{step1_output}
## Goals
1. Verify if existing items are missing important objects
2. Supplement items based on missing objects
3. Continue searching for {topic} related items within {time_range} and supplement
4. Supplement new fields
## Output Requirements
Return structured results directly (do not write files):
### Supplementary Items
- item_name: Brief explanation (why it should be added)
...
### Recommended Supplementary Fields
- field_name: Field description (why this dimension is needed)
...
### Sources
- [Source1](url1)
- [Source2](url2)
"""
```
**One-shot Example** (assuming researching AI Coding History):
```
## Task
Research topic: AI Coding History
Current date: 2025-12-30
Based on the following initial framework, supplement latest items and recommended research fields.
## Existing Framework
### Items List
1. GitHub Copilot: Developed by Microsoft/GitHub, first mainstream AI coding assistant
2. Cursor: AI-first IDE, based on VSCode
...
### Field Framework
- Basic Info: name, release_date, company
- Technical Features: underlying_model, context_window
...
## Goals
1. Verify if existing items are missing important objects
2. Supplement items based on missing objects
3. Continue searching for AI Coding History related items within since 2024 and supplement
4. Supplement new fields
## Output Requirements
Return structured results directly (do not write files):
### Supplementary Items
- item_name: Brief explanation (why it should be added)
...
### Recommended Supplementary Fields
- field_name: Field description (why this dimension is needed)
...
### Sources
- [Source1](url1)
- [Source2](url2)
```
### Step 3: Ask User for Existing Fields
Use AskUserQuestion to ask if user has existing field definition file, if so read and merge.
### Step 4: Generate Outline (Separate Files)
Merge {step1_output}, {step2_output} and user's existing fields, generate two files:
**outline.yaml** (items + config):
- topic: Research topic
- items: Research objects list
- execution: - execution:
- batch_size: 并行agent数量(默认5 - batch_size: Number of parallel agents (confirm with AskUserQuestion)
- items_per_agent: 每个agent调研项目数(默认1,需AskUserQuestion确认) - items_per_agent: Items per agent (confirm with AskUserQuestion)
- output_dir: 结果输出目录(默认./results - output_dir: Results output directory (default: ./results)
**fields.yaml**(字段定义): **fields.yaml** (field definitions):
- 字段分类和定义 - Field categories and definitions
- 每个字段的namedescriptiondetail_level - Each field's name, description, detail_level
- uncertain: 不确定字段列表(保留字段,deep阶段自动填充) - uncertain: Uncertain fields list (reserved field, auto-filled in deep phase)
### Step 5: 输出并确认 ### Step 5: Output and Confirm
- 创建目录: `./{topic_slug}/` - Create directory: `./{topic_slug}/`
- 保存: `outline.yaml` `fields.yaml` - Save: `outline.yaml` and `fields.yaml`
- 展示给用户确认 - Show to user for confirmation
## 输出路径 ## Output Path
``` ```
{当前工作目录}/{topic_slug}/ {current_working_directory}/{topic_slug}/
├── outline.yaml # items列表 + execution配置 ├── outline.yaml # items list + execution config
└── fields.yaml # 字段定义 └── fields.yaml # field definitions
``` ```
## 后续命令 ## Follow-up Commands
- `/research-add-items` - 补充items - `/research-add-items` - Supplement items
- `/research-add-fields` - 补充字段 - `/research-add-fields` - Supplement fields
- `/research-deep` - 开始深度调研 - `/research-deep` - Start deep research
+18 -18
View File
@@ -1,30 +1,30 @@
--- ---
description: 向现有调研outline补充字段定义。 description: Add field definitions to existing research outline.
allowed-tools: Bash, Read, Write, Glob, WebSearch, Task, AskUserQuestion allowed-tools: Bash, Read, Write, Glob, WebSearch, Task, AskUserQuestion
--- ---
# Research Add Fields - 补充调研字段 # Research Add Fields - Supplement Research Fields
## 触发方式 ## Trigger
`/research-add-fields` `/research-add-fields`
## 执行流程 ## Workflow
### Step 1: 自动定位Fields文件 ### Step 1: Auto-locate Fields File
在当前工作目录查找 `*/fields.yaml` 文件,自动读取现有fields定义。 Find `*/fields.yaml` file in current working directory, auto-read existing fields definitions.
### Step 2: 获取补充来源 ### Step 2: Get Supplement Source
询问用户选择: Ask user to choose:
- **A. 用户直接输入**:用户提供字段名称和描述 - **A. User direct input**: User provides field names and descriptions
- **B. Web Search搜索**:启动agent搜索该领域常用字段 - **B. Web Search**: Launch agent to search common fields in this domain
### Step 3: 展示并确认 ### Step 3: Display and Confirm
- 展示建议的新字段列表 - Display suggested new fields list
- 用户确认哪些字段需要添加 - User confirms which fields to add
- 用户指定字段分类和detail_level - User specifies field category and detail_level
### Step 4: 保存更新 ### Step 4: Save Update
将确认的字段追加到fields.yaml,保存文件。 Append confirmed fields to fields.yaml, save file.
## 输出 ## Output
更新后的 `{topic}/fields.yaml` 文件(原地修改,需用户确认) Updated `{topic}/fields.yaml` file (in-place modification, requires user confirmation)
+17 -17
View File
@@ -1,28 +1,28 @@
--- ---
description: 向现有调研outline补充items(调研对象)。 description: Add items (research objects) to existing research outline.
allowed-tools: Bash, Read, Write, Glob, WebSearch, Task, AskUserQuestion allowed-tools: Bash, Read, Write, Glob, WebSearch, Task, AskUserQuestion
--- ---
# Research Add Items - 补充调研对象 # Research Add Items - Supplement Research Objects
## 触发方式 ## Trigger
`/research-add-items` `/research-add-items`
## 执行流程 ## Workflow
### Step 1: 自动定位Outline ### Step 1: Auto-locate Outline
在当前工作目录查找 `*/outline.yaml` 文件,自动读取。 Find `*/outline.yaml` file in current working directory, auto-read.
### Step 2: 并行获取补充来源 ### Step 2: Get Supplement Sources in Parallel
同时进行: Simultaneously:
- **A. 询问用户**:需要补充哪些items?有具体名称吗? - **A. Ask user**: What items to supplement? Any specific names?
- **B. 询问是否需要Web Search**:是否启动agent搜索更多items - **B. Ask if Web Search needed**: Launch agent to search for more items?
### Step 3: 合并更新 ### Step 3: Merge and Update
- 将新items追加到outline.yaml - Append new items to outline.yaml
- 展示给用户确认 - Display to user for confirmation
- 避免重复 - Avoid duplicates
- 保存更新后的outline - Save updated outline
## 输出 ## Output
更新后的 `{topic}/outline.yaml` 文件(原地修改) Updated `{topic}/outline.yaml` file (in-place modification)
+78 -37
View File
@@ -1,55 +1,96 @@
--- ---
description: 读取调研outline,为每个item启动独立agent进行深度调研。禁用task output description: Read research outline, launch independent agent for each item for deep research. Disable task output.
allowed-tools: Bash, Read, Write, Glob, WebSearch, Task allowed-tools: Bash, Read, Write, Glob, WebSearch, Task
--- ---
# Research Deep - 深度调研 # Research Deep - Deep Research
## 触发方式 ## Trigger
`/research-deep` `/research-deep`
## 执行流程 ## Workflow
### Step 1: 自动定位Outline ### Step 1: Auto-locate Outline
在当前工作目录查找 `*/outline.yaml` 文件,读取items列表、execution配置(含items_per_agent)。 Find `*/outline.yaml` file in current working directory, read items list, execution config (including items_per_agent).
### Step 2: 断点续传检查 ### Step 2: Resume Check
- 检查output_dir下已完成的JSON文件 - Check completed JSON files in output_dir
- 跳过已完成的items - Skip completed items
### Step 3: 分批执行 ### Step 3: Batch Execution
- batch_size分批(完成一批需要得到用户同意才可进行下一批) - Batch by batch_size (need user approval before next batch)
- 每个agent负责items_per_agent个项目 - Each agent handles items_per_agent items
- 启动web-search-agent(后台并行,禁用task output),使用以下prompt模板: - Launch web-search-agent (background parallel, disable task output)
``` **Parameter Retrieval**:
## 任务 - `{topic}`: topic field from outline.yaml
调研 {item_related_info},输出结构化JSON到 {output_path} - `{item_name}`: item's name field
- `{item_related_info}`: item's complete yaml content (name + category + description etc.)
- `{output_dir}`: execution.output_dir from outline.yaml (default: ./results)
- `{fields_path}`: absolute path to {topic}/fields.yaml
- `{output_path}`: absolute path to {output_dir}/{item_name}.json
## 字段定义 **Hard Constraint**: The following prompt must be strictly reproduced, only replacing variables in {xxx}, do not modify structure or wording.
读取 {topic}/fields.yaml 获取所有字段定义
## 输出要求 **Prompt Template**:
1. 按fields.yaml定义的字段输出JSON ```python
2. 不确定的字段值标注[不确定] prompt = f"""## Task
3. JSON末尾添加uncertain数组,列出所有不确定的字段名 Research {item_related_info}, output structured JSON to {output_path}
## 输出路径 ## Field Definitions
{output_dir}/{item_name}.json Read {fields_path} to get all field definitions
## Output Requirements
1. Output JSON according to fields defined in fields.yaml
2. Mark uncertain field values with [uncertain]
3. Add uncertain array at the end of JSON, listing all uncertain field names
## Output Path
{output_path}
## Validation
After completing JSON output, run validation script to ensure complete field coverage:
python ~/.claude/commands/research/validate_json.py -f {fields_path} -j {output_path}
Task is complete only after validation passes.
"""
``` ```
### Step 4: 等待与监控 **One-shot Example** (assuming researching GitHub Copilot):
- 等待当前批次完成 ```
- 启动下一批 ## Task
- 显示进度 Research name: GitHub Copilot
category: International Product
description: Developed by Microsoft/GitHub, first mainstream AI coding assistant, ~40% market share, output structured JSON to /home/weizhena/AIcoding/aicoding-history/results/GitHub_Copilot.json
### Step 5: 汇总报告 ## Field Definitions
全部完成后输出: Read /home/weizhena/AIcoding/aicoding-history/fields.yaml to get all field definitions
- 完成数量
- 失败/不确定标记的items
- 输出目录
## Agent配置 ## Output Requirements
- 后台执行: 是 1. Output JSON according to fields defined in fields.yaml
- Task Output: 禁用(agent完成时有明确输出文件) 2. Mark uncertain field values with [uncertain]
- 断点续传: 是 3. Add uncertain array at the end of JSON, listing all uncertain field names
## Output Path
/home/weizhena/AIcoding/aicoding-history/results/GitHub_Copilot.json
## Validation
After completing JSON output, run validation script to ensure complete field coverage:
python ~/.claude/commands/research/validate_json.py -f /home/weizhena/AIcoding/aicoding-history/fields.yaml -j /home/weizhena/AIcoding/aicoding-history/results/GitHub_Copilot.json
Task is complete only after validation passes.
```
### Step 4: Wait and Monitor
- Wait for current batch to complete
- Launch next batch
- Display progress
### Step 5: Summary Report
After all complete, output:
- Completion count
- Failed/uncertain marked items
- Output directory
## Agent Config
- Background execution: Yes
- Task Output: Disabled (agent has explicit output file when complete)
- Resume support: Yes
+60 -20
View File
@@ -1,31 +1,71 @@
--- ---
description: 将deep调研结果汇总为markdown报告,覆盖所有字段,跳过不确定值。 description: Summarize deep research results into markdown report, cover all fields, skip uncertain values.
allowed-tools: Read, Write, Glob, Bash allowed-tools: Read, Write, Glob, Bash
--- ---
# Research Report - 汇总报告 # Research Report - Summary Report
## 触发方式 ## Trigger
`/research-report` `/research-report`
## 执行流程 ## Workflow
### Step 1: 定位结果目录 ### Step 1: Locate Results Directory
在当前工作目录查找 `*/outline.yaml`,读取topic和output_dir配置。 Find `*/outline.yaml` in current working directory, read topic and output_dir config.
### Step 2: 生成Python转换脚本 ### Step 2: Generate Python Conversion Script
`{topic}/` 目录下生成 `generate_report.py`,脚本要求: Generate `generate_report.py` in `{topic}/` directory, script requirements:
- 读取output_dir下所有JSON - Read all JSON from output_dir
- 读取fields.yaml获取字段结构 - Read fields.yaml to get field structure
- 覆盖每个JSON的所有字段值 - Cover all field values from each JSON
- 跳过值包含[不确定]的字段 - Skip fields with values containing [uncertain]
- 跳过uncertain字段 - Skip fields listed in uncertain array
- 生成markdown报告格式:目录(带锚点跳转)+ 详细内容(按字段分类) - Generate markdown report format: Table of contents (with anchor links) + Detailed content (by field category)
- 保存到 `{topic}/report.md` - Save to `{topic}/report.md`
### Step 3: 执行脚本 #### Script Technical Requirements (Must Follow)
运行 `python {topic}/generate_report.py`
## 输出 **1. JSON Structure Compatibility**
- `{topic}/generate_report.py` - 转换脚本 Support two JSON structures:
- `{topic}/report.md` - 汇总报告 - Flat structure: Fields directly at top level `{"name": "xxx", "release_date": "xxx"}`
- Nested structure: Fields in category sub-dict `{"basic_info": {"name": "xxx"}, "technical_features": {...}}`
Field lookup order: Top level -> category mapping key -> Traverse all nested dicts
**2. Category Multi-language Mapping**
fields.yaml category names and JSON keys can be any combination (CN-CN, CN-EN, EN-CN, EN-EN). Must establish bidirectional mapping:
```python
CATEGORY_MAPPING = {
"Basic Info": ["basic_info", "Basic Info"],
"Technical Features": ["technical_features", "technical_characteristics", "Technical Features"],
"Performance Metrics": ["performance_metrics", "performance", "Performance Metrics"],
"Milestone Significance": ["milestone_significance", "milestones", "Milestone Significance"],
"Business Info": ["business_info", "commercial_info", "Business Info"],
"Competition & Ecosystem": ["competition_ecosystem", "competition", "Competition & Ecosystem"],
"History": ["history", "History"],
"Market Positioning": ["market_positioning", "market", "Market Positioning"],
}
```
**3. Complex Value Formatting**
- list of dicts (e.g., key_events, funding_history): Format each dict as one line, separate kv with ` | `
- Normal list: Short lists joined with comma, long lists displayed with line breaks
- Nested dict: Recursive formatting, display with semicolon or line breaks
**4. Extra Fields Collection**
Collect fields that exist in JSON but not defined in fields.yaml, put in "Other Info" category. Note to filter:
- Internal fields: `_source_file`, `uncertain`
- Nested structure top-level keys: `basic_info`, `technical_features` etc.
**5. Uncertain Value Skipping**
Skip conditions:
- Field value contains `[uncertain]` string
- Field name is in `uncertain` array
- Field value is None or empty string
### Step 3: Execute Script
Run `python {topic}/generate_report.py`
## Output
- `{topic}/generate_report.py` - Conversion script
- `{topic}/report.md` - Summary report
+219
View File
@@ -0,0 +1,219 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
JSON Field Validation Script
Validate if JSON files completely cover all fields defined in fields.yaml
"""
import json
import yaml
import sys
from pathlib import Path
from typing import Dict, List, Set, Any, Tuple
# Category mapping (supports both Chinese and English)
CATEGORY_MAPPING = {
"Basic Info": ["basic_info", "Basic Info"],
"Technical Features": ["technical_features", "technical_characteristics", "Technical Features"],
"Performance Metrics": ["performance_metrics", "performance", "Performance Metrics"],
"Milestone Significance": ["milestone_significance", "milestones", "Milestone Significance"],
"Business Info": ["business_info", "commercial_info", "Business Info"],
"Competition & Ecosystem": ["competition_ecosystem", "competition", "Competition & Ecosystem"],
"History": ["history", "History"],
"Market Positioning": ["market_positioning", "market", "Market Positioning"],
}
def load_fields_yaml(fields_path: Path) -> Tuple[Set[str], Set[str], Dict[str, str]]:
"""
Load fields.yaml, returns:
- all_fields: Set of all field names
- required_fields: Set of field names where required=true
- field_categories: Mapping from field name to category
"""
with open(fields_path, 'r', encoding='utf-8') as f:
data = yaml.safe_load(f)
all_fields = set()
required_fields = set()
field_categories = {}
for category_info in data.get("field_categories", []):
category_name = category_info["category"]
for field in category_info.get("fields", []):
field_name = field["name"]
all_fields.add(field_name)
field_categories[field_name] = category_name
if field.get("required", False):
required_fields.add(field_name)
return all_fields, required_fields, field_categories
def extract_json_fields(data: Dict, category_mapping: Dict = None) -> Set[str]:
"""
Extract all field names from JSON (supports flat and nested structures)
"""
if category_mapping is None:
category_mapping = CATEGORY_MAPPING
# Get all possible nested keys
nested_keys = set()
for keys in category_mapping.values():
nested_keys.update(keys)
fields = set()
def collect_fields(d: Dict, is_top_level: bool = True):
for k, v in d.items():
# Skip internal fields
if k in {"_source_file", "uncertain"}:
continue
# If it's a nested structure top-level key, recurse into it
if is_top_level and k in nested_keys:
if isinstance(v, dict):
collect_fields(v, is_top_level=False)
else:
fields.add(k)
if isinstance(v, dict):
collect_fields(v, is_top_level=False)
collect_fields(data)
return fields
def validate_json(json_path: Path, all_fields: Set[str], required_fields: Set[str],
field_categories: Dict[str, str]) -> Dict:
"""
Validate a single JSON file
Returns validation result dictionary
"""
with open(json_path, 'r', encoding='utf-8') as f:
data = json.load(f)
json_fields = extract_json_fields(data)
# Calculate coverage
covered = all_fields & json_fields
missing = all_fields - json_fields
extra = json_fields - all_fields
# Categorize missing fields
missing_required = missing & required_fields
missing_optional = missing - required_fields
# Group missing fields by category
missing_by_category = {}
for field in missing:
cat = field_categories.get(field, "Unknown")
if cat not in missing_by_category:
missing_by_category[cat] = []
missing_by_category[cat].append(field)
return {
"file": json_path.name,
"total_defined": len(all_fields),
"covered": len(covered),
"missing": len(missing),
"extra": len(extra),
"coverage_rate": len(covered) / len(all_fields) * 100 if all_fields else 100,
"missing_required": list(missing_required),
"missing_optional": list(missing_optional),
"missing_by_category": missing_by_category,
"extra_fields": list(extra),
"valid": len(missing_required) == 0, # Valid if all required fields are covered
}
def print_result(result: Dict, verbose: bool = True):
"""Print validation result"""
status = "PASS" if result["valid"] else "FAIL"
print(f"\n{'='*60}")
print(f"[{status}] {result['file']}")
print(f"{'='*60}")
print(f"Coverage: {result['coverage_rate']:.1f}% ({result['covered']}/{result['total_defined']})")
if result["missing_required"]:
print(f"\n[ERROR] Missing required fields ({len(result['missing_required'])}):")
for field in result["missing_required"]:
print(f" - {field}")
if verbose and result["missing_optional"]:
print(f"\n[WARN] Missing optional fields ({len(result['missing_optional'])}):")
for cat, fields in result["missing_by_category"].items():
optional_fields = [f for f in fields if f not in result["missing_required"]]
if optional_fields:
print(f" [{cat}]: {', '.join(optional_fields)}")
if verbose and result["extra_fields"]:
print(f"\n[INFO] Extra fields ({len(result['extra_fields'])}):")
print(f" {', '.join(result['extra_fields'][:10])}")
if len(result["extra_fields"]) > 10:
print(f" ... and {len(result['extra_fields']) - 10} more fields")
def main():
"""Main function"""
import argparse
parser = argparse.ArgumentParser(description="Validate if JSON files cover all fields defined in fields.yaml")
parser.add_argument("--fields", "-f", type=str, help="Path to fields.yaml", default="fields.yaml")
parser.add_argument("--json", "-j", type=str, nargs="*", help="JSON file paths to validate")
parser.add_argument("--dir", "-d", type=str, help="JSON files directory", default="results")
parser.add_argument("--quiet", "-q", action="store_true", help="Show summary only")
args = parser.parse_args()
# Locate fields.yaml
fields_path = Path(args.fields)
if not fields_path.exists():
# Try to find in current and parent directory
for p in [Path.cwd() / "fields.yaml", Path.cwd().parent / "fields.yaml"]:
if p.exists():
fields_path = p
break
if not fields_path.exists():
print(f"[ERROR] fields.yaml not found: {fields_path}")
sys.exit(1)
print(f"Field definitions file: {fields_path}")
all_fields, required_fields, field_categories = load_fields_yaml(fields_path)
print(f"Total fields: {len(all_fields)} (required: {len(required_fields)}, optional: {len(all_fields) - len(required_fields)})")
# Collect JSON files
json_files = []
if args.json:
json_files = [Path(p) for p in args.json]
else:
json_dir = Path(args.dir)
if json_dir.exists():
json_files = sorted(json_dir.glob("*.json"))
if not json_files:
print(f"[WARN] No JSON files found")
sys.exit(0)
# Validate each file
results = []
for json_path in json_files:
if not json_path.exists():
print(f"[WARN] File not found: {json_path}")
continue
result = validate_json(json_path, all_fields, required_fields, field_categories)
results.append(result)
print_result(result, verbose=not args.quiet)
# Summary
print(f"\n{'='*60}")
print("Summary")
print(f"{'='*60}")
passed = sum(1 for r in results if r["valid"])
avg_coverage = sum(r["coverage_rate"] for r in results) / len(results) if results else 0
print(f"Validation passed: {passed}/{len(results)}")
print(f"Average coverage: {avg_coverage:.1f}%")
if passed < len(results):
sys.exit(1)
if __name__ == "__main__":
main()