feat: initial import of all helper scripts from ~/scripts/

- Training data pipeline: convert, export, extract, load-to-db
- Infra tooling: infra-audit, infra-gitea-link
- RAG pipeline: rag-ingest, rag-query
- Fine-tuning: finetune-lora, overnight-qwen3, install-unsloth
- Transcripts: export-transcripts
- Updated README with script index and token reduction strategy
This commit is contained in:
Grace
2026-03-16 22:32:48 -07:00
parent 462f5298e6
commit 014ec8bd5c
15 changed files with 2979 additions and 1 deletions

View File

@@ -1,3 +1,36 @@
# grace-scripts
Grace AI helper scripts — token-efficient homelab automation
Grace AI helper scripts — token-efficient homelab automation.
These scripts replace inline shell construction in AI sessions, reducing token usage and making repeated tasks reproducible and auditable.
## Scripts
| Script | Purpose |
|--------|---------|
| `convert-training-data.py` | Convert raw session exports to unsloth/axolotl-ready JSONL |
| `export-training-data.py` | Export training data from OpenClaw sessions |
| `export-transcripts.py` | Export session transcripts incrementally |
| `extract-sessions.py` | Extract and parse session data |
| `finetune-lora.py` | Run LoRA fine-tuning on local Qwen3-8B |
| `infra-audit.py` | Audit homelab infra state and sync to MongoDB |
| `infra-gitea-link.py` | Link infra components to Gitea issues/repos |
| `load-transcripts-to-db.py` | Sync transcripts to PostgreSQL on DB VM |
| `overnight-qwen3.py` | Overnight batch inference job |
| `overnight-research.sh` | Overnight research and summarization |
| `rag-ingest.py` | Ingest documents into Qdrant for RAG |
| `rag-query.py` | Query Qdrant RAG index |
| `install-unsloth.sh` | Install unsloth fine-tuning framework |
## Credential Protocol
All scripts pull credentials from Infisical. Never hardcode secrets.
IPs and service URLs reference TOOLS.md / BOOTSTRAP.md in the Grace workspace.
## Token Reduction Strategy
Each script replaces multi-step inline shell that Grace would otherwise construct token-by-token in every session. One `exec` call to a script = ~10-50x fewer tokens than building the equivalent inline.
## Issues & Improvements
Use Gitea issues on this repo to track bugs, improvements, and new script requests.