@@ -1,29 +1,114 @@ |
| 1 | | -# dlm-vsc |
| 1 | +# DLM — Document Language Model |
| 2 | 2 | |
| 3 | | -VSCode extension for `.dlm` document authoring. |
| 3 | +> Turn a text file into a fine-tuned LLM. Edit the document, retrain, share. |
| 4 | + |
| 5 | +A `.dlm` file is YAML frontmatter + markdown body with `::instruction::` and `::preference::` sections. Train a LoRA adapter on your document, prompt it, export to Ollama. This extension makes `.dlm` authoring a first-class editor experience. |
| 6 | + |
| 7 | + |
| 4 | 8 | |
| 5 | 9 | ## Features |
| 6 | 10 | |
| 7 | | -- **Syntax highlighting** for `.dlm` files (YAML frontmatter + markdown body + section fences) |
| 8 | | -- **Completions** for base models, adapter types, quantization levels, and section fences |
| 9 | | -- **Hover info** showing base model specs and section metadata |
| 10 | | -- **Diagnostics** for schema errors, unknown base models, and doctor warnings |
| 11 | | -- **Side panel** with: |
| 12 | | - - Quick-insert palette for instruction, preference, image, and audio sections |
| 13 | | - - Source directory picker with native file dialog |
| 14 | | - - Searchable base model browser (click to set) |
| 15 | | - - Template gallery for bootstrapping new documents |
| 16 | | - - Training controls (run/stop with watch mode) |
| 11 | +### Syntax-aware editing |
| 12 | + |
| 13 | +YAML frontmatter, markdown body, and section fences (`::instruction::`, `::preference::`, `::image::`, `::audio::`) get distinct, theme-aware highlighting. Q/A headers inside instruction blocks are styled. |
| 14 | + |
| 15 | + |
| 16 | + |
| 17 | +### Smart completions |
| 18 | + |
| 19 | +Autocomplete for the 26-entry base-model registry, adapter types, quantization levels, and section fences — all driven by the live schema. |
| 20 | + |
| 21 | + |
| 22 | + |
| 23 | +### Hover insight |
| 24 | + |
| 25 | +Hover any base-model key to see params, VRAM footprint, context length, modality, and license. Hover a section fence for the section ID and quick stats. |
| 26 | + |
| 27 | + |
| 28 | + |
| 29 | +### Live diagnostics |
| 30 | + |
| 31 | +Schema errors, unknown base models, and Doctor warnings (VRAM headroom, unsafe combinations) surface inline as you type — no save required. |
| 32 | + |
| 33 | + |
| 34 | + |
| 35 | +### Side panel — frictionless authoring |
| 36 | + |
| 37 | +A dedicated DLM activity-bar view with everything you need to compose a `.dlm`: |
| 38 | + |
| 39 | +- **Quick Insert** — Instruction, Preference, Image, and Audio buttons drop snippet templates with tab stops at the cursor |
| 40 | +- **Source Directory Manager** — native folder picker; relative paths computed and inserted into frontmatter via the LSP |
| 41 | +- **Base Model Browser** — searchable QuickPick over the full registry; click to set `base_model` |
| 42 | +- **Template Gallery** — 8 bundled templates with one-click bootstrap |
| 43 | +- **Document Overview** — section counts, store status, adapter version |
| 44 | +- **Training Controls** — Run / Stop in an integrated terminal |
| 45 | + |
| 46 | + |
| 47 | + |
| 48 | +### Transparent store creation |
| 49 | + |
| 50 | +Open a `.dlm` file. The content-addressed store at `~/.dlm/store/<dlm_id>/` is created on the spot — no manual `dlm init` needed. |
| 17 | 51 | |
| 18 | 52 | ## Requirements |
| 19 | 53 | |
| 20 | | -- [dlm](https://github.com/tenseleyFlow/DocumentLanguageModel) CLI installed |
| 21 | | -- [dlm-lsp](https://github.com/tenseleyFlow/dlm-lsp) language server: `pip install dlm-lsp` |
| 54 | +1. **Python 3.11+** with the `document-language-model` package: |
| 55 | + |
| 56 | + ```bash |
| 57 | + pip install document-language-model |
| 58 | + ``` |
| 59 | + |
| 60 | +2. **`dlm-lsp` language server**: |
| 61 | + |
| 62 | + ```bash |
| 63 | + pip install dlm-lsp |
| 64 | + ``` |
| 65 | + |
| 66 | +3. *(Optional)* For training, choose a hardware-appropriate base. The Doctor panel will tell you what fits. |
| 67 | + |
| 68 | +## Quick start |
| 69 | + |
| 70 | +1. Install the extension. |
| 71 | +2. Create a new file with a `.dlm` extension. The side panel becomes available immediately. |
| 72 | +3. Click **Add Instruction** in the side panel and fill in the Q / A. |
| 73 | +4. Open the **Base Model Browser** and pick a small base (e.g. `qwen2.5-0.5b`) for a fast first run. |
| 74 | +5. Run `DLM: Train Current Document` from the command palette (⇧⌘P / Ctrl+Shift+P). |
| 75 | +6. Run `DLM: Prompt` to chat with your trained adapter. |
| 76 | +7. Run `DLM: Export` to package an Ollama-ready `Modelfile + GGUF`. |
| 77 | + |
| 78 | +## Commands |
| 79 | + |
| 80 | +| Command | What it does | |
| 81 | +| --------------------------------- | ---------------------------------------------- | |
| 82 | +| `DLM: Train Current Document` | Fine-tune the LoRA adapter on the open `.dlm` | |
| 83 | +| `DLM: Export` | Export `base.gguf + adapter.gguf + Modelfile` | |
| 84 | +| `DLM: Synth Instructions` | Generate synthetic Q/A pairs from your sources | |
| 85 | +| `DLM: Show Run History` | Browse training runs for the current document | |
| 86 | +| `DLM: Open Store Directory` | Reveal `~/.dlm/store/<id>` in the file manager | |
| 87 | +| `DLM: Insert Instruction Section` | Drop an instruction block at the cursor | |
| 88 | +| `DLM: Insert Preference Section` | Drop a preference block at the cursor | |
| 89 | + |
| 90 | +## Settings |
| 91 | + |
| 92 | +| Setting | Default | Description | |
| 93 | +| -------------------- | ------------- | ---------------------------------------------------------- | |
| 94 | +| `dlm.command` | `uv run dlm` | Command to invoke the dlm CLI (use `dlm` for system venv) | |
| 95 | +| `dlm.home` | `""` | Override the `~/.dlm` store root | |
| 96 | +| `dlm.defaultBase` | `""` | Default `base_model` for new documents | |
| 97 | +| `dlm.watchOnSave` | `false` | Auto-retrain on save | |
| 98 | +| `dlm.lspPath` | `dlm-lsp` | Path to the `dlm-lsp` binary | |
| 99 | + |
| 100 | +If `dlm` is on your PATH (e.g. installed system-wide), set `dlm.command` to `dlm`. The default `uv run dlm` works inside a `uv`-managed project. |
| 101 | + |
| 102 | +## Multi-editor support |
| 103 | + |
| 104 | +The language server is editor-agnostic. Cookbook entries for Zed, Helix, and Neovim live in the [main repository](https://github.com/tenseleyFlow/DocumentLanguageModel/tree/trunk/docs/cookbook). |
| 105 | + |
| 106 | +## Links |
| 107 | + |
| 108 | +- [DLM main repository](https://github.com/tenseleyFlow/DocumentLanguageModel) |
| 109 | +- [dlm-lsp language server](https://github.com/tenseleyFlow/dlm-lsp) |
| 110 | +- [Issues](https://github.com/tenseleyFlow/dlm-vsc/issues) |
| 22 | 111 | |
| 23 | | -## Development |
| 112 | +## License |
| 24 | 113 | |
| 25 | | -```bash |
| 26 | | -npm install |
| 27 | | -npm run build |
| 28 | | -# Press F5 in VSCode to launch Extension Development Host |
| 29 | | -``` |
| 114 | +MIT |