Commit a053567
committed
Refactor: Improve markdown splitting and chunking logic
- Refactor to handle H1-H6 headings and improve chunking logic.
- Refactor to improve chunking logic and remove unused metadata parameter.
- Refactor to improve chunking logic and remove unused metadata parameter.
- Update tests for all splitters to reflect changes.
- Reduce in from 4000 to 1500.
- Update .clinerules to use backticks for file paths.
- Add (new files).
- Refactor splitter types (new file).1 parent d3aa894 commit a053567
File tree
15 files changed
+999
-365
lines changed- src
- splitter
- splitters
- store
15 files changed
+999
-365
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
4 | | - | |
| 3 | + | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | | - | |
11 | | - | |
| 9 | + | |
| 10 | + | |
12 | 11 | | |
13 | 12 | | |
14 | 13 | | |
15 | | - | |
| 14 | + | |
16 | 15 | | |
17 | 16 | | |
18 | 17 | | |
| |||
0 commit comments