mincomk/risuai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: validate hypav2 data

Browse Source 
# Changelist:
## 1. Types
### MainChunks
Added id(int), and chatMemos(Set<string>)
Id: incremental int starting from 0
chatMemos: A set of UUID, containing which chat has been summarized in it.
### Chunks
mainChunkID: A connection of which mainChunk it has been split from
text: the split text data

## 2. Features
### CleanInvalidChunks
Called every time when chat is updated and Hypamemory is used.

Gets all the memo(UUID)s of current chats, and creates a set.
Then checks if each mainChunk's chatMemos set is subset of the enitre memo set. If not, the summarized part's chat is deleted/edited. The mainChunk is filtered out.
Concurrently, the chunks that are split from that mainChunk is also deleted.
This commit is contained in:
LightningHyperBlaze45654
2024-12-03 23:28:18 -08:00
parent 46502e762b
commit b8bb2330cc
2 changed files with 69 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
src/ts/process/memory/hypamemory.ts
View File

@@ -1,8 +1,7 @@
import localforage from "localforage";
import { globalFetch } from "src/ts/globalApi.svelte";
import { runEmbedding } from "../transformers";
import { alertError } from "src/ts/alert";
import { appendLastPath } from "src/ts/util";
import {globalFetch} from "src/ts/globalApi.svelte";
import {runEmbedding} from "../transformers";
import {appendLastPath} from "src/ts/util";
export class HypaProcesser{
@@ -139,8 +138,7 @@ export class HypaProcesser{
}
async similaritySearchScored(query: string) {
const results = await this.similaritySearchVectorWithScore((await this.getEmbeds(query))[0],);
return results
return await this.similaritySearchVectorWithScore((await this.getEmbeds(query))[0],)
}
private async similaritySearchVectorWithScore(