Commit Graph

3230 Commits

Author SHA1 Message Date
kwaroran
8a4932e19c suggestion: apply 'processScriptFull()' to 'editprocess' mode regex script (#725)
# PR Checklist
- [ ] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [ ] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [ ] Have you added type definitions?

# Description
This PR addresses an issue where the `{{chat_index}}` cbs does not work
correctly in regex scripts of the 'Modify Request Data' (editprocess)
type. To work around this issue, I propose changing the function used in
this section to `processScriptFull()`.

If you have any other suggestions, such as modifying the arguments
instead, please feel free to let me know.

Thanks for your time and consideration.
2025-01-17 22:52:49 +09:00
bangonicdd
172efe4d68 suggestion: apply 'processScriptFull()' to 'editprocess' mode regex script 2025-01-17 03:58:20 +09:00
kwaroran
521a374215 feat: improve HypaV3 (#720)
# PR Checklist
- [x] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [x] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [ ] Have you added type definitions?

# Preview

![preview](https://github.com/user-attachments/assets/9cbd7cd0-fc9d-40d4-bcc7-69c5bfb000a2)

# Description
- refactor: improve array conversion and sorting syntax
- fix: resolve summarize function issues with instruct35
- fix: adjust memory selection order
- fix: restore undefined value from null after importing hypaDataV3
- feat: add expandable chat memo in HypaV3 Data modal
- feat: add important button in HypaV3 Data modal
- fix: message line break display in HypaV3 Data modal
- refactor: extract HypaV3 modal into separate component
2025-01-15 00:00:03 +09:00
Bo26fhmC5M
a10a2c5502 refactor: extract HypaV3 modal into separate component 2025-01-14 13:01:14 +09:00
Bo26fhmC5M
030e802122 fix: message line break display in HypaV3 Data modal 2025-01-14 01:10:06 +09:00
Bo26fhmC5M
f3c6278d21 feat: add important button in HypaV3 Data modal 2025-01-13 23:30:18 +09:00
Bo26fhmC5M
a7c3141cc0 feat: add expandable chat memo in HypaV3 Data modal 2025-01-13 22:33:27 +09:00
Bo26fhmC5M
4176a647a5 fix: restore undefined value from null after importing hypaDataV3 2025-01-13 19:31:24 +09:00
Bo26fhmC5M
a1158ebdb1 fix: adjust memory selection order 2025-01-13 14:09:18 +09:00
Bo26fhmC5M
cfdd5d6bf5 fix: resolve summarize function issues with instruct35 2025-01-13 13:30:10 +09:00
Bo26fhmC5M
04e640a791 refactor: improve array conversion and sorting syntax 2025-01-13 13:02:22 +09:00
kwaroran
3fa199dbfa Improve prompt comparison functionality with usability enhancements (#716)
# PR Checklist
- [ ] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [x] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [x] Have you added type definitions?

# Description
This PR introduces a couple of usability improvements to the prompt
comparison functionality.

1. If the two prompts are identical

Instead of showing the content right away, an alert is displayed saying
the two prompts are the same. Only when the user clicks "Yes" the
content will be shown. This avoids unnecessary confirmation of identical
prompts.

<img width="519" alt="a"
src="https://github.com/user-attachments/assets/bf48420e-bf69-4c8a-b700-754e1c38b6a6"
/>


2. If the two prompts are different

A summary of the number of modified, added, and removed lines is
displayed.

<img width="598" alt="b"
src="https://github.com/user-attachments/assets/c3269ee8-25e7-4ba9-957d-4e377d523795"
/>

When hovering over the summary, a quick overview of the changes appears.
When only a single word or character is modified, it was previously hard
to identify the line that was changed. Now, the overview improves
clarity, making it easier to understand the changes.

<img width="595" alt="e"
src="https://github.com/user-attachments/assets/1b55a3bc-70d2-43fa-ab4c-754e6fd4a85d"
/>

On mobile devices (tested on iOS 16.7), tapping on the 'Differences
detected. Please review the changes.' section will display the changes.

---

As a future improvement, it might be useful to display line numbers or
allow users to click and navigate to the specific parts of the
differences.

I hope these changes make it easier to use, but feel free to reject this
PR if you find it unnecessary or if the code feels too messy. I
completely understand and appreciate your feedback.

Thank you for your time!
2025-01-12 23:59:13 +09:00
kwaroran
876130fc68 feat: Implement HypaV3 ratio-based memory (Variation of HypaV2) (#717)
# PR Checklist
- [x] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [x] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [x] Have you added type definitions?

# Preview

![preview](https://github.com/user-attachments/assets/6555534b-d798-49bf-a0d5-79148c3445b3)

# Description
Implemented HypaV3, a variation of the HypaV2 memory.

---
Key Changes:
- Ratio-based token management instead of fixed token counts
(memoryTokensRatio, extraSummarizationRatio)
- Improved memory selection with adjustable ratios
(recent/similar/random memory)
- Similar memory selection now uses parent summaries instead of
individual chunks
- Selected memories are chronologically ordered
- Control over summary units (maxChatsPerSummary)
- Added orphaned memory preservation option (preserveOrphanedMemory)

---
New Settings:
- memoryTokensRatio: Ratio of memory prompt in max context size
- extraSummarizationRatio: Extra margin ratio of max context size to
keep more free space when summarizing begins
- maxChatsPerSummary: Maximum chats per summary
- similarMemoryRatio: Ratio for similarity-based memory selection
- randomMemoryRatio: Ratio for random memory selection
- enableSimilarityCorrection: Compare chunks with both raw and
summarized version of last 3 chats for better similarity matching
- preserveOrphanedMemory: Keep summaries even when original chats are
deleted

---
Summarization Conditions:
- Start: When current tokens exceed max context size
- End: When either:
- Current tokens drop below maxContextTokens * (1 - memoryTokensRatio -
extraSummarizationRatio)
- Current tokens are within max context size and only minimum required
messages (3) remain

---
Memory Selection Process:
- Recent: Newest summaries up to recentMemoryRatio (1 -
similarMemoryRatio - randomMemoryRatio)
- Random: Unused summaries selected randomly up to randomMemoryRatio
- Similar: Unused summaries selected by similarity to last 3 messages
- Each summary is split into chunks and compared with raw recent
messages for similarity scoring
- When similarity correction enabled, also compares with summarized
version of recent messages
  - Parent summaries of the most similar chunks are selected
- Unused token space from recent and random selections is added to
similar memory's token limit

---
Key Differences from HypaV2:
- Flexible ratio-based memory management instead of absolute values
- Selects parent summaries instead of individual chunks for similar
memory
- UI shows original chat's memos linked to each summary
2025-01-12 23:59:03 +09:00
Bo26fhmC5M
ab7a2aa499 feat: add resummarize button to HypaV3 Data modal 2025-01-12 23:32:20 +09:00
Bo26fhmC5M
be1713df4b feat: add trash button to HypaV3 Data modal 2025-01-12 19:12:04 +09:00
Bo26fhmC5M
f08c31612b fix: prevent over-summarization and improve logging 2025-01-12 17:15:11 +09:00
Bo26fhmC5M
6e49970b76 fix: correct targetTokens calculation in summarization and improve logging 2025-01-12 14:49:01 +09:00
Bo26fhmC5M
85bdf1d544 fix: improve memory token tracking and logging 2025-01-12 13:17:56 +09:00
Bo26fhmC5M
50361d7aa2 feat: Implement HypaV3 ratio-based memory 2025-01-12 11:05:18 +09:00
poroyo
2ee72ca86f feat: add tooltip for differences overview 2025-01-11 18:30:45 +09:00
poroyo
d9f4951c1c feat: add prompt comparison alert for identical prompts 2025-01-11 14:33:06 +09:00
kwaroran
3b533e911f fix backup in non-tauri environment (#713)
# PR Checklist
- [ ] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [ ] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [ ] Have you added type definitions?

# Description
EN
fix non-tauri environment backup issue. (e.g. backwards reset issue)
fix applying duplicate backup file.

KR
Tauri 환경(로컬리스)가 아닌 상황에서 과거 20개의 백업본 이외에 나머지 백업본을 저장하지 않던 문제를 수정합니다.
(뒤로가기 이슈)
백업본 적용 시 모든 백업본을 적용해보던 로직을 개선합니다.
2025-01-11 01:44:58 +09:00
kwaroran
c363d50aa3 fix: gemini multimodal (#712)
# PR Checklist
- [ ] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [ ] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [ ] Have you added type definitions?

# Description
fix gemini multimodal error.
ERROR MESSAGE : Invalid value at 'contents[6].parts[1]' (oneof), oneof
field 'data' is already set. Cannot set 'text'
2025-01-11 01:44:30 +09:00
kwaroran
edb4964da7 fix: Fix message handling and summarization in HypaV2 (#711)
# PR Checklist
- [x] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [x] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [ ] Have you added type definitions?

# Description
This PR addresses several issues in the HypaV2 memory system:

1. Improved Summarization Error Handling
- Before: When unable to summarize messages while exceeding context
size, HypaV2 would continue and make API requests with unintentionally
truncated context
- After: HypaV2 now provides clear error messages asking users to adjust
their settings when:
  - Unable to summarize last 4 messages
  - A single message exceeds chunk size

2. Message Order Preservation
- Before: lastTwoChats logic was causing the first message (index 1) and
user input (index 2) to appear in the wrong order
- After: Fixed by adjusting the initial index to preserve the original
message sequence

3. Message Filtering
- Added logic to skip unnecessary messages during summarization:
  - Excludes [Start a new chat] message at index 0
  - Skips empty messages
<br>
Requesting review from @LightningHyperBlaze45654
2025-01-11 01:44:10 +09:00
kwaroran
9896b68b7a fix: resolve state_unsafe_mutation in getChatVar() (#707)
# PR Checklist
- [ ] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [ ] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [ ] Have you added type definitions?

# Description
While investigating an issue where `{{getvar::A}}` didn't work properly
in background embedding, I encountered a `state_unsafe_mutation` error.
It appears that modifying the null check operator on the relevant line
can resolve this issue. Thank for your time.
2025-01-11 01:44:01 +09:00
Bo26fhmC5M
7c1faee6f4 fix: Fix message handling and summarization in HypaV2
- Add clear error messages when summarization is not possible
- Fix message order issues caused by lastTwoChats logic
- Skip [Start a new chat] and empty messages during summarization
2025-01-08 20:45:19 +09:00
jinook518
07817c9d8b fix backup 2025-01-08 00:37:31 +09:00
donmarble
2d11b05e6c fix: gemini multimodal text 2025-01-07 23:45:01 +09:00
Kwaroran
64b9b89220 fix: remove geminiBlockOff flag from LLM models 2025-01-07 00:14:58 +09:00
Kwaroran
708faeaea8 chore: update application version to 146.1.0 2025-01-07 00:07:24 +09:00
Kwaroran
9fc8978d1d Add GeminiBlockOff 2025-01-07 00:05:20 +09:00
Kwaroran
59d33a9c8b fix: enhance content-type handling in downloadRisuHub function 2025-01-06 23:13:42 +09:00
bangonicdd
6537e0926c fix: add missing check for regex order flag 2025-01-06 15:39:57 +09:00
bangonicdd
806ec4bc97 fix: resolve state_unsafe_mutation in getChatVar() 2025-01-06 09:32:35 +09:00
Kwaroran
3849d1e781 chore: update application version to 146.0.0 2025-01-06 00:30:59 +09:00
Kwaroran
2ec13a4550 Modify some parts of PR 2025-01-06 00:30:33 +09:00
Kwaroran
c0e1311568 Add branches 2025-01-06 00:24:03 +09:00
kwaroran
aacb60eed9 fix: HypaV2 serialization and token calculation (#701)
# PR Checklist
- [x] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [x] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [x] Have you added type definitions?

# Description
This PR addresses two issues in HypaV2:

1. Fixes JSON serialization of chatMemos in HypaV2Data by converting Set
to string array
2. Corrects token calculation by subtracting tokens from previously
summarized chats

Changes:
- Added SerializableHypaV2Data interface with string[] instead of
Set<string>
- Added conversion functions between HypaV2Data and
SerializableHypaV2Data
- Fixed token calculation by properly accounting for removed chats

Note:
I'm not entirely confident about the token calculation changes. A
thorough review would be appreciated, particularly regarding:
- Whether token subtraction for removed chats is the correct approach
- Potential edge cases in the current implementation 

Please review these changes, especially the token calculation logic, to
ensure it aligns with the intended behavior of the memory system.
2025-01-06 00:11:03 +09:00
kwaroran
7d1e9dc9bd feat: add prompt comparison feature (#704)
# PR Checklist
- [ ] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [x] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [x] Have you added type definitions?

# Description
This PR adds a **prompt comparison feature** to the prompt preset
interface. Now, users can compare two different prompts to identify
differences.

## How to Use
1. Open the prompt preset window.
2. Click the diff button (next to the copy button) for the first prompt
to use as the baseline. The button will turn green, indicating
selection.
<img width="487" alt="c"
src="https://github.com/user-attachments/assets/c2dcf5fa-c4ee-4b3f-8e52-3f0866b12bc4"
/>

3. Click the diff button for the second prompt to compare against the
baseline. A diff view will appear.
4. Clicking the same diff button twice will clear the selection.

## Diff Display Details
- Line-level comparison
- Modified lines: blue vertical line.
- Deleted content: red text on red background with red vertical line.
- Added content: green text on light green background with green
vertical line.

<img width="597" alt="b"
src="https://github.com/user-attachments/assets/0d026e9e-a7a0-4a17-9b80-a2b57c74d7f9"
/>


- If the prompt content is identical, the following message will be
displayed at the top

<img width="600" alt="a"
src="https://github.com/user-attachments/assets/dd5f36f2-9e96-4279-9f9f-79a17f9e4c89"
/>


## Implementation Details
1. `handleDiffMode` manages prompt selection and clearing.
2. `checkDiff` compares prompts and uses `highlightChanges` to mark
differences.
3. Special characters are escaped with `escapeHtml` to ensure the text
is displayed as-is.
4. `resultHtml` is rendered via `alertMd`.

## Notes
- This feature uses the `jsdiff` library to compare prompts efficiently.
- The comparison includes the role, type1, and type2 fields (e.g., ##
system; plain; main). Even if the prompts' text is identical,
differences in these fields will be treated as a mismatch.
- The rendering process in `alertMd` appears to sanitize potentially
dangerous content. However, additional escaping is applied to ensure
that the text is displayed as-is.
- `botpreset.svelte` grew significantly due to this feature;
modularization was considered but not implemented.
- The reason for using the "Prompt Preset" window instead of the "Prompt
Preview" feature is that "Prompt Preview" displays the final form with
CBS processing applied. Even if the content in "Prompt Preview" appears
identical, the actual prompts can differ significantly.

If this feature, its implementation, or any other issue doesn't fit the
project's vision, feel free to reject this PR! Thank you for reviewing!
2025-01-06 00:10:51 +09:00
kwaroran
5c8adcb288 fix: replace unsupported Hrkt unicode property to Hira and Kana (#705)
# PR Checklist
- [x] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [x] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [ ] Have you added type definitions?

# Description
`Auto Regenerate On Characterset` feature can throw the following error
when `Hrkt` is enabled:
```
Error while parsing chat message: true, Invalid regular expression: /\p{Script=Hrkt}/gu: Invalid property name
```

This PR fixes regex parsing issues with unicode property:
1. Replaces unsupported `Hrkt` property with separate `Hira` and `Kana`
properties
2. Updates preview text for each property to match the changes
2025-01-06 00:10:35 +09:00
Bo26fhmC5M
bb93e04961 fix: replace unsupported Hrkt unicode property to Hira and Kana 2025-01-05 19:32:31 +09:00
poroyo
495163c9cf style: improve readability for added and removed lines with border-left 2025-01-05 16:11:00 +09:00
poroyo
8ba4417065 feat: add prompt comparison feature 2025-01-05 14:20:20 +09:00
kwaroran
bf8bf02e2f fix: backup size limit (#702)
## Issue track
Previously, the backup loading was depending on ArrayBuffer. In chromium
based browsers, ArrayBuffer's size is limited to 2GB, and anything more
will fail to read the data.

This affects Tauri too, as Tauri uses Edge webview2 which also is based
on chromium.

The proposed change uses file stream instead of ArrayBuffer, effectively
ignoring forced file size limit of 2GB.

# PR Checklist
- [ ] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [x] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [ ] Have you added type definitions?

# Description
Have to revise, if possible. But really didn't changed that much logic
and since it is only a typescript file change, it will work on most
cases.

Worked on node hosted chrome environment with 2.25GB of binary backup
file.
2025-01-03 18:58:26 +09:00
LightningHyperBlaze45654
48b377b1ad fix: backup size limit
## Issue track
Previously, the backup loading was depending on ArrayBuffer. In chromium based browsers, ArrayBuffer's size is limited to 2GB, and anything more will fail to read the data.

This affects Tauri too, as Tauri uses Edge webview2 which also is based on chromium.

The proposed change uses file stream instead of ArrayBuffer, effectively ignoring file size limit/
2025-01-02 11:52:32 -08:00
Bo26fhmC5M
0317a941f1 fix: token calculation for HypaV2 2025-01-02 20:52:37 +09:00
Bo26fhmC5M
2b92582df4 fix: resolve chatMemos serialization issue in HypaV2Data 2025-01-02 18:47:16 +09:00
Kwaroran
821ba1812f Merge branch 'main' of https://github.com/kwaroran/RisuAI 2025-01-02 03:03:03 +09:00
Kwaroran
866996ba1e chore: update version to 145.3.0 2025-01-02 03:02:54 +09:00
kwaroran
e97715843b Add Custom Character Folder Image Feature Using Global Module Assets (#697)
# PR Checklist
- [ ] Have you checked if it works normally in all models? *Ignore this
if it doesn't use models.*
- [x] Have you checked if it works normally in all web, local, and node
hosted versions? If it doesn't, have you blocked it in those versions?
- [x] Have you added type definitions?

# Description
This PR adds a new feature that allows users to add custom images to
character folders. Instead of uploading images directly, this feature
utilizes the existing image pool from **the global module**. Users can
either select from existing images in the global module or upload new
images to the module.

The users can use a context menu on a folder to update its image.

## Why Use the Global Module?

The global module was chosen for several reasons:  
1. It allows the reuse of existing functions and code, minimizing the
need for new implementations.
2. Users can easily select existing images without re-uploading them.  
3. The global module provides reliable support for image upload and
deletion.
4. Images uploaded via the global module are accessible across all
platforms where the shared data is available.

## Note

To retrieve the list of assets from the global module, I used the
following code:

```typescript
let assetPaths: { [key: string]: { path: string } } = {};

const moduleAssets = getModuleAssets();
if (moduleAssets.length > 0) {
  for (const asset of moduleAssets) {
    const assetPath = await getFileSrc(asset[1]);
    assetPaths[asset[0].toLocaleLowerCase()] = {
      path: assetPath,
    };
  }
}
```

I reused the `parseAdditionalAssets` code from
`/src/ts/parser.svelte.ts` because it seemed simple and practical for
this feature. I thought about refactoring it into a shared function but
decided to keep it simple for now. I’m not sure if this is the best
approach, but it can always be changed later if needed.

---

If you are already working on a similar feature, find any issues with
this code, or feel this PR does not align with the project's direction,
I fully understand if this PR is not accepted.

Thank you!

---

One last note: With *transparent background images* and the existing
folder color functionality, users can differentiate categories even when
using the same image by applying different colors!
2025-01-02 02:56:49 +09:00