Refactor: Caching tokenize result (#805)
# PR Checklist - [ ] Have you checked if it works normally in all models? *Ignore this if it doesn't use models.* - [ ] Have you checked if it works normally in all web, local, and node hosted versions? If it doesn't, have you blocked it in those versions? - [ ] Have you added type definitions? # Improved performance  The picture above was taken when I sent a chat. As you can see, the encode function occupies **0.68 seconds** out of 1.8 seconds. This is **36.5%** of sending time  This is a picture taken after adding a caching function. encode time Reduced from **0.68 seconds to 0.12 seconds, an improvement of 27.2%** Since configuration values are also used as key values in the hash function, it is guaranteed that **different results will be produced when the configuration changes**. I tested this myself by changing the bot settings # Implementation I added the `mnemonist` library to implement this. At first, I used lru-cache, but that library recommended using this one if performance is important. It actually performed better too. # Integrity Check  Lastly, the encoding result calculated without caching when sending the first chat, the newly calculated value when sending the second chat, and the result cached when sending the first chat were all the same.
This commit is contained in:
@@ -66,6 +66,7 @@
|
||||
"lucide-svelte": "^0.292.0",
|
||||
"markdown-it": "^14.1.0",
|
||||
"ml-distance": "^4.0.1",
|
||||
"mnemonist": "^0.40.3",
|
||||
"mobile-drag-drop": "3.0.0-rc.0",
|
||||
"msgpackr": "1.10.1",
|
||||
"node-fetch": "2",
|
||||
|
||||
Reference in New Issue
Block a user