kwaroran
a9872b3d5d
Update localModelPath in transformers.ts
2024-01-06 20:25:03 +09:00
kwaroran
1114d5b61b
Update version to 1.71.1
2024-01-06 19:46:19 +09:00
kwaroran
229aeced06
Merge branch 'main' of https://github.com/kwaroran/RisuAI
2024-01-06 19:45:57 +09:00
kwaroran
5a305a8f10
Fix sw register temp
2024-01-06 19:45:50 +09:00
kwaroran
cba3ff802c
Add selectable tokenizer supports on Ooba ( #281 )
...
# PR Checklist
- [ ] Did you check if it works normally in all models? *ignore this
when it dosen't uses models*
- [ ] Did you check if it works normally in all of web, local and node
hosted versions? if it dosen't, did you blocked it in those versions?
- [ ] Did you added a type def?
# Description
I write simple changes on code, which allow user to choose tokenizers.
As I write on https://github.com/kwaroran/RisuAI/issues/280 , differences
in tokenizers makes error when use mistral based models.
As I'm not good at javascript, I simply implement this work by write
name of tokenizer model, and select one on tokenizer.ts file.
I test it on my node RisuAI and I send long context to my own server.
As result, ooba returned 15858 as prompt tokens.
And as I test on official tokenizer implementations, it shows 1k
differences between llama tokenizer and mistral tokenizer.
So I think adding this option will help users use oobabooga with less
error.
2024-01-06 19:16:58 +09:00
kwaroran
8ba0d10d56
Update version to 1.71.0
2024-01-06 19:13:57 +09:00
kwaroran
70f25968b5
add transformers remoteHost again
2024-01-06 19:12:18 +09:00
kwaroran
497e0c6dfb
Update service worker and remote host URLs
2024-01-06 19:10:21 +09:00
kwaroran
f4d6fcb38c
Update characterCards.ts with VITS export support
2024-01-06 07:05:20 +09:00
kwaroran
7344e566f4
Add model selection for VitsModel
2024-01-06 06:45:18 +09:00
kwaroran
0ca4ec3695
Add VITS support
2024-01-06 00:28:06 +09:00
kwaroran
cfdd60bead
Remove TransformersBodyType
2024-01-05 23:58:23 +09:00
justpain02
ff4c67b993
Add selectable tokenizer supports on Ooba
2024-01-05 23:57:59 +09:00
kwaroran
56edb14e44
Update transformers import and add config parameter to runTransformers function
2024-01-05 23:54:01 +09:00
kwaroran
5ae3f35df6
Refactor Transformers
2024-01-05 23:51:37 +09:00
kwaroran
b6787b93a7
Fix extractor initialization in runEmbedding function
2024-01-05 23:33:27 +09:00
kwaroran
dcebac658f
[fix] embedding over 512 tokens
2024-01-05 23:29:57 +09:00
kwaroran
8ebaaca59b
Update version to 1.70.3
2024-01-05 19:58:37 +09:00
kwaroran
846441f04a
fix translator customscript
2024-01-05 19:58:19 +09:00
kwaroran
f373363038
Update version to 1.70.2
2024-01-04 17:48:53 +09:00
kwaroran
0b3c702f62
[fix] writer
2024-01-04 17:47:23 +09:00
kwaroran
495c4df698
[ver] 1.70.1
2024-01-04 17:09:43 +09:00
kwaroran
7c77b55e39
[fix] localwriter
2024-01-04 05:30:06 +09:00
kwaroran
a890a2acb7
[ver] 1.70.0
2024-01-04 03:58:07 +09:00
kwaroran
20bc35d194
[feat] change default italic for high contrast
2024-01-04 03:57:04 +09:00
kwaroran
34a4422af0
[feat] add horde default key
2024-01-04 03:49:21 +09:00
kwaroran
4f77504d3f
[feat] add translate replace regex
2024-01-04 03:37:05 +09:00
kwaroran
5f2275eb22
[feat] better image saving
2024-01-04 03:18:11 +09:00
kwaroran
32e3060b19
[feat] better openrouter support
2024-01-04 02:32:53 +09:00
kwaroran
1d5d69b71b
[feat] finish new welcome
2024-01-04 02:14:05 +09:00
kwaroran
eed172f37e
[feat] recommend new template
2024-01-03 20:17:37 +09:00
kwaroran
d36399e409
[feat] new template
2024-01-03 20:17:03 +09:00
kwaroran
81b75947cd
[ref] apng typing
2024-01-03 19:43:13 +09:00
kwaroran
b10c973efb
[fix] apng support
2024-01-03 19:42:26 +09:00
kwaroran
23ef56cb33
[feat] fix horde support
2024-01-03 19:36:47 +09:00
kwaroran
99743f581a
[feat] add top_k for openrouter
2024-01-03 19:03:24 +09:00
kwaroran
a8e50dcd77
[feat] new pngchunk encode/decoderr
2024-01-03 04:13:19 +09:00
kwaroran
239d819383
[feat] add mobile writer support
2024-01-03 02:41:28 +09:00
kwaroran
8f496bc627
[fix] mobile save syste
2024-01-03 02:32:23 +09:00
kwaroran
2442b287ff
[feat] android support
2024-01-01 23:14:58 +09:00
kwaroran
0d1dd45cdf
[feat] new year
2023-12-31 22:19:57 +09:00
kwaroran
a304d555ba
[fix] use two tildes
2023-12-31 03:41:24 +09:00
kwaroran
85bf4f254b
[fix] add lorebook to dataset
2023-12-29 00:17:04 +09:00
kwaroran
772c21fe43
[ver] 1.69.5
2023-12-29 00:01:47 +09:00
kwaroran
76b903c927
[fix] saving error
2023-12-29 00:01:20 +09:00
kwaroran
3f1e9777b1
[ver] 1.69.4
2023-12-28 23:49:24 +09:00
kwaroran
8b2a2f9ce8
[feat] add export as dataset
2023-12-28 23:48:48 +09:00
kwaroran
64d109508d
[fix] backupcheck process
2023-12-28 23:15:39 +09:00
kwaroran
8ba0bfb992
[ver] 1.69.3
2023-12-28 23:12:32 +09:00
kwaroran
00c8ab0204
[fix] backupcheck process
2023-12-28 23:11:34 +09:00
