Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 1e37436

Browse files
hanishkvchanishkvc
authored
SimpleChat: a simple and dumb web front end for testing /chat/completions and /completions end points and try chat (ggml-org#7350)
* SimpleChat: Add a skeletal html page Contains a div placeholder for showing chat messages till now a text-input for allowing user to enter next chat message/query to the model. a submit button to allow sending of the user entered message and chat till now to the model. * SimpleChat: A js skeleton with SimpleChat class Allows maintaining an array of chat message. Allows adding chat message (from any of the roles be it system, user, assistant, ...) Allows showing chat messages till now, in a given div element. * SimpleChat: request_json, globals, startme * SimpleChatJS: Roles Class, submitClick Define Role class with static members corresponding to the roles. Update startme to * Get hold of the ui elements. * Attach a click handler to submit button, which adds the user input to xchats array and shows the chat messages till now in chat div element. Trap DOMContentLoaded to trigger startme * SimpleChat:HTML: Bring in the js file * SimpleChat: Rather value wrt input text element * SimpleChat: Also add completions related prompt * SimpleChat: Use common helper logic wrt json data * SimpleChat: Move handling of submit request into its own func * SimpleChat: Try handshake with llm over its web service endpoint * SimpleChat:JS: Extract model response and show to user * SimpleChat:JS: Messages/Prompt, indicate working to end user * SimpleChat: Try keep input element in view * SimpleChat: Diff user/assistant msgs, Make input wider Also show a default message to user Also add some metas * SimpleChat: Move into its own sub directory to avoid confusion * SimpleChat:sh: Add simple shell script to run python3 http.server So one needs to run the llm server locally then run this script and access it using a local browser * SimpleChat:JS: Try trap enter key press wrt input text field So user can either press submit button or press enter key * SimpleChat: Allow user to select chat or completion mode * SimpleChat: Dont submit if already submitted and waiting Also make chat the default selection wrt mode * SimpleChat:JS: Handle difference in response Try read the assistance response from appropriate field in the response got. Also examples/server seems to return the response in a slightly different field, so try account for that also. * SimpleChat:JS: Force completion mode be single message by default * SimpleChat: Add a simple readme file * SimpleChat:HTML: Cleanup/structure UI a bit, Add input for system * SimpleChat:Allow system prompt to be set, if provided before user * SimpleChat: Ignore empty user input, without trimming * SimpleChat:Alert user if they provide sysprompt late or change it * SimpleChat: Move handling systemprompt into its own func * SimpleChat:HTML: Add a style for system role message * SimpleChat: Update the readme file * SimpleChat:CSS: Move style info into its own css file To keep it simple, clean and seperate so that things are not unnecessarily cluttered. * SimpleChat:CSS: Allow for chat div to be scrollable * SimpleChat:JS: Try ensure the last entry in chat is visible Needed because now only the chat div is scrollable and not the full page. In last commit the chat div size was fixed to 75% vertical height, so the full page no longer scrolls, so the old bring user-input element to view wont work, instead now the last element in the chat div should be brought into view. * SimpleChat:JS: bottom of element visible, Set focus to user input As the generated text could be multiple lines and occupy more space that the full scrollable div's vertical space, make the bottom of the last element (which can be such a generated text) in the div visible by scrolling. Ensure that the user input box has focus * SimpleChat: Update notes a bit. Try keep browser happy Avoid browser quirk mode with DOCTYPE. Help with accessibility a bit by specifying the language explicitly. Specify the char encoding explicitly, inturn utf-8 is a safe bet, even with intermixing of languages if reqd in future. Add a cache-control http-equiv meta tag, which in all probability will be ignored. Defer js loading and execution, just for fun and future, not that critical here as it stands now. * SimpleChat:HTML:Group user input+btn together; Note about multichat * SimpleChat:JS: Allow for changing system prompt anytime for future * SimpleChat:Readme: Note about handle_systemprompt begin/anytime * SimpleChat:HTML: Add viewport meta for better mobile friendliness Without this the page content may look too small. * SimpleChat:HtmlCss: Cleanup UI flow set margin wrt vmin rather than vw or vh so portrait/landscape ok. Use flex and flex-grow to put things on the same line as well as distribute available space as needed. Given two main elements/line so it remains simple. In each line have one element with grows and one sits with a basic comfortably fixed size. * SimpleChat: textarea for multiline user chat, inturn shift+enter 4 enter * SimpleChat: Make vertical layout better responsive (flex based) Also needed to make things cleaner and properly usable whether landscape or portrait, after changing to multiline textarea rather than single line user input. Avoid hardcoding the chat-till-now display area height, instead make it a flex-growable within a flex column of ui elements within a fixed vertical area. * SimpleChat: Rename simplechat.html to index.html, update readme Instead of providing a seperate shell script, update the readme wrt how to run/use this web front end. * SimpleChat: Screen fixed view and scrolling, Printing full * SimpleChat:JS:CI: Avoid space at end of jsdoc param line * SimpleChat:JS: MultiChat initial skeleton Will help maintain multiple independent chats in future * SimpleChat:JS: Move system prompt begin/anytime into SimpleChat * SimpleChat:JS:Keep MultiChatUI simple for now Worry about different chats with different servers for later. * SimpleChat:JS: Move handle submit into MultiChat, build on same Create an instance of MultiChatUI and inturn a instance of chat session, which is what the UI will inturn work on. * SimpleChat:JS: Move to dictionary of SimpleChat, instead of array * SimpleChat: Move ui elements into MultiChatUI, Update el IDs Move ui elements into MultiChatUI, so that current handleUserSubmit doesnt need to take the element arguments. Also in future, when user is allowed to switch between different chat sessions, the UI can be updated as needed by using the elements in UI already known to MultiChatUI instance. Rename the element ids' so that they follow a common convention, as well as one can identify what the element represents in a more consistant manner. * SimpleChat:MCUI:Show available chat sessions, try switch btw them Previous commits brought in / consolidated existing logic into MultiChatUI class. Now start adding logic towards multichat support * show buttons indicating available chat sessions * on sessin button click, try switch to that session * SimpleChat:MCUI: Store and use current chat session id Also allow to switch chat session optionally, wrt some of the related helpers. setup for two chat sessions by default. * SimpleChat:MCUI: Delay enabling user-input to avoid race Re-enable user-input, only after response to a user query has been updated to the chat-div. This ensures that if user tries to switch chat session, it wont be allowed till chat-request-response flow is done. * SimpleChat: Take care of system prompt Helper to get the latest system prompt and inturn use same to set the system prompt ui, when switching. Ensure that system prompt is set if and when enter key is pressed. * SimpleChat:GetSystemLatest, fix a oversight. * SimpleChat:MCUI: Allow selected chat-session btn to be highlighted Also have a general helper for setting class of children. * SimpleChat:Cleanup corners Show system prompt in chat space, when it is set by pressing enter, as a feedback to user. Alert user, if they try to switch chat session in the middle of waiting for a response from the ai model. * SimpleChat:MCUI: Ensure req-resp failure doesnt lock up things * SimpleChat:MCUI: Support for new chat sessions Also a general create button helper. * SimpleChat:MCUI: CreateSessionBtn helper, use wrt NewChat Also fix a oversight wrt using stale data wrt the list of chat sessions. * SimpleChat:MCUI: NewChat btn first before existing chat sessions * SimpleChat:MCUI:CornerCases:Skip new chat, show only if current Skip NewChat if user cancels or if one waiting for response from the ai model. Dont show a chat with newly got ai model response, if current chat session has changed, some how. Chat session shouldnt be allowed to change, if there is a pending response, but still as a additional sanity check. * SimpleChat: Update readme, title, show usage if no chat to show * SimpleChat: Cleanup the log/dialog messages a bit
1 parent 197ff91 commit 1e37436

File tree

4 files changed

+672
-0
lines changed

4 files changed

+672
-0
lines changed

‎examples/server/public_simplechat/index.html

+52
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
<!DOCTYPE html>
2+
<html lang="en">
3+
<head>
4+
<title>SimpleChat (LlamaCPP, ...) </title>
5+
<meta charset="UTF-8" />
6+
<meta name="viewport" content="width=device-width, initial-scale=1" />
7+
<meta name="message" content="Save Nature Save Earth" />
8+
<meta name="description" content="SimpleChat: trigger LLM web service endpoints /chat/completions and /completions, single/multi chat sessions" />
9+
<meta name="author" content="by Humans for All" />
10+
<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate" />
11+
<script src="simplechat.js" defer></script>
12+
<link rel="stylesheet" href="simplechat.css" />
13+
</head>
14+
<body>
15+
<div class="samecolumn" id="fullbody">
16+
17+
<div class="sameline">
18+
<p class="heading flex-grow" > <b> SimpleChat </b> </p>
19+
<div class="sameline">
20+
<label for="api-ep">Mode:</label>
21+
<select name="api-ep" id="api-ep">
22+
<option value="chat" selected>Chat</option>
23+
<option value="completion">Completion</option>
24+
</select>
25+
</div>
26+
</div>
27+
28+
<div id="sessions-div" class="sameline"></div>
29+
30+
<hr>
31+
<div class="sameline">
32+
<label for="system-in">System</label>
33+
<input type="text" name="system" id="system-in" class="flex-grow"/>
34+
</div>
35+
36+
<hr>
37+
<div id="chat-div">
38+
<p> Enter the system prompt above, before entering/submitting any user query.</p>
39+
<p> Enter your text to the ai assistant below.</p>
40+
<p> Use shift+enter for inserting enter.</p>
41+
<p> Refresh the page to start over fresh.</p>
42+
</div>
43+
44+
<hr>
45+
<div class="sameline">
46+
<textarea id="user-in" class="flex-grow" rows="3"></textarea>
47+
<button id="user-btn">submit</button>
48+
</div>
49+
50+
</div>
51+
</body>
52+
</html>

‎examples/server/public_simplechat/readme.md

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
2+
# SimpleChat
3+
4+
by Humans for All.
5+
6+
7+
## overview
8+
9+
This simple web frontend, allows triggering/testing the server's /completions or /chat/completions endpoints
10+
in a simple way with minimal code from a common code base. Inturn additionally it tries to allow single or
11+
multiple independent back and forth chatting to an extent, with the ai llm model at a basic level, with their
12+
own system prompts.
13+
14+
The UI follows a responsive web design so that the layout can adapt to available display space in a usable
15+
enough manner, in general.
16+
17+
NOTE: Given that the idea is for basic minimal testing, it doesnt bother with any model context length and
18+
culling of old messages from the chat.
19+
20+
NOTE: It doesnt set any parameters other than temperature for now. However if someone wants they can update
21+
the js file as needed.
22+
23+
24+
## usage
25+
26+
One could run this web frontend directly using server itself or if anyone is thinking of adding a built in web
27+
frontend to configure the server over http(s) or so, then run this web frontend using something like python's
28+
http module.
29+
30+
### running using examples/server
31+
32+
bin/server -m path/model.gguf --path ../examples/server/public_simplechat [--port PORT]
33+
34+
### running using python3's server module
35+
36+
first run examples/server
37+
* bin/server -m path/model.gguf
38+
39+
next run this web front end in examples/server/public_simplechat
40+
* cd ../examples/server/public_simplechat
41+
* python3 -m http.server PORT
42+
43+
### using the front end
44+
45+
Open this simple web front end from your local browser
46+
* http://127.0.0.1:PORT/index.html
47+
48+
Once inside
49+
* Select between chat and completion mode. By default it is set to chat mode.
50+
* If you want to provide a system prompt, then ideally enter it first, before entering any user query.
51+
* if chat.add_system_begin is used
52+
* you cant change the system prompt, after it is has been submitted once along with user query.
53+
* you cant set a system prompt, after you have submitted any user query
54+
* if chat.add_system_anytime is used
55+
* one can change the system prompt any time during chat, by changing the contents of system prompt.
56+
* inturn the updated/changed system prompt will be inserted into the chat session.
57+
* this allows for the subsequent user chatting to be driven by the new system prompt set above.
58+
* Enter your query and either press enter or click on the submit button.
59+
If you want to insert enter (\n) as part of your chat/query to ai model, use shift+enter.
60+
* Wait for the logic to communicate with the server and get the response.
61+
* the user is not allowed to enter any fresh query during this time.
62+
* the user input box will be disabled and a working message will be shown in it.
63+
* just refresh the page, to reset wrt the chat history and or system prompt and start afresh.
64+
* Using NewChat one can start independent chat sessions.
65+
* two independent chat sessions are setup by default.
66+
67+
68+
## Devel note
69+
70+
Sometimes the browser may be stuborn with caching of the file, so your updates to html/css/js
71+
may not be visible. Also remember that just refreshing/reloading page in browser or for that
72+
matter clearing site data, dont directly override site caching in all cases. Worst case you may
73+
have to change port. Or in dev tools of browser, you may be able to disable caching fully.
74+
75+
Concept of multiple chat sessions with different servers, as well as saving and restoring of
76+
those across browser usage sessions, can be woven around the SimpleChat/MultiChatUI class and
77+
its instances relatively easily, however given the current goal of keeping this simple, it has
78+
not been added, for now.
79+
80+
By switching between chat.add_system_begin/anytime, one can control whether one can change
81+
the system prompt, anytime during the conversation or only at the beginning.

‎examples/server/public_simplechat/simplechat.css

+61
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
/**
2+
* the styling of the simplechat web frontend
3+
* by Humans for All
4+
*/
5+
6+
#fullbody {
7+
height: 98vh;
8+
}
9+
10+
.heading {
11+
background-color: lightgray;
12+
}
13+
14+
.session-selected {
15+
background-color: lightblue;
16+
}
17+
18+
.role-system {
19+
background-color: lightblue;
20+
}
21+
.role-user {
22+
background-color: lightgray;
23+
}
24+
25+
.flex-grow {
26+
flex-grow: 1;
27+
}
28+
.float-right {
29+
float: right;
30+
}
31+
32+
#chat-div {
33+
overflow: scroll;
34+
flex-grow: 1;
35+
flex-shrink: 1;
36+
min-height: 40vh;
37+
}
38+
button {
39+
min-width: 8vw;
40+
}
41+
42+
.sameline {
43+
display: flex;
44+
flex-direction: row;
45+
}
46+
.samecolumn {
47+
display: flex;
48+
flex-direction: column;
49+
}
50+
51+
* {
52+
margin: 0.6vmin;
53+
}
54+
55+
@media print {
56+
57+
#fullbody {
58+
height: auto;
59+
}
60+
61+
}

0 commit comments

Comments
 (0)