Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@leseb
Copy link
Contributor

@leseb leseb commented Mar 14, 2024

After #559, a new message was added in the chat when the context size is too small for both prompt and response.
We now have a test that validates this behavior.

@leseb
Copy link
Contributor Author

leseb commented Mar 14, 2024

@russellb 👋🏻

@leseb
Copy link
Contributor Author

leseb commented Mar 14, 2024

Can someone trigger the CI for this change? Thanks!

@leseb
Copy link
Contributor Author

leseb commented Mar 14, 2024

Local run:

+ for cmd in lab expect
+ type -p lab
/home/leseb/cli/test_run_Bw3jRiLC/venv/bin/lab
+ for cmd in lab expect
+ type -p expect
/usr/bin/expect
+ PID1=
+ PID2=
+ PID_SERVE=
+ PID_CHAT=
+ trap cleanup EXIT QUIT INT TERM
+ rm -f config.yaml
+ echo -e '\n\n\n'
+ lab init
Welcome to InstructLab CLI. This guide will help you to setup your environment.
Please provide the following values to initiate the environment:
Path to taxonomy repo [taxonomy]: Path to your model [models/merlinite-7b-Q4_K_M.gguf]: Generating `config.yaml` in the current directory...
Initialization completed successfully, you're ready to start using `lab`. Enjoy!
+ lab download
Downloading model from ibm/merlinite-7b-GGUF@main to models...
+ test_ctx_size
+ PID_SERVE=1514751
+ set +e
+ lab serve --max-ctx-size 1
+ PID_CHAT=1514752
+ wait_for_pid_to_disappear 1514752
+ lab chat
++ seq 1 20
+ for i in $(seq 1 20)
+ test -d /proc/1514752
+ '[' 1 -eq 20 ']'
+ sleep 1
+ for i in $(seq 1 20)
+ test -d /proc/1514752
+ '[' 2 -eq 20 ']'
+ sleep 1
+ for i in $(seq 1 20)
+ test -d /proc/1514752
+ '[' 3 -eq 20 ']'
+ sleep 1
+ for i in $(seq 1 20)
+ test -d /proc/1514752
+ break
+ PID_CHAT=
+ set -e
+ timeout 10 bash -c '
        until grep -q "exceed context window of" test_ctx_size_lab_serve_output.txt; do
        echo "waiting for context size error"
        sleep 1
    done
'
+ timeout 10 bash -c '
        until grep -q "Message too large for context size." test_ctx_size_lab_chat_output.txt; do
        echo "waiting for chat error"
        sleep 1
    done
'
+ exit 0
+ cleanup
+ set +e
+ for pid in $PID1 $PID2 $PID_SERVE $PID_CHAT
+ '[' -n 1514751 ']'
+ kill 1514751
+ rm -f test_ctx_size_lab_serve_output.txt test_ctx_size_lab_chat_output.txt

@russellb
Copy link
Member

russellb commented Mar 14, 2024

Can someone trigger the CI for this change? Thanks!

The reason it isn't running is that scripts/functional_tests.sh isn't listed as a path to trigger the workflow. I added that change to #561 because it would have conflicted with another change I had in that branch if I posted it separately.

This is the commit: 7ff733a

@leseb leseb force-pushed the add-test-for-context-too-large branch 3 times, most recently from 3ca0fac to a0b3aa1 Compare March 18, 2024 11:31
After instructlab/instructlab#559, a new message was
added in the chat when the context size is too small for both prompt and
response.
We now have a test that validates this behavior.

Signed-off-by: Sébastien Han <[email protected]>
@xukai92 xukai92 added the testing Relates to testing label Mar 19, 2024
@xukai92 xukai92 added this to the March 21 Musts milestone Mar 21, 2024
@xukai92
Copy link
Member

xukai92 commented Mar 21, 2024

@leseb can you resolve the conflicts?

@xukai92
Copy link
Member

xukai92 commented Mar 22, 2024

I resolved the conflicts and all tests passed.

@xukai92 xukai92 merged commit 79d5a3e into instructlab:main Mar 22, 2024
@leseb leseb deleted the add-test-for-context-too-large branch March 22, 2024 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Relates to testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants