public/demo-ai-inference
6
0
Fork 0
Code Issues Pull requests 2 Activity

swap model to Qwen2.5 0.5B for CPU-friendly demo #2

Open
xenushka wants to merge 4 commits from qwen2.5-model-swap into main
pull from: qwen2.5-model-swap
merge into: public:main
Conversation 0 Commits 4 Files changed 4 +99 -44
xenushka commented 2026-05-15 02:18:42 +00:00 (Migrated from codeberg.org)
Copy link

This PR swaps the demo from Phi-3.1-mini to a smaller Qwen2.5 0.5B model. The goal is a lower-risk, CPU-friendly demo rather than maximizing model quality.

It also updates local dev behavior so the model download is checked, browser testing works against the local llama.cpp server, and the chat UI streams responses so users see text appear instead of waiting on the typing dots.

While I was at it, I also refreshed the visible model copy in the chat window footer, and updated the Caution CLI link so it links to the correct CLI repo.

This PR swaps the demo from Phi-3.1-mini to a smaller Qwen2.5 0.5B model. The goal is a lower-risk, CPU-friendly demo rather than maximizing model quality. It also updates local dev behavior so the model download is checked, browser testing works against the local llama.cpp server, and the chat UI streams responses so users see text appear instead of waiting on the typing dots. While I was at it, I also refreshed the visible model copy in the chat window footer, and updated the Caution CLI link so it links to the [correct CLI repo](https://codeberg.org/caution/platform/src/branch/main/src/cli).
This pull request has changes conflicting with the target branch.
  • Containerfile
  • Makefile
  • Procfile
  • index.html
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin qwen2.5-model-swap:qwen2.5-model-swap
git switch qwen2.5-model-swap

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff qwen2.5-model-swap
git switch qwen2.5-model-swap
git rebase main
git switch main
git merge --ff-only qwen2.5-model-swap
git switch qwen2.5-model-swap
git rebase main
git switch main
git merge --no-ff qwen2.5-model-swap
git switch main
git merge --squash qwen2.5-model-swap
git switch main
git merge --ff-only qwen2.5-model-swap
git switch main
git merge qwen2.5-model-swap
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
public/demo-ai-inference!2
No description provided.