mirror of
https://github.com/deepseek-ai/deepseek-recipe.git
synced 2026-10-28 17:17:08 +08:00
93b0fd229c
Co-authored-by: xinchengxx <xinchengxx@deepseek.com> Co-authored-by: zhu-he <hezhu@deepseek.com>
77 lines
3.3 KiB
Markdown
77 lines
3.3 KiB
Markdown
# server-py
|
|
|
|
A FastAPI example with Python mock inference. The server combines the Rust
|
|
components exposed by `deepseek_recipe` for request conversion, prompt
|
|
rendering, image resolution, and response processing. Request preparation and
|
|
HTTP transport are implemented in [main.py](main.py).
|
|
|
|
## Usage
|
|
|
|
Use Python 3.10+ in a virtual environment and install the
|
|
[Rust and OpenCV prerequisites](../docs/development.md#native-dependencies).
|
|
From the repository root:
|
|
|
|
```sh
|
|
python3 -m venv .venv
|
|
. .venv/bin/activate
|
|
python3 -m pip install ./deepseek-recipe-python -r server-py/requirements.txt
|
|
python3 -m uvicorn main:app --app-dir server-py --host 127.0.0.1 --port 7777
|
|
```
|
|
|
|
Use `--host` and `--port` to change the listening address.
|
|
On Windows PowerShell, activate with `.venv\Scripts\Activate.ps1` instead.
|
|
|
|
## API
|
|
|
|
- `POST /v1/chat/completions`
|
|
- `POST /v1/responses`
|
|
- `POST /v1/messages`
|
|
|
|
All three protocols support JSON for `stream=false` and SSE for `stream=true`.
|
|
|
|
For example, in a second terminal:
|
|
|
|
```sh
|
|
curl http://127.0.0.1:7777/v1/chat/completions \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"model":"deepseek-flash","messages":[{"role":"user","content":"Hello"}],"stream":false}'
|
|
```
|
|
|
|
The response contains the mock answer `Hello world!`. Set `"stream":true` and
|
|
add `-N` to curl to inspect SSE output. No API key or model weights are
|
|
required.
|
|
|
|
## Custom inference
|
|
|
|
Use `create_app(infer=...)` in [main.py](main.py) to supply a backend. The
|
|
callback receives the server's `PreparedRequest`, including the rendered
|
|
prompt, inference options, and processed images, and produces an asynchronous
|
|
iterator of `InferenceChunk` values. The server's stream processor has no
|
|
tokenizer, so the callback must produce `text` chunks: a `tokens` chunk fails
|
|
the response, and usage does not count `reasoning_tokens`.
|
|
`inference_options.max_tokens` is `None` when the request sets none; a backend
|
|
that encodes the prompt resolves the value to send with
|
|
`conversation_request.resolve_max_tokens(prompt_tokens, context_limit)`, where
|
|
`prompt_tokens` adds `image_token_adjustment` to the encoded token count.
|
|
|
|
Use `create_app(options=...)` to change the Rust conversion options, starting
|
|
from `conversion_options()` in [main.py](main.py). `conversion_options()` drops
|
|
Responses `web_search` declarations and rejects Messages `web_search`
|
|
declarations; `with_responses_web_search(WebSearchBehavior.Reject)` and
|
|
`with_messages_web_search(WebSearchBehavior.Drop)` reverse that behavior.
|
|
Conversion resolves the model version from the request's
|
|
model name through the options' `model_versions_by_name`, and rejects a request
|
|
whose model is not listed. `conversion_options()` accepts the library default
|
|
names `DeepSeek-V4-Pro` and `DeepSeek-V4.1-Flash` and the DeepSeek API names
|
|
`deepseek-v4-pro` and `deepseek-flash`. The model version selects the prompt
|
|
encoding: `ModelVersion.DeepseekV41Flash` renders prompts with
|
|
`DeepseekV41Encoding`, and `ModelVersion.DeepseekV4Pro` renders them with
|
|
`DeepseekV4Encoding`.
|
|
|
|
## Deployment scope
|
|
|
|
This example has no authentication and uses an image fetcher that does not
|
|
filter private network addresses. Keep it on loopback for local use; see the
|
|
[deployment considerations](../docs/development.md#example-servers) before
|
|
exposing it to other users. The Rust example uses the same default port.
|