coffee/deepseek-recipe
1
0
Fork 0
mirror of https://github.com/deepseek-ai/deepseek-recipe.git synced 2026-10-28 17:17:08 +08:00
Code Issues Packages Projects Releases Wiki Activity

Initial commit

Browse Source 
Co-authored-by: xinchengxx <xinchengxx@deepseek.com>
Co-authored-by: XieJiSS <xiejiss@deepseek.com>
Co-authored-by: yqaty <jiayu@deepseek.com>
Co-authored-by: zhu-he <hezhu@deepseek.com>
Co-authored-by: season-guo <seasonguo@deepseek.com>
This commit is contained in:
icrim 2026-09-10 18:07:01 +08:00
commit 57b9c84242
125 changed files with 555904 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
.gitignore vendored Normal file
View File

@ -0,0 +1,11 @@
**/target/
.venv/
__pycache__/
.pytest_cache/
*.so
*.pyd
*.dSYM/
*.egg-info/
dist/
dist-repaired/
.vscode/

3205
Cargo.lock generated Normal file
View File

File diff suppressed because it is too large Load Diff

54
Cargo.toml Normal file
View File

@ -0,0 +1,54 @@
[workspace]
members = [
"deepseek-recipe",
"deepseek-recipe-core",
"deepseek-recipe-encoding",
"deepseek-recipe-image",
"deepseek-recipe-python",
"encoding-decoding-demo",
"server-rs",
]
resolver = "3"
[workspace.package]
edition = "2024"
readme = "README.md"
license = "MIT"
repository = "https://github.com/deepseek-ai/deepseek-recipe"
[workspace.dependencies]
async-stream = "0.3.6"
axum = "0.8.7"
base64 = "0.22.1"
data-url = "0.3.2"
deepseek-recipe = { version = "0.1.0", path = "deepseek-recipe" }
deepseek-recipe-core = { version = "0.1.0", path = "deepseek-recipe-core" }
deepseek-recipe-encoding = { version = "0.1.0", path = "deepseek-recipe-encoding" }
deepseek-recipe-image = { version = "0.1.0", path = "deepseek-recipe-image" }
futures = "0.3.31"
image = { version = "0.25.0", default-features = false, features = [
"gif",
"jpeg",
"png",
"webp",
] }
infer = "0.19.0"
itertools = "0.14.0"
jsonschema = { version = "0.18.3", default-features = false }
opencv = { version = "0.93", default-features = false, features = [
"clang-runtime",
"imgcodecs",
"imgproc",
] }
reqwest = { version = "0.12.28", default-features = false, features = [
"rustls-tls",
"stream",
] }
serde = { version = "1.0.228", features = ["derive"] }
serde_json = { version = "1.0.145", features = ["preserve_order"] }
strum_macros = "0.28.0"
thiserror = "2.0.17"
tokenizers = { version = "0.23.2", features = ["http"] }
tokio = { version = "1.48.0", features = ["macros", "rt-multi-thread"] }
tokio-stream = "0.1.17"
tracing = "0.1.41"

21
LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 DeepSeek
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

137
README.md Normal file
View File

@ -0,0 +1,137 @@
<div align="center">
<img src="static/deepseek-whale.svg" alt="DeepSeek" width="240">
</div>
# deepseek-recipe
**English** | [中文](README.zh.md)
deepseek-recipe is a collection of Rust libraries and Python bindings that
uniformly convert API requests in different formats into the Conversation
format, encode them into prompts for DeepSeek models, and convert model output
into responses in the corresponding format. Use these components to connect an
inference backend to API services that support multiple formats. Model
inference, tool execution, and HTTP transport must be provided externally.
[Getting started](#using-deepseek-recipe) · [Streaming](docs/streaming.md) ·
[Use with tokenizer](docs/tokenizer.md) · [Contributing](CONTRIBUTING.md) ·
[License](#license)
## Supported scope
- **Request/response formats:** Conversion of Messages, Chat Completions, and
Responses requests, [streamed responses](docs/streaming.md), and complete
responses. Supports text, images, thinking, and client tool calls.
- **Prompts:** Encoding of DeepSeek V4 and V4.1 conversations into prompts or token IDs.
- **Generation settings:** Thinking mode, reasoning effort, `temperature`, `top_p`,
and output token limits.
- **Output parsing:** Thinking, tool calls, JSON object output, and stop sequences.
- **Images:** Provided as base64 or external URLs. The image component provides
DeepSeek V4.1 preprocessing with OpenCV.
- **Tool definitions:** Function tools; the Responses API also supports tool
namespaces and the `apply_patch` custom tool.
## Not yet supported
- Token probabilities (`logprobs` and `top_logprobs`).
- Document content, audio/video input, and file retrieval by `file_id`.
- Server tool execution, such as `web_search`.
- JSON Schema and regex output constraints, or enforcement of tool `strict` settings.
- Multiple completions per Chat Completions request (`n > 1`).
- Responses custom tool definitions other than `apply_patch`.
- Responses conversation storage and context retrieval through `previous_response_id`.
- Responses encrypted thinking content (`encrypted_content`).
## Using deepseek-recipe
To convert a Chat Completions request into a DeepSeek V4.1 prompt:
### Python
#### Installation
Python 3.10+:
```sh
python3 -m pip install deepseek-recipe
```
#### Example
```python
from deepseek_recipe import ChatCompletionRequest, ConversionOptions, DeepseekV41Encoding
request = ChatCompletionRequest({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
})
converted = request.convert(ConversionOptions())
rendered = DeepseekV41Encoding().render_conversation(converted.conversation)
print(rendered.prompt)
```
### Rust
#### Installation
```sh
cargo add deepseek-recipe@0.1 deepseek-recipe-encoding@0.1
```
See the [development guide](docs/development.md) for source builds and image dependencies.
#### Example
```rust
use deepseek_recipe::openai::ChatCompletionRequest;
use deepseek_recipe::request::{ConversionOptions, ProtocolRequest};
use deepseek_recipe_encoding::PromptEncoding;
use deepseek_recipe_encoding::v4::dsv41::DeepseekV41Encoding;
use serde_json::json;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let request: ChatCompletionRequest = serde_json::from_value(json!({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
}))?;
let converted = request.convert(ConversionOptions::default())?;
let rendered = DeepseekV41Encoding::new().render_conversation(&converted.conversation);
println!("{}", rendered.prompt);
Ok(())
}
```
### Encoding & Decoding Demo
Run the encoding and decoding demo from the repository root:
```sh
cargo run -p encoding-decoding-demo --locked
```
Open [http://127.0.0.1:7778](http://127.0.0.1:7778).
### More examples
Rust and Python both support converting model output into streaming responses.
See the [streaming examples](docs/streaming.md). To encode conversations into
token IDs or decode backend token IDs, see
[use with tokenizer](docs/tokenizer.md).
## Packages and example projects
| Package | Purpose |
| --- | --- |
| [`deepseek-recipe`](https://docs.rs/deepseek-recipe) | Protocol conversion and model output parsing. |
| [`deepseek-recipe-core`](https://docs.rs/deepseek-recipe-core) | Shared conversation, message, image, and tool types. |
| [`deepseek-recipe-encoding`](https://docs.rs/deepseek-recipe-encoding) | DeepSeek V4 and V4.1 prompt rendering and token encoding. |
| [`deepseek-recipe-image`](https://docs.rs/deepseek-recipe-image) | Image fetching and preprocessing. |
| [`deepseek-recipe-python`](deepseek-recipe-python/README.md) | Python bindings, imported as `deepseek_recipe`. |
| [encoding-decoding-demo](encoding-decoding-demo/README.md) | A web interface for encoding prompts, inspecting special tokens, and decoding complete model output into Chat Completions, Responses, or Messages. |
| [server-rs](server-rs/README.md) | An Axum API example with mock inference. |
| [server-py](server-py/README.md) | A FastAPI example with mock inference. |
## License
Project code and public documentation are licensed under the [MIT License](LICENSE).
Bundled tokenizer notices are in [static/tokenizers/README.md](static/tokenizers/README.md).

122
README.zh.md Normal file
View File

@ -0,0 +1,122 @@
<div align="center">
<img src="static/deepseek-whale.svg" alt="DeepSeek" width="240">
</div>
# deepseek-recipe
[English](README.md) | **中文**
deepseek-recipe 包含一套 Rust 库以及对应的 Python bindings：支持将不同格式的 API 请求统一转换为 Conversation 格式，编码成适用于 DeepSeek 模型的 prompt，并支持将模型输出转换为对应格式的响应。
可用于把推理后端接入兼容多种格式的 API 服务；模型推理、工具执行和 HTTP 传输需由外部提供。
[快速上手](#使用-deepseek-recipe) · [流式响应](docs/streaming.zh.md) · [使用 tokenizer](docs/tokenizer.zh.md) · [贡献指南](CONTRIBUTING.md) · [许可证](#许可证)
## 支持范围
- **请求/响应格式：** Messages、Chat Completions 与 Responses 的请求转换、[流式响应](docs/streaming.zh.md)和完整响应，支持文本、图片、思考和客户端工具调用。
- **Prompt：** 支持将 DeepSeek V4 与 V4.1 的对话编码为 prompt 或 token IDs。
- **生成参数：** 思考模式、reasoning effort、`temperature`、`top_p` 和输出 token 上限。
- **输出解析：** 思考、工具调用、JSON object 输出和 stop sequence。
- **图片：** 支持以 base64 或外部 URL 传入。提供基于 OpenCV 的 DeepSeek V4.1 图片预处理。
- **工具定义：** 支持函数工具；Responses API 还支持工具命名空间和 `apply_patch` 自定义工具。
## 尚未支持
- Token 概率（`logprobs` 和 `top_logprobs`）。
- 文档内容、音视频输入和通过 `file_id` 获取文件。
- 服务端工具执行，例如 `web_search`。
- JSON Schema、正则表达式输出约束，以及工具 `strict` 设置的强制执行。
- Chat Completions 单次请求生成多个候选回复（`n > 1`）。
- Responses 中除 `apply_patch` 外的自定义工具定义。
- Responses 的会话存储，以及通过 `previous_response_id` 获取上下文。
- Responses 的加密思考内容（`encrypted_content`）。
## 使用 deepseek-recipe
将 Chat Completions 请求转换为 DeepSeek V4.1 prompt：
### Python
#### 安装
Python 3.10+：
```sh
python3 -m pip install deepseek-recipe
```
#### 示例
```python
from deepseek_recipe import ChatCompletionRequest, ConversionOptions, DeepseekV41Encoding
request = ChatCompletionRequest({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
})
converted = request.convert(ConversionOptions())
rendered = DeepseekV41Encoding().render_conversation(converted.conversation)
print(rendered.prompt)
```
### Rust
#### 安装
```sh
cargo add deepseek-recipe@0.1 deepseek-recipe-encoding@0.1
```
源码构建与图片依赖见[开发指南](docs/development.md)。
#### 示例
```rust
use deepseek_recipe::openai::ChatCompletionRequest;
use deepseek_recipe::request::{ConversionOptions, ProtocolRequest};
use deepseek_recipe_encoding::PromptEncoding;
use deepseek_recipe_encoding::v4::dsv41::DeepseekV41Encoding;
use serde_json::json;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let request: ChatCompletionRequest = serde_json::from_value(json!({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
}))?;
let converted = request.convert(ConversionOptions::default())?;
let rendered = DeepseekV41Encoding::new().render_conversation(&converted.conversation);
println!("{}", rendered.prompt);
Ok(())
}
```
### Encoding & Decoding Demo
在仓库根目录运行编解码 demo：
```sh
cargo run -p encoding-decoding-demo --locked
```
打开 [http://127.0.0.1:7778](http://127.0.0.1:7778)。
### 更多示例
Rust 和 Python 均支持将模型输出转换为流式响应，见[流式响应示例](docs/streaming.zh.md)；将对话编码为 token IDs、以及直接解码后端返回的 token IDs 的示例见[使用 tokenizer](docs/tokenizer.zh.md)。
## 包与示例
| 包 | 用途 |
| --- | --- |
| [`deepseek-recipe`](https://docs.rs/deepseek-recipe) | 协议转换与模型输出解析。 |
| [`deepseek-recipe-core`](https://docs.rs/deepseek-recipe-core) | 共享的对话、消息、图片与工具类型。 |
| [`deepseek-recipe-encoding`](https://docs.rs/deepseek-recipe-encoding) | DeepSeek V4 与 V4.1 的 prompt 渲染与 token 编码。 |
| [`deepseek-recipe-image`](https://docs.rs/deepseek-recipe-image) | 图片拉取与预处理。 |
| [`deepseek-recipe-python`](deepseek-recipe-python/README.md) | Python bindings，以 `deepseek_recipe` 导入。 |
| [encoding-decoding-demo](encoding-decoding-demo/README.md) | 编码 prompt、查看 special token，并将完整模型输出解码为 Chat Completions、Responses 或 Messages 的 Web 界面。 |
| [server-rs](server-rs/README.md) | 使用 mock 推理的 Axum API 示例。 |
| [server-py](server-py/README.md) | 支持 mock 推理的 FastAPI 示例。 |
## 许可证
项目代码和公开文档采用 [MIT License](LICENSE)。随仓库分发的 tokenizer 声明见 [static/tokenizers/README.md](static/tokenizers/README.md)。

13
deepseek-recipe-core/Cargo.toml Normal file
View File

@ -0,0 +1,13 @@
[package]
name = "deepseek-recipe-core"
version = "0.1.0"
edition.workspace = true
description = "Shared conversation, message, and tool types for DeepSeek models."
readme.workspace = true
documentation = "https://docs.rs/deepseek-recipe-core"
license.workspace = true
repository.workspace = true
[dependencies]
serde = { workspace = true }
serde_json = { workspace = true }

21
deepseek-recipe-core/LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 DeepSeek
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

55
deepseek-recipe-core/src/conversation.rs Normal file
View File

@ -0,0 +1,55 @@
use serde::{Deserialize, Serialize};
use crate::messages::InputMessage;
use crate::tools::{ToolChoice, ToolDefinition};
/// A conversation and its prompt configuration.
///
/// Sampling parameters and transport options belong to the caller. Thinking is
/// enabled by default.
#[derive(Debug, Clone)]
pub struct Conversation {
/// Messages in conversation order.
pub messages: Vec<InputMessage>,
/// Enable thinking mode for the next assistant turn.
pub thinking_mode: bool,
/// Available client tools, in declaration order.
pub tools: Vec<ToolDefinition>,
/// Tool selection after protocol-specific normalization.
pub tool_choice: ToolChoice,
/// Reasoning effort used when thinking mode is enabled.
pub reasoning_effort: Option<ReasoningEffort>,
/// Requested answer format used during prompt rendering.
pub response_format: ResponseFormat,
}
impl Default for Conversation {
fn default() -> Self {
Self {
messages: Vec::new(),
thinking_mode: true,
tools: Vec::new(),
tool_choice: ToolChoice::Auto,
reasoning_effort: None,
response_format: ResponseFormat::Text,
}
}
}
/// Output format described in the inference prompt.
#[derive(Debug, Clone, Default)]
pub enum ResponseFormat {
#[default]
Text,
JsonObject,
}
/// Protocol-independent reasoning effort.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ReasoningEffort {
Low,
High,
Xhigh,
Max,
}

8
deepseek-recipe-core/src/lib.rs Normal file
View File

@ -0,0 +1,8 @@
//! Shared conversation, image, and tool types for protocol adapters and prompt
//! rendering.
pub mod conversation;
pub mod messages;
pub mod multimodal;
pub mod tools;
pub mod util;

34
deepseek-recipe-core/src/messages.rs Normal file
View File

@ -0,0 +1,34 @@
use crate::multimodal::ImageSource;
/// A message in the conversation supplied for prompt rendering.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum InputMessage {
System {
content: String,
},
User {
content: String,
image_sources: Vec<ImageSource>,
},
Assistant {
content: String,
reasoning_content: Option<String>,
tool_calls: Option<Vec<ToolCall>>,
},
Tool {
content: String,
image_sources: Vec<ImageSource>,
tool_call_id: String,
},
LatestReminder {
content: String,
},
}
/// A historical tool call and its arguments serialized as JSON.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ToolCall {
pub id: String,
pub name: String,
pub arguments: String,
}

197
deepseek-recipe-core/src/multimodal/mod.rs Normal file
View File

@ -0,0 +1,197 @@
//! Image input types and DeepSeek V4.1 image token accounting.
//!
//! A caller supplies images as [`ImageSource`] values. Resolving an image source
//! produces an [`ImageInfo`] carrying the bytes sent to the inference backend.
//! Every image contributes one [`IMAGE_SPECIAL_TOKEN`] placeholder to the prompt
//! text, in the order the images appear in the conversation.
use serde::{Deserialize, Serialize};
pub mod token_spec;
pub use token_spec::{CalcResizeError, ImageTokenSpec, ResizeResult};
/// Placeholder written into the prompt for every image.
pub const IMAGE_SPECIAL_TOKEN: &str = "<｜image｜>";
/// Token ID that the V4.1 tokenizer assigns to [`IMAGE_SPECIAL_TOKEN`]. A caller
/// that encodes the prompt into token IDs can compare against this ID.
pub const IMAGE_SPECIAL_TOKEN_ID: u32 = 129264;
/// Maximum UTF-8 byte length of an external image URL during conversion.
pub const MAX_IMAGE_URL_LEN: usize = 8192;
/// Media type accepted for image input.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ImageMediaType {
Jpeg,
Png,
Gif,
Webp,
}
impl ImageMediaType {
/// Return the media type of this image media type.
pub const fn mime(self) -> &'static str {
match self {
Self::Jpeg => "image/jpeg",
Self::Png => "image/png",
Self::Gif => "image/gif",
Self::Webp => "image/webp",
}
}
/// Resolve a media type string. Parameters after `;` are ignored, matching
/// is case-insensitive, and `image/jpg` is accepted as an alias of
/// `image/jpeg`. Returns `None` for an unsupported media type.
pub fn from_mime(mime: &str) -> Option<Self> {
let mime = mime.split(';').next().unwrap_or_default().trim();
let (kind, subtype) = mime.split_once('/')?;
if !kind.eq_ignore_ascii_case("image") {
return None;
}
if subtype.eq_ignore_ascii_case("jpeg") || subtype.eq_ignore_ascii_case("jpg") {
return Some(Self::Jpeg);
}
if subtype.eq_ignore_ascii_case("png") {
return Some(Self::Png);
}
if subtype.eq_ignore_ascii_case("gif") {
return Some(Self::Gif);
}
if subtype.eq_ignore_ascii_case("webp") {
return Some(Self::Webp);
}
None
}
}
/// Requested image detail level.
///
/// The OpenCV preprocessor applies an additional long-side limit for
/// [`Low`](Self::Low) before fitting the token budget. All levels still undergo
/// token-budget fitting and WebP encoding, which can resize the image.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Deserialize, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum ImageDetail {
Low,
#[default]
High,
Original,
Auto,
}
impl ImageDetail {
/// Return whether the additional low-detail long-side limit applies.
pub const fn is_low(self) -> bool {
matches!(self, Self::Low)
}
}
/// One image supplied with a conversation, before resolution.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ImageSource {
/// Data URL with a base64 or percent-encoded body.
DataUrl {
data_url: String,
detail: ImageDetail,
},
/// External image URL. Conversion and resolution check the case-insensitive
/// `http` prefix; the fetcher is responsible for validating the full URL.
Url { url: String, detail: ImageDetail },
/// Encoded image bytes supplied by the caller.
Bytes { data: Vec<u8>, detail: ImageDetail },
}
impl ImageSource {
/// Return the requested detail level.
pub const fn detail(&self) -> ImageDetail {
match self {
Self::DataUrl { detail, .. }
| Self::Url { detail, .. }
| Self::Bytes { detail, .. } => *detail,
}
}
}
/// Return whether a value is an external image URL.
///
/// Checks whether the first four bytes spell `http`, ignoring ASCII case.
/// This classifies external image references during conversion and resolution;
/// it does not validate URL syntax, the complete scheme, or the destination.
pub fn is_http_url(url: &str) -> bool {
url.get(..4)
.is_some_and(|scheme| scheme.eq_ignore_ascii_case("http"))
}
/// A preprocessed image sent to the inference backend.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ImageInfo {
/// Encoded image bytes.
pub data: Vec<u8>,
/// Width in pixels after preprocessing.
pub width: u32,
/// Height in pixels after preprocessing.
pub height: u32,
}
/// Images attached to one conversation request.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct MultiModalData {
/// Images in the order their placeholders appear in the prompt.
pub images: Vec<ImageInfo>,
}
impl MultiModalData {
/// Return whether no image is attached.
pub fn is_empty(&self) -> bool {
self.images.is_empty()
}
/// Return the number of prompt tokens beyond one per placeholder.
///
/// Every image contributes one placeholder token to the prompt and
/// [`ImageTokenSpec::calc_token_len`] tokens to the inference input. The
/// adjustment is the difference over all images.
///
/// # Errors
///
/// Returns [`CalcResizeError`] when the token length of an image does not
/// converge.
pub fn image_token_adjustment(&self) -> Result<usize, CalcResizeError> {
let spec = ImageTokenSpec::v41();
let mut adjustment = 0usize;
for image in &self.images {
adjustment += spec.calc_token_len(image.width as usize, image.height as usize)? - 1;
}
Ok(adjustment)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn token_adjustment_counts_one_placeholder_per_image() {
let spec = ImageTokenSpec::v41();
let data = MultiModalData {
images: vec![
ImageInfo {
data: Vec::new(),
width: 1920,
height: 1080,
},
ImageInfo {
data: Vec::new(),
width: 640,
height: 480,
},
],
};
let expected =
spec.calc_token_len(1920, 1080).unwrap() + spec.calc_token_len(640, 480).unwrap() - 2;
assert_eq!(data.image_token_adjustment().unwrap(), expected);
assert!(MultiModalData::default().image_token_adjustment().unwrap() == 0);
}
}

314
deepseek-recipe-core/src/multimodal/token_spec.rs Normal file
View File

@ -0,0 +1,314 @@
//! Image token accounting for DeepSeek V4.1.
use std::fmt;
/// Result of fitting one image into the token budget.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ResizeResult {
/// Number of token rows in the model token grid.
pub n_llm_h: usize,
/// Number of token columns in the model token grid.
pub n_llm_w: usize,
/// Resized image height in pixels.
pub best_height: usize,
/// Resized image width in pixels.
pub best_width: usize,
/// Total number of model tokens for the image.
pub num_tokens: usize,
}
/// Failure to fit an image into the token budget.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum CalcResizeError {
/// Fitting the image did not reach a fixed point within the iteration limit.
NotConverged {
max_iter: usize,
last_result: ResizeResult,
},
}
impl fmt::Display for CalcResizeError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::NotConverged {
max_iter,
last_result,
} => write!(
f,
"image resize did not converge after {max_iter} iterations, last result: {last_result:?}"
),
}
}
}
impl std::error::Error for CalcResizeError {}
/// DeepSeek V4.1 image token specification.
///
/// The specification maps an image size to the number of model tokens the image
/// occupies. V4.1 uses 14-pixel patches and a spatial downsample ratio of 3.
/// Images with a positive area below 544 * 544 pixels are upscaled before
/// patch alignment and token-budget fitting. This is an area target, not a
/// minimum for each dimension; token-budget fitting can reduce it further.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ImageTokenSpec {
patch_size: usize,
downsample_ratio: usize,
max_n_token: usize,
min_pixels: usize,
}
impl ImageTokenSpec {
/// Return the specification of DeepSeek V4.1 image tokens.
///
/// Images are divided into 14-pixel patches, downsampled by 3 in each
/// dimension, and limited to 1024 model tokens. Images with a positive area
/// below 544 * 544 pixels are upscaled before alignment and budget fitting.
pub const fn v41() -> Self {
Self {
patch_size: 14,
downsample_ratio: 3,
max_n_token: 1024,
min_pixels: 544 * 544,
}
}
/// Return the maximum number of model tokens one image may occupy.
pub const fn max_n_token(&self) -> usize {
self.max_n_token
}
/// Return the pixel size of one patch.
pub const fn patch_size(&self) -> usize {
self.patch_size
}
/// Return the spatial downsampling ratio between the patch grid and the
/// model token grid.
pub const fn downsample_ratio(&self) -> usize {
self.downsample_ratio
}
/// Return the model tokens one image of `width`×`height` pixels occupies.
///
/// # Errors
///
/// Returns [`CalcResizeError`] when fitting the image does not converge.
pub fn calc_token_len(&self, width: usize, height: usize) -> Result<usize, CalcResizeError> {
self.calc_resize(width, height)
.map(|result| result.num_tokens)
}
/// Return whether token-budget fitting leaves `width`×`height` unchanged.
/// This checks dimensions only; it does not inspect encoded image bytes.
pub fn is_image_valid(&self, width: usize, height: usize) -> bool {
self.calc_resize(width, height)
.map(|result| result.best_width == width && result.best_height == height)
.unwrap_or(false)
}
/// Return the target size and token length for an image of `width`×`height`
/// pixels.
///
/// # Errors
///
/// Returns [`CalcResizeError`] when fitting the image does not converge.
pub fn calc_resize(
&self,
width: usize,
height: usize,
) -> Result<ResizeResult, CalcResizeError> {
const MAX_ITER: usize = 10;
let mut result = self.calc_resize_once(width, height);
for _ in 1..MAX_ITER {
let next = self.calc_resize_once(result.best_width, result.best_height);
if next == result {
return Ok(result);
}
result = next;
}
Err(CalcResizeError::NotConverged {
max_iter: MAX_ITER,
last_result: result,
})
}
/// Upscale to the minimum area, align to patches, and fit the token budget.
fn calc_resize_once(&self, width: usize, height: usize) -> ResizeResult {
let mut width = width;
let mut height = height;
let current_pixels = width * height;
if current_pixels < self.min_pixels && current_pixels > 0 {
let ratio = (self.min_pixels as f64 / current_pixels as f64).sqrt();
width = (width as f64 * ratio) as usize;
height = (height as f64 * ratio) as usize;
}
let best_width = width.div_ceil(self.patch_size) * self.patch_size;
let best_height = height.div_ceil(self.patch_size) * self.patch_size;
self.fit_token_budget(height, width, best_height, best_width)
}
/// Reduce `best_width`×`best_height` until the token budget is satisfied.
fn fit_token_budget(
&self,
height: usize,
width: usize,
best_height: usize,
best_width: usize,
) -> ResizeResult {
let mut result = self.resize_result(best_height, best_width);
if result.num_tokens > self.max_n_token {
result = self.solve_resize_ratio(height, width, self.max_n_token);
assert!(
result.num_tokens <= self.max_n_token,
"token budget must be satisfied after one solve"
);
}
result
}
/// Return the token length of a patch grid of `best_height`×`best_width`
/// pixels.
fn resize_result(&self, best_height: usize, best_width: usize) -> ResizeResult {
let n_llm_h = (best_height / self.patch_size).div_ceil(self.downsample_ratio);
let n_llm_w = (best_width / self.patch_size).div_ceil(self.downsample_ratio);
ResizeResult {
n_llm_h,
n_llm_w,
best_height,
best_width,
num_tokens: self.calc_num_tokens(n_llm_h, n_llm_w),
}
}
/// Return the model tokens of a `n_llm_h`×`n_llm_w` token grid.
///
/// Each row holds one newline token in addition to its image tokens, and the
/// image is delimited by a start and an end token.
fn calc_num_tokens(&self, n_llm_h: usize, n_llm_w: usize) -> usize {
n_llm_h * (n_llm_w + 1) + 2
}
/// Solve for the largest aspect-preserving size within `max_n_token`.
fn solve_resize_ratio(&self, height: usize, width: usize, max_n_token: usize) -> ResizeResult {
let ratio = height as f64 / width as f64;
let max_w_float = ((max_n_token as f64 - 2.0) / ratio + 0.25).sqrt() - 0.5;
let max_h_float = max_w_float * ratio;
let (best_height, best_width);
if max_w_float < 1.0 {
let max_w = 1;
let max_h = (max_n_token - 2) / (max_w + 1);
best_width = max_w * self.patch_size * self.downsample_ratio;
best_height = max_h * self.patch_size * self.downsample_ratio;
} else if max_h_float < 1.0 {
let max_h = 1;
let max_w = (max_n_token - 2) / max_h - 1;
assert!(max_w > 1, "token budget must allow a two-column grid");
best_width = max_w * self.patch_size * self.downsample_ratio;
best_height = max_h * self.patch_size * self.downsample_ratio;
} else {
let max_w = max_w_float as usize;
let max_h = max_h_float as usize;
let beta_w = (max_w * self.patch_size * self.downsample_ratio) as f64 / width as f64;
let beta_h = (max_h * self.patch_size * self.downsample_ratio) as f64 / height as f64;
let beta = beta_w.min(beta_h);
best_width = (width as f64 * beta / self.patch_size as f64) as usize * self.patch_size;
best_height =
(height as f64 * beta / self.patch_size as f64) as usize * self.patch_size;
}
self.resize_result(best_height, best_width)
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Expected target sizes and token lengths of the V4.1 preprocessing.
const CASES: &[(usize, usize, usize, usize, usize, usize, usize)] = &[
// width, height, num_tokens, best_width, best_height, n_llm_h, n_llm_w
(978, 479, 302, 980, 490, 12, 24),
(272, 300, 198, 518, 574, 14, 13),
(516, 517, 184, 546, 546, 13, 13),
(1091, 397, 272, 1092, 406, 10, 26),
(743, 870, 401, 756, 882, 21, 18),
(906, 740, 416, 910, 742, 18, 22),
(1107, 873, 590, 1120, 882, 21, 27),
(493, 243, 202, 784, 392, 10, 19),
(354, 476, 197, 476, 630, 15, 12),
(766, 642, 322, 770, 644, 16, 19),
(972, 672, 402, 980, 672, 16, 24),
(1167, 486, 350, 1176, 490, 12, 28),
(1134, 389, 282, 1134, 392, 10, 27),
(275, 346, 197, 490, 616, 15, 12),
(1033, 141, 187, 1484, 210, 5, 36),
(125, 751, 226, 224, 1344, 32, 6),
(560, 1029, 377, 560, 1036, 25, 14),
(482, 725, 236, 490, 728, 18, 12),
(782, 879, 422, 784, 882, 21, 19),
(434, 272, 200, 700, 434, 11, 17),
(2, 165, 356, 70, 4942, 118, 2),
(165, 2, 240, 4942, 70, 2, 118),
(158, 10000, 762, 126, 7966, 190, 3),
(10000, 158, 962, 10010, 168, 4, 239),
(40, 50, 197, 490, 616, 15, 12),
(50, 40, 194, 616, 490, 12, 15),
(1, 1, 184, 546, 546, 13, 13),
(100, 100, 184, 546, 546, 13, 13),
(16809, 11841, 990, 1540, 1092, 26, 37),
(1920, 330, 378, 1932, 336, 8, 46),
(8192, 8192, 994, 1302, 1302, 31, 31),
(4096, 4096, 994, 1302, 1302, 31, 31),
(1920, 1080, 968, 1708, 966, 23, 41),
(1080, 1920, 986, 966, 1708, 41, 23),
(3024, 4032, 1010, 1134, 1512, 36, 27),
(640, 480, 206, 644, 490, 12, 16),
];
#[test]
fn calc_resize_matches_v41_preprocessing() {
let spec = ImageTokenSpec::v41();
for &(width, height, num_tokens, best_width, best_height, n_llm_h, n_llm_w) in CASES {
let result = spec.calc_resize(width, height).expect("resize converges");
assert_eq!(
result,
ResizeResult {
n_llm_h,
n_llm_w,
best_height,
best_width,
num_tokens,
},
"calc_resize({width}, {height})"
);
assert!(
result.num_tokens <= spec.max_n_token(),
"({width}, {height}) exceeds the budget"
);
}
}
#[test]
fn fitted_images_are_valid() {
let spec = ImageTokenSpec::v41();
for &(_, _, _, best_width, best_height, _, _) in CASES {
assert!(
spec.is_image_valid(best_width, best_height),
"{best_width}x{best_height}"
);
}
}
#[test]
fn token_len_has_a_floor_and_a_ceiling() {
let spec = ImageTokenSpec::v41();
assert_eq!(spec.calc_token_len(1, 1).expect("resize converges"), 184);
assert_eq!(
spec.calc_token_len(16809, 11841).expect("resize converges"),
990
);
}
}

26
deepseek-recipe-core/src/tools.rs Normal file
View File

@ -0,0 +1,26 @@
//! Client tool definitions used when constructing a conversation.
/// A function the caller may execute after receiving a tool call.
#[derive(Debug, Clone)]
pub struct ToolDefinition {
/// Function name.
pub name: String,
/// Human-readable purpose of the function.
pub description: Option<String>,
/// JSON Schema describing the input object.
pub parameters: serde_json::Value,
/// Requested strict argument validation. Enforcement belongs to the caller.
pub strict: Option<bool>,
}
/// Normalized tool selection for a rendered conversation.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub enum ToolChoice {
/// The model may answer or call a tool.
#[default]
Auto,
/// Render the prompt with tool definitions omitted.
None,
/// Start the assistant output inside a tool-call block.
Required,
}

104
deepseek-recipe-core/src/util/json_formatter.rs Normal file
View File

@ -0,0 +1,104 @@
//! JSON serialization in Python's `json.dumps` style.
use std::io;
use serde::Serialize;
use serde_json::ser::{CompactFormatter, Formatter};
/// Uses spaced separators and normalizes small negative float exponents.
struct PythonStyleFormatter;
impl Formatter for PythonStyleFormatter {
fn begin_array_value<W>(&mut self, writer: &mut W, first: bool) -> io::Result<()>
where
W: ?Sized + io::Write,
{
writer.write_all(if first { b"" } else { b", " })
}
fn begin_object_key<W>(&mut self, writer: &mut W, first: bool) -> io::Result<()>
where
W: ?Sized + io::Write,
{
writer.write_all(if first { b"" } else { b", " })
}
fn begin_object_value<W>(&mut self, writer: &mut W) -> io::Result<()>
where
W: ?Sized + io::Write,
{
writer.write_all(b": ")
}
fn write_f64<W>(&mut self, writer: &mut W, value: f64) -> io::Result<()>
where
W: ?Sized + io::Write,
{
let mut buf = Vec::with_capacity(32);
CompactFormatter.write_f64(&mut buf, value)?;
if !buf.contains(&b'e') {
let (sign, rest): (&[u8], &[u8]) = match buf.strip_prefix(b"-") {
Some(rest) => (b"-", rest),
None => (b"", &buf),
};
if let Some(fraction) = rest.strip_prefix(b"0.") {
let zeros = fraction.iter().take_while(|&&b| b == b'0').count();
if let [first_digit, rest_digits @ ..] = &fraction[zeros..]
&& zeros >= 4
{
writer.write_all(sign)?;
writer.write_all(std::slice::from_ref(first_digit))?;
if !rest_digits.is_empty() {
writer.write_all(b".")?;
writer.write_all(rest_digits)?;
}
return write!(writer, "e-{:02}", zeros + 1);
}
}
} else if let [mantissa @ .., b'e', b'-', digit] = buf.as_slice() {
writer.write_all(mantissa)?;
return write!(writer, "e-0{}", *digit as char);
}
writer.write_all(&buf)
}
}
/// Serialize JSON with spaced separators and Python-style negative float
/// exponents. Non-ASCII text remains unescaped; other string and number
/// formatting follows `serde_json`.
///
/// The input is a JSON value, whose serialization to memory produces valid
/// JSON.
pub fn stringify_python_style(value: &serde_json::Value) -> String {
let mut buf = Vec::with_capacity(128);
let mut ser = serde_json::ser::Serializer::with_formatter(&mut buf, PythonStyleFormatter);
value
.serialize(&mut ser)
.expect("JSON values serialize to memory");
String::from_utf8(buf).expect("JSON serialization produces UTF-8")
}
#[cfg(test)]
mod tests {
use super::stringify_python_style;
#[test]
fn spaced_separators() {
let value = serde_json::json!({
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The location user interested in"
}
},
"required": ["location"],
"additionalProperties": false
});
assert_eq!(
stringify_python_style(&value),
r#"{"type": "object", "properties": {"location": {"type": "string", "description": "The location user interested in"}}, "required": ["location"], "additionalProperties": false}"#
);
}
}

3
deepseek-recipe-core/src/util/mod.rs Normal file
View File

@ -0,0 +1,3 @@
//! Shared helpers used by protocol adapters and prompt rendering.
pub mod json_formatter;

17
deepseek-recipe-encoding/Cargo.toml Normal file
View File

@ -0,0 +1,17 @@
[package]
name = "deepseek-recipe-encoding"
version = "0.1.0"
edition.workspace = true
description = "Conversation rendering and token encoding for DeepSeek V4 and V4.1 prompts."
readme.workspace = true
documentation = "https://docs.rs/deepseek-recipe-encoding"
license.workspace = true
repository.workspace = true
[dependencies]
deepseek-recipe-core = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
thiserror = { workspace = true }
tokenizers = { workspace = true }
tracing = { workspace = true }

21
deepseek-recipe-encoding/LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 DeepSeek
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

47
deepseek-recipe-encoding/src/lib.rs Normal file
View File

@ -0,0 +1,47 @@
//! Model-specific conversation rendering and token encoding.
//!
//! V4 and V4.1 provide prompt rendering. Attach a tokenizer to an encoding to
//! encode conversations into token IDs.
use deepseek_recipe_core::conversation::Conversation;
use deepseek_recipe_core::multimodal::ImageSource;
pub use tokenizer::TokenizerEncoder;
pub use v4::{dsv4, dsv41};
pub mod v4;
mod tokenizer;
/// A rendered prompt and the images its placeholders refer to.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RenderedPrompt {
/// Model input string.
pub prompt: String,
/// Image sources in the order their placeholders appear in the prompt.
pub image_sources: Vec<ImageSource>,
}
/// Failure to encode a conversation.
#[derive(Debug, thiserror::Error)]
pub enum EncodingError {
/// The rendered prompt could not be encoded.
#[error("failed to encode conversation: {0}")]
Encode(String),
/// The encoding has no attached tokenizer.
#[error("no tokenizer is attached to the encoding")]
MissingTokenizer,
}
/// Render model prompts and encode conversations into token IDs.
pub trait PromptEncoding {
/// Encode a conversation into model token IDs using the attached tokenizer.
///
/// The tokenizer must be attached to the encoding before this call, through
/// the encoding's `with_tokenizer` method. Returns
/// [`EncodingError::MissingTokenizer`] when no tokenizer is attached.
fn encode(&self, conversation: &Conversation) -> Result<Vec<u32>, EncodingError>;
/// Render the conversation and the prefix for the next assistant turn.
fn render_conversation(&self, conversation: &Conversation) -> RenderedPrompt;
}

28
deepseek-recipe-encoding/src/tokenizer.rs Normal file
View File

@ -0,0 +1,28 @@
//! Encode rendered prompts into token ids.
use std::sync::Arc;
use tokenizers::Tokenizer;
/// Encode text into model token ids.
///
/// Implemented for [`Tokenizer`]; other tokenizers can implement it to back a
/// prompt encoding.
pub trait TokenizerEncoder: Send + Sync {
/// Encode `text` into token ids without added special tokens.
fn encode_ids(&self, text: &str) -> Result<Vec<u32>, String>;
}
impl TokenizerEncoder for Tokenizer {
fn encode_ids(&self, text: &str) -> Result<Vec<u32>, String> {
self.encode(text, false)
.map(|encoding| encoding.get_ids().to_vec())
.map_err(|error| error.to_string())
}
}
impl<T: TokenizerEncoder> TokenizerEncoder for Arc<T> {
fn encode_ids(&self, text: &str) -> Result<Vec<u32>, String> {
(**self).encode_ids(text)
}
}

85
deepseek-recipe-encoding/src/v4/dsv4.rs Normal file
View File

@ -0,0 +1,85 @@
use deepseek_recipe_core::conversation::ReasoningEffort;
use super::EncodingV4;
use crate::TokenizerEncoder;
const REASONING_EFFORT_HIGH: &str = r"Reasoning Effort: Absolute maximum with no shortcuts permitted.
You MUST be very thorough in your thinking and comprehensively decompose the problem to resolve the root cause, rigorously stress-testing your logic against all potential paths, edge cases, and adversarial scenarios.
Explicitly write out your entire deliberation process, documenting every intermediate step, considered alternative, and rejected hypothesis to ensure absolutely no assumption is left unchecked.
";
const REASONING_EFFORT_MAX: &str = r"Reasoning Effort: Beyond maximum — exhaustive, relentless, and uncompromising.
You MUST reason with the utmost depth and rigor, leaving absolutely nothing to chance: exhaustively decompose the problem into its most fundamental components, trace every causal chain to its root, and resolve the underlying cause rather than any surface symptom.
Do not stop reasoning until you have independently verified the solution from multiple angles and are certain that no assumption remains unchecked and no error remains undiscovered.
";
/// Prompt rendering and token encoding for DeepSeek V4.
pub struct DeepseekV4Encoding {
tokenizer: Option<Box<dyn TokenizerEncoder>>,
}
impl DeepseekV4Encoding {
/// Create an encoding without an attached tokenizer.
///
/// Token encoding requires a tokenizer, attached with `with_tokenizer`.
pub fn new() -> Self {
Self { tokenizer: None }
}
/// Attach the tokenizer used by token encoding.
pub fn with_tokenizer(mut self, tokenizer: impl TokenizerEncoder + 'static) -> Self {
self.tokenizer = Some(Box::new(tokenizer));
self
}
}
impl Default for DeepseekV4Encoding {
fn default() -> Self {
Self::new()
}
}
impl EncodingV4 for DeepseekV4Encoding {
fn tokenizer(&self) -> Option<&dyn TokenizerEncoder> {
self.tokenizer.as_deref()
}
fn supports_mid_conversation_system(&self) -> bool {
false
}
fn system_token(&self) -> &'static str {
""
}
fn tool_calls_block_name(&self) -> &'static str {
"tool_calls"
}
fn tool_call_tag_name(&self) -> &'static str {
"invoke"
}
fn tool_parameter_tag_name(&self) -> &'static str {
"parameter"
}
fn render_reasoning_effort(
&self,
index: usize,
thinking_mode: bool,
effort: Option<ReasoningEffort>,
) -> String {
if index != 0 || !thinking_mode {
return String::new();
}
match effort {
Some(ReasoningEffort::Low) => String::new(),
None | Some(ReasoningEffort::High) | Some(ReasoningEffort::Xhigh) => {
REASONING_EFFORT_HIGH.to_string()
}
Some(ReasoningEffort::Max) => REASONING_EFFORT_MAX.to_string(),
}
}
}

79
deepseek-recipe-encoding/src/v4/dsv41.rs Normal file
View File

@ -0,0 +1,79 @@
use deepseek_recipe_core::conversation::ReasoningEffort;
use super::{EncodingV4, SYSTEM_SP_TOKEN};
use crate::TokenizerEncoder;
fn reasoning_effort_template(effort: Option<ReasoningEffort>) -> String {
let effort_score = match effort {
Some(ReasoningEffort::Low) => 50,
None | Some(ReasoningEffort::High) | Some(ReasoningEffort::Xhigh) => 75,
Some(ReasoningEffort::Max) => 100,
};
format!(
"Reasoning Effort: {effort_score} (range 1-100, the higher the value, the more thorough the reasoning)\n\n"
)
}
/// Prompt rendering and token encoding for DeepSeek V4.1.
pub struct DeepseekV41Encoding {
tokenizer: Option<Box<dyn TokenizerEncoder>>,
}
impl DeepseekV41Encoding {
/// Create an encoding without an attached tokenizer.
///
/// Token encoding requires a tokenizer, attached with `with_tokenizer`.
pub fn new() -> Self {
Self { tokenizer: None }
}
/// Attach the tokenizer used by token encoding.
pub fn with_tokenizer(mut self, tokenizer: impl TokenizerEncoder + 'static) -> Self {
self.tokenizer = Some(Box::new(tokenizer));
self
}
}
impl Default for DeepseekV41Encoding {
fn default() -> Self {
Self::new()
}
}
impl EncodingV4 for DeepseekV41Encoding {
fn tokenizer(&self) -> Option<&dyn TokenizerEncoder> {
self.tokenizer.as_deref()
}
fn supports_mid_conversation_system(&self) -> bool {
true
}
fn system_token(&self) -> &'static str {
SYSTEM_SP_TOKEN
}
fn tool_calls_block_name(&self) -> &'static str {
" calls"
}
fn tool_call_tag_name(&self) -> &'static str {
" invoke"
}
fn tool_parameter_tag_name(&self) -> &'static str {
" parameter"
}
fn render_reasoning_effort(
&self,
index: usize,
thinking_mode: bool,
effort: Option<ReasoningEffort>,
) -> String {
if index == 0 && thinking_mode {
return reasoning_effort_template(effort);
}
String::new()
}
}

435
deepseek-recipe-encoding/src/v4/mod.rs Normal file
View File

@ -0,0 +1,435 @@
use deepseek_recipe_core::conversation::{Conversation, ReasoningEffort, ResponseFormat};
use deepseek_recipe_core::messages::{InputMessage, ToolCall};
use deepseek_recipe_core::tools::{ToolChoice, ToolDefinition};
use deepseek_recipe_core::util::json_formatter::stringify_python_style;
use crate::EncodingError;
use crate::PromptEncoding;
use crate::RenderedPrompt;
use crate::TokenizerEncoder;
pub mod dsv4;
pub mod dsv41;
/// Marks the start of the prompt.
pub const BOS_TOKEN: &str = "<｜begin▁of▁sentence｜>";
/// Starts the reasoning content of an assistant turn.
pub const THINKING_START_TOKEN: &str = "<think>";
/// Ends the reasoning content of an assistant turn.
pub const THINKING_END_TOKEN: &str = "</think>";
/// Starts a system message in a V4.1 prompt.
pub const SYSTEM_SP_TOKEN: &str = "<｜System｜>";
/// Starts a user message.
pub const USER_SP_TOKEN: &str = "<｜User｜>";
/// Starts an assistant message.
pub const ASSISTANT_SP_TOKEN: &str = "<｜Assistant｜>";
/// Starts the latest reminder message.
pub const LATEST_REMINDER_SP_TOKEN: &str = "<｜latest_reminder｜>";
/// Terminates a message.
pub const EOS_TOKEN: &str = "<｜end▁of▁sentence｜>";
/// Tag name prefix of the markup that structures tool calls. The angle brackets
/// come from the surrounding template, as in `<｜DSML｜tool_calls>` for V4 and
/// `<｜DSML｜ calls>` for V4.1.
pub const DSML_SP_TOKEN: &str = "｜DSML｜";
fn parameter_template(
dsml_token: &str,
tool_parameter_tag_name: &str,
key: &str,
is_str: &str,
value: &str,
) -> String {
format!(
"<{dsml_token}{tool_parameter_tag_name} name=\"{key}\" string=\"{is_str}\">{value}</{dsml_token}{tool_parameter_tag_name}>"
)
}
fn render_tool_arguments(tool_call: &ToolCall, tool_parameter_tag_name: &str) -> String {
let arguments =
serde_json::from_str::<serde_json::Map<String, serde_json::Value>>(&tool_call.arguments)
.unwrap_or_else(|err| {
tracing::warn!(?err, "invalid tool call arguments");
serde_json::Map::from_iter([(
"arguments".to_owned(),
tool_call.arguments.clone().into(),
)])
});
arguments
.iter()
.map(|(key, value)| {
let (is_str, kv_str) = match value.as_str() {
Some(s) => ("true", s.to_owned()),
None => ("false", stringify_python_style(value)),
};
parameter_template(DSML_SP_TOKEN, tool_parameter_tag_name, key, is_str, &kv_str)
})
.collect::<Vec<_>>()
.join("\n")
}
pub(crate) trait EncodingV4 {
fn tokenizer(&self) -> Option<&dyn TokenizerEncoder>;
fn supports_mid_conversation_system(&self) -> bool;
fn system_token(&self) -> &'static str;
fn tool_calls_block_name(&self) -> &'static str;
fn tool_call_tag_name(&self) -> &'static str;
fn tool_parameter_tag_name(&self) -> &'static str;
fn render_reasoning_effort(
&self,
index: usize,
thinking_mode: bool,
effort: Option<ReasoningEffort>,
) -> String;
}
fn tool_call_template(encoding: &impl EncodingV4, name: &str, arguments: &str) -> String {
let tool_call_tag_name = encoding.tool_call_tag_name();
format!(
"<{DSML_SP_TOKEN}{tool_call_tag_name} name=\"{name}\">\n{arguments}\n</{DSML_SP_TOKEN}{tool_call_tag_name}>"
)
}
fn tool_calls_template(encoding: &impl EncodingV4, tool_calls: &str) -> String {
let tc_block_name = encoding.tool_calls_block_name();
format!("<{DSML_SP_TOKEN}{tc_block_name}>\n{tool_calls}\n</{DSML_SP_TOKEN}{tc_block_name}>")
}
fn render_tool_calls(encoding: &impl EncodingV4, tool_calls: &[ToolCall]) -> String {
tool_calls
.iter()
.map(|tool_call| {
tool_call_template(
encoding,
&tool_call.name,
&render_tool_arguments(tool_call, encoding.tool_parameter_tag_name()),
)
})
.collect::<Vec<_>>()
.join("\n")
}
fn render_message(
encoding: &impl EncodingV4,
messages: &[InputMessage],
index: usize,
thinking_mode: bool,
reasoning_effort: Option<ReasoningEffort>,
) -> String {
let msg = &messages[index];
let prev = messages[..index].last();
let reasoning_effort_prompt =
encoding.render_reasoning_effort(index, thinking_mode, reasoning_effort);
let mut prompt = if index == 0
&& (!reasoning_effort_prompt.is_empty() || matches!(msg, InputMessage::System { .. }))
{
encoding.system_token().to_string()
} else {
String::new()
};
prompt += &reasoning_effort_prompt;
match msg {
InputMessage::System { content } => {
if index > 0 && encoding.supports_mid_conversation_system() {
prompt += encoding.system_token();
}
prompt += content;
}
InputMessage::User { content, .. } => {
if matches!(
prev,
Some(InputMessage::User { .. } | InputMessage::Tool { .. })
) {
prompt += "\n\n";
} else {
prompt += USER_SP_TOKEN;
}
prompt += content;
}
InputMessage::LatestReminder { content } => {
prompt += LATEST_REMINDER_SP_TOKEN;
prompt += content;
}
InputMessage::Tool { content, .. } => {
if matches!(
prev,
Some(InputMessage::User { .. } | InputMessage::Tool { .. })
) {
prompt += "\n\n";
} else {
prompt += USER_SP_TOKEN;
}
prompt += &format!("<tool_result>{content}</tool_result>");
}
InputMessage::Assistant {
content,
reasoning_content,
tool_calls,
} => {
let tool_calls_content = match tool_calls {
Some(tool_calls) if !tool_calls.is_empty() => {
format!(
"\n\n{}",
tool_calls_template(encoding, &render_tool_calls(encoding, tool_calls))
)
}
_ => String::new(),
};
let mut thinking_part = String::new();
if thinking_mode && index > 0 {
if let Some(reasoning_content) = reasoning_content {
thinking_part += reasoning_content;
}
thinking_part += THINKING_END_TOKEN;
}
prompt += ASSISTANT_SP_TOKEN;
prompt += if !thinking_part.is_empty() {
THINKING_START_TOKEN
} else {
THINKING_END_TOKEN
};
prompt += &thinking_part;
prompt += content;
prompt += &tool_calls_content;
prompt += EOS_TOKEN;
}
}
prompt
}
impl<T: EncodingV4> PromptEncoding for T {
fn encode(&self, conversation: &Conversation) -> Result<Vec<u32>, EncodingError> {
let tokenizer = self.tokenizer().ok_or(EncodingError::MissingTokenizer)?;
let rendered = self.render_conversation(conversation);
tokenizer
.encode_ids(&rendered.prompt)
.map_err(EncodingError::Encode)
}
fn render_conversation(&self, conversation: &Conversation) -> RenderedPrompt {
let mut messages = normalize_messages(self, &conversation.messages);
let has_tools =
conversation.tool_choice != ToolChoice::None && !conversation.tools.is_empty();
let format_schema = match &conversation.response_format {
ResponseFormat::Text => None,
ResponseFormat::JsonObject => Some(stringify_python_style(&serde_json::json!({
"type": "json_object"
}))),
};
if has_tools || format_schema.is_some() {
if !matches!(messages.first(), Some(InputMessage::System { .. })) {
messages.insert(
0,
InputMessage::System {
content: String::new(),
},
);
}
if let Some(InputMessage::System { content }) = messages.first_mut() {
if has_tools {
content.push_str("\n\n");
content.push_str(&render_tool_prompt(self, &conversation.tools));
}
if let Some(schema) = format_schema {
content.push_str("\n\n## Response Format:\n\nYou MUST strictly adhere to the following schema to reply:\n");
content.push_str(&schema);
}
}
}
let mut prompt = BOS_TOKEN.to_string();
for index in 0..messages.len() {
prompt += &render_message(
self,
&messages,
index,
conversation.thinking_mode,
conversation.reasoning_effort,
);
}
prompt.push_str(ASSISTANT_SP_TOKEN);
prompt.push_str(if conversation.thinking_mode {
THINKING_START_TOKEN
} else {
THINKING_END_TOKEN
});
if conversation.tool_choice == ToolChoice::Required && !conversation.tools.is_empty() {
prompt.push_str(&format!(
"\n\n<{DSML_SP_TOKEN}{}>\n",
self.tool_calls_block_name()
));
}
let image_sources = messages
.iter()
.filter_map(|message| match message {
InputMessage::User { image_sources, .. }
| InputMessage::Tool { image_sources, .. } => Some(image_sources.as_slice()),
_ => None,
})
.flatten()
.cloned()
.collect();
RenderedPrompt {
prompt,
image_sources,
}
}
}
fn render_tool_prompt(encoding: &impl EncodingV4, tools: &[ToolDefinition]) -> String {
let tool_schemas = tools
.iter()
.map(|tool| {
stringify_python_style(&serde_json::json!({
"name": tool.name,
"description": tool.description.as_deref().unwrap_or_default(),
"parameters": tool.parameters,
}))
})
.collect::<Vec<_>>()
.join("\n");
let dsml_token = DSML_SP_TOKEN;
let tc_block_name = encoding.tool_calls_block_name();
let tool_call_tag_name = encoding.tool_call_tag_name();
let tool_parameter_tag_name = encoding.tool_parameter_tag_name();
let thinking_start_token = THINKING_START_TOKEN;
let thinking_end_token = THINKING_END_TOKEN;
format!(
r#"## Tools
You have access to a set of tools to help answer the user's question. You can invoke tools by writing a "<{dsml_token}{tc_block_name}>" block like the following:
<{dsml_token}{tc_block_name}>
<{dsml_token}{tool_call_tag_name} name="$TOOL_NAME">
<{dsml_token}{tool_parameter_tag_name} name="$PARAMETER_NAME" string="true|false">$PARAMETER_VALUE</{dsml_token}{tool_parameter_tag_name}>
...
</{dsml_token}{tool_call_tag_name}>
<{dsml_token}{tool_call_tag_name} name="$TOOL_NAME2">
...
</{dsml_token}{tool_call_tag_name}>
</{dsml_token}{tc_block_name}>
String parameters should be specified as is and set `string="true"`. For all other types (numbers, booleans, arrays, objects), pass the value in JSON format and set `string="false"`.
If thinking_mode is enabled (triggered by {thinking_start_token}), you MUST output your complete reasoning inside {thinking_start_token}...{thinking_end_token} BEFORE any tool calls or final response.
Otherwise, output directly after {thinking_end_token} with tool calls or final response.
### Available Tool Schemas
{tool_schemas}
You MUST strictly follow the above defined tool name and parameter schemas to invoke tool calls.
"#
)
}
fn normalize_messages(encoding: &impl EncodingV4, messages: &[InputMessage]) -> Vec<InputMessage> {
let mut normalized = Vec::new();
let mut has_non_system = false;
for message in messages.iter().cloned() {
match message {
InputMessage::System { content } if !encoding.supports_mid_conversation_system() => {
if !has_non_system {
if let Some(InputMessage::System { content: head }) = normalized.last_mut() {
if !head.is_empty() && !content.is_empty() {
head.push_str("\n\n");
}
head.push_str(&content);
} else {
normalized.push(InputMessage::System { content });
}
} else if !content.is_empty() {
normalized.push(InputMessage::User {
content,
image_sources: Vec::new(),
});
}
}
InputMessage::User {
content,
image_sources,
} => {
has_non_system = true;
if let Some(InputMessage::User {
content: previous,
image_sources: previous_image_sources,
}) = normalized.last_mut()
{
previous.push_str("\n\n");
previous.push_str(&content);
previous_image_sources.extend(image_sources);
} else {
normalized.push(InputMessage::User {
content,
image_sources,
});
}
}
message => {
has_non_system |= !matches!(message, InputMessage::System { .. });
normalized.push(message);
}
}
}
sort_tool_results_by_call_order(&mut normalized);
normalized
}
fn sort_tool_results_by_call_order(messages: &mut [InputMessage]) {
let mut order: Vec<String> = Vec::new();
let mut idx = 0;
while idx < messages.len() {
match &messages[idx] {
InputMessage::Assistant {
tool_calls: Some(tool_calls),
..
} if !tool_calls.is_empty() => {
order = tool_calls.iter().map(|tc| tc.id.clone()).collect();
idx += 1;
}
InputMessage::User { .. } | InputMessage::Tool { .. } => {
let start = idx;
while idx < messages.len()
&& matches!(
messages[idx],
InputMessage::User { .. } | InputMessage::Tool { .. }
)
{
idx += 1;
}
let tool_idxs: Vec<usize> = (start..idx)
.filter(|&i| matches!(messages[i], InputMessage::Tool { .. }))
.collect();
if tool_idxs.len() > 1 && !order.is_empty() {
let mut tools: Vec<InputMessage> = tool_idxs
.iter()
.map(|&i| {
std::mem::replace(
&mut messages[i],
InputMessage::LatestReminder {
content: String::new(),
},
)
})
.collect();
tools.sort_by_key(|m| match m {
InputMessage::Tool { tool_call_id, .. } => {
order.iter().position(|id| id == tool_call_id).unwrap_or(0)
}
_ => 0,
});
for (&i, tool) in tool_idxs.iter().zip(tools) {
messages[i] = tool;
}
}
}
_ => idx += 1,
}
}
}

30
deepseek-recipe-image/Cargo.toml Normal file
View File

@ -0,0 +1,30 @@
[package]
name = "deepseek-recipe-image"
version = "0.1.0"
edition.workspace = true
description = "Image fetching and preprocessing for DeepSeek V4.1 model input."
readme.workspace = true
documentation = "https://docs.rs/deepseek-recipe-image"
license.workspace = true
repository.workspace = true
[dependencies]
data-url = { workspace = true }
deepseek-recipe-core = { workspace = true }
futures = { workspace = true }
image = { workspace = true, optional = true }
infer = { workspace = true, optional = true }
opencv = { workspace = true, optional = true }
reqwest = { workspace = true, optional = true }
thiserror = { workspace = true }
tokio = { workspace = true }
tokio-stream = { workspace = true }
tracing = { workspace = true }
[features]
default = ["opencv-preprocess", "reqwest-fetch"]
# Supply `OpenCvImagePreprocessor` for DeepSeek V4.1 preprocessing. Disable this
# feature to build without OpenCV when the caller supplies a preprocessor.
opencv-preprocess = ["dep:image", "dep:infer", "dep:opencv"]
# Supply `ReqwestImageFetcher` for downloading images from external URLs.
reqwest-fetch = ["dep:reqwest"]

21
deepseek-recipe-image/LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 DeepSeek
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

112
deepseek-recipe-image/src/error.rs Normal file
View File

@ -0,0 +1,112 @@
//! Failures while resolving or preprocessing an image.
use deepseek_recipe_core::multimodal::CalcResizeError;
/// Failure to fetch or preprocess one image.
#[derive(Debug, thiserror::Error)]
pub enum ImageError {
/// The image source needs a capability this implementation does not provide.
#[error("unsupported image source: {0}")]
Unsupported(&'static str),
/// The external image URL is too long.
#[error("external link length {len} too long, max link length {max}")]
UrlTooLong {
/// Length of the rejected URL.
len: usize,
/// Maximum accepted length.
max: usize,
},
/// The external image reference lacks the required `http` prefix.
#[error("invalid url: {url:?}")]
InvalidUrl {
/// Rejected URL.
url: String,
},
/// The request carries more images than the limit allows.
#[error("too many images: max {max} images per request, got {count}")]
TooManyImages {
/// Maximum number of images per request.
max: usize,
/// Number of images in the request.
count: usize,
},
/// One image is larger than the limit allows.
#[error("image size {size} bytes exceeds the limit of {max} bytes")]
ImageTooLarge {
/// Size of the rejected image.
size: usize,
/// Maximum accepted size.
max: usize,
},
/// The images together are larger than the limit allows.
#[error("total image size {size} bytes exceeds the limit of {max} bytes")]
TotalSizeTooLarge {
/// Total size of all images.
size: usize,
/// Maximum accepted total size.
max: usize,
},
/// The data URL or its encoded body is malformed.
#[error("invalid data url: {0}")]
InvalidDataUrl(String),
/// The image bytes are empty.
#[error("input image data is empty")]
EmptyImage,
/// The image media type is unsupported.
#[error("unsupported image media type: {0}")]
UnsupportedMediaType(String),
/// The image dimensions exceed the limit.
#[error("image dimensions exceed the maximum allowed")]
ImageDimensionsTooLarge,
/// The image could not be decoded.
#[error("failed to decode image: {0}")]
Decode(String),
/// The image could not be resized.
#[error("failed to resize image: {0}")]
Resize(String),
/// The image could not be encoded.
#[error("failed to encode image: {0}")]
Encode(String),
/// The image encoder reported failure without an error message.
#[error("failed to encode image")]
EncodeFailed,
/// Fitting the image into the token budget failed.
#[error(transparent)]
TokenBudget(#[from] CalcResizeError),
/// The image could not be fetched.
#[error("failed to fetch image from {url}: {source}")]
Fetch {
/// Rejected URL.
url: String,
/// Underlying fetcher failure, including wrapped input or size errors.
#[source]
source: Box<dyn std::error::Error + Send + Sync>,
},
/// The image server returned a non-success status.
#[error("failed to fetch image from {url}: unexpected status {status}")]
FetchStatus {
/// Rejected URL.
url: String,
/// HTTP status code of the response.
status: u16,
},
/// The HTTP client could not be constructed.
#[error("failed to create the HTTP client: {0}")]
Client(String),
}
impl ImageError {
/// Return whether another attempt of the same operation may succeed.
///
/// `Fetch` is retryable regardless of its source; `FetchStatus` is retryable
/// for status codes at least 500. Other variants are not retryable. The
/// resolver evaluates the fetcher's error before wrapping it in `Fetch`,
/// so a returned wrapper may have a different classification.
pub fn is_retryable(&self) -> bool {
match self {
Self::Fetch { .. } => true,
Self::FetchStatus { status, .. } => *status >= 500,
_ => false,
}
}
}

125
deepseek-recipe-image/src/fetcher.rs Normal file
View File

@ -0,0 +1,125 @@
//! Image fetching over HTTP with the `reqwest` client.
use std::time::Duration;
use tokio_stream::StreamExt;
use crate::ImageFetcher;
use crate::error::ImageError;
use crate::limits::{ImageByteBudget, ImageLimits};
/// Downloads images from external URLs with `reqwest`.
///
/// Clients created by [`Self::new`] and [`Self::with_max_bytes`] follow at most
/// five redirects and use a ten-second connection timeout and a sixty-second
/// request timeout. [`Self::from_client`] preserves the supplied client's
/// settings. All constructors enforce the configured response body size limit.
/// A response body is read as a stream, and the bytes that are kept are reserved
/// from the budget of the resolve call, so a body that would exceed the total
/// limit of the request stops the download.
///
/// We strongly recommend that you implement SSRF protection, such as rejecting
/// private, loopback, and link-local addresses, when callers may supply image
/// URLs; this fetcher does not filter its targets, because we think it is beyond
/// the scope of the library.
pub struct ReqwestImageFetcher {
client: reqwest::Client,
max_bytes: usize,
}
impl ReqwestImageFetcher {
/// Create a fetcher with the default image size limit.
///
/// # Errors
///
/// Returns [`ImageError`] when the HTTP client cannot be constructed.
pub fn new() -> Result<Self, ImageError> {
Self::with_max_bytes(ImageLimits::default().max_image_bytes)
}
/// Create a fetcher with an explicit size limit in bytes.
///
/// # Errors
///
/// Returns [`ImageError`] when the HTTP client cannot be constructed.
pub fn with_max_bytes(max_bytes: usize) -> Result<Self, ImageError> {
let client = reqwest::Client::builder()
.connect_timeout(Duration::from_secs(10))
.timeout(Duration::from_secs(60))
.redirect(reqwest::redirect::Policy::limited(5))
.build()
.map_err(|err| ImageError::Client(err.to_string()))?;
Ok(Self { client, max_bytes })
}
/// Create a fetcher from an existing client.
pub fn from_client(client: reqwest::Client, max_bytes: usize) -> Self {
Self { client, max_bytes }
}
/// Return the encoded size limit this fetcher applies to one image.
///
/// The resolver can apply a smaller limit through the
/// [`ImageByteBudget`] of the resolve call.
pub const fn max_bytes(&self) -> usize {
self.max_bytes
}
}
impl ImageFetcher for ReqwestImageFetcher {
async fn fetch(&self, url: &str, budget: &ImageByteBudget) -> Result<Vec<u8>, ImageError> {
let response = self
.client
.get(url)
.send()
.await
.map_err(|err| ImageError::Fetch {
url: url.to_string(),
source: Box::new(err),
})?;
if !response.status().is_success() {
return Err(ImageError::FetchStatus {
url: url.to_string(),
status: response.status().as_u16(),
});
}
// The limit of the fetcher and the limit of the resolve call both apply.
let max_bytes = self.max_bytes.min(budget.max_image_bytes());
if let Some(length) = response.content_length()
&& length > max_bytes as u64
{
return Err(ImageError::ImageTooLarge {
size: length as usize,
max: max_bytes,
});
}
let mut body = Vec::new();
let mut stream = response.bytes_stream();
while let Some(chunk) = stream.next().await {
let chunk = match chunk {
Ok(chunk) => chunk,
Err(err) => {
// A response that fails while its body is read is the
// retryable failure that can occur after bytes were
// reserved, so the retry reserves them once.
budget.release(body.len());
return Err(ImageError::Fetch {
url: url.to_string(),
source: Box::new(err),
});
}
};
let image_bytes = body.len() + chunk.len();
if image_bytes > max_bytes {
return Err(ImageError::ImageTooLarge {
size: image_bytes,
max: max_bytes,
});
}
budget.reserve(chunk.len())?;
body.extend_from_slice(&chunk);
}
Ok(body)
}
}

94
deepseek-recipe-image/src/lib.rs Normal file
View File

@ -0,0 +1,94 @@
//! Image fetching and preprocessing for DeepSeek V4.1 model input.
//!
//! [`ImageResolver`] turns the [`ImageSource`](deepseek_recipe_core::multimodal::ImageSource)
//! values produced by protocol conversion into
//! [`MultiModalData`](deepseek_recipe_core::multimodal::MultiModalData). Fetching
//! external URLs and preprocessing are supplied by [`ImageFetcher`] and
//! [`ImagePreprocessor`] implementations.
//!
//! The default `opencv-preprocess` feature provides `OpenCvImagePreprocessor`,
//! and the default `reqwest-fetch` feature provides `ReqwestImageFetcher`. A
//! caller that supplies its own preprocessor disables default features and
//! builds without OpenCV.
use deepseek_recipe_core::multimodal::ImageInfo;
mod error;
#[cfg(feature = "reqwest-fetch")]
mod fetcher;
mod limits;
#[cfg(feature = "opencv-preprocess")]
mod opencv;
mod resolver;
mod retry;
pub use error::ImageError;
#[cfg(feature = "reqwest-fetch")]
pub use fetcher::ReqwestImageFetcher;
pub use limits::{ImageByteBudget, ImageLimits, ImageQuota, PreprocessOptions};
#[cfg(feature = "opencv-preprocess")]
pub use opencv::OpenCvImagePreprocessor;
pub use resolver::ImageResolver;
pub use retry::RetryPolicy;
use std::future::Future;
/// Fetches the bytes of an image identified by an external URL.
///
/// The default implementation rejects every URL. Supply an implementation when
/// requests may reference external images.
pub trait ImageFetcher: Send + Sync {
/// Fetch the bytes of `url`.
///
/// The resolver checks a case-insensitive `http` prefix before calling this
/// method. The fetcher must validate the complete URL and allowed destinations.
///
/// The fetches of one resolve call run concurrently, so an implementation
/// keeps their total size within the limit by reserving every chunk it keeps
/// from `budget` before it keeps the chunk, and it rejects a body larger than
/// [`ImageByteBudget::max_image_bytes`]. An implementation releases the bytes
/// it has reserved before it returns a retryable error, so the retry of that
/// attempt reserves them once. The resolver checks the size of the returned
/// bytes against [`ImageLimits`] as well.
///
/// # Errors
///
/// Returns [`ImageError`] when the URL cannot be fetched or the image
/// exceeds a limit of `budget`.
fn fetch(
&self,
url: &str,
budget: &ImageByteBudget,
) -> impl Future<Output = Result<Vec<u8>, ImageError>> + Send {
let _ = (url, budget);
async {
Err(ImageError::Unsupported(
"image fetching requires an ImageFetcher implementation",
))
}
}
}
/// Decodes, resizes, and encodes one image for the inference backend.
///
/// The default implementation rejects every image.
pub trait ImagePreprocessor: Send + Sync {
/// Preprocess one encoded image.
///
/// # Errors
///
/// Returns [`ImageError`] when the image is unsupported, exceeds a limit, or
/// cannot be preprocessed.
fn preprocess(
&self,
data: Vec<u8>,
options: PreprocessOptions,
) -> impl Future<Output = Result<ImageInfo, ImageError>> + Send {
let _ = (data, options);
async {
Err(ImageError::Unsupported(
"image preprocessing requires an ImagePreprocessor implementation",
))
}
}
}

246
deepseek-recipe-image/src/limits.rs Normal file
View File

@ -0,0 +1,246 @@
//! Limits applied while resolving image sources.
use std::sync::atomic::{AtomicUsize, Ordering};
use deepseek_recipe_core::multimodal::{ImageDetail, MAX_IMAGE_URL_LEN};
use crate::error::ImageError;
/// Limits applied to the images of one request.
#[derive(Debug, Clone, Copy)]
pub struct ImageLimits {
/// Maximum number of images in one request.
pub max_images: usize,
/// Maximum UTF-8 byte length of an external image URL.
pub max_url_len: usize,
/// Maximum encoded size of one image.
pub max_image_bytes: usize,
/// Maximum total encoded size of the images of one request.
pub max_total_bytes: usize,
/// Maximum number of sources processed concurrently by one resolve call.
/// A value of zero is treated as one.
pub max_concurrent_sources: usize,
/// Maximum width or height in pixels when a request has few images.
pub max_dimension_px: u32,
/// Maximum width or height in pixels when a request has many images.
pub max_dimension_on_many_images_px: u32,
/// Image count at which [`max_dimension_on_many_images_px`](Self::max_dimension_on_many_images_px)
/// applies.
pub many_images_threshold: usize,
/// Maximum long side before token-budget fitting at low detail.
/// The token specification may subsequently upscale the image.
pub low_detail_max_dimension_px: u32,
}
impl Default for ImageLimits {
fn default() -> Self {
Self {
max_images: 600,
max_url_len: MAX_IMAGE_URL_LEN,
max_image_bytes: 32 * 1024 * 1024,
max_total_bytes: 64 * 1024 * 1024,
max_concurrent_sources: 8,
max_dimension_px: 8192,
max_dimension_on_many_images_px: 4096,
many_images_threshold: 15,
low_detail_max_dimension_px: 512,
}
}
}
impl ImageLimits {
/// Return the maximum accepted width or height for a request carrying
/// `image_count` images.
pub const fn max_dimension(&self, image_count: usize) -> u32 {
if image_count < self.many_images_threshold {
self.max_dimension_px
} else {
self.max_dimension_on_many_images_px
}
}
/// Return the preprocessing options for a request carrying `image_count`
/// images at the given detail level.
pub const fn preprocess_options(
&self,
detail: ImageDetail,
image_count: usize,
) -> PreprocessOptions {
PreprocessOptions {
detail,
max_dimension_px: self.max_dimension(image_count),
low_detail_max_dimension_px: self.low_detail_max_dimension_px,
}
}
}
/// Usage accumulated over the resolve calls of one request.
///
/// The image count and total source byte limits are enforced across
/// [`resolve`](crate::ImageResolver::resolve) calls. Create one quota per
/// request and pass the same quota to every call. A call records its sources
/// after preprocessing every one of them, so a failed call adds nothing.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct ImageQuota {
images: usize,
bytes: usize,
}
impl ImageQuota {
/// Return an empty quota.
pub const fn new() -> Self {
Self {
images: 0,
bytes: 0,
}
}
/// Return the number of sources recorded by the completed calls.
pub const fn image_count(&self) -> usize {
self.images
}
/// Return the encoded source bytes recorded by the completed calls.
pub const fn byte_size(&self) -> usize {
self.bytes
}
/// Record `images` preprocessed sources of `bytes` encoded size.
pub(crate) fn add(&mut self, images: usize, bytes: usize) {
self.images += images;
self.bytes += bytes;
}
}
/// The encoded bytes still available to the image sources of one resolve call.
///
/// [`ImageResolver::resolve`](crate::ImageResolver::resolve) creates one budget
/// for each call and passes it to every fetch of that call. A fetch reserves the
/// bytes it keeps as its body arrives, so the fetches running concurrently
/// cannot together exceed [`ImageLimits::max_total_bytes`]. A budget is not
/// shared between calls: the images of earlier calls are accounted for through
/// the [`ImageQuota`] the call was given.
#[derive(Debug)]
pub struct ImageByteBudget {
/// Bytes still available to the sources of one resolve call.
remaining: AtomicUsize,
/// Maximum encoded size of one image.
max_image_bytes: usize,
/// Maximum total encoded size of the images of one request.
max_total_bytes: usize,
}
impl ImageByteBudget {
/// Create the budget of one request that has already accumulated
/// `used_bytes`.
///
/// `max_image_bytes` limits one image, and `max_total_bytes` limits every
/// image of the request.
pub const fn new(max_image_bytes: usize, max_total_bytes: usize, used_bytes: usize) -> Self {
Self {
remaining: AtomicUsize::new(max_total_bytes.saturating_sub(used_bytes)),
max_image_bytes,
max_total_bytes,
}
}
/// Return the maximum encoded size of one image.
pub const fn max_image_bytes(&self) -> usize {
self.max_image_bytes
}
/// Reserve `bytes` of one image.
///
/// # Errors
///
/// Returns [`ImageError::TotalSizeTooLarge`] when the request would exceed
/// its total encoded size. A failed reservation reserves nothing.
pub fn reserve(&self, bytes: usize) -> Result<(), ImageError> {
let mut available = self.remaining.load(Ordering::Relaxed);
loop {
if available < bytes {
let used = self.max_total_bytes.saturating_sub(available);
return Err(ImageError::TotalSizeTooLarge {
size: used.saturating_add(bytes),
max: self.max_total_bytes,
});
}
match self.remaining.compare_exchange_weak(
available,
available - bytes,
Ordering::Relaxed,
Ordering::Relaxed,
) {
Ok(_) => return Ok(()),
Err(current) => available = current,
}
}
}
/// Return `bytes` reserved by an attempt that failed.
pub fn release(&self, bytes: usize) {
self.remaining.fetch_add(bytes, Ordering::Relaxed);
}
}
/// Options for one image preprocessing call.
#[derive(Debug, Clone, Copy)]
pub struct PreprocessOptions {
/// Requested detail level.
pub detail: ImageDetail,
/// Maximum accepted source width or height in pixels, before resizing.
pub max_dimension_px: u32,
/// Maximum long side before token-budget fitting at low detail.
/// The token specification may subsequently upscale the image.
pub low_detail_max_dimension_px: u32,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reserving_bytes_reduces_the_available_bytes() {
let budget = ImageByteBudget::new(8, 10, 0);
assert_eq!(budget.max_image_bytes(), 8);
budget.reserve(6).expect("the total limit is available");
let error = budget
.reserve(5)
.expect_err("the reservation exceeds the total limit");
assert!(
matches!(error, ImageError::TotalSizeTooLarge { size: 11, max: 10 }),
"{error}"
);
budget
.reserve(4)
.expect("the remaining bytes are available");
}
#[test]
fn a_budget_accounts_for_the_bytes_of_earlier_calls() {
let budget = ImageByteBudget::new(8, 10, 6);
let error = budget
.reserve(5)
.expect_err("earlier calls used six bytes of the total limit");
assert!(
matches!(error, ImageError::TotalSizeTooLarge { size: 11, max: 10 }),
"{error}"
);
budget
.reserve(4)
.expect("the remaining bytes are available");
}
#[test]
fn a_release_returns_the_bytes_of_a_failed_attempt() {
let budget = ImageByteBudget::new(8, 10, 0);
budget.reserve(10).expect("the total limit is available");
budget.release(10);
budget
.reserve(10)
.expect("the released bytes are available again");
}
}

456
deepseek-recipe-image/src/opencv.rs Normal file
View File

@ -0,0 +1,456 @@
//! The default image preprocessor for DeepSeek V4.1.
use std::io::Cursor;
use deepseek_recipe_core::multimodal::{ImageInfo, ImageMediaType, ImageTokenSpec};
use image::{ImageDecoder, Limits};
use opencv::core::{Mat, MatTraitConst, Scalar, Vector, VectorToVec};
use opencv::imgcodecs;
use opencv::imgproc;
use opencv::prelude::*;
use crate::ImagePreprocessor;
use crate::error::ImageError;
use crate::limits::PreprocessOptions;
/// Padding color (BGR) matching the model image transform mean of 0.5.
const PAD_COLOR: Scalar = Scalar::new(127.0, 127.0, 127.0, 0.0);
/// Background (BGR) applied where an image has transparency: `#FDFDFD`.
const INFERENCE_BACKGROUND: (u8, u8, u8) = (0xfd, 0xfd, 0xfd);
/// Maximum size of one dimension of a WebP image.
const WEBP_MAX_DIMENSION: i32 = 16383;
/// Largest allocation allowed while reading only an image header.
const MAX_HEADER_ALLOC: u64 = 16 * 1024 * 1024;
/// WebP quality of a preprocessed image.
const WEBP_QUALITY: i32 = 90;
/// Preprocesses images with OpenCV.
///
/// Preprocessing decodes the image, applies the low-detail limit, fits the image
/// into the V4.1 token budget, and encodes it as WebP at quality 90.
#[derive(Debug, Clone, Copy, Default)]
pub struct OpenCvImagePreprocessor;
impl ImagePreprocessor for OpenCvImagePreprocessor {
async fn preprocess(
&self,
data: Vec<u8>,
options: PreprocessOptions,
) -> Result<ImageInfo, ImageError> {
tokio::task::spawn_blocking(move || preprocess(&data, options))
.await
.map_err(|err| ImageError::Decode(err.to_string()))?
}
}
/// Preprocess one encoded image.
fn preprocess(data: &[u8], options: PreprocessOptions) -> Result<ImageInfo, ImageError> {
if data.is_empty() {
return Err(ImageError::EmptyImage);
}
let media_type = detect_media_type(data)?;
check_dimensions(data, options.max_dimension_px)?;
let mut mat = decode_image_mat(data, media_type)?;
if options.detail.is_low() {
mat = limit_image_mat(mat, options.low_detail_max_dimension_px)?;
}
preprocess_mat(mat, options.max_dimension_px)
}
/// Return the media type of the image bytes, read from the file header.
fn detect_media_type(data: &[u8]) -> Result<ImageMediaType, ImageError> {
let kind = infer::get(data)
.ok_or_else(|| ImageError::UnsupportedMediaType("unknown media type".to_string()))?;
let mime = kind.mime_type();
ImageMediaType::from_mime(mime)
.ok_or_else(|| ImageError::UnsupportedMediaType(mime.to_string()))
}
/// Reject an image whose declared dimensions exceed `max_dimension`, reading the
/// file header only.
fn check_dimensions(data: &[u8], max_dimension: u32) -> Result<(), ImageError> {
let mut reader = image::ImageReader::new(Cursor::new(data))
.with_guessed_format()
.map_err(|err| ImageError::Decode(err.to_string()))?;
let mut limits = Limits::default();
limits.max_image_width = Some(max_dimension);
limits.max_image_height = Some(max_dimension);
limits.max_alloc = Some(MAX_HEADER_ALLOC);
reader.limits(limits);
let decoder = match reader.into_decoder() {
Ok(decoder) => decoder,
Err(err) => return check_dimensions_without_limits(err, data, max_dimension),
};
let (width, height) = decoder.dimensions();
if width > max_dimension || height > max_dimension {
return Err(ImageError::ImageDimensionsTooLarge);
}
Ok(())
}
/// Handle a header read that failed under the configured limits.
///
/// Decoders that do not support limits report the limit as unsupported. Their
/// dimensions are read directly instead.
fn check_dimensions_without_limits(
err: image::ImageError,
data: &[u8],
max_dimension: u32,
) -> Result<(), ImageError> {
let image::ImageError::Limits(limit_error) = &err else {
return Err(ImageError::Decode(err.to_string()));
};
match limit_error.kind() {
image::error::LimitErrorKind::DimensionError => Err(ImageError::ImageDimensionsTooLarge),
image::error::LimitErrorKind::Unsupported { .. } => {
let reader = image::ImageReader::new(Cursor::new(data))
.with_guessed_format()
.map_err(|err| ImageError::Decode(err.to_string()))?;
let decoder = reader
.into_decoder()
.map_err(|err| ImageError::Decode(err.to_string()))?;
let (width, height) = decoder.dimensions();
if width > max_dimension || height > max_dimension {
return Err(ImageError::ImageDimensionsTooLarge);
}
Ok(())
}
_ => Err(ImageError::Decode(err.to_string())),
}
}
/// Decode one image into an 8-bit BGR matrix.
///
/// GIF images are decoded with the `image` crate because OpenCV does not support
/// them. Images with transparency are blended onto the inference background.
fn decode_image_mat(data: &[u8], media_type: ImageMediaType) -> Result<Mat, ImageError> {
let mat = if media_type == ImageMediaType::Gif {
gif_to_mat(data)?
} else {
imgcodecs::imdecode(&Vector::from_slice(data), imgcodecs::IMREAD_UNCHANGED)
.map_err(|err| ImageError::Decode(err.to_string()))?
};
let size = mat
.size()
.map_err(|err| ImageError::Decode(err.to_string()))?;
if size.width == 0 || size.height == 0 {
return Err(ImageError::EmptyImage);
}
normalize_decoded_mat(mat)
}
/// Reduce a decoded matrix to 8-bit BGR.
fn normalize_decoded_mat(mat: Mat) -> Result<Mat, ImageError> {
let mat = if mat.depth() == opencv::core::CV_16U {
let mut mat8 = Mat::default();
mat.convert_to(&mut mat8, opencv::core::CV_8U, 1.0 / 257.0, 0.0)
.map_err(|err| ImageError::Decode(err.to_string()))?;
mat8
} else {
mat
};
match mat.channels() {
3 => Ok(mat),
4 => alpha_blend(&mat),
1 => {
let mut channels = Vector::<Mat>::new();
channels.push(mat.clone());
channels.push(mat.clone());
channels.push(mat);
let mut bgr = Mat::default();
opencv::core::merge(&channels, &mut bgr)
.map_err(|err| ImageError::Decode(err.to_string()))?;
Ok(bgr)
}
2 => {
let mut channels = Vector::<Mat>::new();
opencv::core::split(&mat, &mut channels)
.map_err(|err| ImageError::Decode(err.to_string()))?;
let gray = channels
.get(0)
.map_err(|err| ImageError::Decode(err.to_string()))?;
let alpha = channels
.get(1)
.map_err(|err| ImageError::Decode(err.to_string()))?;
let mut bgra_channels = Vector::<Mat>::new();
bgra_channels.push(gray.clone());
bgra_channels.push(gray.clone());
bgra_channels.push(gray);
bgra_channels.push(alpha);
let mut bgra = Mat::default();
opencv::core::merge(&bgra_channels, &mut bgra)
.map_err(|err| ImageError::Decode(err.to_string()))?;
alpha_blend(&bgra)
}
channels => Err(ImageError::Decode(format!(
"unsupported image channel count: {channels}"
))),
}
}
/// Blend a 4-channel BGRA matrix onto the inference background.
fn alpha_blend(bgra: &Mat) -> Result<Mat, ImageError> {
if !bgra.is_continuous() {
return Err(ImageError::Decode(
"image matrix is not continuous".to_string(),
));
}
let (rows, cols) = (bgra.rows(), bgra.cols());
// SAFETY: the allocation is owned by the returned matrix and is fully
// written by the loop below.
let mut blended = unsafe { Mat::new_rows_cols(rows, cols, opencv::core::CV_8UC3) }
.map_err(|err| ImageError::Decode(err.to_string()))?;
let source = bgra
.data_bytes()
.map_err(|err| ImageError::Decode(err.to_string()))?;
let target = blended
.data_bytes_mut()
.map_err(|err| ImageError::Decode(err.to_string()))?;
let (background_b, background_g, background_r) = INFERENCE_BACKGROUND;
let (background_b, background_g, background_r) = (
u32::from(background_b),
u32::from(background_g),
u32::from(background_r),
);
for (source_pixel, target_pixel) in source.chunks_exact(4).zip(target.chunks_exact_mut(3)) {
let alpha = u32::from(source_pixel[3]);
let inverse_alpha = 255 - alpha;
target_pixel[0] =
((u32::from(source_pixel[0]) * alpha + background_b * inverse_alpha + 127) / 255) as u8;
target_pixel[1] =
((u32::from(source_pixel[1]) * alpha + background_g * inverse_alpha + 127) / 255) as u8;
target_pixel[2] =
((u32::from(source_pixel[2]) * alpha + background_r * inverse_alpha + 127) / 255) as u8;
}
Ok(blended)
}
/// Scale an image down so that its long side is at most `max_dimension`.
fn limit_image_mat(mat: Mat, max_dimension: u32) -> Result<Mat, ImageError> {
let size = mat
.size()
.map_err(|err| ImageError::Resize(err.to_string()))?;
let long_side = size.width.max(size.height);
if long_side <= max_dimension as i32 {
return Ok(mat);
}
let ratio = max_dimension as f64 / long_side as f64;
let width = ((size.width as f64 * ratio).round() as i32).max(1);
let height = ((size.height as f64 * ratio).round() as i32).max(1);
direct_resize(&mat, width, height)
}
/// Fit an image into the V4.1 token budget and encode it as WebP.
fn preprocess_mat(mat: Mat, max_dimension_px: u32) -> Result<ImageInfo, ImageError> {
let size = mat
.size()
.map_err(|err| ImageError::Resize(err.to_string()))?;
// `check_dimensions` reads the size the header declares. The decoded matrix
// is the image the request carries, so it is checked against the limit too.
let max_dimension = max_dimension_px as i32;
if size.width > max_dimension || size.height > max_dimension {
return Err(ImageError::ImageDimensionsTooLarge);
}
let (best_width, best_height) =
fit_webp_target_size(size.width as usize, size.height as usize)?;
if best_width <= 0 || best_height <= 0 {
return Err(ImageError::Resize(format!(
"invalid target size: {best_width}x{best_height}"
)));
}
let resized = if size.width != best_width || size.height != best_height {
resize_to_best(&mat, size.width, size.height, best_width, best_height)
.or_else(|_| direct_resize(&mat, best_width, best_height))?
} else {
mat
};
let mut encoded = Vector::new();
let params = Vector::from_slice(&[imgcodecs::IMWRITE_WEBP_QUALITY, WEBP_QUALITY]);
// A false return value reports an encoding failure, leaving the output
// buffer without an encoded image.
let encode_succeeded = imgcodecs::imencode(".webp", &resized, &mut encoded, &params)
.map_err(|err| ImageError::Encode(err.to_string()))?;
if !encode_succeeded {
return Err(ImageError::EncodeFailed);
}
Ok(ImageInfo {
data: encoded.to_vec(),
width: best_width as u32,
height: best_height as u32,
})
}
/// Return the V4.1 target size of an image of `width`×`height` pixels that WebP
/// can encode.
///
/// [`ImageTokenSpec::v41`] fits the token budget without a dimension limit, and
/// an elongated image can require a side above [`WEBP_MAX_DIMENSION`]. A size
/// outside the limit is scaled into it and placed on the patch grid, and fitting
/// is repeated for that size, so the returned size is both a size the
/// specification leaves unchanged and a size WebP encodes.
fn fit_webp_target_size(width: usize, height: usize) -> Result<(i32, i32), ImageError> {
// The specification scales a size below its minimum area back up, so the
// limit is reached after a few rounds that each raise the short side.
const MAX_FIT_ROUNDS: usize = 6;
let spec = ImageTokenSpec::v41();
let limit = WEBP_MAX_DIMENSION as usize;
let mut source = (width, height);
for _ in 0..MAX_FIT_ROUNDS {
let fitted = spec.calc_resize(source.0, source.1)?;
if fitted.best_width <= limit && fitted.best_height <= limit {
return Ok((fitted.best_width as i32, fitted.best_height as i32));
}
source = scale_into_webp_limit(spec.patch_size(), fitted.best_width, fitted.best_height);
}
Err(ImageError::Resize(format!(
"no V4.1 target size within {WEBP_MAX_DIMENSION} pixels for an image of {width}x{height} pixels"
)))
}
/// Scale a size into the WebP dimension limit.
///
/// The long side is placed on the patch grid at or below the limit, and the
/// short side is placed on the patch grid at or above its scaled value. Raising
/// the short side keeps the scaled size at or above the minimum area of the
/// specification, which otherwise scales the long side past the limit again.
fn scale_into_webp_limit(patch_size: usize, width: usize, height: usize) -> (usize, usize) {
let limit = WEBP_MAX_DIMENSION as usize;
let long_side = width.max(height);
if long_side <= limit {
return (width, height);
}
let scaled_long = limit / patch_size * patch_size;
let scaled_short = (width.min(height) as f64 * scaled_long as f64 / long_side as f64).round();
let short_side = (scaled_short as usize).max(1).div_ceil(patch_size) * patch_size;
if width >= height {
(scaled_long, short_side)
} else {
(short_side, scaled_long)
}
}
/// Scale an image to fit `best_width`×`best_height` while preserving its aspect
/// ratio, and center it on a background of [`PAD_COLOR`].
fn resize_to_best(
mat: &Mat,
source_width: i32,
source_height: i32,
best_width: i32,
best_height: i32,
) -> Result<Mat, ImageError> {
let image_ratio = source_width as f64 / source_height as f64;
let target_ratio = best_width as f64 / best_height as f64;
let (width, height) = if image_ratio > target_ratio {
(
best_width,
round_half_even(source_height as f64 / source_width as f64 * best_width as f64).max(1),
)
} else if image_ratio < target_ratio {
(
round_half_even(source_width as f64 / source_height as f64 * best_height as f64).max(1),
best_height,
)
} else {
(best_width, best_height)
};
let mut scaled = Mat::default();
imgproc::resize(
mat,
&mut scaled,
opencv::core::Size::new(width, height),
0.0,
0.0,
imgproc::INTER_CUBIC,
)
.map_err(|err| ImageError::Resize(err.to_string()))?;
let left = round_half_even((best_width - width) as f64 * 0.5);
let top = round_half_even((best_height - height) as f64 * 0.5);
let right = best_width - width - left;
let bottom = best_height - height - top;
let mut padded = Mat::default();
opencv::core::copy_make_border(
&scaled,
&mut padded,
top,
bottom,
left,
right,
opencv::core::BORDER_CONSTANT,
PAD_COLOR,
)
.map_err(|err| ImageError::Resize(err.to_string()))?;
Ok(padded)
}
/// Scale an image directly to `width`×`height`.
fn direct_resize(mat: &Mat, width: i32, height: i32) -> Result<Mat, ImageError> {
let mut resized = Mat::default();
imgproc::resize(
mat,
&mut resized,
opencv::core::Size::new(width, height),
0.0,
0.0,
imgproc::INTER_CUBIC,
)
.map_err(|err| ImageError::Resize(err.to_string()))?;
Ok(resized)
}
/// Decode the first frame of a GIF into a 4-channel BGRA matrix.
fn gif_to_mat(data: &[u8]) -> Result<Mat, ImageError> {
let reader = image::ImageReader::new(Cursor::new(data))
.with_guessed_format()
.map_err(|err| ImageError::Decode(err.to_string()))?;
let format = reader
.format()
.ok_or_else(|| ImageError::Decode("unknown image format".to_string()))?;
if format != image::ImageFormat::Gif {
return Err(ImageError::Decode(format!("expected GIF, got {format:?}")));
}
let frame = reader
.decode()
.map_err(|err| ImageError::Decode(err.to_string()))?;
let (width, height) = (frame.width(), frame.height());
if width == 0 || height == 0 {
return Err(ImageError::EmptyImage);
}
// OpenCV expects BGRA; the `image` crate decodes to RGBA.
let mut bgra = frame.to_rgba8().into_raw();
for pixel in bgra.chunks_exact_mut(4) {
pixel.swap(0, 2);
}
// SAFETY: the allocation is owned by the returned matrix and is fully
// written from `bgra` immediately below.
let mut mat = unsafe { Mat::new_rows_cols(height as i32, width as i32, opencv::core::CV_8UC4) }
.map_err(|err| ImageError::Decode(err.to_string()))?;
mat.data_bytes_mut()
.map_err(|err| ImageError::Decode(err.to_string()))?
.copy_from_slice(&bgra);
Ok(mat)
}
/// Round to the nearest integer, with a value exactly between two integers
/// rounding to the even one.
fn round_half_even(value: f64) -> i32 {
let floor = value.floor();
let fraction = value - floor;
if fraction < 0.5 || (fraction == 0.5 && (floor as i64) % 2 == 0) {
floor as i32
} else {
(floor + 1.0) as i32
}
}

449
deepseek-recipe-image/src/resolver.rs Normal file
View File

@ -0,0 +1,449 @@
//! Resolve image sources into the images of one request.
use futures::StreamExt;
use futures::stream::{self, TryStreamExt};
use deepseek_recipe_core::multimodal::{ImageMediaType, ImageSource, MultiModalData, is_http_url};
use crate::error::ImageError;
use crate::limits::{ImageByteBudget, ImageLimits, ImageQuota};
use crate::retry::RetryPolicy;
use crate::{ImageFetcher, ImagePreprocessor};
/// Resolves image sources into preprocessed images.
///
/// Supports external URLs, data URLs, and encoded bytes. Applies request limits
/// and preserves source order. The caller supplies the fetcher and preprocessor.
/// At most [`ImageLimits::max_concurrent_sources`] sources are resolved and
/// preprocessed concurrently.
pub struct ImageResolver<F, P> {
fetcher: F,
preprocessor: P,
limits: ImageLimits,
retry: RetryPolicy,
}
impl<F, P> ImageResolver<F, P>
where
F: ImageFetcher,
P: ImagePreprocessor,
{
/// Combine an image fetcher with an image preprocessor.
pub fn new(fetcher: F, preprocessor: P) -> Self {
Self {
fetcher,
preprocessor,
limits: ImageLimits::default(),
retry: RetryPolicy::download(),
}
}
/// Replace the default limits.
pub fn with_limits(mut self, limits: ImageLimits) -> Self {
self.limits = limits;
self
}
/// Replace the default retry policy of image downloads.
pub fn with_retry(mut self, retry: RetryPolicy) -> Self {
self.retry = retry;
self
}
/// Resolve sources concurrently and preserve their order in the result.
///
/// Pass one [`ImageQuota`] per request and reuse it across calls, so the
/// count and total byte limits apply across calls. Dimension limits use the
/// accumulated count including this call; earlier images are not rechecked.
/// The call reserves the bytes it keeps from one [`ImageByteBudget`] it
/// shares with every fetch, so the sources of the call are limited together.
///
/// # Errors
///
/// Returns [`ImageError`] when a source cannot be fetched, decoded, or
/// preprocessed, or when the request exceeds a limit. A failed call leaves
/// `quota` unchanged: the sources of the call are recorded once every one of
/// them is preprocessed.
pub async fn resolve(
&self,
sources: &[ImageSource],
quota: &mut ImageQuota,
) -> Result<MultiModalData, ImageError> {
let request_image_count = quota.image_count() + sources.len();
if request_image_count > self.limits.max_images {
return Err(ImageError::TooManyImages {
max: self.limits.max_images,
count: request_image_count,
});
}
// The bytes of earlier calls of this request are recorded in `quota`
// rather than reserved again in this budget.
let budget = ImageByteBudget::new(
self.limits.max_image_bytes,
self.limits.max_total_bytes,
quota.byte_size(),
);
let concurrency = self.limits.max_concurrent_sources.max(1);
// The futures are created before they are polled, so the sources are
// resolved at most `concurrency` at a time.
let resolutions: Vec<_> = sources
.iter()
.map(|source| self.encoded_bytes(source, &budget))
.collect();
let encoded: Vec<Vec<u8>> = stream::iter(resolutions)
.buffered(concurrency)
.try_collect()
.await?;
// A fetch reserves the bytes it keeps while it reads a body. This check
// rejects an implementation that returns more than it reserved.
let mut request_bytes = quota.byte_size();
for data in &encoded {
if data.len() > self.limits.max_image_bytes {
return Err(ImageError::ImageTooLarge {
size: data.len(),
max: self.limits.max_image_bytes,
});
}
request_bytes += data.len();
if request_bytes > self.limits.max_total_bytes {
return Err(ImageError::TotalSizeTooLarge {
size: request_bytes,
max: self.limits.max_total_bytes,
});
}
}
let admitted_bytes = request_bytes - quota.byte_size();
let preprocesses: Vec<_> = encoded
.into_iter()
.zip(sources)
.map(|(data, source)| {
let options = self
.limits
.preprocess_options(source.detail(), request_image_count);
self.preprocessor.preprocess(data, options)
})
.collect();
let images = stream::iter(preprocesses)
.buffered(concurrency)
.try_collect()
.await?;
// The call records its sources only once all of them are preprocessed, so
// a failure or a cancellation leaves the quota unchanged.
quota.add(sources.len(), admitted_bytes);
Ok(MultiModalData { images })
}
/// Return the encoded bytes of one image source.
async fn encoded_bytes(
&self,
source: &ImageSource,
budget: &ImageByteBudget,
) -> Result<Vec<u8>, ImageError> {
let data = match source {
ImageSource::DataUrl { data_url, .. } => decode_data_url(data_url, budget)?,
ImageSource::Url { url, .. } => {
// Direct sources receive the same HTTP prefix check as
// protocol input. The fetcher validates the full URL.
if !is_http_url(url) {
return Err(ImageError::InvalidUrl { url: url.clone() });
}
if url.len() > self.limits.max_url_len {
return Err(ImageError::UrlTooLong {
len: url.len(),
max: self.limits.max_url_len,
});
}
self.retry
.execute(
|_| async { self.fetcher.fetch(url, budget).await },
ImageError::is_retryable,
)
.await
.map_err(|error| match error {
// The fetcher already reports the rejected URL.
error @ (ImageError::Fetch { .. } | ImageError::FetchStatus { .. }) => {
error
}
error => ImageError::Fetch {
url: url.clone(),
source: Box::new(error),
},
})?
}
ImageSource::Bytes { data, .. } => {
reserve_image_bytes(budget, data.len())?;
data.clone()
}
};
if data.is_empty() {
return Err(ImageError::EmptyImage);
}
Ok(data)
}
}
/// Reserve the bytes of one image that is already in memory.
fn reserve_image_bytes(budget: &ImageByteBudget, bytes: usize) -> Result<(), ImageError> {
let max = budget.max_image_bytes();
if bytes > max {
return Err(ImageError::ImageTooLarge { size: bytes, max });
}
budget.reserve(bytes)
}
/// Decode a base64 or percent-encoded data URL with a supported image media type.
fn decode_data_url(data_url: &str, budget: &ImageByteBudget) -> Result<Vec<u8>, ImageError> {
let parsed = data_url::DataUrl::process(data_url)
.map_err(|err| ImageError::InvalidDataUrl(err.to_string()))?;
let media_type = parsed.mime_type().to_string();
if ImageMediaType::from_mime(&media_type).is_none() {
return Err(ImageError::UnsupportedMediaType(media_type));
}
// A data URL carries the image in the request itself, so the decoded size is
// checked before the body is decoded and again once it is known.
let estimated = estimated_decoded_len(data_url);
let max_image_bytes = budget.max_image_bytes();
if estimated > max_image_bytes {
return Err(ImageError::ImageTooLarge {
size: estimated,
max: max_image_bytes,
});
}
let (data, _) = parsed
.decode_to_vec()
.map_err(|err| ImageError::InvalidDataUrl(err.to_string()))?;
reserve_image_bytes(budget, data.len())?;
Ok(data)
}
/// Return the decoded size a data URL body reports before it is decoded.
///
/// A base64 body occupies four characters for every three bytes it carries, so
/// the result can be up to two bytes smaller than the decoded body. A
/// percent-encoded byte occupies three characters.
fn estimated_decoded_len(data_url: &str) -> usize {
let Some((header, body)) = data_url.split_once(',') else {
return 0;
};
if header.ends_with(";base64") {
body.len() / 4 * 3
} else {
body.len().saturating_sub(2 * body.matches('%').count())
}
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use deepseek_recipe_core::multimodal::{ImageDetail, ImageInfo};
use super::*;
use crate::limits::PreprocessOptions;
/// A fetcher that returns a body of a fixed size and ignores the budget.
struct FixedFetcher {
body_len: usize,
}
impl ImageFetcher for FixedFetcher {
async fn fetch(
&self,
_url: &str,
_budget: &ImageByteBudget,
) -> Result<Vec<u8>, ImageError> {
Ok(vec![0; self.body_len])
}
}
/// A fetcher that records how many of its calls run at the same time.
#[derive(Clone, Default)]
struct CountingFetcher {
active: Arc<AtomicUsize>,
max_active: Arc<AtomicUsize>,
}
impl ImageFetcher for CountingFetcher {
async fn fetch(&self, _url: &str, budget: &ImageByteBudget) -> Result<Vec<u8>, ImageError> {
let active = self.active.fetch_add(1, Ordering::SeqCst) + 1;
self.max_active.fetch_max(active, Ordering::SeqCst);
tokio::task::yield_now().await;
self.active.fetch_sub(1, Ordering::SeqCst);
budget.reserve(1)?;
Ok(vec![0])
}
}
/// A preprocessor that rejects every image.
struct NoPreprocessor;
impl ImagePreprocessor for NoPreprocessor {}
/// A preprocessor that accepts every image without inspecting its bytes.
struct AcceptingPreprocessor;
impl ImagePreprocessor for AcceptingPreprocessor {
async fn preprocess(
&self,
data: Vec<u8>,
_options: PreprocessOptions,
) -> Result<ImageInfo, ImageError> {
Ok(ImageInfo {
data,
width: 1,
height: 1,
})
}
}
fn limits(max_image_bytes: usize, max_total_bytes: usize) -> ImageLimits {
ImageLimits {
max_image_bytes,
max_total_bytes,
..ImageLimits::default()
}
}
fn bytes_source(len: usize) -> ImageSource {
ImageSource::Bytes {
data: vec![0; len],
detail: ImageDetail::High,
}
}
fn url_source() -> ImageSource {
ImageSource::Url {
url: "https://example.com/image.webp".to_string(),
detail: ImageDetail::High,
}
}
#[tokio::test]
async fn bytes_over_the_image_limit_are_rejected() {
let resolver = ImageResolver::new(FixedFetcher { body_len: 0 }, NoPreprocessor)
.with_limits(limits(4, 64));
let error = resolver
.resolve(&[bytes_source(5)], &mut ImageQuota::new())
.await
.expect_err("the image exceeds the image limit");
assert!(
matches!(error, ImageError::ImageTooLarge { size: 5, max: 4 }),
"{error}"
);
}
#[tokio::test]
async fn bytes_over_the_total_limit_are_rejected() {
let resolver = ImageResolver::new(FixedFetcher { body_len: 0 }, NoPreprocessor)
.with_limits(limits(8, 10));
let error = resolver
.resolve(&[bytes_source(6), bytes_source(6)], &mut ImageQuota::new())
.await
.expect_err("the images exceed the total limit");
assert!(
matches!(error, ImageError::TotalSizeTooLarge { size: 12, max: 10 }),
"{error}"
);
}
#[tokio::test]
async fn a_fetcher_that_ignores_the_budget_is_rejected() {
let resolver = ImageResolver::new(FixedFetcher { body_len: 5 }, NoPreprocessor)
.with_limits(limits(4, 64));
let error = resolver
.resolve(&[url_source()], &mut ImageQuota::new())
.await
.expect_err("the fetched image exceeds the image limit");
assert!(
matches!(error, ImageError::ImageTooLarge { size: 5, max: 4 }),
"{error}"
);
}
#[tokio::test]
async fn data_urls_over_the_image_limit_are_rejected_without_decoding() {
let resolver = ImageResolver::new(FixedFetcher { body_len: 0 }, NoPreprocessor)
.with_limits(limits(4, 64));
let source = ImageSource::DataUrl {
data_url: format!("data:image/webp;base64,{}", "A".repeat(8)),
detail: ImageDetail::High,
};
let error = resolver
.resolve(&[source], &mut ImageQuota::new())
.await
.expect_err("the data url exceeds the image limit");
assert!(
matches!(error, ImageError::ImageTooLarge { size: 6, max: 4 }),
"{error}"
);
}
#[tokio::test]
async fn sources_within_the_limits_reach_preprocessing() {
let resolver = ImageResolver::new(FixedFetcher { body_len: 4 }, NoPreprocessor)
.with_limits(limits(4, 64));
let error = resolver
.resolve(&[url_source()], &mut ImageQuota::new())
.await
.expect_err("the preprocessor rejects every image");
assert!(matches!(error, ImageError::Unsupported(_)), "{error}");
}
#[tokio::test]
async fn a_completed_call_records_its_sources() {
let resolver = ImageResolver::new(FixedFetcher { body_len: 0 }, AcceptingPreprocessor)
.with_limits(limits(8, 64));
let mut quota = ImageQuota::new();
resolver
.resolve(&[bytes_source(6), bytes_source(6)], &mut quota)
.await
.expect("the preprocessor accepts every image");
assert_eq!(quota.image_count(), 2);
assert_eq!(quota.byte_size(), 12);
}
#[tokio::test]
async fn a_preprocessing_failure_leaves_the_quota_unchanged() {
let resolver = ImageResolver::new(FixedFetcher { body_len: 0 }, NoPreprocessor)
.with_limits(limits(8, 64));
let mut quota = ImageQuota::new();
resolver
.resolve(&[bytes_source(6), bytes_source(6)], &mut quota)
.await
.expect_err("the preprocessor rejects every image");
assert_eq!(quota, ImageQuota::new());
}
#[tokio::test]
async fn resolutions_are_limited_to_max_concurrent_sources() {
const SOURCES: usize = 6;
let fetcher = CountingFetcher::default();
let resolver =
ImageResolver::new(fetcher.clone(), NoPreprocessor).with_limits(ImageLimits {
max_concurrent_sources: 3,
..limits(8, 64)
});
let sources = vec![url_source(); SOURCES];
resolver
.resolve(&sources, &mut ImageQuota::new())
.await
.expect_err("the preprocessor rejects every image");
assert_eq!(
fetcher.max_active.load(Ordering::SeqCst),
3,
"the resolver starts at most max_concurrent_sources fetches at a time"
);
}
}

76
deepseek-recipe-image/src/retry.rs Normal file
View File

@ -0,0 +1,76 @@
//! Retrying an operation that may fail.
use std::future::Future;
use std::time::Duration;
/// Delays applied before repeated attempts of an operation.
///
/// A policy with `n` delays performs at most `n + 1` attempts.
#[derive(Debug, Clone, Default)]
pub struct RetryPolicy {
delays: Vec<Duration>,
}
impl RetryPolicy {
/// Return a policy that waits `delays` between attempts.
pub fn new(delays: Vec<Duration>) -> Self {
Self { delays }
}
/// Return a policy that performs one attempt.
pub const fn none() -> Self {
Self { delays: Vec::new() }
}
/// Return the policy used for image downloads: one retry after 0.3 seconds,
/// one after 0.6 seconds, and one after 1.0 second.
pub fn download() -> Self {
Self {
delays: vec![
Duration::from_millis(300),
Duration::from_millis(600),
Duration::from_secs(1),
],
}
}
/// Return the maximum number of attempts.
pub fn max_attempts(&self) -> usize {
self.delays.len() + 1
}
/// Run `operation` until it succeeds, fails without retrying, or runs out of
/// attempts.
///
/// `retryable` decides whether a failure starts another attempt. `attempt`
/// passed to `operation` is the zero-based attempt index.
///
/// # Errors
///
/// Returns the error of the final attempt.
pub async fn execute<T, E, F, Fut>(
&self,
mut operation: F,
retryable: impl Fn(&E) -> bool,
) -> Result<T, E>
where
F: FnMut(usize) -> Fut,
Fut: Future<Output = Result<T, E>>,
{
let mut attempt = 0;
loop {
let error = match operation(attempt).await {
Ok(value) => return Ok(value),
Err(error) => error,
};
let Some(delay) = self.delays.get(attempt) else {
return Err(error);
};
if !retryable(&error) {
return Err(error);
}
tokio::time::sleep(*delay).await;
attempt += 1;
}
}
}

27
deepseek-recipe-python/Cargo.toml Normal file
View File

@ -0,0 +1,27 @@
[package]
name = "deepseek-recipe-python"
version = "0.1.0"
edition = "2024"
description = "Python bindings for deepseek-recipe components."
readme = "README.md"
license.workspace = true
[lib]
crate-type = ["cdylib"]
name = "_native"
doc = false
[dependencies]
deepseek-recipe = { workspace = true }
deepseek-recipe-core = { workspace = true }
deepseek-recipe-encoding = { workspace = true }
deepseek-recipe-image = { workspace = true, features = [
"opencv-preprocess",
"reqwest-fetch",
] }
futures = { workspace = true }
pyo3 = { version = "0.27", features = ["abi3-py310"] }
serde = { workspace = true }
serde_json = { workspace = true }
tokenizers = { workspace = true }
tokio = { workspace = true }

21
deepseek-recipe-python/LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 DeepSeek
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

34
deepseek-recipe-python/README.md Normal file
View File

@ -0,0 +1,34 @@
# deepseek-recipe-python
Python bindings for deepseek-recipe: conversion of Messages, Chat Completions,
and Responses requests, DeepSeek V4/V4.1 prompt and token encoding, image
preprocessing, and response streaming. Model inference, tool execution, and HTTP
transport are provided by the application.
## Installation
Python 3.10 or later:
```sh
python3 -m pip install deepseek-recipe
```
Import as `deepseek_recipe`. Source builds require Rust, OpenCV 4.x, and Clang/libclang.
## Example
```python
from deepseek_recipe import ChatCompletionRequest, ConversionOptions, DeepseekV41Encoding
request = ChatCompletionRequest({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
})
converted = request.convert(ConversionOptions())
rendered = DeepseekV41Encoding().render_conversation(converted.conversation)
print(rendered.prompt)
```
## License
MIT. The distribution includes the license text.

578
deepseek-recipe-python/THIRD_PARTY_LICENSES Normal file
View File

@ -0,0 +1,578 @@
Third-party licenses of the libraries bundled in this distribution
==================================================================
The native extension of this package links the libraries below, whose builds are
bundled in the wheel under "deepseek_recipe/.dylibs" on macOS and
"deepseek_recipe.libs" on Linux. Each library keeps its own license, and the
texts are reproduced in full below.
OpenCV 4.12.0 Apache License 2.0
libjpeg-turbo (compiled into OpenCV imgcodecs) BSD 3-Clause and IJG
libpng (compiled into OpenCV imgcodecs) PNG Reference Library version 2
libwebp (compiled into OpenCV imgcodecs) BSD 3-Clause
zlib (compiled into OpenCV imgcodecs) zlib License
------------------------------------------------------------------------
OpenCV 4.12.0 - Apache License 2.0
------------------------------------------------------------------------
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
------------------------------------------------------------------------
OpenCV 4.12.0 - copyright notices
------------------------------------------------------------------------
Copyright (C) 2000-2022, Intel Corporation, all rights reserved.
Copyright (C) 2009-2011, Willow Garage Inc., all rights reserved.
Copyright (C) 2009-2016, NVIDIA Corporation, all rights reserved.
Copyright (C) 2010-2013, Advanced Micro Devices, Inc., all rights reserved.
Copyright (C) 2015-2023, OpenCV Foundation, all rights reserved.
Copyright (C) 2008-2016, Itseez Inc., all rights reserved.
Copyright (C) 2019-2023, Xperience AI, all rights reserved.
Copyright (C) 2019-2022, Shenzhen Institute of Artificial Intelligence and Robotics for Society, all rights reserved.
Copyright (C) 2022-2023, Southern University of Science And Technology, all rights reserved.
Third party copyrights are property of their respective owners.
------------------------------------------------------------------------
libjpeg-turbo - BSD 3-Clause and IJG
------------------------------------------------------------------------
libjpeg-turbo Licenses
======================
libjpeg-turbo is covered by two compatible BSD-style open source licenses:
- The IJG (Independent JPEG Group) License, which is listed in
[README.ijg](README.ijg)
This license applies to the libjpeg API library and associated programs,
including any code inherited from libjpeg and any modifications to that
code. Note that the libjpeg-turbo SIMD source code bears the
[zlib License](https://opensource.org/licenses/Zlib), but in the context of
the overall libjpeg API library, the terms of the zlib License are subsumed
by the terms of the IJG License.
- The Modified (3-clause) BSD License, which is listed below
This license applies to the TurboJPEG API library and associated programs, as
well as the build system. Note that the TurboJPEG API library wraps the
libjpeg API library, so in the context of the overall TurboJPEG API library,
both the terms of the IJG License and the terms of the Modified (3-clause)
BSD License apply.
Complying with the libjpeg-turbo Licenses
=========================================
This section provides a roll-up of the libjpeg-turbo licensing terms, to the
best of our understanding. This is not a license in and of itself. It is
intended solely for clarification.
1. If you are distributing a modified version of the libjpeg-turbo source,
then:
1. You cannot alter or remove any existing copyright or license notices
from the source.
**Origin**
- Clause 1 of the IJG License
- Clause 1 of the Modified BSD License
- Clauses 1 and 3 of the zlib License
2. You must add your own copyright notice to the header of each source
file you modified, so others can tell that you modified that file. (If
there is not an existing copyright header in that file, then you can
simply add a notice stating that you modified the file.)
**Origin**
- Clause 1 of the IJG License
- Clause 2 of the zlib License
3. You must include the IJG README file, and you must not alter any of the
copyright or license text in that file.
**Origin**
- Clause 1 of the IJG License
2. If you are distributing only libjpeg-turbo binaries without the source, or
if you are distributing an application that statically links with
libjpeg-turbo, then:
1. Your product documentation must include a message stating:
This software is based in part on the work of the Independent JPEG
Group.
**Origin**
- Clause 2 of the IJG license
2. If your binary distribution includes or uses the TurboJPEG API, then
your product documentation must include the text of the Modified BSD
License (see below.)
**Origin**
- Clause 2 of the Modified BSD License
3. You cannot use the name of the IJG or The libjpeg-turbo Project or the
contributors thereof in advertising, publicity, etc.
**Origin**
- IJG License
- Clause 3 of the Modified BSD License
4. The IJG and The libjpeg-turbo Project do not warrant libjpeg-turbo to be
free of defects, nor do we accept any liability for undesirable
consequences resulting from your use of the software.
**Origin**
- IJG License
- Modified BSD License
- zlib License
The Modified (3-clause) BSD License
===================================
Copyright (C)2009-2024 D. R. Commander. All Rights Reserved.<br>
Copyright (C)2015 Viktor Szathmáry. All Rights Reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
- Neither the name of the libjpeg-turbo Project nor the names of its
contributors may be used to endorse or promote products derived from this
software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS",
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
Why Two Licenses?
=================
The zlib License could have been used instead of the Modified (3-clause) BSD
License, and since the IJG License effectively subsumes the distribution
conditions of the zlib License, this would have effectively placed
libjpeg-turbo binary distributions under the IJG License. However, the IJG
License specifically refers to the Independent JPEG Group and does not extend
attribution and endorsement protections to other entities. Thus, it was
desirable to choose a license that granted us the same protections for new code
that were granted to the IJG for code derived from their software.
------------------------------------------------------------------------
libpng - PNG Reference Library version 2
------------------------------------------------------------------------
COPYRIGHT NOTICE, DISCLAIMER, and LICENSE
=========================================
PNG Reference Library License version 2
---------------------------------------
* Copyright (c) 1995-2024 The PNG Reference Library Authors.
* Copyright (c) 2018-2024 Cosmin Truta.
* Copyright (c) 2000-2002, 2004, 2006-2018 Glenn Randers-Pehrson.
* Copyright (c) 1996-1997 Andreas Dilger.
* Copyright (c) 1995-1996 Guy Eric Schalnat, Group 42, Inc.
The software is supplied "as is", without warranty of any kind,
express or implied, including, without limitation, the warranties
of merchantability, fitness for a particular purpose, title, and
non-infringement. In no event shall the Copyright owners, or
anyone distributing the software, be liable for any damages or
other liability, whether in contract, tort or otherwise, arising
from, out of, or in connection with the software, or the use or
other dealings in the software, even if advised of the possibility
of such damage.
Permission is hereby granted to use, copy, modify, and distribute
this software, or portions hereof, for any purpose, without fee,
subject to the following restrictions:
1. The origin of this software must not be misrepresented; you
must not claim that you wrote the original software. If you
use this software in a product, an acknowledgment in the product
documentation would be appreciated, but is not required.
2. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
3. This Copyright notice may not be removed or altered from any
source or altered source distribution.
PNG Reference Library License version 1 (for libpng 0.5 through 1.6.35)
-----------------------------------------------------------------------
libpng versions 1.0.7, July 1, 2000, through 1.6.35, July 15, 2018 are
Copyright (c) 2000-2002, 2004, 2006-2018 Glenn Randers-Pehrson, are
derived from libpng-1.0.6, and are distributed according to the same
disclaimer and license as libpng-1.0.6 with the following individuals
added to the list of Contributing Authors:
Simon-Pierre Cadieux
Eric S. Raymond
Mans Rullgard
Cosmin Truta
Gilles Vollant
James Yu
Mandar Sahastrabuddhe
Google Inc.
Vadim Barkov
and with the following additions to the disclaimer:
There is no warranty against interference with your enjoyment of
the library or against infringement. There is no warranty that our
efforts or the library will fulfill any of your particular purposes
or needs. This library is provided with all faults, and the entire
risk of satisfactory quality, performance, accuracy, and effort is
with the user.
Some files in the "contrib" directory and some configure-generated
files that are distributed with libpng have other copyright owners, and
are released under other open source licenses.
libpng versions 0.97, January 1998, through 1.0.6, March 20, 2000, are
Copyright (c) 1998-2000 Glenn Randers-Pehrson, are derived from
libpng-0.96, and are distributed according to the same disclaimer and
license as libpng-0.96, with the following individuals added to the
list of Contributing Authors:
Tom Lane
Glenn Randers-Pehrson
Willem van Schaik
libpng versions 0.89, June 1996, through 0.96, May 1997, are
Copyright (c) 1996-1997 Andreas Dilger, are derived from libpng-0.88,
and are distributed according to the same disclaimer and license as
libpng-0.88, with the following individuals added to the list of
Contributing Authors:
John Bowler
Kevin Bracey
Sam Bushell
Magnus Holmgren
Greg Roelofs
Tom Tanner
Some files in the "scripts" directory have other copyright owners,
but are released under this license.
libpng versions 0.5, May 1995, through 0.88, January 1996, are
Copyright (c) 1995-1996 Guy Eric Schalnat, Group 42, Inc.
For the purposes of this copyright and license, "Contributing Authors"
is defined as the following set of individuals:
Andreas Dilger
Dave Martindale
Guy Eric Schalnat
Paul Schmidt
Tim Wegner
The PNG Reference Library is supplied "AS IS". The Contributing
Authors and Group 42, Inc. disclaim all warranties, expressed or
implied, including, without limitation, the warranties of
merchantability and of fitness for any purpose. The Contributing
Authors and Group 42, Inc. assume no liability for direct, indirect,
incidental, special, exemplary, or consequential damages, which may
result from the use of the PNG Reference Library, even if advised of
the possibility of such damage.
Permission is hereby granted to use, copy, modify, and distribute this
source code, or portions hereof, for any purpose, without fee, subject
to the following restrictions:
1. The origin of this source code must not be misrepresented.
2. Altered versions must be plainly marked as such and must not
be misrepresented as being the original source.
3. This Copyright notice may not be removed or altered from any
source or altered source distribution.
The Contributing Authors and Group 42, Inc. specifically permit,
without fee, and encourage the use of this source code as a component
to supporting the PNG file format in commercial products. If you use
this source code in a product, acknowledgment is not required but would
be appreciated.
------------------------------------------------------------------------
libwebp - BSD 3-Clause
------------------------------------------------------------------------
Copyright (c) 2010, Google Inc. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with the
distribution.
* Neither the name of Google nor the names of its contributors may
be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
------------------------------------------------------------------------
zlib - zlib License
------------------------------------------------------------------------
Copyright notice:
(C) 1995-2022 Jean-loup Gailly and Mark Adler
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
Jean-loup Gailly Mark Adler
jloup@gzip.org madler@alumni.caltech.edu

8
deepseek-recipe-python/build.rs Normal file
View File

@ -0,0 +1,8 @@
//! Configure native library linking for Python wheels.
fn main() {
if std::env::var("CARGO_CFG_TARGET_OS").as_deref() == Ok("macos") {
// Wheel repair writes longer dependency paths into Mach-O load commands.
println!("cargo::rustc-link-arg-cdylib=-Wl,-headerpad_max_install_names");
}
}

34
deepseek-recipe-python/pyproject.toml Normal file
View File

@ -0,0 +1,34 @@
[build-system]
requires = ["maturin>=1.9,<2"]
build-backend = "maturin"
[project]
# The distribution name of PyPI; the import name stays `deepseek_recipe`. The
# Cargo package of this crate, and its directory, keep their own name.
name = "deepseek-recipe"
# The version of the distribution; the Cargo packages keep their own version.
version = "0.1.1"
description = "Python bindings for deepseek-recipe components"
readme = "README.md"
requires-python = ">=3.10"
license = "MIT"
license-files = ["LICENSE", "THIRD_PARTY_LICENSES"]
[project.optional-dependencies]
test = ["pytest>=8,<10"]
# PyPI shows these links on the project page. The instance serves them once the
# repository is public.
[project.urls]
Homepage = "https://github.com/deepseek-ai/deepseek-recipe"
Repository = "https://github.com/deepseek-ai/deepseek-recipe"
Documentation = "https://github.com/deepseek-ai/deepseek-recipe/tree/main/docs"
Issues = "https://github.com/deepseek-ai/deepseek-recipe/issues"
[tool.maturin]
python-source = "python"
module-name = "deepseek_recipe._native"
features = ["pyo3/extension-module"]
# Importing the package from the source tree leaves `__pycache__` behind, and the
# python-source walk picks it up. Exclude it from the wheel and the sdist.
exclude = ["**/__pycache__/**"]

129
deepseek-recipe-python/python/deepseek_recipe/__init__.py Normal file
View File

@ -0,0 +1,129 @@
"""Direct Python bindings for Rust request, rendering, image, and response components."""
from ._native import (
ASSISTANT_SP_TOKEN,
BOS_TOKEN,
DSML_SP_TOKEN,
EOS_TOKEN,
IMAGE_SPECIAL_TOKEN,
IMAGE_SPECIAL_TOKEN_ID,
LATEST_REMINDER_SP_TOKEN,
SYSTEM_SP_TOKEN,
THINKING_END_TOKEN,
THINKING_START_TOKEN,
USER_SP_TOKEN,
AssistantMessage,
BytesImageSource,
CalcResizeError,
ChatCompletionChunk,
ChatCompletionChunkGenerator,
ChatCompletionRequest,
ChatCompletionResponse,
Conversation,
ConversationRequest,
ConversionError,
ConversionOptions,
DataUrlImageSource,
DeepseekV41Encoding,
DeepseekV4Encoding,
ImageError,
ImageInfo,
ImageQuota,
ImageResolver,
ImageSource,
InferenceChunk,
InferenceFinishReason,
InferenceOptions,
LatestReminderMessage,
Message,
MessagesChunkGenerator,
MessagesRequest,
MessagesResponse,
MessagesStreamEvent,
MultiModalData,
OpenCvImagePreprocessor,
ParsingOptions,
PreprocessOptions,
PromptUsage,
ReasoningStage,
RenderedPrompt,
ReqwestImageFetcher,
ResponsesChunkGenerator,
ResponsesRequest,
ResponsesResponse,
ResponsesStreamEvent,
StreamProcessor,
SystemMessage,
Tokenizer,
ToolCall,
ToolDefinition,
ToolMessage,
UrlImageSource,
UserMessage,
WebSearchBehavior,
__version__,
)
__all__ = [
"ASSISTANT_SP_TOKEN",
"BOS_TOKEN",
"DSML_SP_TOKEN",
"EOS_TOKEN",
"IMAGE_SPECIAL_TOKEN",
"IMAGE_SPECIAL_TOKEN_ID",
"LATEST_REMINDER_SP_TOKEN",
"SYSTEM_SP_TOKEN",
"THINKING_END_TOKEN",
"THINKING_START_TOKEN",
"USER_SP_TOKEN",
"AssistantMessage",
"BytesImageSource",
"CalcResizeError",
"ChatCompletionChunk",
"ChatCompletionChunkGenerator",
"ChatCompletionRequest",
"ChatCompletionResponse",
"Conversation",
"ConversationRequest",
"ConversionError",
"ConversionOptions",
"DataUrlImageSource",
"DeepseekV41Encoding",
"DeepseekV4Encoding",
"ImageError",
"ImageInfo",
"ImageQuota",
"ImageResolver",
"ImageSource",
"InferenceChunk",
"InferenceFinishReason",
"InferenceOptions",
"LatestReminderMessage",
"Message",
"MessagesChunkGenerator",
"MessagesRequest",
"MessagesResponse",
"MessagesStreamEvent",
"MultiModalData",
"OpenCvImagePreprocessor",
"ParsingOptions",
"PreprocessOptions",
"PromptUsage",
"ReasoningStage",
"RenderedPrompt",
"ReqwestImageFetcher",
"ResponsesChunkGenerator",
"ResponsesRequest",
"ResponsesResponse",
"ResponsesStreamEvent",
"StreamProcessor",
"SystemMessage",
"Tokenizer",
"ToolCall",
"ToolDefinition",
"ToolMessage",
"UrlImageSource",
"UserMessage",
"WebSearchBehavior",
"__version__",
]

581
deepseek-recipe-python/python/deepseek_recipe/_native.pyi Normal file
View File

@ -0,0 +1,581 @@
from collections.abc import Sequence
from typing import Any, Literal, TypeAlias
__version__: str
BOS_TOKEN: str
EOS_TOKEN: str
SYSTEM_SP_TOKEN: str
USER_SP_TOKEN: str
ASSISTANT_SP_TOKEN: str
LATEST_REMINDER_SP_TOKEN: str
THINKING_START_TOKEN: str
THINKING_END_TOKEN: str
DSML_SP_TOKEN: str
IMAGE_SPECIAL_TOKEN: str
IMAGE_SPECIAL_TOKEN_ID: int
ToolChoice: TypeAlias = Literal["auto", "none", "required"]
ReasoningEffort: TypeAlias = Literal["low", "high", "xhigh", "max"]
ResponseFormat: TypeAlias = Literal["text", "json_object"]
ImageDetail: TypeAlias = Literal["low", "high", "original", "auto"]
Role: TypeAlias = Literal["system", "user", "assistant", "tool", "latest_reminder"]
ImageSourceKind: TypeAlias = Literal["data_url", "url", "bytes"]
RequestBody: TypeAlias = bytes | str | dict[str, Any]
class ConversionError(Exception):
"""Raised when a protocol request is invalid or cannot be converted."""
status_code: int
body: str
class ImageError(Exception):
"""A Rust image error with its original variant and retry classification."""
kind: str
is_retryable: bool
class CalcResizeError(ValueError):
"""The Rust image token calculation did not converge."""
class ToolCall:
"""A historical tool call and its arguments."""
def __init__(self, id: str, name: str, arguments: str) -> None: ...
@property
def id(self) -> str: ...
@property
def name(self) -> str: ...
@property
def arguments(self) -> str:
"""Caller-supplied arguments, expected to encode a JSON object; not validated."""
class ToolDefinition:
"""A client tool definition."""
def __init__(
self,
name: str,
description: str | None = None,
parameters: Any = None,
strict: bool | None = None,
) -> None: ...
@property
def name(self) -> str: ...
@property
def description(self) -> str | None: ...
@property
def parameters(self) -> Any:
"""Input JSON Schema of the tool."""
@property
def strict(self) -> bool | None: ...
class ImageSource:
"""Base class of the images supplied with a conversation."""
@property
def kind(self) -> ImageSourceKind: ...
@property
def detail(self) -> ImageDetail: ...
@property
def data_url(self) -> str | None: ...
@property
def url(self) -> str | None: ...
@property
def data(self) -> bytes | None: ...
class DataUrlImageSource(ImageSource):
"""An image referenced by a data URL with a base64 or percent-encoded body."""
def __init__(self, data_url: str, detail: ImageDetail = "high") -> None: ...
class UrlImageSource(ImageSource):
"""An image referenced by an external URL."""
def __init__(self, url: str, detail: ImageDetail = "high") -> None: ...
class BytesImageSource(ImageSource):
"""An image supplied as encoded bytes."""
def __init__(self, data: bytes, detail: ImageDetail = "high") -> None: ...
class Message:
"""Base class of the messages in a conversation."""
@property
def role(self) -> Role: ...
@property
def content(self) -> str: ...
@property
def reasoning_content(self) -> str | None: ...
@property
def tool_calls(self) -> list[ToolCall] | None: ...
@property
def tool_call_id(self) -> str | None: ...
@property
def image_sources(self) -> list[ImageSource]: ...
class SystemMessage(Message):
"""A system message."""
def __init__(self, content: str) -> None: ...
class UserMessage(Message):
"""A user message."""
def __init__(
self, content: str, image_sources: Sequence[ImageSource] | None = None
) -> None: ...
class AssistantMessage(Message):
"""An assistant message."""
def __init__(
self,
content: str,
reasoning_content: str | None = None,
tool_calls: Sequence[ToolCall] | None = None,
) -> None: ...
class ToolMessage(Message):
"""A tool result message."""
def __init__(
self,
content: str,
tool_call_id: str,
image_sources: Sequence[ImageSource] | None = None,
) -> None: ...
class LatestReminderMessage(Message):
"""A reminder message that carries additional instructions."""
def __init__(self, content: str) -> None: ...
class Conversation:
"""A conversation and its prompt configuration."""
def __init__(
self,
messages: Sequence[Message] | None = None,
*,
thinking_mode: bool = True,
tools: Sequence[ToolDefinition] | None = None,
tool_choice: ToolChoice = "auto",
reasoning_effort: ReasoningEffort | None = None,
response_format: ResponseFormat = "text",
) -> None: ...
@property
def messages(self) -> list[Message]: ...
@property
def thinking_mode(self) -> bool: ...
@property
def tools(self) -> list[ToolDefinition]: ...
@property
def tool_choice(self) -> ToolChoice: ...
@property
def reasoning_effort(self) -> ReasoningEffort | None: ...
@property
def response_format(self) -> ResponseFormat: ...
class RenderedPrompt:
"""A rendered prompt and the images its placeholders refer to."""
@property
def prompt(self) -> str: ...
@property
def image_sources(self) -> list[ImageSource]: ...
class DeepseekV4Encoding:
"""The Rust V4 prompt renderer. Token encoding requires an attached tokenizer."""
def __init__(self) -> None: ...
def with_tokenizer(self, tokenizer: Tokenizer) -> DeepseekV4Encoding: ...
def render_conversation(self, conversation: Conversation) -> RenderedPrompt: ...
def encode(self, conversation: Conversation) -> list[int]: ...
class DeepseekV41Encoding:
"""The Rust V4.1 prompt renderer. Token encoding requires an attached tokenizer."""
def __init__(self) -> None: ...
def with_tokenizer(self, tokenizer: Tokenizer) -> DeepseekV41Encoding: ...
def render_conversation(self, conversation: Conversation) -> RenderedPrompt: ...
def encode(self, conversation: Conversation) -> list[int]: ...
class WebSearchBehavior:
"""Handling of supported web-search declarations, choices, and Messages blocks."""
Ignore: WebSearchBehavior
Reject: WebSearchBehavior
class ConversionOptions:
"""Conversion defaults: Responses web search is ignored; Messages rejects it."""
def __init__(
self,
*,
default_thinking_mode: bool = True,
responses_web_search: WebSearchBehavior = WebSearchBehavior.Ignore,
messages_web_search: WebSearchBehavior = WebSearchBehavior.Reject,
) -> None: ...
@property
def default_thinking_mode(self) -> bool: ...
@property
def responses_web_search(self) -> WebSearchBehavior: ...
@property
def messages_web_search(self) -> WebSearchBehavior: ...
def with_default_thinking_mode(self, default_thinking_mode: bool) -> ConversionOptions: ...
def with_responses_web_search(
self, responses_web_search: WebSearchBehavior
) -> ConversionOptions: ...
def with_messages_web_search(
self, messages_web_search: WebSearchBehavior
) -> ConversionOptions: ...
class ConversationRequest:
"""A Rust conversation request with inference and parsing options."""
def __init__(self, conversation: Conversation) -> None: ...
@property
def conversation(self) -> Conversation: ...
@property
def inference_options(self) -> InferenceOptions: ...
@property
def parsing_options(self) -> ParsingOptions: ...
@property
def model(self) -> str | None: ...
@property
def stream(self) -> bool: ...
class ChatCompletionRequest:
"""A Rust Chat Completions request; even failed conversion consumes it."""
def __init__(self, body: RequestBody) -> None: ...
def include_usage(self) -> bool:
"""Read the stream usage setting before consuming this request."""
def convert(self, options: ConversionOptions) -> ConversationRequest: ...
@staticmethod
def chunk_generator(
request: ConversationRequest, id: str, model: str
) -> ChatCompletionChunkGenerator: ...
class ResponsesRequest:
"""A Rust Responses request; even failed conversion consumes it."""
def __init__(self, body: RequestBody) -> None: ...
def custom_tool_names(self) -> set[str]:
"""Read the declared custom tool names before consuming this request."""
def convert(self, options: ConversionOptions) -> ConversationRequest: ...
@staticmethod
def chunk_generator(
request: ConversationRequest, id: str, model: str
) -> ResponsesChunkGenerator: ...
class MessagesRequest:
"""A Rust Messages request; even failed conversion consumes it."""
def __init__(self, body: RequestBody) -> None: ...
def convert(self, options: ConversionOptions) -> ConversationRequest: ...
@staticmethod
def chunk_generator(
request: ConversationRequest, id: str, model: str
) -> MessagesChunkGenerator: ...
class InferenceOptions:
"""Converted inference parameters; the backend applies unspecified defaults."""
@property
def max_tokens(self) -> int | None: ...
@property
def temperature(self) -> float | None: ...
@property
def top_p(self) -> float | None: ...
@property
def thinking_budget_tokens(self) -> int | None: ...
@property
def disable_parallel_tool_use(self) -> bool | None: ...
class ImageInfo:
"""Encoded image bytes and their dimensions in pixels."""
def __init__(self, data: bytes, width: int, height: int) -> None: ...
@property
def data(self) -> bytes: ...
@property
def width(self) -> int: ...
@property
def height(self) -> int: ...
class MultiModalData:
"""Images in the order their placeholders appear in a prompt."""
def __init__(self, images: Sequence[ImageInfo] | None = None) -> None: ...
@property
def images(self) -> list[ImageInfo]: ...
def is_empty(self) -> bool: ...
def image_token_adjustment(self) -> int:
"""Image input tokens beyond one token per prompt placeholder."""
class ImageQuota:
"""Source count and encoded bytes recorded across resolve calls.
A call records its sources after it preprocesses every one of them, so a
failed call adds nothing.
"""
def __init__(self) -> None: ...
def image_count(self) -> int: ...
def byte_size(self) -> int: ...
class PreprocessOptions:
"""Options passed to one Rust image preprocessing operation."""
def __init__(
self, detail: ImageDetail, max_dimension_px: int, low_detail_max_dimension_px: int
) -> None: ...
@property
def detail(self) -> ImageDetail: ...
@property
def max_dimension_px(self) -> int: ...
@property
def low_detail_max_dimension_px(self) -> int: ...
class ReqwestImageFetcher:
"""Fetches external image URLs through the Rust HTTP client."""
def __init__(self) -> None: ...
@staticmethod
def with_max_bytes(max_bytes: int) -> ReqwestImageFetcher: ...
def fetch(self, url: str) -> bytes: ...
class OpenCvImagePreprocessor:
"""Preprocesses encoded images through the Rust OpenCV implementation."""
def __init__(self) -> None: ...
def preprocess(self, data: bytes, options: PreprocessOptions) -> ImageInfo: ...
class ImageResolver:
"""Resolves sources using explicit Rust image components and request quota."""
def __init__(
self, fetcher: ReqwestImageFetcher, preprocessor: OpenCvImagePreprocessor
) -> None: ...
def resolve(self, sources: Sequence[ImageSource], quota: ImageQuota) -> MultiModalData:
"""Resolve images and update the supplied quota; a failed call leaves it unchanged."""
class PromptUsage:
"""Prompt-side token counts from the backend."""
def __init__(
self, *, prompt_tokens: int = 0, prompt_cache_hit_tokens: int = 0,
) -> None: ...
@property
def prompt_tokens(self) -> int: ...
@property
def prompt_cache_hit_tokens(self) -> int: ...
class InferenceFinishReason:
"""Backend finish reasons; output parsing determines the protocol finish reason."""
Stop: InferenceFinishReason
Length: InferenceFinishReason
ContentFilter: InferenceFinishReason
class InferenceChunk:
"""One backend update, constructed with ready(), text(), token(), or finish()."""
@staticmethod
def ready(
*, prompt_usage: PromptUsage | None = None, system_fingerprint: str | None = None,
) -> InferenceChunk: ...
@staticmethod
def text(text: str, content_tokens: int = 0) -> InferenceChunk: ...
@staticmethod
def token(token_id: int) -> InferenceChunk: ...
@staticmethod
def finish(finish_reason: InferenceFinishReason) -> InferenceChunk: ...
@property
def kind(self) -> Literal["ready", "text", "token", "finish"]: ...
@property
def content(self) -> str | None: ...
@property
def token_id(self) -> int | None: ...
@property
def finish_reason(self) -> InferenceFinishReason | None: ...
@property
def prompt_usage(self) -> PromptUsage | None: ...
@property
def content_tokens(self) -> int | None:
"""The tokens this chunk's content accounts for; text chunks only."""
@property
def system_fingerprint(self) -> str | None: ...
class ReasoningStage:
"""Position within reasoning content when parsing begins."""
Start: ReasoningStage
Reasoning: ReasoningStage
Content: ReasoningStage
def start_from_reasoning(self) -> bool: ...
class ParsingOptions:
"""Settings passed directly to the Rust stream parser."""
def __init__(
self,
*,
parse_tool_calls: bool = True,
tool_call_initial_stage: bool = False,
parse_json_output: bool = False,
reasoning_initial_stage: ReasoningStage | None = None,
stop_sequences: Sequence[str] | None = None,
) -> None: ...
parse_tool_calls: bool
tool_call_initial_stage: bool
parse_json_output: bool
reasoning_initial_stage: ReasoningStage | None
@property
def stop_sequences(self) -> list[str]: ...
@stop_sequences.setter
def stop_sequences(self, value: Sequence[str]) -> None: ...
class ChatCompletionChunkGenerator:
"""A Rust Chat Completions generator consumed by a stream processor."""
def __init__(
self, id: str, model: str, include_usage: bool, thinking_mode: bool
) -> None: ...
def with_include_usage(self, include_usage: bool) -> ChatCompletionChunkGenerator:
"""Consume this generator and construct one with the supplied usage setting."""
class ResponsesChunkGenerator:
"""A Rust Responses generator consumed by a stream processor."""
def __init__(self, id: str, model: str) -> None: ...
def with_custom_tool_names(
self, custom_tool_names: set[str] | frozenset[str]
) -> ResponsesChunkGenerator:
"""Consume this generator and construct one with the supplied custom tool names."""
class MessagesChunkGenerator:
"""A Rust Messages generator consumed by a stream processor."""
def __init__(self, id: str, model: str, thinking_mode: bool) -> None: ...
def with_signature(self, signature: str) -> MessagesChunkGenerator:
"""Consume this generator and construct one with the supplied signature."""
class ChatCompletionChunk:
"""A Rust Chat Completions chunk consumed by response accumulation."""
def to_json(self) -> str: ...
class ResponsesStreamEvent:
"""A Rust Responses event consumed by response accumulation."""
def to_json(self) -> str: ...
class MessagesStreamEvent:
"""A Rust Messages event consumed by response accumulation."""
def to_json(self) -> str: ...
class ChatCompletionResponse:
"""A Rust Chat Completions response with explicit accumulation."""
def __init__(
self, id: str, model: str, created: int, prompt_tokens: int, prompt_cache_hit_tokens: int
) -> None: ...
def append(self, chunk: ChatCompletionChunk) -> None:
"""Consume this chunk and apply its delta to the response."""
def to_json(self) -> str: ...
@staticmethod
def chunk_event_type(chunk: ChatCompletionChunk) -> str | None: ...
@staticmethod
def done_message() -> str | None: ...
class ResponsesResponse:
"""A Rust Responses response with explicit accumulation."""
def __init__(
self, id: str, model: str, created: int, prompt_tokens: int, prompt_cache_hit_tokens: int
) -> None: ...
def append(self, chunk: ResponsesStreamEvent) -> None:
"""Consume this event and apply its delta to the response."""
def to_json(self) -> str: ...
@staticmethod
def chunk_event_type(chunk: ResponsesStreamEvent) -> str | None: ...
@staticmethod
def done_message() -> str | None: ...
class MessagesResponse:
"""A Rust Messages response with explicit accumulation."""
def __init__(
self, id: str, model: str, created: int, prompt_tokens: int, prompt_cache_hit_tokens: int
) -> None: ...
def append(self, chunk: MessagesStreamEvent) -> None:
"""Consume this event and apply its delta to the response."""
def to_json(self) -> str: ...
@staticmethod
def chunk_event_type(chunk: MessagesStreamEvent) -> str | None: ...
@staticmethod
def done_message() -> str | None: ...
class Tokenizer:
"""A HuggingFace tokenizer loaded from a `tokenizer.json` file."""
@staticmethod
def from_file(path: str) -> Tokenizer:
"""Load a tokenizer from a `tokenizer.json` file."""
@staticmethod
def from_str(text: str) -> Tokenizer:
"""Load a tokenizer from the contents of a `tokenizer.json` file."""
@staticmethod
def from_pretrained(
identifier: str, revision: str | None = None, token: str | None = None
) -> Tokenizer:
"""Download and cache `tokenizer.json` from the HuggingFace Hub."""
def encode(self, text: str) -> list[int]: ...
class StreamProcessor:
"""Adapt Python inference chunks to the Rust processor's input stream."""
def __init__(
self,
generator: ChatCompletionChunkGenerator | ResponsesChunkGenerator | MessagesChunkGenerator,
options: ParsingOptions,
tokenizer: Tokenizer | None = None,
) -> None:
"""Consume the generator and construct a Rust stream processor.
The optional tokenizer decodes token-id chunks. Without one, a token
chunk raises RuntimeError and closes the processor.
"""
@property
def finished(self) -> bool:
"""Whether processing completed normally and the processor remains unclosed."""
def push(
self, chunk: InferenceChunk
) -> list[ChatCompletionChunk | ResponsesStreamEvent | MessagesStreamEvent]:
"""Process one chunk; raises RuntimeError after completion, close, or decode failure."""
def finish(self) -> list[ChatCompletionChunk | ResponsesStreamEvent | MessagesStreamEvent]:
"""Process EOF; repeated calls after completion yield no chunks.
Calls after close or a decoding failure raise RuntimeError.
"""
def close(self) -> None:
"""Release processing state without producing successful completion."""

0
deepseek-recipe-python/python/deepseek_recipe/py.typed Normal file
View File

880
deepseek-recipe-python/src/conversation.rs Normal file
View File

@ -0,0 +1,880 @@
//! Python bindings for the conversation types that prompt rendering consumes.
use deepseek_recipe_core::conversation::{Conversation, ReasoningEffort, ResponseFormat};
use deepseek_recipe_core::messages::{InputMessage, ToolCall};
use deepseek_recipe_core::multimodal::{ImageDetail, ImageSource};
use deepseek_recipe_core::tools::{ToolChoice, ToolDefinition};
use deepseek_recipe_encoding::RenderedPrompt;
use pyo3::exceptions::PyValueError;
use pyo3::prelude::*;
use serde_json::Value;
use crate::json::{json_to_python, python_to_json};
/// Parse a `tool_choice` string.
fn parse_tool_choice(value: &str) -> PyResult<ToolChoice> {
match value {
"auto" => Ok(ToolChoice::Auto),
"none" => Ok(ToolChoice::None),
"required" => Ok(ToolChoice::Required),
other => Err(PyValueError::new_err(format!(
"invalid tool_choice {other:?}; expected \"auto\", \"none\", or \"required\""
))),
}
}
/// Parse a `reasoning_effort` string.
fn parse_reasoning_effort(value: &str) -> PyResult<ReasoningEffort> {
match value {
"low" => Ok(ReasoningEffort::Low),
"high" => Ok(ReasoningEffort::High),
"xhigh" => Ok(ReasoningEffort::Xhigh),
"max" => Ok(ReasoningEffort::Max),
other => Err(PyValueError::new_err(format!(
"invalid reasoning_effort {other:?}; expected \"low\", \"high\", \"xhigh\", or \"max\""
))),
}
}
/// Parse a `response_format` string.
fn parse_response_format(value: &str) -> PyResult<ResponseFormat> {
match value {
"text" => Ok(ResponseFormat::Text),
"json_object" => Ok(ResponseFormat::JsonObject),
other => Err(PyValueError::new_err(format!(
"invalid response_format {other:?}; expected \"text\" or \"json_object\""
))),
}
}
/// Parse an image `detail` string.
pub(crate) fn parse_image_detail(value: &str) -> PyResult<ImageDetail> {
match value {
"low" => Ok(ImageDetail::Low),
"high" => Ok(ImageDetail::High),
"original" => Ok(ImageDetail::Original),
"auto" => Ok(ImageDetail::Auto),
other => Err(PyValueError::new_err(format!(
"invalid detail {other:?}; expected \"low\", \"high\", \"original\", or \"auto\""
))),
}
}
/// Return the detail level string of an image source.
pub(crate) fn image_detail_str(detail: ImageDetail) -> &'static str {
match detail {
ImageDetail::Low => "low",
ImageDetail::High => "high",
ImageDetail::Original => "original",
ImageDetail::Auto => "auto",
}
}
/// Base class of the messages in a conversation.
///
/// Every message has a role and content. Fields that belong to one role are
/// `None` for the other roles.
#[pyclass(name = "Message", module = "deepseek_recipe._native", subclass, frozen)]
pub(crate) struct PyMessage {
pub(crate) inner: InputMessage,
}
#[pymethods]
impl PyMessage {
/// Role of this message: `system`, `user`, `assistant`, `tool`, or
/// `latest_reminder`.
#[getter]
fn role(&self) -> &'static str {
match &self.inner {
InputMessage::System { .. } => "system",
InputMessage::User { .. } => "user",
InputMessage::Assistant { .. } => "assistant",
InputMessage::Tool { .. } => "tool",
InputMessage::LatestReminder { .. } => "latest_reminder",
}
}
/// Message text.
#[getter]
fn content(&self) -> &str {
match &self.inner {
InputMessage::System { content }
| InputMessage::User { content, .. }
| InputMessage::Assistant { content, .. }
| InputMessage::Tool { content, .. }
| InputMessage::LatestReminder { content } => content,
}
}
/// Reasoning content of an assistant message.
#[getter]
fn reasoning_content(&self) -> Option<&str> {
match &self.inner {
InputMessage::Assistant {
reasoning_content, ..
} => reasoning_content.as_deref(),
_ => None,
}
}
/// Tool calls of an assistant message.
#[getter]
fn tool_calls(&self, py: Python<'_>) -> PyResult<Option<Vec<Py<PyToolCall>>>> {
let tool_calls = match &self.inner {
InputMessage::Assistant { tool_calls, .. } => tool_calls.as_deref(),
_ => None,
};
tool_calls
.map(|tool_calls| {
tool_calls
.iter()
.map(|tool_call| Py::new(py, PyToolCall::from_core(tool_call.clone())))
.collect()
})
.transpose()
}
/// Call ID of a tool message.
#[getter]
fn tool_call_id(&self) -> Option<&str> {
match &self.inner {
InputMessage::Tool { tool_call_id, .. } => Some(tool_call_id),
_ => None,
}
}
/// Image sources of a user or tool message.
#[getter]
fn image_sources(&self, py: Python<'_>) -> PyResult<Vec<Py<PyAny>>> {
let image_sources = match &self.inner {
InputMessage::User { image_sources, .. } | InputMessage::Tool { image_sources, .. } => {
image_sources.as_slice()
}
_ => &[],
};
image_sources
.iter()
.map(|source| image_source_to_py(py, source))
.collect()
}
fn __repr__(&self) -> String {
match &self.inner {
InputMessage::System { content } => format!("SystemMessage({content:?})"),
InputMessage::User {
content,
image_sources,
} => format!(
"UserMessage({content:?}, image_sources={})",
image_sources.len()
),
InputMessage::Assistant {
content,
reasoning_content,
tool_calls,
} => format!(
"AssistantMessage({content:?}, reasoning_content={}, tool_calls={})",
reasoning_content.is_some(),
tool_calls.as_ref().map_or(0, Vec::len),
),
InputMessage::Tool {
content,
image_sources,
tool_call_id,
} => format!(
"ToolMessage({content:?}, tool_call_id={tool_call_id:?}, image_sources={})",
image_sources.len()
),
InputMessage::LatestReminder { content } => {
format!("LatestReminderMessage({content:?})")
}
}
}
}
/// A system message.
#[pyclass(
name = "SystemMessage",
module = "deepseek_recipe._native",
extends = PyMessage,
frozen
)]
pub(crate) struct PySystemMessage;
impl PySystemMessage {
fn from_core(content: String) -> PyClassInitializer<Self> {
PyClassInitializer::from(PyMessage {
inner: InputMessage::System { content },
})
.add_subclass(Self)
}
}
#[pymethods]
impl PySystemMessage {
#[new]
fn new(content: String) -> PyClassInitializer<Self> {
Self::from_core(content)
}
}
/// A user message.
#[pyclass(
name = "UserMessage",
module = "deepseek_recipe._native",
extends = PyMessage,
frozen
)]
pub(crate) struct PyUserMessage;
impl PyUserMessage {
fn from_core(content: String, image_sources: Vec<ImageSource>) -> PyClassInitializer<Self> {
PyClassInitializer::from(PyMessage {
inner: InputMessage::User {
content,
image_sources,
},
})
.add_subclass(Self)
}
}
#[pymethods]
impl PyUserMessage {
#[new]
#[pyo3(signature = (content, image_sources=None))]
fn new(
content: String,
image_sources: Option<Vec<Py<PyImageSource>>>,
) -> PyClassInitializer<Self> {
Self::from_core(content, image_sources_from_py(image_sources))
}
}
/// An assistant message.
#[pyclass(
name = "AssistantMessage",
module = "deepseek_recipe._native",
extends = PyMessage,
frozen
)]
pub(crate) struct PyAssistantMessage;
impl PyAssistantMessage {
fn from_core(
content: String,
reasoning_content: Option<String>,
tool_calls: Option<Vec<ToolCall>>,
) -> PyClassInitializer<Self> {
PyClassInitializer::from(PyMessage {
inner: InputMessage::Assistant {
content,
reasoning_content,
tool_calls,
},
})
.add_subclass(Self)
}
}
#[pymethods]
impl PyAssistantMessage {
#[new]
#[pyo3(signature = (content, reasoning_content=None, tool_calls=None))]
fn new(
content: String,
reasoning_content: Option<String>,
tool_calls: Option<Vec<Py<PyToolCall>>>,
) -> PyClassInitializer<Self> {
let tool_calls = tool_calls.map(|tool_calls| {
tool_calls
.iter()
.map(|tool_call| tool_call.get().inner.clone())
.collect()
});
Self::from_core(content, reasoning_content, tool_calls)
}
}
/// A tool result message.
#[pyclass(
name = "ToolMessage",
module = "deepseek_recipe._native",
extends = PyMessage,
frozen
)]
pub(crate) struct PyToolMessage;
impl PyToolMessage {
fn from_core(
content: String,
tool_call_id: String,
image_sources: Vec<ImageSource>,
) -> PyClassInitializer<Self> {
PyClassInitializer::from(PyMessage {
inner: InputMessage::Tool {
content,
image_sources,
tool_call_id,
},
})
.add_subclass(Self)
}
}
#[pymethods]
impl PyToolMessage {
#[new]
#[pyo3(signature = (content, tool_call_id, image_sources=None))]
fn new(
content: String,
tool_call_id: String,
image_sources: Option<Vec<Py<PyImageSource>>>,
) -> PyClassInitializer<Self> {
Self::from_core(content, tool_call_id, image_sources_from_py(image_sources))
}
}
/// A reminder message that carries additional instructions.
#[pyclass(
name = "LatestReminderMessage",
module = "deepseek_recipe._native",
extends = PyMessage,
frozen
)]
pub(crate) struct PyLatestReminderMessage;
impl PyLatestReminderMessage {
fn from_core(content: String) -> PyClassInitializer<Self> {
PyClassInitializer::from(PyMessage {
inner: InputMessage::LatestReminder { content },
})
.add_subclass(Self)
}
}
#[pymethods]
impl PyLatestReminderMessage {
#[new]
fn new(content: String) -> PyClassInitializer<Self> {
Self::from_core(content)
}
}
/// Collect the core image sources of Python image source objects.
fn image_sources_from_py(image_sources: Option<Vec<Py<PyImageSource>>>) -> Vec<ImageSource> {
image_sources
.unwrap_or_default()
.iter()
.map(|source| source.get().inner.clone())
.collect()
}
/// A historical tool call and its arguments.
#[pyclass(name = "ToolCall", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyToolCall {
pub(crate) inner: ToolCall,
}
impl PyToolCall {
fn from_core(inner: ToolCall) -> Self {
Self { inner }
}
}
#[pymethods]
impl PyToolCall {
#[new]
fn new(id: String, name: String, arguments: String) -> Self {
Self::from_core(ToolCall {
id,
name,
arguments,
})
}
/// Call ID that a tool result refers to.
#[getter]
fn id(&self) -> &str {
&self.inner.id
}
/// Name of the called tool.
#[getter]
fn name(&self) -> &str {
&self.inner.name
}
/// Caller-supplied arguments, expected to encode a JSON object.
/// Construction does not validate the string.
#[getter]
fn arguments(&self) -> &str {
&self.inner.arguments
}
fn __repr__(&self) -> String {
format!(
"ToolCall(id={:?}, name={:?}, arguments={:?})",
self.inner.id, self.inner.name, self.inner.arguments
)
}
}
/// A client tool definition.
#[pyclass(name = "ToolDefinition", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyToolDefinition {
pub(crate) inner: ToolDefinition,
}
#[pymethods]
impl PyToolDefinition {
/// Construct a tool definition.
///
/// `parameters` is the input JSON Schema. Python `None` maps to JSON `null`.
/// `strict` requests strict argument validation and is left to the caller.
#[new]
#[pyo3(signature = (name, description=None, parameters=None, strict=None))]
fn new(
py: Python<'_>,
name: String,
description: Option<String>,
parameters: Option<Py<PyAny>>,
strict: Option<bool>,
) -> PyResult<Self> {
let parameters = match parameters {
Some(parameters) => python_to_json(parameters.bind(py))?,
None => Value::Null,
};
Ok(Self {
inner: ToolDefinition {
name,
description,
parameters,
strict,
},
})
}
/// Tool name.
#[getter]
fn name(&self) -> &str {
&self.inner.name
}
/// Human-readable purpose of the tool.
#[getter]
fn description(&self) -> Option<&str> {
self.inner.description.as_deref()
}
/// Input JSON Schema of the tool.
#[getter]
fn parameters(&self, py: Python<'_>) -> PyResult<Py<PyAny>> {
json_to_python(py, &self.inner.parameters)
}
/// Requested strict argument validation.
#[getter]
fn strict(&self) -> Option<bool> {
self.inner.strict
}
fn __repr__(&self) -> String {
format!(
"ToolDefinition(name={:?}, description={:?}, strict={:?})",
self.inner.name, self.inner.description, self.inner.strict
)
}
}
/// Base class of the images supplied with a conversation.
#[pyclass(
name = "ImageSource",
module = "deepseek_recipe._native",
subclass,
frozen
)]
pub(crate) struct PyImageSource {
pub(crate) inner: ImageSource,
}
#[pymethods]
impl PyImageSource {
/// Kind of image source: `data_url`, `url`, or `bytes`.
#[getter]
fn kind(&self) -> &'static str {
match &self.inner {
ImageSource::DataUrl { .. } => "data_url",
ImageSource::Url { .. } => "url",
ImageSource::Bytes { .. } => "bytes",
}
}
/// Requested image detail level.
#[getter]
fn detail(&self) -> &'static str {
image_detail_str(self.inner.detail())
}
/// Data URL of a data URL source, with a base64 or percent-encoded body.
#[getter]
fn data_url(&self) -> Option<&str> {
match &self.inner {
ImageSource::DataUrl { data_url, .. } => Some(data_url),
_ => None,
}
}
/// External URL of a URL source.
#[getter]
fn url(&self) -> Option<&str> {
match &self.inner {
ImageSource::Url { url, .. } => Some(url),
_ => None,
}
}
/// Encoded bytes of a bytes source.
#[getter]
fn data(&self) -> Option<&[u8]> {
match &self.inner {
ImageSource::Bytes { data, .. } => Some(data),
_ => None,
}
}
fn __repr__(&self) -> String {
match &self.inner {
ImageSource::DataUrl { data_url, detail } => format!(
"DataUrlImageSource({data_url:?}, detail={:?})",
image_detail_str(*detail)
),
ImageSource::Url { url, detail } => {
format!(
"UrlImageSource({url:?}, detail={:?})",
image_detail_str(*detail)
)
}
ImageSource::Bytes { data, detail } => format!(
"BytesImageSource(<{} bytes>, detail={:?})",
data.len(),
image_detail_str(*detail)
),
}
}
}
/// An image referenced by a data URL with a base64 or percent-encoded body.
#[pyclass(
name = "DataUrlImageSource",
module = "deepseek_recipe._native",
extends = PyImageSource,
frozen
)]
pub(crate) struct PyDataUrlImageSource;
impl PyDataUrlImageSource {
fn from_core(data_url: String, detail: ImageDetail) -> PyClassInitializer<Self> {
PyClassInitializer::from(PyImageSource {
inner: ImageSource::DataUrl { data_url, detail },
})
.add_subclass(Self)
}
}
#[pymethods]
impl PyDataUrlImageSource {
#[new]
#[pyo3(signature = (data_url, detail="high"))]
fn new(data_url: String, detail: &str) -> PyResult<PyClassInitializer<Self>> {
Ok(Self::from_core(data_url, parse_image_detail(detail)?))
}
}
/// An image referenced by an external URL.
#[pyclass(
name = "UrlImageSource",
module = "deepseek_recipe._native",
extends = PyImageSource,
frozen
)]
pub(crate) struct PyUrlImageSource;
impl PyUrlImageSource {
fn from_core(url: String, detail: ImageDetail) -> PyClassInitializer<Self> {
PyClassInitializer::from(PyImageSource {
inner: ImageSource::Url { url, detail },
})
.add_subclass(Self)
}
}
#[pymethods]
impl PyUrlImageSource {
#[new]
#[pyo3(signature = (url, detail="high"))]
fn new(url: String, detail: &str) -> PyResult<PyClassInitializer<Self>> {
Ok(Self::from_core(url, parse_image_detail(detail)?))
}
}
/// An image supplied as encoded bytes.
#[pyclass(
name = "BytesImageSource",
module = "deepseek_recipe._native",
extends = PyImageSource,
frozen
)]
pub(crate) struct PyBytesImageSource;
impl PyBytesImageSource {
fn from_core(data: Vec<u8>, detail: ImageDetail) -> PyClassInitializer<Self> {
PyClassInitializer::from(PyImageSource {
inner: ImageSource::Bytes { data, detail },
})
.add_subclass(Self)
}
}
#[pymethods]
impl PyBytesImageSource {
#[new]
#[pyo3(signature = (data, detail="high"))]
fn new(data: Vec<u8>, detail: &str) -> PyResult<PyClassInitializer<Self>> {
Ok(Self::from_core(data, parse_image_detail(detail)?))
}
}
/// Construct the Python image source object for a core image source.
fn image_source_to_py(py: Python<'_>, source: &ImageSource) -> PyResult<Py<PyAny>> {
Ok(match source {
ImageSource::DataUrl { data_url, detail } => Py::new(
py,
PyDataUrlImageSource::from_core(data_url.clone(), *detail),
)?
.into_any(),
ImageSource::Url { url, detail } => {
Py::new(py, PyUrlImageSource::from_core(url.clone(), *detail))?.into_any()
}
ImageSource::Bytes { data, detail } => {
Py::new(py, PyBytesImageSource::from_core(data.clone(), *detail))?.into_any()
}
})
}
/// Construct the Python message object for a core message.
fn message_to_py(py: Python<'_>, message: &InputMessage) -> PyResult<Py<PyAny>> {
Ok(match message {
InputMessage::System { content } => {
Py::new(py, PySystemMessage::from_core(content.clone()))?.into_any()
}
InputMessage::User {
content,
image_sources,
} => Py::new(
py,
PyUserMessage::from_core(content.clone(), image_sources.clone()),
)?
.into_any(),
InputMessage::Assistant {
content,
reasoning_content,
tool_calls,
} => Py::new(
py,
PyAssistantMessage::from_core(
content.clone(),
reasoning_content.clone(),
tool_calls.clone(),
),
)?
.into_any(),
InputMessage::Tool {
content,
image_sources,
tool_call_id,
} => Py::new(
py,
PyToolMessage::from_core(content.clone(), tool_call_id.clone(), image_sources.clone()),
)?
.into_any(),
InputMessage::LatestReminder { content } => {
Py::new(py, PyLatestReminderMessage::from_core(content.clone()))?.into_any()
}
})
}
/// A conversation and its prompt configuration.
#[pyclass(name = "Conversation", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyConversation {
pub(crate) inner: Conversation,
}
impl PyConversation {
pub(crate) fn from_core(inner: Conversation) -> Self {
Self { inner }
}
}
#[pymethods]
impl PyConversation {
/// Construct a conversation.
///
/// Thinking mode is enabled by default, tool selection defaults to `auto`,
/// and the response format defaults to `text`.
#[new]
#[pyo3(signature = (
messages=None,
*,
thinking_mode=true,
tools=None,
tool_choice="auto",
reasoning_effort=None,
response_format="text"
))]
fn new(
messages: Option<Vec<Py<PyMessage>>>,
thinking_mode: bool,
tools: Option<Vec<Py<PyToolDefinition>>>,
tool_choice: &str,
reasoning_effort: Option<&str>,
response_format: &str,
) -> PyResult<Self> {
let messages = messages
.unwrap_or_default()
.iter()
.map(|message| message.get().inner.clone())
.collect();
let tools = tools
.unwrap_or_default()
.iter()
.map(|tool| tool.get().inner.clone())
.collect();
let reasoning_effort = reasoning_effort.map(parse_reasoning_effort).transpose()?;
Ok(Self {
inner: Conversation {
messages,
thinking_mode,
tools,
tool_choice: parse_tool_choice(tool_choice)?,
reasoning_effort,
response_format: parse_response_format(response_format)?,
},
})
}
/// Messages in conversation order.
#[getter]
fn messages(&self, py: Python<'_>) -> PyResult<Vec<Py<PyAny>>> {
self.inner
.messages
.iter()
.map(|message| message_to_py(py, message))
.collect()
}
/// Whether thinking mode is enabled for the next assistant turn.
#[getter]
fn thinking_mode(&self) -> bool {
self.inner.thinking_mode
}
/// Available client tools, in declaration order.
#[getter]
fn tools(&self, py: Python<'_>) -> PyResult<Vec<Py<PyToolDefinition>>> {
self.inner
.tools
.iter()
.map(|tool| {
Py::new(
py,
PyToolDefinition {
inner: tool.clone(),
},
)
})
.collect()
}
/// Tool selection: `auto`, `none`, or `required`.
#[getter]
fn tool_choice(&self) -> &'static str {
match self.inner.tool_choice {
ToolChoice::Auto => "auto",
ToolChoice::None => "none",
ToolChoice::Required => "required",
}
}
/// Reasoning effort used when thinking mode is enabled.
#[getter]
fn reasoning_effort(&self) -> Option<&'static str> {
self.inner.reasoning_effort.map(|effort| match effort {
ReasoningEffort::Low => "low",
ReasoningEffort::High => "high",
ReasoningEffort::Xhigh => "xhigh",
ReasoningEffort::Max => "max",
})
}
/// Requested answer format: `text` or `json_object`.
#[getter]
fn response_format(&self) -> &'static str {
match self.inner.response_format {
ResponseFormat::Text => "text",
ResponseFormat::JsonObject => "json_object",
}
}
fn __repr__(&self) -> String {
format!(
"Conversation(messages={}, thinking_mode={}, tools={}, tool_choice={:?}, \
reasoning_effort={:?}, response_format={:?})",
self.inner.messages.len(),
self.inner.thinking_mode,
self.inner.tools.len(),
self.tool_choice(),
self.reasoning_effort(),
self.response_format(),
)
}
}
/// A rendered prompt and the images its placeholders refer to.
#[pyclass(name = "RenderedPrompt", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyRenderedPrompt {
inner: RenderedPrompt,
}
impl PyRenderedPrompt {
pub(crate) fn from_core(inner: RenderedPrompt) -> Self {
Self { inner }
}
}
#[pymethods]
impl PyRenderedPrompt {
/// Model input string.
#[getter]
fn prompt(&self) -> &str {
&self.inner.prompt
}
/// Image sources in the order their placeholders appear in the prompt.
#[getter]
fn image_sources(&self, py: Python<'_>) -> PyResult<Vec<Py<PyAny>>> {
self.inner
.image_sources
.iter()
.map(|source| image_source_to_py(py, source))
.collect()
}
fn __repr__(&self) -> String {
format!(
"RenderedPrompt(prompt=<{} characters>, image_sources={})",
self.inner.prompt.chars().count(),
self.inner.image_sources.len(),
)
}
}

66
deepseek-recipe-python/src/encoding.rs Normal file
View File

@ -0,0 +1,66 @@
//! Direct bindings for the Rust model-specific prompt renderers.
use std::sync::Arc;
use deepseek_recipe_encoding::PromptEncoding;
use deepseek_recipe_encoding::v4::dsv4::DeepseekV4Encoding;
use deepseek_recipe_encoding::v4::dsv41::DeepseekV41Encoding;
use pyo3::exceptions::PyRuntimeError;
use pyo3::prelude::*;
use crate::conversation::{PyConversation, PyRenderedPrompt};
use crate::tokenizer::PyTokenizer;
macro_rules! prompt_encoding {
($python:ident, $rust:ty, $name:literal) => {
/// A Rust prompt renderer. Token encoding requires an attached tokenizer.
#[pyclass(name = $name, module = "deepseek_recipe._native", frozen)]
pub(crate) struct $python {
inner: $rust,
}
#[pymethods]
impl $python {
#[new]
fn new() -> Self {
Self {
inner: <$rust>::new(),
}
}
/// Attach a tokenizer for `encode`, returning a new encoding.
fn with_tokenizer(&self, tokenizer: PyRef<'_, PyTokenizer>) -> Self {
Self {
inner: <$rust>::new().with_tokenizer(Arc::clone(&tokenizer.inner)),
}
}
/// Render a conversation through the Rust model's prompt template.
fn render_conversation(
&self,
py: Python<'_>,
conversation: &PyConversation,
) -> PyRenderedPrompt {
let rendered = py.detach(|| self.inner.render_conversation(&conversation.inner));
PyRenderedPrompt::from_core(rendered)
}
/// Encode a conversation into model token IDs using the attached tokenizer.
fn encode(&self, py: Python<'_>, conversation: &PyConversation) -> PyResult<Vec<u32>> {
py.detach(|| self.inner.encode(&conversation.inner))
.map_err(|error| PyRuntimeError::new_err(error.to_string()))
}
}
};
}
prompt_encoding!(
PyDeepseekV4Encoding,
DeepseekV4Encoding,
"DeepseekV4Encoding"
);
prompt_encoding!(
PyDeepseekV41Encoding,
DeepseekV41Encoding,
"DeepseekV41Encoding"
);

34
deepseek-recipe-python/src/error.rs Normal file
View File

@ -0,0 +1,34 @@
//! The exception raised when a protocol request cannot be converted.
use deepseek_recipe::request::ConversionError as RustConversionError;
use pyo3::exceptions::PyException;
use pyo3::prelude::*;
pyo3::create_exception!(
_native,
ConversionError,
PyException,
"Raised when a protocol request is invalid or cannot be converted."
);
/// Build the exception for a failed conversion.
///
/// The exception message is the conversion error message. Its `status_code` and
/// `body` attributes hold the HTTP status and serialized JSON error response.
pub(crate) fn request_error(py: Python<'_>, error: RustConversionError) -> PyErr {
let status_code = error.status_code();
let response = error.into_response();
let body = match serde_json::to_string(&response) {
Ok(body) => body,
Err(error) => return pyo3::exceptions::PyRuntimeError::new_err(error.to_string()),
};
let exception = ConversionError::new_err(response.error.message);
let value = exception.value(py);
if let Err(error) = value
.setattr("status_code", status_code)
.and_then(|()| value.setattr("body", body))
{
return error;
}
exception
}

350
deepseek-recipe-python/src/image.rs Normal file
View File

@ -0,0 +1,350 @@
//! Python bindings for image fetching, preprocessing, and token accounting.
use std::future::Future;
use std::sync::Arc;
use deepseek_recipe_core::multimodal::{ImageInfo, MultiModalData};
use deepseek_recipe_image::{
ImageByteBudget, ImageError as RustImageError, ImageFetcher, ImagePreprocessor, ImageQuota,
ImageResolver, OpenCvImagePreprocessor, PreprocessOptions, ReqwestImageFetcher,
};
use pyo3::exceptions::{PyException, PyRuntimeError, PyValueError};
use pyo3::prelude::*;
use pyo3::types::PyBytes;
use crate::conversation::{PyImageSource, image_detail_str, parse_image_detail};
pyo3::create_exception!(
_native,
ImageError,
PyException,
"An image fetching, decoding, or preprocessing failure."
);
pyo3::create_exception!(
_native,
CalcResizeError,
PyValueError,
"The Rust image token calculation did not converge."
);
/// Preserve the Rust error variant and retry classification in Python.
fn image_error(py: Python<'_>, error: RustImageError) -> PyErr {
let kind = match &error {
RustImageError::Unsupported(_) => "Unsupported",
RustImageError::UrlTooLong { .. } => "UrlTooLong",
RustImageError::InvalidUrl { .. } => "InvalidUrl",
RustImageError::TooManyImages { .. } => "TooManyImages",
RustImageError::ImageTooLarge { .. } => "ImageTooLarge",
RustImageError::TotalSizeTooLarge { .. } => "TotalSizeTooLarge",
RustImageError::InvalidDataUrl(_) => "InvalidDataUrl",
RustImageError::EmptyImage => "EmptyImage",
RustImageError::UnsupportedMediaType(_) => "UnsupportedMediaType",
RustImageError::ImageDimensionsTooLarge => "ImageDimensionsTooLarge",
RustImageError::Decode(_) => "Decode",
RustImageError::Resize(_) => "Resize",
RustImageError::Encode(_) => "Encode",
RustImageError::TokenBudget(_) => "TokenBudget",
RustImageError::Fetch { .. } => "Fetch",
RustImageError::FetchStatus { .. } => "FetchStatus",
RustImageError::Client(_) => "Client",
RustImageError::EncodeFailed => "EncodeFailed",
};
let exception = ImageError::new_err(error.to_string());
let value = exception.value(py);
if let Err(error) = value
.setattr("kind", kind)
.and_then(|()| value.setattr("is_retryable", error.is_retryable()))
{
return error;
}
exception
}
/// Run an asynchronous Rust image operation with the GIL released.
fn run_image_operation<T: Send>(
py: Python<'_>,
future: impl Future<Output = Result<T, RustImageError>> + Send,
) -> PyResult<T> {
let result = py
.detach(|| {
let runtime = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()?;
Ok::<_, std::io::Error>(runtime.block_on(future))
})
.map_err(|error| PyRuntimeError::new_err(error.to_string()))?;
result.map_err(|error| image_error(py, error))
}
/// Encoded image bytes and their dimensions in pixels.
#[pyclass(name = "ImageInfo", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyImageInfo {
inner: ImageInfo,
}
#[pymethods]
impl PyImageInfo {
#[new]
fn new(data: Vec<u8>, width: u32, height: u32) -> Self {
Self {
inner: ImageInfo {
data,
width,
height,
},
}
}
/// Encoded image bytes.
#[getter]
fn data(&self, py: Python<'_>) -> Py<PyBytes> {
PyBytes::new(py, &self.inner.data).unbind()
}
/// Image width in pixels.
#[getter]
fn width(&self) -> u32 {
self.inner.width
}
/// Image height in pixels.
#[getter]
fn height(&self) -> u32 {
self.inner.height
}
}
/// Images in the order their placeholders appear in a prompt.
#[pyclass(name = "MultiModalData", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyMultiModalData {
inner: MultiModalData,
}
#[pymethods]
impl PyMultiModalData {
#[new]
#[pyo3(signature = (images=None))]
fn new(images: Option<Vec<Py<PyImageInfo>>>) -> Self {
Self {
inner: MultiModalData {
images: images
.unwrap_or_default()
.iter()
.map(|image| image.get().inner.clone())
.collect(),
},
}
}
/// Images in placeholder order.
#[getter]
fn images(&self) -> Vec<PyImageInfo> {
self.inner
.images
.iter()
.map(|image| PyImageInfo {
inner: image.clone(),
})
.collect()
}
/// Return whether no image is attached.
fn is_empty(&self) -> bool {
self.inner.is_empty()
}
/// Return the number of prompt tokens beyond one per image placeholder.
fn image_token_adjustment(&self) -> PyResult<usize> {
self.inner
.image_token_adjustment()
.map_err(|error| CalcResizeError::new_err(error.to_string()))
}
}
/// Image count and encoded size accumulated across resolve calls.
/// A call records its sources after preprocessing all of them, so a failed call
/// adds nothing.
#[pyclass(name = "ImageQuota", module = "deepseek_recipe._native")]
pub(crate) struct PyImageQuota {
inner: ImageQuota,
}
#[pymethods]
impl PyImageQuota {
#[new]
fn new() -> Self {
Self {
inner: ImageQuota::new(),
}
}
/// Return the number of sources recorded by the completed calls.
fn image_count(&self) -> usize {
self.inner.image_count()
}
/// Return the encoded source bytes recorded by the completed calls.
fn byte_size(&self) -> usize {
self.inner.byte_size()
}
}
/// Options passed to one image preprocessing operation.
#[pyclass(name = "PreprocessOptions", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyPreprocessOptions {
inner: PreprocessOptions,
}
#[pymethods]
impl PyPreprocessOptions {
#[new]
fn new(
detail: &str,
max_dimension_px: u32,
low_detail_max_dimension_px: u32,
) -> PyResult<Self> {
Ok(Self {
inner: PreprocessOptions {
detail: parse_image_detail(detail)?,
max_dimension_px,
low_detail_max_dimension_px,
},
})
}
/// Requested image detail level.
#[getter]
fn detail(&self) -> &'static str {
image_detail_str(self.inner.detail)
}
/// Maximum accepted source width or height in pixels, before resizing.
#[getter]
fn max_dimension_px(&self) -> u32 {
self.inner.max_dimension_px
}
/// Maximum long side before token-budget fitting at low detail.
#[getter]
fn low_detail_max_dimension_px(&self) -> u32 {
self.inner.low_detail_max_dimension_px
}
}
/// Fetches external image URLs through the Rust HTTP client.
#[derive(Clone)]
#[pyclass(
name = "ReqwestImageFetcher",
module = "deepseek_recipe._native",
frozen
)]
pub(crate) struct PyReqwestImageFetcher {
inner: Arc<ReqwestImageFetcher>,
}
impl ImageFetcher for PyReqwestImageFetcher {
fn fetch(
&self,
url: &str,
budget: &ImageByteBudget,
) -> impl Future<Output = Result<Vec<u8>, RustImageError>> + Send {
self.inner.fetch(url, budget)
}
}
#[pymethods]
impl PyReqwestImageFetcher {
/// Construct the Rust fetcher with its default image size limit.
#[new]
fn new(py: Python<'_>) -> PyResult<Self> {
ReqwestImageFetcher::new()
.map(|inner| Self {
inner: Arc::new(inner),
})
.map_err(|error| image_error(py, error))
}
/// Construct the Rust fetcher with an explicit encoded size limit.
#[staticmethod]
fn with_max_bytes(py: Python<'_>, max_bytes: usize) -> PyResult<Self> {
ReqwestImageFetcher::with_max_bytes(max_bytes)
.map(|inner| Self {
inner: Arc::new(inner),
})
.map_err(|error| image_error(py, error))
}
/// Fetch encoded image bytes with the GIL released.
///
/// A standalone fetch applies the size limit of this fetcher to one image.
fn fetch(&self, py: Python<'_>, url: &str) -> PyResult<Py<PyBytes>> {
let max_bytes = self.inner.max_bytes();
let budget = ImageByteBudget::new(max_bytes, max_bytes, 0);
let data = run_image_operation(py, self.inner.fetch(url, &budget))?;
Ok(PyBytes::new(py, &data).unbind())
}
}
/// Preprocesses encoded images through the Rust OpenCV implementation.
#[pyclass(
name = "OpenCvImagePreprocessor",
module = "deepseek_recipe._native",
frozen
)]
pub(crate) struct PyOpenCvImagePreprocessor {
inner: OpenCvImagePreprocessor,
}
#[pymethods]
impl PyOpenCvImagePreprocessor {
#[new]
fn new() -> Self {
Self {
inner: OpenCvImagePreprocessor,
}
}
/// Preprocess encoded image bytes with the GIL released.
fn preprocess(
&self,
py: Python<'_>,
data: Vec<u8>,
options: &PyPreprocessOptions,
) -> PyResult<PyImageInfo> {
run_image_operation(py, self.inner.preprocess(data, options.inner))
.map(|inner| PyImageInfo { inner })
}
}
/// Resolves image sources using explicit fetching and preprocessing components.
#[pyclass(name = "ImageResolver", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyImageResolver {
inner: ImageResolver<PyReqwestImageFetcher, OpenCvImagePreprocessor>,
}
#[pymethods]
impl PyImageResolver {
#[new]
fn new(fetcher: &PyReqwestImageFetcher, preprocessor: &PyOpenCvImagePreprocessor) -> Self {
Self {
inner: ImageResolver::new(fetcher.clone(), preprocessor.inner),
}
}
/// Resolve image sources and update the caller-supplied request quota.
/// A failed call leaves the quota unchanged.
fn resolve(
&self,
py: Python<'_>,
sources: Vec<Py<PyImageSource>>,
mut quota: PyRefMut<'_, PyImageQuota>,
) -> PyResult<PyMultiModalData> {
let sources: Vec<_> = sources
.iter()
.map(|source| source.get().inner.clone())
.collect();
run_image_operation(py, self.inner.resolve(&sources, &mut quota.inner))
.map(|inner| PyMultiModalData { inner })
}
}

246
deepseek-recipe-python/src/inference.rs Normal file
View File

@ -0,0 +1,246 @@
//! Python representations of backend inference chunks and usage snapshots.
use deepseek_recipe::stream::{InferenceChunk, InferenceFinishReason, PromptUsage};
use pyo3::prelude::*;
/// Prompt-side token counts supplied by the inference backend.
///
/// Unspecified counts default to zero. The binding does not infer token counts
/// from generated text.
#[pyclass(name = "PromptUsage", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyPromptUsage {
inner: PromptUsage,
}
#[pymethods]
impl PyPromptUsage {
#[new]
#[pyo3(signature = (*, prompt_tokens = 0, prompt_cache_hit_tokens = 0))]
fn new(prompt_tokens: usize, prompt_cache_hit_tokens: usize) -> Self {
Self {
inner: PromptUsage {
prompt_tokens,
prompt_cache_hit_tokens,
},
}
}
/// Total input tokens, including tokens read from the prompt cache.
#[getter]
fn prompt_tokens(&self) -> usize {
self.inner.prompt_tokens
}
/// Input tokens read from the prompt cache.
#[getter]
fn prompt_cache_hit_tokens(&self) -> usize {
self.inner.prompt_cache_hit_tokens
}
}
/// The finish reason reported by the inference backend.
#[derive(Clone, Copy, PartialEq, Eq)]
#[pyclass(
name = "InferenceFinishReason",
module = "deepseek_recipe._native",
eq,
frozen
)]
pub(crate) enum PyInferenceFinishReason {
Stop,
Length,
ContentFilter,
}
impl PyInferenceFinishReason {
fn to_core(self) -> InferenceFinishReason {
match self {
Self::Stop => InferenceFinishReason::Stop,
Self::Length => InferenceFinishReason::Length,
Self::ContentFilter => InferenceFinishReason::ContentFilter,
}
}
fn from_core(reason: InferenceFinishReason) -> Self {
match reason {
InferenceFinishReason::Stop => Self::Stop,
InferenceFinishReason::Length => Self::Length,
InferenceFinishReason::ContentFilter => Self::ContentFilter,
}
}
}
/// One backend inference chunk, constructed with `ready`, `text`, `token`, or
/// `finish`.
///
/// Ready chunks carry the prompt usage snapshot; text chunks carry the tokens
/// their content accounts for. Omitted counts default to zero.
#[pyclass(name = "InferenceChunk", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyInferenceChunk {
inner: InferenceChunk,
}
impl PyInferenceChunk {
pub(crate) fn to_core(&self) -> InferenceChunk {
match &self.inner {
InferenceChunk::Ready {
system_fingerprint,
prompt_usage,
} => InferenceChunk::Ready {
system_fingerprint: system_fingerprint.clone(),
prompt_usage: *prompt_usage,
},
InferenceChunk::Text {
content,
content_tokens,
} => InferenceChunk::Text {
content: content.clone(),
content_tokens: *content_tokens,
},
InferenceChunk::Token { token_id } => InferenceChunk::Token {
token_id: *token_id,
},
InferenceChunk::Finish { finish_reason } => InferenceChunk::Finish {
finish_reason: *finish_reason,
},
}
}
}
#[pymethods]
impl PyInferenceChunk {
/// Construct the backend's initial metadata chunk.
///
/// Omitted usage defaults to zero. The system fingerprint is carried into
/// protocol responses that expose it, such as OpenAI Chat Completions.
#[staticmethod]
#[pyo3(signature = (*, prompt_usage = None, system_fingerprint = None))]
fn ready(
prompt_usage: Option<PyRef<'_, PyPromptUsage>>,
system_fingerprint: Option<String>,
) -> Self {
Self {
inner: InferenceChunk::Ready {
system_fingerprint,
prompt_usage: prompt_usage.map(|usage| usage.inner).unwrap_or_default(),
},
}
}
/// Construct one text chunk from the backend.
///
/// `content_tokens` is the number of tokens this chunk's content accounts
/// for; the stream processor accumulates it across chunks.
#[staticmethod]
#[pyo3(signature = (text, content_tokens = 0))]
fn text(text: String, content_tokens: usize) -> Self {
Self {
inner: InferenceChunk::Text {
content: text,
content_tokens,
},
}
}
/// Construct one token-id chunk, decoded by the processor's tokenizer.
#[staticmethod]
fn token(token_id: u32) -> Self {
Self {
inner: InferenceChunk::Token { token_id },
}
}
/// Construct the backend's terminal chunk.
#[staticmethod]
fn finish(finish_reason: PyRef<'_, PyInferenceFinishReason>) -> Self {
Self {
inner: InferenceChunk::Finish {
finish_reason: (*finish_reason).to_core(),
},
}
}
/// The chunk kind: `ready`, `text`, `token`, or `finish`.
#[getter]
fn kind(&self) -> &'static str {
match &self.inner {
InferenceChunk::Ready { .. } => "ready",
InferenceChunk::Text { .. } => "text",
InferenceChunk::Token { .. } => "token",
InferenceChunk::Finish { .. } => "finish",
}
}
/// The backend content, present only on text chunks.
#[getter]
fn content(&self) -> Option<&str> {
match &self.inner {
InferenceChunk::Text { content, .. } => Some(content),
InferenceChunk::Ready { .. }
| InferenceChunk::Token { .. }
| InferenceChunk::Finish { .. } => None,
}
}
/// The backend token id, present only on token chunks.
#[getter]
fn token_id(&self) -> Option<u32> {
match &self.inner {
InferenceChunk::Token { token_id } => Some(*token_id),
InferenceChunk::Ready { .. }
| InferenceChunk::Text { .. }
| InferenceChunk::Finish { .. } => None,
}
}
/// The backend finish reason, present only on finish chunks.
#[getter]
fn finish_reason(&self) -> Option<PyInferenceFinishReason> {
match &self.inner {
InferenceChunk::Finish { finish_reason, .. } => {
Some(PyInferenceFinishReason::from_core(*finish_reason))
}
InferenceChunk::Ready { .. }
| InferenceChunk::Text { .. }
| InferenceChunk::Token { .. } => None,
}
}
/// The prompt usage snapshot, present only on ready chunks.
#[getter]
fn prompt_usage(&self) -> Option<PyPromptUsage> {
match &self.inner {
InferenceChunk::Ready { prompt_usage, .. } => Some(PyPromptUsage {
inner: *prompt_usage,
}),
InferenceChunk::Text { .. }
| InferenceChunk::Token { .. }
| InferenceChunk::Finish { .. } => None,
}
}
/// The tokens this chunk's content accounts for, present only on text
/// chunks.
#[getter]
fn content_tokens(&self) -> Option<usize> {
match &self.inner {
InferenceChunk::Text { content_tokens, .. } => Some(*content_tokens),
InferenceChunk::Ready { .. }
| InferenceChunk::Token { .. }
| InferenceChunk::Finish { .. } => None,
}
}
/// The backend fingerprint, present only on ready chunks.
#[getter]
fn system_fingerprint(&self) -> Option<&str> {
match &self.inner {
InferenceChunk::Ready {
system_fingerprint, ..
} => system_fingerprint.as_deref(),
InferenceChunk::Text { .. }
| InferenceChunk::Token { .. }
| InferenceChunk::Finish { .. } => None,
}
}
}

143
deepseek-recipe-python/src/json.rs Normal file
View File

@ -0,0 +1,143 @@
//! Conversion between Python objects and JSON values.
use pyo3::exceptions::{PyOverflowError, PyRecursionError, PyTypeError, PyValueError};
use pyo3::prelude::*;
use pyo3::types::{PyBool, PyDict, PyFloat, PyInt, PyList, PyString, PyTuple};
use serde_json::{Map, Number, Value};
/// Maximum number of nested Python containers converted into one JSON value.
const MAX_JSON_DEPTH: usize = 128;
/// Convert a Python object into a JSON value.
///
/// Accepts `None`, `bool`, `int`, `float`, `str`, `list`, `tuple`, and `dict`
/// with string keys.
///
/// # Errors
///
/// Raises `TypeError` for unsupported types, `OverflowError` for an integer
/// outside the JSON number range, and `ValueError` for a non-finite float or a
/// circular reference. More than 128 nested containers raises `RecursionError`.
pub(crate) fn python_to_json(value: &Bound<'_, PyAny>) -> PyResult<Value> {
python_to_json_with_parents(value, &mut Vec::new())
}
fn python_to_json_with_parents(
value: &Bound<'_, PyAny>,
parents: &mut Vec<*mut pyo3::ffi::PyObject>,
) -> PyResult<Value> {
if value.is_none() {
return Ok(Value::Null);
}
if let Ok(boolean) = value.cast::<PyBool>() {
return Ok(Value::Bool(boolean.is_true()));
}
if let Ok(text) = value.cast::<PyString>() {
return Ok(Value::String(text.to_str()?.to_owned()));
}
if let Ok(integer) = value.cast::<PyInt>() {
if let Ok(number) = integer.extract::<i64>() {
return Ok(Value::Number(Number::from(number)));
}
if let Ok(number) = integer.extract::<u64>() {
return Ok(Value::Number(Number::from(number)));
}
return Err(PyOverflowError::new_err(
"integer is outside the JSON number range",
));
}
if let Ok(float) = value.cast::<PyFloat>() {
let number = float.value();
return Number::from_f64(number)
.map(Value::Number)
.ok_or_else(|| PyValueError::new_err(format!("{number} is not a finite JSON number")));
}
let pointer = value.as_ptr();
if parents.contains(&pointer) {
return Err(PyValueError::new_err("circular reference in JSON input"));
}
if parents.len() >= MAX_JSON_DEPTH {
return Err(PyRecursionError::new_err(format!(
"JSON input exceeds the maximum nesting depth of {MAX_JSON_DEPTH}"
)));
}
parents.push(pointer);
let result = python_container_to_json(value, parents);
parents.pop();
result
}
fn python_container_to_json(
value: &Bound<'_, PyAny>,
parents: &mut Vec<*mut pyo3::ffi::PyObject>,
) -> PyResult<Value> {
if let Ok(list) = value.cast::<PyList>() {
return Ok(Value::Array(
list.iter()
.map(|item| python_to_json_with_parents(&item, parents))
.collect::<PyResult<_>>()?,
));
}
if let Ok(tuple) = value.cast::<PyTuple>() {
return Ok(Value::Array(
tuple
.iter()
.map(|item| python_to_json_with_parents(&item, parents))
.collect::<PyResult<_>>()?,
));
}
if let Ok(dict) = value.cast::<PyDict>() {
let mut entries = Map::new();
for (key, item) in dict.iter() {
let key = key
.cast_into::<PyString>()
.map_err(|_| PyTypeError::new_err("JSON object keys must be str"))?;
entries.insert(
key.to_str()?.to_owned(),
python_to_json_with_parents(&item, parents)?,
);
}
return Ok(Value::Object(entries));
}
Err(PyTypeError::new_err(format!(
"{} is not a JSON value",
value.get_type().name()?
)))
}
/// Convert a JSON value into a Python object.
pub(crate) fn json_to_python(py: Python<'_>, value: &Value) -> PyResult<Py<PyAny>> {
Ok(match value {
Value::Null => py.None(),
Value::Bool(boolean) => boolean.into_pyobject(py)?.to_owned().into_any().unbind(),
Value::Number(number) => {
if let Some(integer) = number.as_i64() {
integer.into_pyobject(py)?.into_any().unbind()
} else if let Some(integer) = number.as_u64() {
integer.into_pyobject(py)?.into_any().unbind()
} else {
number
.as_f64()
.unwrap_or_default()
.into_pyobject(py)?
.into_any()
.unbind()
}
}
Value::String(text) => PyString::new(py, text).into_any().unbind(),
Value::Array(items) => {
let list = PyList::empty(py);
for item in items {
list.append(json_to_python(py, item)?)?;
}
list.into_any().unbind()
}
Value::Object(entries) => {
let dict = PyDict::new(py);
for (key, item) in entries {
dict.set_item(key, json_to_python(py, item)?)?;
}
dict.into_any().unbind()
}
})
}

114
deepseek-recipe-python/src/lib.rs Normal file
View File

@ -0,0 +1,114 @@
//! Python bindings for Rust request, rendering, image, and response components.
//!
//! Bindings adapt Python values and ownership to the corresponding Rust types.
use deepseek_recipe_core::multimodal::{IMAGE_SPECIAL_TOKEN, IMAGE_SPECIAL_TOKEN_ID};
use deepseek_recipe_encoding::v4::{
ASSISTANT_SP_TOKEN, BOS_TOKEN, DSML_SP_TOKEN, EOS_TOKEN, LATEST_REMINDER_SP_TOKEN,
SYSTEM_SP_TOKEN, THINKING_END_TOKEN, THINKING_START_TOKEN, USER_SP_TOKEN,
};
use pyo3::prelude::*;
use crate::conversation::{
PyAssistantMessage, PyBytesImageSource, PyConversation, PyDataUrlImageSource, PyImageSource,
PyLatestReminderMessage, PyMessage, PyRenderedPrompt, PySystemMessage, PyToolCall,
PyToolDefinition, PyToolMessage, PyUrlImageSource, PyUserMessage,
};
use crate::encoding::{PyDeepseekV4Encoding, PyDeepseekV41Encoding};
use crate::error::ConversionError;
use crate::image::{
CalcResizeError, ImageError, PyImageInfo, PyImageQuota, PyImageResolver, PyMultiModalData,
PyOpenCvImagePreprocessor, PyPreprocessOptions, PyReqwestImageFetcher,
};
use crate::inference::{PyInferenceChunk, PyInferenceFinishReason, PyPromptUsage};
use crate::parsing::{PyParsingOptions, PyReasoningStage};
use crate::request::{
PyChatCompletionRequest, PyConversationRequest, PyConversionOptions, PyInferenceOptions,
PyMessagesRequest, PyResponsesRequest, PyWebSearchBehavior,
};
use crate::response::{
PyChatCompletionChunk, PyChatCompletionChunkGenerator, PyChatCompletionResponse,
PyMessagesChunkGenerator, PyMessagesResponse, PyMessagesStreamEvent, PyResponsesChunkGenerator,
PyResponsesResponse, PyResponsesStreamEvent, PyStreamProcessor,
};
use crate::tokenizer::PyTokenizer;
mod conversation;
mod encoding;
mod error;
mod image;
mod inference;
mod json;
mod parsing;
mod request;
mod response;
mod tokenizer;
#[pymodule]
fn _native(m: &Bound<'_, PyModule>) -> PyResult<()> {
m.add("__version__", env!("CARGO_PKG_VERSION"))?;
m.add("BOS_TOKEN", BOS_TOKEN)?;
m.add("EOS_TOKEN", EOS_TOKEN)?;
m.add("SYSTEM_SP_TOKEN", SYSTEM_SP_TOKEN)?;
m.add("USER_SP_TOKEN", USER_SP_TOKEN)?;
m.add("ASSISTANT_SP_TOKEN", ASSISTANT_SP_TOKEN)?;
m.add("LATEST_REMINDER_SP_TOKEN", LATEST_REMINDER_SP_TOKEN)?;
m.add("THINKING_START_TOKEN", THINKING_START_TOKEN)?;
m.add("THINKING_END_TOKEN", THINKING_END_TOKEN)?;
m.add("DSML_SP_TOKEN", DSML_SP_TOKEN)?;
m.add("IMAGE_SPECIAL_TOKEN", IMAGE_SPECIAL_TOKEN)?;
m.add("IMAGE_SPECIAL_TOKEN_ID", IMAGE_SPECIAL_TOKEN_ID)?;
m.add("ConversionError", m.py().get_type::<ConversionError>())?;
m.add("ImageError", m.py().get_type::<ImageError>())?;
m.add("CalcResizeError", m.py().get_type::<CalcResizeError>())?;
m.add_class::<PyChatCompletionRequest>()?;
m.add_class::<PyResponsesRequest>()?;
m.add_class::<PyMessagesRequest>()?;
m.add_class::<PyConversionOptions>()?;
m.add_class::<PyWebSearchBehavior>()?;
m.add_class::<PyConversationRequest>()?;
m.add_class::<PyInferenceOptions>()?;
m.add_class::<PyDeepseekV4Encoding>()?;
m.add_class::<PyDeepseekV41Encoding>()?;
m.add_class::<PyImageInfo>()?;
m.add_class::<PyMultiModalData>()?;
m.add_class::<PyImageQuota>()?;
m.add_class::<PyPreprocessOptions>()?;
m.add_class::<PyReqwestImageFetcher>()?;
m.add_class::<PyOpenCvImagePreprocessor>()?;
m.add_class::<PyImageResolver>()?;
m.add_class::<PyInferenceChunk>()?;
m.add_class::<PyPromptUsage>()?;
m.add_class::<PyInferenceFinishReason>()?;
m.add_class::<PyReasoningStage>()?;
m.add_class::<PyParsingOptions>()?;
m.add_class::<PyChatCompletionChunkGenerator>()?;
m.add_class::<PyResponsesChunkGenerator>()?;
m.add_class::<PyMessagesChunkGenerator>()?;
m.add_class::<PyChatCompletionChunk>()?;
m.add_class::<PyResponsesStreamEvent>()?;
m.add_class::<PyMessagesStreamEvent>()?;
m.add_class::<PyChatCompletionResponse>()?;
m.add_class::<PyResponsesResponse>()?;
m.add_class::<PyMessagesResponse>()?;
m.add_class::<PyTokenizer>()?;
m.add_class::<PyStreamProcessor>()?;
m.add_class::<PyConversation>()?;
m.add_class::<PyRenderedPrompt>()?;
m.add_class::<PyMessage>()?;
m.add_class::<PySystemMessage>()?;
m.add_class::<PyUserMessage>()?;
m.add_class::<PyAssistantMessage>()?;
m.add_class::<PyToolMessage>()?;
m.add_class::<PyLatestReminderMessage>()?;
m.add_class::<PyToolCall>()?;
m.add_class::<PyToolDefinition>()?;
m.add_class::<PyImageSource>()?;
m.add_class::<PyDataUrlImageSource>()?;
m.add_class::<PyUrlImageSource>()?;
m.add_class::<PyBytesImageSource>()?;
Ok(())
}

130
deepseek-recipe-python/src/parsing.rs Normal file
View File

@ -0,0 +1,130 @@
//! Python representations of Rust stream parsing options.
use deepseek_recipe::stream::state_machine::{ParsingOptions, ReasoningStage};
use pyo3::prelude::*;
/// Position within reasoning content when parsing begins.
#[derive(Clone, Copy, PartialEq, Eq)]
#[pyclass(
name = "ReasoningStage",
module = "deepseek_recipe._native",
eq,
frozen
)]
pub(crate) enum PyReasoningStage {
Start,
Reasoning,
Content,
}
impl PyReasoningStage {
fn to_core(self) -> ReasoningStage {
match self {
Self::Start => ReasoningStage::Start,
Self::Reasoning => ReasoningStage::Reasoning,
Self::Content => ReasoningStage::Content,
}
}
fn from_core(stage: ReasoningStage) -> Self {
match stage {
ReasoningStage::Start => Self::Start,
ReasoningStage::Reasoning => Self::Reasoning,
ReasoningStage::Content => Self::Content,
}
}
}
#[pymethods]
impl PyReasoningStage {
/// Whether the initial stage contains reasoning content.
fn start_from_reasoning(&self) -> bool {
self.to_core().start_from_reasoning()
}
}
/// Settings passed directly to the Rust stream parser.
#[pyclass(name = "ParsingOptions", module = "deepseek_recipe._native")]
pub(crate) struct PyParsingOptions {
pub(crate) inner: ParsingOptions,
}
impl PyParsingOptions {
pub(crate) fn from_core(inner: ParsingOptions) -> Self {
Self { inner }
}
}
#[pymethods]
impl PyParsingOptions {
#[new]
#[pyo3(signature = (*, parse_tool_calls = true, tool_call_initial_stage = false,
parse_json_output = false, reasoning_initial_stage = None, stop_sequences = None))]
fn new(
parse_tool_calls: bool,
tool_call_initial_stage: bool,
parse_json_output: bool,
reasoning_initial_stage: Option<PyRef<'_, PyReasoningStage>>,
stop_sequences: Option<Vec<String>>,
) -> Self {
Self::from_core(ParsingOptions {
parse_tool_calls,
tool_call_initial_stage,
parse_json_output,
reasoning_initial_stage: reasoning_initial_stage.map(|stage| (*stage).to_core()),
stop_sequences: stop_sequences.unwrap_or_default(),
})
}
#[getter]
fn parse_tool_calls(&self) -> bool {
self.inner.parse_tool_calls
}
#[setter]
fn set_parse_tool_calls(&mut self, value: bool) {
self.inner.parse_tool_calls = value;
}
#[getter]
fn tool_call_initial_stage(&self) -> bool {
self.inner.tool_call_initial_stage
}
#[setter]
fn set_tool_call_initial_stage(&mut self, value: bool) {
self.inner.tool_call_initial_stage = value;
}
#[getter]
fn parse_json_output(&self) -> bool {
self.inner.parse_json_output
}
#[setter]
fn set_parse_json_output(&mut self, value: bool) {
self.inner.parse_json_output = value;
}
#[getter]
fn reasoning_initial_stage(&self) -> Option<PyReasoningStage> {
self.inner
.reasoning_initial_stage
.map(PyReasoningStage::from_core)
}
#[setter]
fn set_reasoning_initial_stage(&mut self, value: Option<PyRef<'_, PyReasoningStage>>) {
self.inner.reasoning_initial_stage = value.map(|stage| (*stage).to_core());
}
#[getter]
fn stop_sequences(&self) -> Vec<String> {
self.inner.stop_sequences.clone()
}
#[setter]
fn set_stop_sequences(&mut self, value: Vec<String>) {
self.inner.stop_sequences = value;
}
}

296
deepseek-recipe-python/src/request.rs Normal file
View File

@ -0,0 +1,296 @@
//! Python representations of Rust protocol requests and conversion results.
use std::collections::HashSet;
use deepseek_recipe::anthropic::MessagesRequest;
use deepseek_recipe::openai::{ChatCompletionRequest, ResponsesRequest};
use deepseek_recipe::request::{
ConversationRequest, ConversionOptions, InferenceOptions, ProtocolRequest, WebSearchBehavior,
};
use pyo3::exceptions::{PyRuntimeError, PyValueError};
use pyo3::prelude::*;
use pyo3::types::{PyBytes, PyString};
use serde::de::DeserializeOwned;
use serde_json::Value;
use crate::conversation::PyConversation;
use crate::error::request_error;
use crate::json::python_to_json;
use crate::parsing::PyParsingOptions;
use crate::response::{
PyChatCompletionChunkGenerator, PyMessagesChunkGenerator, PyResponsesChunkGenerator,
};
/// Rust conversion behavior for supported web-search declarations and content.
#[derive(Clone, Copy, PartialEq, Eq)]
#[pyclass(
name = "WebSearchBehavior",
module = "deepseek_recipe._native",
eq,
frozen
)]
pub(crate) enum PyWebSearchBehavior {
Ignore,
Reject,
}
impl PyWebSearchBehavior {
fn to_core(self) -> WebSearchBehavior {
match self {
Self::Ignore => WebSearchBehavior::Ignore,
Self::Reject => WebSearchBehavior::Reject,
}
}
fn from_core(behavior: WebSearchBehavior) -> PyResult<Self> {
match behavior {
WebSearchBehavior::Ignore => Ok(Self::Ignore),
WebSearchBehavior::Reject => Ok(Self::Reject),
other => Err(PyValueError::new_err(format!(
"unsupported web search behavior: {other:?}"
))),
}
}
}
/// Defaults passed to Rust protocol request conversion.
#[pyclass(name = "ConversionOptions", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyConversionOptions {
inner: ConversionOptions,
}
#[pymethods]
impl PyConversionOptions {
/// Construct conversion defaults.
///
/// Every argument defaults to the Rust default: thinking enabled, Responses
/// `web_search` ignored, and Messages `web_search` rejected.
#[new]
#[pyo3(signature = (
*,
default_thinking_mode=true,
responses_web_search=PyWebSearchBehavior::Ignore,
messages_web_search=PyWebSearchBehavior::Reject
))]
fn new(
default_thinking_mode: bool,
responses_web_search: PyWebSearchBehavior,
messages_web_search: PyWebSearchBehavior,
) -> Self {
Self {
inner: ConversionOptions::new()
.with_default_thinking_mode(default_thinking_mode)
.with_responses_web_search(responses_web_search.to_core())
.with_messages_web_search(messages_web_search.to_core()),
}
}
#[getter]
fn default_thinking_mode(&self) -> bool {
self.inner.default_thinking_mode
}
fn with_default_thinking_mode(&self, default_thinking_mode: bool) -> Self {
Self {
inner: self.inner.with_default_thinking_mode(default_thinking_mode),
}
}
/// Responses web-search behavior, defaulting to `Ignore`.
#[getter]
fn responses_web_search(&self) -> PyResult<PyWebSearchBehavior> {
PyWebSearchBehavior::from_core(self.inner.responses_web_search)
}
/// Messages web-search behavior, defaulting to `Reject`.
#[getter]
fn messages_web_search(&self) -> PyResult<PyWebSearchBehavior> {
PyWebSearchBehavior::from_core(self.inner.messages_web_search)
}
/// Construct options with the supplied Responses web-search behavior.
fn with_responses_web_search(&self, responses_web_search: PyWebSearchBehavior) -> Self {
Self {
inner: self
.inner
.with_responses_web_search(responses_web_search.to_core()),
}
}
/// Construct options with the supplied Messages web-search behavior.
fn with_messages_web_search(&self, messages_web_search: PyWebSearchBehavior) -> Self {
Self {
inner: self
.inner
.with_messages_web_search(messages_web_search.to_core()),
}
}
}
/// Inference parameters extracted by Rust protocol conversion.
#[pyclass(
name = "InferenceOptions",
module = "deepseek_recipe._native",
frozen,
get_all
)]
pub(crate) struct PyInferenceOptions {
max_tokens: Option<u32>,
temperature: Option<f32>,
top_p: Option<f32>,
thinking_budget_tokens: Option<u64>,
disable_parallel_tool_use: Option<bool>,
}
impl PyInferenceOptions {
fn from_core(options: &InferenceOptions) -> Self {
Self {
max_tokens: options.max_tokens,
temperature: options.temperature,
top_p: options.top_p,
thinking_budget_tokens: options.thinking_budget_tokens,
disable_parallel_tool_use: options.disable_parallel_tool_use,
}
}
}
/// A Rust conversation request with inference and parsing options.
#[pyclass(
name = "ConversationRequest",
module = "deepseek_recipe._native",
frozen
)]
pub(crate) struct PyConversationRequest {
pub(crate) inner: ConversationRequest,
}
#[pymethods]
impl PyConversationRequest {
#[new]
fn new(conversation: &PyConversation) -> Self {
Self {
inner: ConversationRequest::new(conversation.inner.clone()),
}
}
#[getter]
fn conversation(&self) -> PyConversation {
PyConversation::from_core(self.inner.conversation.clone())
}
#[getter]
fn inference_options(&self) -> PyInferenceOptions {
PyInferenceOptions::from_core(&self.inner.inference_options)
}
#[getter]
fn parsing_options(&self) -> PyParsingOptions {
PyParsingOptions::from_core(self.inner.parsing_options.clone())
}
#[getter]
fn model(&self) -> Option<&str> {
self.inner.model.as_deref()
}
#[getter]
fn stream(&self) -> bool {
self.inner.stream
}
}
macro_rules! protocol_request {
($python:ident, $rust:ty, $name:literal, $generator:ty, $($extra:item)*) => {
/// A Rust protocol request deserialized from a Python dictionary or JSON.
///
/// Conversion consumes the request even if conversion fails. Keep caller
/// metadata from the original body; `include_usage` is exposed separately
/// for Chat Completions and must be read before calling `convert`.
#[pyclass(name = $name, module = "deepseek_recipe._native")]
pub(crate) struct $python {
inner: Option<$rust>,
}
#[pymethods]
impl $python {
#[new]
fn new(body: &Bound<'_, PyAny>) -> PyResult<Self> {
Ok(Self {
inner: Some(deserialize(body)?),
})
}
fn convert(
&mut self,
py: Python<'_>,
options: &PyConversionOptions,
) -> PyResult<PyConversationRequest> {
let request = self.inner.take().ok_or_else(|| {
PyRuntimeError::new_err("protocol request has been consumed")
})?;
let options = options.inner;
let inner = py
.detach(|| request.convert(options))
.map_err(|error| request_error(py, error))?;
Ok(PyConversationRequest { inner })
}
#[staticmethod]
fn chunk_generator(
request: &PyConversationRequest,
id: String,
model: String,
) -> $generator {
<$generator>::from_core(<$rust>::chunk_generator(&request.inner, id, model))
}
$($extra)*
}
};
}
protocol_request!(
PyChatCompletionRequest,
ChatCompletionRequest,
"ChatCompletionRequest",
PyChatCompletionChunkGenerator,
fn include_usage(&self) -> PyResult<bool> {
self.inner
.as_ref()
.map(ChatCompletionRequest::include_usage)
.ok_or_else(|| PyRuntimeError::new_err("protocol request has been consumed"))
}
);
protocol_request!(
PyResponsesRequest,
ResponsesRequest,
"ResponsesRequest",
PyResponsesChunkGenerator,
fn custom_tool_names(&self) -> PyResult<HashSet<String>> {
self.inner
.as_ref()
.map(ResponsesRequest::custom_tool_names)
.ok_or_else(|| PyRuntimeError::new_err("protocol request has been consumed"))
}
);
protocol_request!(
PyMessagesRequest,
MessagesRequest,
"MessagesRequest",
PyMessagesChunkGenerator,
);
/// Deserialize the Rust request using Python-to-JSON representation conversion.
fn deserialize<T: DeserializeOwned>(body: &Bound<'_, PyAny>) -> PyResult<T> {
let value: Value = if let Ok(text) = body.cast::<PyString>() {
serde_json::from_str(text.to_str()?)
.map_err(|error| PyValueError::new_err(format!("invalid JSON request: {error}")))?
} else if let Ok(bytes) = body.cast::<PyBytes>() {
serde_json::from_slice(bytes.as_bytes())
.map_err(|error| PyValueError::new_err(format!("invalid JSON request: {error}")))?
} else {
python_to_json(body)?
};
serde_json::from_value(value)
.map_err(|error| PyValueError::new_err(format!("invalid request body: {error}")))
}

480
deepseek-recipe-python/src/response.rs Normal file
View File

@ -0,0 +1,480 @@
//! Python bindings for Rust response generators, accumulation, and streams.
use std::collections::HashSet;
use std::pin::Pin;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::task::Poll;
use deepseek_recipe::anthropic::messages::response::{
MessagesChunkGenerator, MessagesResponse, MessagesStreamEvent,
};
use deepseek_recipe::openai::chat_completion::response::{
ChatCompletionChunk, ChatCompletionChunkGenerator, ChatCompletionResponse,
};
use deepseek_recipe::openai::responses::response::{
ResponsesChunkGenerator, ResponsesResponse, ResponsesStreamEvent,
};
use deepseek_recipe::response::ProtocolResponse;
use deepseek_recipe::stream::state_machine::ParsingOptions;
use deepseek_recipe::stream::{ChunkGenerator, InferenceChunk, StreamError, StreamProcessor};
use deepseek_recipe::util::append_delta::AppendDelta;
use futures::Stream;
use futures::channel::mpsc;
use futures::executor::block_on;
use futures::future::poll_fn;
use pyo3::exceptions::{PyRuntimeError, PyTypeError};
use pyo3::prelude::*;
use serde::Serialize;
use tokenizers::Tokenizer;
use crate::inference::PyInferenceChunk;
use crate::parsing::PyParsingOptions;
use crate::tokenizer::PyTokenizer;
/// A Rust Chat Completions generator consumed by a stream processor.
#[pyclass(
name = "ChatCompletionChunkGenerator",
module = "deepseek_recipe._native"
)]
pub(crate) struct PyChatCompletionChunkGenerator {
inner: Option<ChatCompletionChunkGenerator>,
}
impl PyChatCompletionChunkGenerator {
pub(crate) fn from_core(inner: ChatCompletionChunkGenerator) -> Self {
Self { inner: Some(inner) }
}
}
#[pymethods]
impl PyChatCompletionChunkGenerator {
#[new]
fn new(id: String, model: String, include_usage: bool, thinking_mode: bool) -> Self {
Self::from_core(ChatCompletionChunkGenerator::new(
id,
model,
include_usage,
thinking_mode,
))
}
/// Consume this generator and apply the Rust usage setting.
fn with_include_usage(&mut self, include_usage: bool) -> PyResult<Self> {
Ok(Self::from_core(
take_generator(&mut self.inner)?.with_include_usage(include_usage),
))
}
}
/// A Rust Responses generator consumed by a stream processor.
#[pyclass(name = "ResponsesChunkGenerator", module = "deepseek_recipe._native")]
pub(crate) struct PyResponsesChunkGenerator {
inner: Option<ResponsesChunkGenerator>,
}
impl PyResponsesChunkGenerator {
pub(crate) fn from_core(inner: ResponsesChunkGenerator) -> Self {
Self { inner: Some(inner) }
}
}
#[pymethods]
impl PyResponsesChunkGenerator {
#[new]
fn new(id: String, model: String) -> Self {
Self::from_core(ResponsesChunkGenerator::new(id, model))
}
/// Consume this generator and configure the custom tool names.
fn with_custom_tool_names(&mut self, custom_tool_names: HashSet<String>) -> PyResult<Self> {
Ok(Self::from_core(
take_generator(&mut self.inner)?.with_custom_tool_names(custom_tool_names),
))
}
}
/// A Rust Messages generator consumed by a stream processor.
#[pyclass(name = "MessagesChunkGenerator", module = "deepseek_recipe._native")]
pub(crate) struct PyMessagesChunkGenerator {
inner: Option<MessagesChunkGenerator>,
}
impl PyMessagesChunkGenerator {
pub(crate) fn from_core(inner: MessagesChunkGenerator) -> Self {
Self { inner: Some(inner) }
}
}
#[pymethods]
impl PyMessagesChunkGenerator {
#[new]
fn new(id: String, model: String, thinking_mode: bool) -> Self {
Self::from_core(MessagesChunkGenerator::new(id, model, thinking_mode))
}
/// Consume this generator and apply the Rust thinking signature setting.
fn with_signature(&mut self, signature: String) -> PyResult<Self> {
Ok(Self::from_core(
take_generator(&mut self.inner)?.with_signature(signature),
))
}
}
fn take_generator<G>(generator: &mut Option<G>) -> PyResult<G> {
generator
.take()
.ok_or_else(|| PyRuntimeError::new_err("chunk generator has been consumed"))
}
fn to_json<T: Serialize>(value: &T) -> PyResult<String> {
serde_json::to_string(value)
.map_err(|error| PyRuntimeError::new_err(format!("failed to serialize response: {error}")))
}
// Each wrapper delegates serialization and accumulation to its Rust type.
macro_rules! response_bindings {
(
$py_chunk:ident, $chunk:ident, $chunk_name:literal,
$py_response:ident, $response:ident, $response_name:literal
) => {
/// A protocol chunk consumed by response accumulation.
#[pyclass(name = $chunk_name, module = "deepseek_recipe._native")]
pub(crate) struct $py_chunk {
inner: Option<$chunk>,
}
#[pymethods]
impl $py_chunk {
/// Serialize this chunk with the Rust protocol representation.
fn to_json(&self) -> PyResult<String> {
to_json(self.as_core()?)
}
}
impl $py_chunk {
fn as_core(&self) -> PyResult<&$chunk> {
self.inner
.as_ref()
.ok_or_else(|| PyRuntimeError::new_err("protocol chunk has been consumed"))
}
}
/// A Rust protocol response with explicit accumulation.
#[pyclass(name = $response_name, module = "deepseek_recipe._native")]
pub(crate) struct $py_response {
inner: $response,
}
#[pymethods]
impl $py_response {
#[new]
fn new(
id: String,
model: String,
created: u64,
prompt_tokens: usize,
prompt_cache_hit_tokens: usize,
) -> Self {
Self {
inner: $response::new(
id,
model,
created,
prompt_tokens,
prompt_cache_hit_tokens,
),
}
}
/// Consume a chunk and delegate to Rust `AppendDelta::append`.
fn append(&mut self, chunk: &mut $py_chunk) -> PyResult<()> {
let chunk = chunk
.inner
.take()
.ok_or_else(|| PyRuntimeError::new_err("protocol chunk has been consumed"))?;
self.inner.append(chunk);
Ok(())
}
/// Serialize the current Rust response, including partial state.
fn to_json(&self) -> PyResult<String> {
to_json(&self.inner)
}
/// Return the event name supplied by the Rust protocol response.
#[staticmethod]
fn chunk_event_type(chunk: &$py_chunk) -> PyResult<Option<&'static str>> {
Ok($response::chunk_event_type(chunk.as_core()?))
}
/// Return the transport sentinel supplied by the Rust response.
#[staticmethod]
fn done_message() -> Option<String> {
$response::done_message()
}
}
};
}
response_bindings!(
PyChatCompletionChunk,
ChatCompletionChunk,
"ChatCompletionChunk",
PyChatCompletionResponse,
ChatCompletionResponse,
"ChatCompletionResponse"
);
response_bindings!(
PyResponsesStreamEvent,
ResponsesStreamEvent,
"ResponsesStreamEvent",
PyResponsesResponse,
ResponsesResponse,
"ResponsesResponse"
);
response_bindings!(
PyMessagesStreamEvent,
MessagesStreamEvent,
"MessagesStreamEvent",
PyMessagesResponse,
MessagesResponse,
"MessagesResponse"
);
enum ProtocolChunk {
ChatCompletion(ChatCompletionChunk),
Responses(ResponsesStreamEvent),
Messages(MessagesStreamEvent),
}
impl ProtocolChunk {
fn into_python(self, py: Python<'_>) -> PyResult<Py<PyAny>> {
match self {
Self::ChatCompletion(chunk) => {
Ok(Py::new(py, PyChatCompletionChunk { inner: Some(chunk) })?.into_any())
}
Self::Responses(chunk) => {
Ok(Py::new(py, PyResponsesStreamEvent { inner: Some(chunk) })?.into_any())
}
Self::Messages(chunk) => {
Ok(Py::new(py, PyMessagesStreamEvent { inner: Some(chunk) })?.into_any())
}
}
}
}
type ChunkStream = Pin<Box<dyn Stream<Item = Result<ProtocolChunk, StreamError>> + Send>>;
struct Processor {
input: Option<mpsc::Sender<InferenceChunk>>,
events: Option<ChunkStream>,
input_pending: Arc<AtomicBool>,
finished: bool,
closed: bool,
}
/// Adapt Python input to the Rust processor's inference stream.
///
/// Construction consumes the generator. Each push produces the protocol chunks
/// available before the processor needs another input. Finish closes the input
/// normally; close releases state without generating successful completion.
#[pyclass(name = "StreamProcessor", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyStreamProcessor {
state: Mutex<Processor>,
}
impl PyStreamProcessor {
fn from_generator<G>(
generator: G,
options: ParsingOptions,
tokenizer: Option<Arc<Tokenizer>>,
wrap: fn(G::Chunk) -> ProtocolChunk,
) -> Self
where
G: ChunkGenerator,
G::Chunk: Send,
{
let (sender, mut receiver) = mpsc::channel(1);
let input_pending = Arc::new(AtomicBool::new(false));
let receiver_pending = Arc::clone(&input_pending);
let input = futures::stream::poll_fn(move |cx| {
let result = Pin::new(&mut receiver).poll_next(cx);
if result.is_pending() {
receiver_pending.store(true, Ordering::Relaxed);
}
result
});
let processor = StreamProcessor::new(generator, options);
let processor = match tokenizer {
Some(tokenizer) => processor.with_tokenizer(tokenizer),
None => processor,
};
let events =
futures::StreamExt::map(processor.process(input), move |event| event.map(wrap));
Self {
state: Mutex::new(Processor {
input: Some(sender),
events: Some(Box::pin(events)),
input_pending,
finished: false,
closed: false,
}),
}
}
fn with_state<T: Send>(
&self,
py: Python<'_>,
operation: impl FnOnce(&mut Processor) -> PyResult<T> + Send,
) -> PyResult<T> {
py.detach(|| {
let mut state = self
.state
.lock()
.map_err(|_| PyRuntimeError::new_err("stream processor lock poisoned"))?;
operation(&mut state)
})
}
}
#[pymethods]
impl PyStreamProcessor {
#[new]
#[pyo3(signature = (generator, options, tokenizer = None))]
fn new(
generator: &Bound<'_, PyAny>,
options: &PyParsingOptions,
tokenizer: Option<PyRef<'_, PyTokenizer>>,
) -> PyResult<Self> {
let tokenizer = tokenizer.map(|tokenizer| tokenizer.inner.clone());
if let Ok(mut generator) =
generator.extract::<PyRefMut<'_, PyChatCompletionChunkGenerator>>()
{
Ok(Self::from_generator(
take_generator(&mut generator.inner)?,
options.inner.clone(),
tokenizer,
ProtocolChunk::ChatCompletion,
))
} else if let Ok(mut generator) =
generator.extract::<PyRefMut<'_, PyResponsesChunkGenerator>>()
{
Ok(Self::from_generator(
take_generator(&mut generator.inner)?,
options.inner.clone(),
tokenizer,
ProtocolChunk::Responses,
))
} else if let Ok(mut generator) =
generator.extract::<PyRefMut<'_, PyMessagesChunkGenerator>>()
{
Ok(Self::from_generator(
take_generator(&mut generator.inner)?,
options.inner.clone(),
tokenizer,
ProtocolChunk::Messages,
))
} else {
Err(PyTypeError::new_err("expected a protocol chunk generator"))
}
}
/// Process one inference chunk until Rust needs another input.
/// Raises `RuntimeError` after completion or close, or on a decoding failure.
fn push(&self, py: Python<'_>, chunk: &PyInferenceChunk) -> PyResult<Vec<Py<PyAny>>> {
let chunk = chunk.to_core();
self.with_state(py, |state| state.push(chunk))?
.into_iter()
.map(|chunk| chunk.into_python(py))
.collect()
}
/// Close the input normally and process its buffered output.
/// Repeated calls after completion produce no chunks. Calls after close or
/// a decoding failure raise `RuntimeError`.
fn finish(&self, py: Python<'_>) -> PyResult<Vec<Py<PyAny>>> {
self.with_state(py, Processor::finish)?
.into_iter()
.map(|chunk| chunk.into_python(py))
.collect()
}
/// Whether the Rust stream reached normal completion and has not been closed.
#[getter]
fn finished(&self, py: Python<'_>) -> PyResult<bool> {
self.with_state(py, |state| Ok(state.finished && !state.closed))
}
/// Release state without producing normal completion chunks.
fn close(&self, py: Python<'_>) -> PyResult<()> {
self.with_state(py, |state| {
state.close();
Ok(())
})
}
}
impl Processor {
fn drain(&mut self) -> PyResult<Vec<ProtocolChunk>> {
let mut chunks = Vec::new();
block_on(poll_fn(|cx| {
loop {
// Only a pending input read ends this batch. Other pending
// futures retain their wakeups and continue on this executor.
self.input_pending.store(false, Ordering::Relaxed);
let event = self.events.as_mut().unwrap().as_mut().poll_next(cx);
match event {
Poll::Ready(Some(Ok(chunk))) => chunks.push(chunk),
Poll::Ready(Some(Err(error))) => {
self.close();
return Poll::Ready(Err(PyRuntimeError::new_err(error.to_string())));
}
Poll::Ready(None) => {
self.finished = true;
self.input = None;
self.events = None;
return Poll::Ready(Ok(std::mem::take(&mut chunks)));
}
Poll::Pending if self.input_pending.load(Ordering::Relaxed) => {
return Poll::Ready(Ok(std::mem::take(&mut chunks)));
}
Poll::Pending => return Poll::Pending,
}
}
}))
}
fn push(&mut self, chunk: InferenceChunk) -> PyResult<Vec<ProtocolChunk>> {
if self.closed {
return Err(PyRuntimeError::new_err("stream processor is closed"));
}
if self.finished {
return Err(PyRuntimeError::new_err("stream processor has finished"));
}
if let Err(error) = self.input.as_mut().unwrap().try_send(chunk) {
self.close();
return Err(PyRuntimeError::new_err(format!(
"failed to submit inference chunk: {error}"
)));
}
self.drain()
}
fn finish(&mut self) -> PyResult<Vec<ProtocolChunk>> {
if self.closed {
return Err(PyRuntimeError::new_err("stream processor is closed"));
}
if self.finished {
return Ok(Vec::new());
}
self.input = None;
self.drain()
}
fn close(&mut self) {
self.closed = true;
self.input = None;
self.events = None;
}
}

69
deepseek-recipe-python/src/tokenizer.rs Normal file
View File

@ -0,0 +1,69 @@
//! Bindings for tokenizers used to decode token-id inference chunks.
use std::sync::Arc;
use pyo3::exceptions::PyRuntimeError;
use pyo3::prelude::*;
use tokenizers::{FromPretrainedParameters, Tokenizer};
/// A HuggingFace tokenizer loaded from a `tokenizer.json` file.
///
/// A stream processor uses it to decode token-id inference chunks.
#[pyclass(name = "Tokenizer", module = "deepseek_recipe._native", frozen)]
pub(crate) struct PyTokenizer {
pub(crate) inner: Arc<Tokenizer>,
}
impl PyTokenizer {
fn from_tokenizer(inner: Tokenizer) -> Self {
Self {
inner: Arc::new(inner),
}
}
}
#[pymethods]
impl PyTokenizer {
/// Load a tokenizer from a `tokenizer.json` file.
#[staticmethod]
fn from_file(path: &str) -> PyResult<Self> {
Tokenizer::from_file(path)
.map(Self::from_tokenizer)
.map_err(|error| PyRuntimeError::new_err(error.to_string()))
}
/// Load a tokenizer from the contents of a `tokenizer.json` file.
#[staticmethod]
fn from_str(text: &str) -> PyResult<Self> {
Tokenizer::from_bytes(text)
.map(Self::from_tokenizer)
.map_err(|error| PyRuntimeError::new_err(error.to_string()))
}
/// Download and cache `tokenizer.json` from the HuggingFace Hub.
#[staticmethod]
#[pyo3(signature = (identifier, revision = None, token = None))]
#[allow(deprecated)]
fn from_pretrained(
py: Python<'_>,
identifier: &str,
revision: Option<String>,
token: Option<String>,
) -> PyResult<Self> {
let mut params = FromPretrainedParameters::default();
if let Some(revision) = revision {
params.revision = revision;
}
params.token = token;
py.detach(|| Tokenizer::from_pretrained(identifier, Some(params)))
.map(Self::from_tokenizer)
.map_err(|error| PyRuntimeError::new_err(error.to_string()))
}
/// Encode text into token ids without added special tokens.
fn encode(&self, py: Python<'_>, text: &str) -> PyResult<Vec<u32>> {
py.detach(|| self.inner.encode(text, false))
.map(|encoding| encoding.get_ids().to_vec())
.map_err(|error| PyRuntimeError::new_err(error.to_string()))
}
}

335
deepseek-recipe-python/tests/test_bindings.py Normal file
View File

@ -0,0 +1,335 @@
"""Smoke tests for the Rust conversion, rendering, response, and image bindings."""
import json
from pathlib import Path
import pytest
from deepseek_recipe import (
IMAGE_SPECIAL_TOKEN,
THINKING_END_TOKEN,
ChatCompletionRequest,
ChatCompletionResponse,
Conversation,
ConversionError,
ConversionOptions,
DeepseekV41Encoding,
ImageQuota,
ImageResolver,
InferenceChunk,
InferenceFinishReason,
MessagesRequest,
OpenCvImagePreprocessor,
PromptUsage,
ReqwestImageFetcher,
ResponsesRequest,
StreamProcessor,
Tokenizer,
ToolDefinition,
UserMessage,
WebSearchBehavior,
)
REQUEST = {
"model": "smoke-model",
"messages": [{"role": "user", "content": "hi"}],
}
TOKENIZER_PATH = (
Path(__file__).resolve().parents[2] / "static" / "tokenizers" / "v41" / "tokenizer.json"
)
def test_rendering():
converted = ChatCompletionRequest(REQUEST).convert(ConversionOptions())
encoding = DeepseekV41Encoding()
rendered = encoding.render_conversation(Conversation(messages=[UserMessage("hi")]))
assert converted.conversation.messages[0].content == "hi"
assert "hi" in rendered.prompt
assert rendered.image_sources == []
assert encoding.render_conversation(converted.conversation).prompt == rendered.prompt
def test_encoding_with_tokenizer():
converted = ChatCompletionRequest(REQUEST).convert(ConversionOptions())
tokenizer = Tokenizer.from_file(str(TOKENIZER_PATH))
with pytest.raises(RuntimeError, match="no tokenizer"):
DeepseekV41Encoding().encode(converted.conversation)
encoding = DeepseekV41Encoding().with_tokenizer(tokenizer)
prompt_ids = encoding.encode(converted.conversation)
assert prompt_ids == tokenizer.encode(encoding.render_conversation(converted.conversation).prompt)
def test_web_search_conversion_options():
defaults = ConversionOptions()
constructed = ConversionOptions(
default_thinking_mode=False,
responses_web_search=WebSearchBehavior.Reject,
messages_web_search=WebSearchBehavior.Ignore,
)
configured = (
defaults.with_default_thinking_mode(False)
.with_responses_web_search(WebSearchBehavior.Reject)
.with_messages_web_search(WebSearchBehavior.Ignore)
)
assert defaults.default_thinking_mode
assert defaults.responses_web_search == WebSearchBehavior.Ignore
assert defaults.messages_web_search == WebSearchBehavior.Reject
for options in (constructed, configured):
assert not options.default_thinking_mode
assert options.responses_web_search == WebSearchBehavior.Reject
assert options.messages_web_search == WebSearchBehavior.Ignore
responses = {
"model": "smoke-model",
"input": "hi",
"tools": [{"type": "web_search"}],
}
messages = {
**REQUEST,
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
}
assert ResponsesRequest(responses).convert(defaults).conversation.tools == []
with pytest.raises(ConversionError, match="server tools are not supported"):
MessagesRequest(messages).convert(defaults)
converted = MessagesRequest(messages).convert(configured)
assert converted.conversation.tools == []
assert not converted.conversation.thinking_mode
with pytest.raises(ConversionError, match="Server tools are not supported"):
ResponsesRequest(responses).convert(configured)
def test_custom_inference():
converted = ChatCompletionRequest({**REQUEST, "temperature": 0.25}).convert(
ConversionOptions()
)
generator = ChatCompletionRequest.chunk_generator(converted, "smoke-response", "smoke-model")
processor = StreamProcessor(generator, converted.parsing_options)
response = ChatCompletionResponse("smoke-response", "smoke-model", 0, 0, 0)
try:
assert converted.inference_options.temperature == 0.25
for chunk in (
InferenceChunk.ready(prompt_usage=PromptUsage(prompt_tokens=4)),
InferenceChunk.text(
THINKING_END_TOKEN + "Custom answer", content_tokens=2
),
InferenceChunk.finish(InferenceFinishReason.Stop),
):
for output in processor.push(chunk):
response.append(output)
for output in processor.finish():
response.append(output)
body = json.loads(response.to_json())
assert processor.finished
assert body["id"] == "smoke-response"
assert body["choices"][0]["message"]["content"] == "Custom answer"
assert body["usage"]["prompt_tokens"] == 4
assert body["usage"]["completion_tokens"] == 2
finally:
processor.close()
def test_token_chunks():
converted = ChatCompletionRequest(REQUEST).convert(ConversionOptions())
generator = ChatCompletionRequest.chunk_generator(converted, "token-response", "smoke-model")
tokenizer = Tokenizer.from_file(str(TOKENIZER_PATH))
token_ids = tokenizer.encode(THINKING_END_TOKEN + "Custom answer")
processor = StreamProcessor(generator, converted.parsing_options, tokenizer)
response = ChatCompletionResponse("token-response", "smoke-model", 0, 0, 0)
try:
for chunk in (
InferenceChunk.ready(),
*(InferenceChunk.token(token_id) for token_id in token_ids),
InferenceChunk.finish(InferenceFinishReason.Stop),
):
for output in processor.push(chunk):
response.append(output)
for output in processor.finish():
response.append(output)
body = json.loads(response.to_json())
assert body["choices"][0]["message"]["content"] == "Custom answer"
assert body["usage"]["completion_tokens"] == len(token_ids)
finally:
processor.close()
def test_split_character_tokens():
# A character outside the vocabulary decodes from several byte tokens; the
# ids buffered while it was incomplete still count as completion tokens.
converted = ChatCompletionRequest(REQUEST).convert(ConversionOptions())
generator = ChatCompletionRequest.chunk_generator(converted, "token-response", "smoke-model")
tokenizer = Tokenizer.from_file(str(TOKENIZER_PATH))
token_ids = tokenizer.encode(THINKING_END_TOKEN + "🦀")
assert len(token_ids) > 2
processor = StreamProcessor(generator, converted.parsing_options, tokenizer)
response = ChatCompletionResponse("token-response", "smoke-model", 0, 0, 0)
try:
for chunk in (
InferenceChunk.ready(),
*(InferenceChunk.token(token_id) for token_id in token_ids),
InferenceChunk.finish(InferenceFinishReason.Stop),
):
for output in processor.push(chunk):
response.append(output)
for output in processor.finish():
response.append(output)
body = json.loads(response.to_json())
assert body["choices"][0]["message"]["content"] == "🦀"
assert body["usage"]["completion_tokens"] == len(token_ids)
finally:
processor.close()
def test_undecodable_token_ignored():
converted = ChatCompletionRequest(REQUEST).convert(ConversionOptions())
generator = ChatCompletionRequest.chunk_generator(converted, "token-response", "smoke-model")
tokenizer = Tokenizer.from_file(str(TOKENIZER_PATH))
processor = StreamProcessor(generator, converted.parsing_options, tokenizer)
response = ChatCompletionResponse("token-response", "smoke-model", 0, 0, 0)
try:
for chunk in (
InferenceChunk.ready(),
InferenceChunk.token(999_999_999),
InferenceChunk.text(THINKING_END_TOKEN + "Hi", content_tokens=1),
InferenceChunk.finish(InferenceFinishReason.Stop),
):
for output in processor.push(chunk):
response.append(output)
for output in processor.finish():
response.append(output)
body = json.loads(response.to_json())
assert body["choices"][0]["message"]["content"] == "Hi"
assert body["usage"]["completion_tokens"] == 1
finally:
processor.close()
def test_token_chunk_without_tokenizer_errors():
converted = ChatCompletionRequest(REQUEST).convert(ConversionOptions())
generator = ChatCompletionRequest.chunk_generator(converted, "token-response", "smoke-model")
processor = StreamProcessor(generator, converted.parsing_options)
try:
with pytest.raises(RuntimeError, match="without a tokenizer"):
processor.push(InferenceChunk.token(1))
finally:
processor.close()
def test_tokenizer_from_str():
# Loading from the file's contents gives the same ids as loading the file.
text = TOKENIZER_PATH.read_text()
assert Tokenizer.from_str(text).encode("Custom answer") == Tokenizer.from_file(
str(TOKENIZER_PATH)
).encode("Custom answer")
def test_tokenizer_from_str_errors():
with pytest.raises(RuntimeError):
Tokenizer.from_str("not json")
def test_foreign_tokenizer_errors():
converted = ChatCompletionRequest(REQUEST).convert(ConversionOptions())
generator = ChatCompletionRequest.chunk_generator(converted, "py-response", "smoke-model")
with pytest.raises(TypeError):
StreamProcessor(generator, converted.parsing_options, object())
def test_image_resolution():
# One 50-by-40 RGB PNG exercises the native image decoder and WebP encoder.
png = (
"iVBORw0KGgoAAAANSUhEUgAAADIAAAAoCAIAAAAzED4bAAAAPElEQVR4nO3OUQkAIBBA"
"sQvht/2jGMsYPmSwAJu1T9A8H2hpaWlpaTVoaWkVaGlpFWhpaRVoaWkVaH3QuqTYo"
"YpppISkAAAAAElFTkSuQmCC"
)
converted = ChatCompletionRequest({
**REQUEST,
"messages": [{"role": "user", "content": [{
"type": "image_url",
"image_url": {"url": "data:image/png;base64," + png},
}]}],
}).convert(ConversionOptions())
rendered = DeepseekV41Encoding().render_conversation(converted.conversation)
resolver = ImageResolver(ReqwestImageFetcher(), OpenCvImagePreprocessor())
multimodal = resolver.resolve(rendered.image_sources, ImageQuota())
assert IMAGE_SPECIAL_TOKEN in rendered.prompt
(image,) = multimodal.images
assert image.width > 0 and image.height > 0
assert image.data[:4] == b"RIFF" and image.data[8:12] == b"WEBP"
assert multimodal.image_token_adjustment() > 0
def test_circular_dict_rejected():
# A self-referencing container must raise instead of exhausting the stack.
parameters = {}
parameters["self"] = parameters
with pytest.raises(ValueError, match="circular reference"):
ToolDefinition(name="f", parameters=parameters)
def test_circular_list_rejected():
parameters = []
parameters.append(parameters)
with pytest.raises(ValueError, match="circular reference"):
ToolDefinition(name="f", parameters=parameters)
def test_indirect_cycle_rejected():
outer = {}
inner = {"outer": outer}
outer["inner"] = inner
with pytest.raises(ValueError, match="circular reference"):
ToolDefinition(name="f", parameters=outer)
def test_cycle_through_tuple_rejected():
parameters = {"items": []}
parameters["items"].append((parameters,))
with pytest.raises(ValueError, match="circular reference"):
ToolDefinition(name="f", parameters=parameters)
def test_circular_request_body_rejected():
body = dict(REQUEST)
body["self"] = body
with pytest.raises(ValueError, match="circular reference"):
ChatCompletionRequest(body)
def test_deep_nesting_rejected():
parameters = {}
node = parameters
for _ in range(200):
child = {}
node["next"] = child
node = child
with pytest.raises(RecursionError):
ToolDefinition(name="f", parameters=parameters)
def test_nesting_within_limit_accepted():
parameters = {}
node = parameters
for _ in range(127):
child = {}
node["next"] = child
node = child
assert ToolDefinition(name="f", parameters=parameters).parameters == parameters
def test_shared_subtree_accepted():
# A reference used twice is a shared subtree, not a circular reference.
shared = {"type": "string"}
parameters = {"type": "object", "properties": {"a": shared, "b": shared}}
assert ToolDefinition(name="f", parameters=parameters).parameters == parameters

24
deepseek-recipe/Cargo.toml Normal file
View File

@ -0,0 +1,24 @@
[package]
name = "deepseek-recipe"
version = "0.1.0"
edition.workspace = true
description = "API request conversion and streamed output parsing for DeepSeek models."
readme.workspace = true
documentation = "https://docs.rs/deepseek-recipe"
license.workspace = true
repository.workspace = true
[dependencies]
async-stream = { workspace = true }
base64 = { workspace = true }
deepseek-recipe-core = { workspace = true }
itertools = { workspace = true }
jsonschema = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
strum_macros = { workspace = true }
tokenizers = { workspace = true }
tokio-stream = { workspace = true }
[dev-dependencies]
tokio = { workspace = true }

21
deepseek-recipe/LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 DeepSeek
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

48
deepseek-recipe/src/error_response.rs Normal file
View File

@ -0,0 +1,48 @@
use serde::Serialize;
use crate::request::ConversionError;
#[derive(Debug, Serialize)]
pub struct ErrorDetail {
pub message: String,
pub r#type: String,
pub param: Option<String>,
pub code: String,
}
#[derive(Debug, Serialize)]
pub struct ErrorResponse {
pub error: ErrorDetail,
}
impl ConversionError {
/// The HTTP status code for this error.
pub fn status_code(&self) -> u16 {
match self {
ConversionError::BadRequest { .. } => 400,
ConversionError::Internal { .. } => 500,
}
}
/// Build the error response body for this error.
pub fn into_response(self) -> ErrorResponse {
let status_code = self.status_code();
let (type_, detail) = match self {
ConversionError::BadRequest { detail } => ("invalid_request_error", detail),
ConversionError::Internal { detail } => ("internal_error", detail),
};
ErrorResponse {
error: ErrorDetail {
message: detail,
r#type: type_.to_string(),
param: None,
code: if status_code >= 500 {
type_.to_string()
} else {
"invalid_request_error".to_owned()
},
},
}
}
}

16
deepseek-recipe/src/lib.rs Normal file
View File

@ -0,0 +1,16 @@
//! API protocol conversion and inference output parsing.
//!
//! [`request::ProtocolRequest::convert`] produces a shared
//! [`request::ConversationRequest`]. [`stream::StreamProcessor`] parses backend
//! output and passes it to the selected protocol's event generator. The caller
//! supplies prompt rendering, token encoding, inference, usage, and transport.
pub use protocol::anthropic;
pub use protocol::openai;
pub mod error_response;
mod protocol;
pub mod request;
pub mod response;
pub mod stream;
pub mod util;

25
deepseek-recipe/src/protocol/anthropic/messages/mod.rs Normal file
View File

@ -0,0 +1,25 @@
//! Anthropic Messages request conversion and response events.
//!
//! Supports text, images, client tools, and thinking blocks. Document content
//! and hosted tool execution are outside the supported scope. Server tools are
//! rejected by default; web-search handling is configurable through
//! [`crate::request::ConversionOptions::messages_web_search`].
//!
//! Input text must not spell out the image placeholder. Every placeholder in a
//! prompt corresponds to one image source, and the adapter inserts them for
//! image blocks only, so message text, tool references, thinking blocks, tool
//! result text, the top-level `system` field, tool definitions, and historical
//! tool calls are rejected when they carry the placeholder spelling.
//!
//! Use [`crate::request::ProtocolRequest::convert`] to produce a shared
//! [`crate::request::ConversationRequest`], and
//! [`crate::request::ProtocolRequest::chunk_generator`] to construct response
//! events. [`crate::stream::StreamProcessor`] parses backend output into typed
//! [`response::MessagesStreamEvent`] values, which can also be accumulated into
//! a complete [`response::MessagesResponse`].
//!
//! The caller supplies model inference, tool execution, token usage, and HTTP
//! transport.
pub mod request;
pub mod response;

622
deepseek-recipe/src/protocol/anthropic/messages/request/convert.rs Normal file
View File

@ -0,0 +1,622 @@
use std::collections::{BTreeMap, HashSet};
use deepseek_recipe_core::conversation::{Conversation, ReasoningEffort};
use deepseek_recipe_core::messages::{InputMessage, ToolCall};
use deepseek_recipe_core::multimodal::{IMAGE_SPECIAL_TOKEN, ImageSource};
use deepseek_recipe_core::tools::{ToolChoice, ToolDefinition};
use deepseek_recipe_core::util::json_formatter::stringify_python_style;
use itertools::Itertools;
use crate::protocol::image::{
UNSUPPORTED_DOCUMENT_PLACEHOLDER, base64_image_source, external_image_source,
file_id_unsupported, image_not_allowed,
};
use crate::protocol::validation::{
image_special_token_not_allowed, validate_max_tokens, validate_sampling, validate_tool_name,
validate_tool_parameters, validate_tool_text,
};
use crate::request::{
ConversationRequest, ConversionError, ConversionOptions, InferenceOptions, ProtocolRequest,
WebSearchBehavior,
};
use crate::stream::state_machine::{ParsingOptions, ReasoningStage};
use super::super::response::{MessagesChunkGenerator, MessagesResponse};
use super::schema::{
MessagesContent, MessagesContentBlock, MessagesImageSource, MessagesMessage,
MessagesReasoningEffort, MessagesRequest, MessagesTextBlock, MessagesTextOrTextBlocks,
MessagesThinkingType, MessagesTool, MessagesToolChoice,
};
impl ProtocolRequest for MessagesRequest {
type Response = MessagesResponse;
/// Validate and convert Messages input into a conversation request.
///
/// Explicit thinking overrides effort, which overrides the supplied default.
/// Missing historical thinking is accepted, and supplied thinking is preserved.
fn convert(self, options: ConversionOptions) -> Result<ConversationRequest, ConversionError> {
let (thinking_mode, reasoning_effort) = self.resolve_thinking(options);
let inference_options = self.inference_options()?;
let stop_sequences = self.stop_sequences.unwrap_or_default();
let (tools, tool_choice) =
convert_tools(self.tools, self.tool_choice, thinking_mode, &options)?;
let parsing_options =
Self::parsing_options(thinking_mode, &tools, tool_choice, stop_sequences);
let messages = transform_messages(self.messages, self.system, &options)?;
validate_tool_text(&tools, &messages)?;
Ok(ConversationRequest {
conversation: Conversation {
messages,
thinking_mode,
tools,
tool_choice,
reasoning_effort,
..Conversation::default()
},
inference_options,
parsing_options,
model: Some(self.model),
stream: self.stream.unwrap_or(false),
})
}
fn chunk_generator(
request: &ConversationRequest,
id: String,
model: String,
) -> MessagesChunkGenerator {
MessagesChunkGenerator::new(id, model, request.conversation.thinking_mode)
}
}
impl MessagesRequest {
fn resolve_thinking(&self, options: ConversionOptions) -> (bool, Option<ReasoningEffort>) {
let effort = self
.output_config
.as_ref()
.and_then(|config| config.effort)
.map(|effort| match effort {
MessagesReasoningEffort::Low => ReasoningEffort::Low,
MessagesReasoningEffort::Medium | MessagesReasoningEffort::High => {
ReasoningEffort::High
}
MessagesReasoningEffort::Xhigh => ReasoningEffort::Xhigh,
MessagesReasoningEffort::Ultra | MessagesReasoningEffort::Max => {
ReasoningEffort::Max
}
});
let thinking = match self.thinking.as_ref().map(|thinking| thinking.kind) {
Some(MessagesThinkingType::Enabled) => true,
Some(MessagesThinkingType::Disabled) => false,
None => effort.is_some() || options.default_thinking_mode,
};
(
thinking,
thinking.then_some(effort.unwrap_or(ReasoningEffort::High)),
)
}
fn parsing_options(
thinking: bool,
tools: &[ToolDefinition],
tool_choice: ToolChoice,
stop_sequences: Vec<String>,
) -> ParsingOptions {
ParsingOptions {
parse_tool_calls: !tools.is_empty(),
reasoning_initial_stage: thinking.then_some(ReasoningStage::Start),
stop_sequences,
tool_call_initial_stage: tool_choice == ToolChoice::Required,
..ParsingOptions::default()
}
}
fn inference_options(&self) -> Result<InferenceOptions, ConversionError> {
validate_max_tokens(self.max_tokens, "max_tokens")?;
validate_sampling(self.temperature, self.top_p)?;
if let Some(stops) = &self.stop_sequences
&& (stops.len() > 16 || stops.iter().any(String::is_empty))
{
return Err(ConversionError::bad_request(
"stop_sequences must contain at most 16 non-empty strings",
));
}
let disable_parallel_tool_use = match &self.tool_choice {
Some(
MessagesToolChoice::Auto {
disable_parallel_tool_use,
}
| MessagesToolChoice::Any {
disable_parallel_tool_use,
}
| MessagesToolChoice::Tool {
disable_parallel_tool_use,
..
},
) => *disable_parallel_tool_use,
_ => None,
};
Ok(InferenceOptions {
max_tokens: self.max_tokens,
temperature: self.temperature,
top_p: self.top_p,
thinking_budget_tokens: self
.thinking
.as_ref()
.and_then(|thinking| thinking.budget_tokens),
disable_parallel_tool_use,
})
}
}
fn convert_tools(
tools: Option<Vec<MessagesTool>>,
choice: Option<MessagesToolChoice>,
thinking: bool,
options: &ConversionOptions,
) -> Result<(Vec<ToolDefinition>, ToolChoice), ConversionError> {
let mut names = HashSet::new();
let mut definitions = Vec::new();
let mut ignored_server_tool_names = HashSet::new();
for (index, tool) in tools.unwrap_or_default().into_iter().enumerate() {
if let Some(kind) = &tool.kind {
if kind.starts_with("web_search") {
match options.messages_web_search {
WebSearchBehavior::Ignore => {
ignored_server_tool_names.insert(tool.name.clone());
continue;
}
WebSearchBehavior::Reject => {
return Err(ConversionError::bad_request(format!(
"tools.{index}: server tools are not supported"
)));
}
}
}
return Err(ConversionError::bad_request(format!(
"tools.{index}: server tools are not supported"
)));
}
validate_tool_name(&tool.name, &format!("tools.{index}.name"))?;
if !names.insert(tool.name.clone()) {
return Err(ConversionError::bad_request("Tool names must be unique"));
}
let parameters = tool.input_schema.ok_or_else(|| {
ConversionError::bad_request(format!("tools.{index}: missing input_schema"))
})?;
validate_tool_parameters(&parameters, &format!("tools.{index}.input_schema"))?;
definitions.push(ToolDefinition {
name: tool.name,
description: tool.description,
parameters,
strict: None,
});
}
let choice = match choice {
Some(MessagesToolChoice::None) => {
definitions.clear();
ToolChoice::None
}
Some(MessagesToolChoice::Tool { name, .. }) => {
if ignored_server_tool_names.contains(&name) {
// The named server tool was dropped; drop the choice too.
ToolChoice::Auto
} else if !names.contains(&name) {
return Err(ConversionError::bad_request(format!(
"tool_choice: no tool named '{name}' was specified"
)));
} else if thinking {
return Err(ConversionError::bad_request(
"Thinking mode does not support this tool_choice",
));
} else {
definitions.retain(|tool| tool.name == name);
ToolChoice::Required
}
}
_ => ToolChoice::Auto,
};
Ok((definitions, choice))
}
impl MessagesContentBlock {
/// Convert text and tool references, substitute document placeholders, and
/// collect images with one [`IMAGE_SPECIAL_TOKEN`] placeholder per image.
fn into_content(self, path: &str) -> Result<(String, Vec<ImageSource>), ConversionError> {
match self {
MessagesContentBlock::Text { text } => Ok((text, Vec::new())),
MessagesContentBlock::ToolReference { tool_name } => Ok((tool_name, Vec::new())),
MessagesContentBlock::Image { source } => Ok((
IMAGE_SPECIAL_TOKEN.to_string(),
vec![image_source(source, path)?],
)),
MessagesContentBlock::Document => {
Ok((UNSUPPORTED_DOCUMENT_PLACEHOLDER.to_string(), Vec::new()))
}
_ => Err(ConversionError::internal("unexpected content block type")),
}
}
/// Convert textual content and document placeholders, rejecting images.
fn into_text(self, path: &str, role: &str) -> Result<String, ConversionError> {
let (content, image_sources) = self.into_content(path)?;
if !image_sources.is_empty() {
return Err(image_not_allowed(role));
}
Ok(content)
}
}
impl MessagesTextBlock {
/// Convert text and tool references, substitute document placeholders, and
/// collect images with one [`IMAGE_SPECIAL_TOKEN`] placeholder per image.
fn into_content(self, path: &str) -> Result<(String, Vec<ImageSource>), ConversionError> {
match self {
MessagesTextBlock::Text { text } => Ok((text, Vec::new())),
MessagesTextBlock::ToolReference { tool_name } => Ok((tool_name, Vec::new())),
MessagesTextBlock::Image { source } => Ok((
IMAGE_SPECIAL_TOKEN.to_string(),
vec![image_source(source, path)?],
)),
MessagesTextBlock::Document => {
Ok((UNSUPPORTED_DOCUMENT_PLACEHOLDER.to_string(), Vec::new()))
}
MessagesTextBlock::Unsupported => Err(unsupported_block()),
}
}
}
impl MessagesTextOrTextBlocks {
/// Join textual content and document placeholders, collecting image sources.
fn into_content(self, path: &str) -> Result<(String, Vec<ImageSource>), ConversionError> {
match self {
MessagesTextOrTextBlocks::Raw(content) => Ok((content, Vec::new())),
MessagesTextOrTextBlocks::Blocks(blocks) => {
let mut texts = Vec::new();
let mut image_sources = Vec::new();
for (block_index, block) in blocks.into_iter().enumerate() {
let (text, images) = block.into_content(&format!("{path}[{block_index}]"))?;
texts.push(text);
image_sources.extend(images);
}
Ok((texts.join("\n\n"), image_sources))
}
}
}
/// Join textual content and document placeholders, rejecting images.
fn into_text(self, path: &str, role: &str) -> Result<String, ConversionError> {
let (content, image_sources) = self.into_content(path)?;
if !image_sources.is_empty() {
return Err(image_not_allowed(role));
}
Ok(content)
}
}
/// Convert one Messages image reference.
fn image_source(source: MessagesImageSource, path: &str) -> Result<ImageSource, ConversionError> {
match source {
MessagesImageSource::Base64 { media_type, data } => {
base64_image_source(&media_type, data, path)
}
MessagesImageSource::Url { url } => external_image_source(url, None, path),
MessagesImageSource::File { file_id: _ } => Err(file_id_unsupported(path)),
}
}
fn check_tool_use_resolved(
message_idx: usize,
tool_use_ids: &BTreeMap<String, usize>,
) -> Result<(), ConversionError> {
if tool_use_ids.is_empty() {
return Ok(());
}
Err(ConversionError::bad_request(format!(
"messages.{message_idx}: `tool_use` ids were found without `tool_result` blocks immediately after: {}. Each `tool_use` block \
must have a corresponding `tool_result` block in the next message.",
tool_use_ids.keys().join(", ")
)))
}
/// Return whether any message text or top-level system text carries
/// [`IMAGE_SPECIAL_TOKEN`].
///
/// Message text, tool references, thinking blocks, and tool result text are
/// checked. Tool call names and arguments are checked after conversion by
/// [`validate_tool_text`].
fn messages_contain_image_special_token(
messages: &[MessagesMessage],
system: Option<&MessagesTextOrTextBlocks>,
) -> bool {
messages.iter().any(|message| match message {
MessagesMessage::User(content) | MessagesMessage::Assistant(content) => {
content_contains_image_special_token(content)
}
MessagesMessage::System(content) => {
text_or_text_blocks_contain_image_special_token(content)
}
}) || system.is_some_and(text_or_text_blocks_contain_image_special_token)
}
/// Return whether one message content carries [`IMAGE_SPECIAL_TOKEN`].
fn content_contains_image_special_token(content: &MessagesContent) -> bool {
match content {
MessagesContent::Raw(content) => content.contains(IMAGE_SPECIAL_TOKEN),
MessagesContent::Blocks(blocks) => blocks.iter().any(block_contains_image_special_token),
}
}
/// Return whether one message content block carries [`IMAGE_SPECIAL_TOKEN`].
fn block_contains_image_special_token(block: &MessagesContentBlock) -> bool {
match block {
MessagesContentBlock::Text { text } => text.contains(IMAGE_SPECIAL_TOKEN),
MessagesContentBlock::ToolReference { tool_name } => {
tool_name.contains(IMAGE_SPECIAL_TOKEN)
}
MessagesContentBlock::Thinking { thinking, .. } => thinking.contains(IMAGE_SPECIAL_TOKEN),
MessagesContentBlock::ToolResult { content, .. } => content
.as_ref()
.is_some_and(text_or_text_blocks_contain_image_special_token),
_ => false,
}
}
/// Return whether one text or text-block value carries [`IMAGE_SPECIAL_TOKEN`].
fn text_or_text_blocks_contain_image_special_token(content: &MessagesTextOrTextBlocks) -> bool {
match content {
MessagesTextOrTextBlocks::Raw(content) => content.contains(IMAGE_SPECIAL_TOKEN),
MessagesTextOrTextBlocks::Blocks(blocks) => blocks.iter().any(|block| match block {
MessagesTextBlock::Text { text } => text.contains(IMAGE_SPECIAL_TOKEN),
MessagesTextBlock::ToolReference { tool_name } => {
tool_name.contains(IMAGE_SPECIAL_TOKEN)
}
_ => false,
}),
}
}
fn transform_messages(
messages: Vec<MessagesMessage>,
system: Option<MessagesTextOrTextBlocks>,
options: &ConversionOptions,
) -> Result<Vec<InputMessage>, ConversionError> {
if messages.is_empty() {
return Err(ConversionError::bad_request(
"messages: at least one message is required",
));
}
if messages_contain_image_special_token(&messages, system.as_ref()) {
return Err(image_special_token_not_allowed());
}
let mut transformed_messages = Vec::new();
if let Some(system) = system {
transformed_messages.push(InputMessage::System {
content: system.into_text("system", "system")?,
});
}
let mut tool_use_ids = BTreeMap::new();
let messages_len = messages.len();
for (message_idx, message) in messages.into_iter().enumerate() {
match message {
MessagesMessage::User(MessagesContent::Raw(content)) => {
check_tool_use_resolved(message_idx, &tool_use_ids)?;
transformed_messages.push(InputMessage::User {
content,
image_sources: Vec::new(),
});
}
MessagesMessage::Assistant(MessagesContent::Raw(content)) => {
check_tool_use_resolved(message_idx, &tool_use_ids)?;
transformed_messages.push(InputMessage::Assistant {
content,
reasoning_content: None,
tool_calls: None,
});
}
MessagesMessage::System(content) => {
check_tool_use_resolved(message_idx, &tool_use_ids)?;
let (content, image_sources) =
content.into_content(&format!("messages.{message_idx}"))?;
transformed_messages.push(InputMessage::User {
content: format!("<system-reminder>\n{content}\n</system-reminder>"),
image_sources,
});
}
MessagesMessage::User(MessagesContent::Blocks(blocks)) => {
if blocks.is_empty() {
return Err(ConversionError::bad_request(format!(
"messages.{message_idx}: all messages must have non-empty content"
)));
}
let mut matched_tool_use_ids: HashSet<String> = HashSet::new();
for (block_idx, block) in blocks.into_iter().enumerate() {
let path = format!("messages.{message_idx}.content[{block_idx}]");
match block {
MessagesContentBlock::Unsupported => return Err(unsupported_block()),
MessagesContentBlock::Text { .. }
| MessagesContentBlock::ToolReference { .. } => {
check_tool_use_resolved(message_idx, &tool_use_ids)?;
let content = block.into_text(&path, "user")?;
transformed_messages.push(InputMessage::User {
content,
image_sources: Vec::new(),
});
}
MessagesContentBlock::Image { source } => {
check_tool_use_resolved(message_idx, &tool_use_ids)?;
transformed_messages.push(InputMessage::User {
content: IMAGE_SPECIAL_TOKEN.to_string(),
image_sources: vec![image_source(source, &path)?],
});
}
MessagesContentBlock::Document => {
check_tool_use_resolved(message_idx, &tool_use_ids)?;
transformed_messages.push(InputMessage::User {
content: UNSUPPORTED_DOCUMENT_PLACEHOLDER.to_string(),
image_sources: Vec::new(),
});
}
MessagesContentBlock::ToolUse { .. } => {
return Err(ConversionError::bad_request(format!(
"messages.{message_idx}: `tool_use` blocks can only be in `assistant` messages"
)));
}
MessagesContentBlock::ToolResult {
tool_use_id,
content,
..
} => {
if matched_tool_use_ids.contains(tool_use_id.as_str()) {
return Err(ConversionError::bad_request(format!(
"messages.{message_idx}.content.{block_idx}: each tool_use must have a single result. Found \
multiple `tool_result` blocks with id: {tool_use_id}"
)));
} else if !tool_use_ids.contains_key(tool_use_id.as_str()) {
return Err(ConversionError::bad_request(format!(
"unexpected `messages.{message_idx}.content.{block_idx}: tool_use_id` found in `tool_result` \
blocks: {tool_use_id}. Each `tool_result` block must have a corresponding `tool_use` block in \
the previous message."
)));
} else {
tool_use_ids.remove(&tool_use_id);
matched_tool_use_ids.insert(tool_use_id.clone());
}
let (content, image_sources) = match content {
Some(content) => content.into_content(&path)?,
None => (String::new(), Vec::new()),
};
transformed_messages.push(InputMessage::Tool {
content,
image_sources,
tool_call_id: tool_use_id,
});
}
MessagesContentBlock::Thinking { .. } => {
return Err(ConversionError::bad_request(format!(
"messages.{message_idx}.content: thinking blocks may only be in `assistant` messages"
)));
}
MessagesContentBlock::ServerToolUse { .. } => {
return Err(ConversionError::bad_request(format!(
"messages.{message_idx}: `server_tool_use` blocks can only be in `assistant` messages"
)));
}
MessagesContentBlock::WebSearchToolResult { .. } => {
return Err(ConversionError::bad_request(format!(
"messages.{message_idx}: `web_search_tool_result` blocks can only be in `assistant` messages"
)));
}
}
}
if !tool_use_ids.is_empty() {
return Err(ConversionError::bad_request(format!(
"messages.{}:`tool_use` ids were found without `tool_result` blocks immediately after: {}. Each `tool_use` \
block must have a corresponding `tool_result` block in the next message.",
message_idx - 1,
tool_use_ids.keys().join(", ")
)));
}
}
MessagesMessage::Assistant(MessagesContent::Blocks(blocks)) => {
check_tool_use_resolved(message_idx, &tool_use_ids)?;
if blocks.is_empty() {
return Err(ConversionError::bad_request(format!(
"messages.{message_idx}: all messages must have non-empty content"
)));
}
let mut content = String::new();
let mut reasoning_content: Option<String> = None;
let mut tool_calls: Option<Vec<ToolCall>> = None;
for (block_idx, block) in blocks.into_iter().enumerate() {
let path = format!("messages.{message_idx}.content[{block_idx}]");
match block {
MessagesContentBlock::Unsupported => return Err(unsupported_block()),
MessagesContentBlock::Text { .. }
| MessagesContentBlock::ToolReference { .. }
| MessagesContentBlock::Image { .. }
| MessagesContentBlock::Document => {
if tool_calls.is_some() {
return Err(ConversionError::bad_request(format!(
"messages.{message_idx}.{block_idx}: `tool_use` ids were found without `tool_result` blocks \
immediately after: {}. Each `tool_use` block must have a corresponding `tool_result` block in \
the next message.",
tool_use_ids.keys().join(", ")
)));
}
if !content.is_empty() {
content.push_str("\n\n");
}
content.push_str(&block.into_text(&path, "assistant")?);
}
MessagesContentBlock::ToolUse { id, name, input } => {
let tool_use_id = id.clone();
let tool_call = ToolCall {
id,
name,
arguments: stringify_python_style(&input),
};
tool_calls.get_or_insert_with(Vec::new).push(tool_call);
if tool_use_ids.insert(tool_use_id, block_idx).is_some() {
return Err(ConversionError::bad_request(format!(
"messages.{message_idx}.content.{block_idx}: `tool_use` ids must be unique"
)));
}
}
MessagesContentBlock::ToolResult { .. } => {
return Err(ConversionError::bad_request(format!(
"messages.{message_idx}: `tool_result` blocks can only be in `user` messages"
)));
}
MessagesContentBlock::Thinking { thinking, .. } => {
if let Some(reasoning_content) = &mut reasoning_content {
reasoning_content.push_str("\n\n");
reasoning_content.push_str(&thinking);
} else {
reasoning_content = Some(thinking);
}
}
MessagesContentBlock::ServerToolUse { .. }
| MessagesContentBlock::WebSearchToolResult { .. } => {
if options.messages_web_search == WebSearchBehavior::Reject {
return Err(ConversionError::bad_request(
"Unsupported content block: server tools are not supported",
));
}
}
}
}
transformed_messages.push(InputMessage::Assistant {
content,
reasoning_content,
tool_calls,
});
}
}
}
if !tool_use_ids.is_empty() {
return Err(ConversionError::bad_request(format!(
"messages.{}:`tool_use` ids were found without `tool_result` blocks immediately after: {}. Each `tool_use` block must \
have a corresponding `tool_result` block in the next message.",
messages_len - 1,
tool_use_ids.keys().join(", ")
)));
}
Ok(transformed_messages)
}
fn unsupported_block() -> ConversionError {
ConversionError::bad_request("Unsupported content block")
}

4
deepseek-recipe/src/protocol/anthropic/messages/request/mod.rs Normal file
View File

@ -0,0 +1,4 @@
pub use schema::*;
mod convert;
mod schema;

192
deepseek-recipe/src/protocol/anthropic/messages/request/schema.rs Normal file
View File

@ -0,0 +1,192 @@
use serde::Deserialize;
/// An Anthropic Messages request. Unknown top-level fields are ignored.
///
/// Model selection, execution and usage accounting are the caller's
/// responsibility. Images are supported. Document blocks become the text
/// `[Unsupported Document]`; their contents are discarded. `web_search` server
/// tools are rejected by default; see
/// [`crate::request::WebSearchBehavior`].
#[derive(Debug, Deserialize)]
pub struct MessagesRequest {
pub model: String,
pub messages: Vec<MessagesMessage>,
pub system: Option<MessagesTextOrTextBlocks>,
pub max_tokens: Option<u32>,
pub temperature: Option<f32>,
pub top_p: Option<f32>,
pub stop_sequences: Option<Vec<String>>,
pub stream: Option<bool>,
pub tools: Option<Vec<MessagesTool>>,
pub tool_choice: Option<MessagesToolChoice>,
pub thinking: Option<MessagesThinking>,
pub output_config: Option<MessagesOutputConfig>,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "role", content = "content", rename_all = "snake_case")]
pub enum MessagesMessage {
User(MessagesContent),
Assistant(MessagesContent),
/// Compatibility extension, rendered as a user system-reminder.
System(MessagesTextOrTextBlocks),
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
pub enum MessagesContent {
Raw(String),
Blocks(Vec<MessagesContentBlock>),
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
pub enum MessagesTextOrTextBlocks {
Raw(String),
Blocks(Vec<MessagesTextBlock>),
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum MessagesTextBlock {
Text {
text: String,
},
ToolReference {
tool_name: String,
},
Image {
source: MessagesImageSource,
},
/// Documents are rendered as an unsupported-document placeholder.
Document,
/// Recognized during deserialization, rejected during conversion.
#[serde(other)]
Unsupported,
}
/// One image referenced by inline data, URL, or file identifier. A file
/// identifier requires a file service and is rejected during conversion.
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum MessagesImageSource {
Base64 {
/// Declared media type, validated against the supported image types.
media_type: String,
data: String,
},
Url {
url: String,
},
File {
file_id: String,
},
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum MessagesContentBlock {
Text {
text: String,
},
ToolReference {
tool_name: String,
},
ToolUse {
id: String,
name: String,
input: serde_json::Value,
},
ToolResult {
tool_use_id: String,
content: Option<MessagesTextOrTextBlocks>,
/// Accepted for compatibility; the result text is preserved either way.
is_error: Option<bool>,
},
Thinking {
/// Signature data available to the caller before conversion.
signature: Option<String>,
thinking: String,
},
Image {
source: MessagesImageSource,
},
/// A server tool invocation. Assistant messages only; dropped or rejected
/// per [`crate::request::ConversionOptions::messages_web_search`].
ServerToolUse {
id: String,
name: String,
input: serde_json::Value,
},
/// A server tool result. Assistant messages only; dropped or rejected per
/// [`crate::request::ConversionOptions::messages_web_search`].
WebSearchToolResult {
tool_use_id: String,
content: serde_json::Value,
},
/// Documents are rendered as an unsupported-document placeholder.
Document,
#[serde(other)]
Unsupported,
}
/// A client tool declaration. A `type` starting with `web_search` is dropped or
/// rejected per [`crate::request::ConversionOptions::messages_web_search`];
/// other `type` discriminators are rejected as unsupported.
#[derive(Debug, Deserialize)]
pub struct MessagesTool {
pub name: String,
pub description: Option<String>,
pub input_schema: Option<serde_json::Value>,
#[serde(rename = "type")]
pub kind: Option<String>,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum MessagesToolChoice {
Auto {
disable_parallel_tool_use: Option<bool>,
},
/// For compatibility, `any` permits both text replies and tool calls.
Any {
disable_parallel_tool_use: Option<bool>,
},
Tool {
name: String,
disable_parallel_tool_use: Option<bool>,
},
None,
}
#[derive(Debug, Deserialize)]
pub struct MessagesThinking {
#[serde(rename = "type")]
pub kind: MessagesThinkingType,
/// Thinking budget passed through for the caller to enforce.
pub budget_tokens: Option<u64>,
}
#[derive(Debug, Clone, Copy, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum MessagesThinkingType {
#[serde(alias = "adaptive")]
Enabled,
Disabled,
}
#[derive(Debug, Deserialize)]
pub struct MessagesOutputConfig {
pub effort: Option<MessagesReasoningEffort>,
}
#[derive(Debug, Clone, Copy, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum MessagesReasoningEffort {
Low,
Medium,
High,
Xhigh,
Ultra,
Max,
}

207
deepseek-recipe/src/protocol/anthropic/messages/response/chunk_generator.rs Normal file
View File

@ -0,0 +1,207 @@
use crate::response::ProtocolResponse;
use crate::stream::{ChunkGenerator, OutputChunk, PromptUsage};
use super::schema::{
MessagesDelta, MessagesResponse, MessagesResponseContent, MessagesStopInfo,
MessagesStreamEvent, MessagesUsage,
};
/// Converts parsed inference output to Messages events.
#[derive(Debug)]
pub struct MessagesChunkGenerator {
id: String,
model: String,
thinking_mode: bool,
signature: String,
content_index: usize,
tool_call_index: usize,
active: Option<MessagesContentBlockKind>,
prompt_usage: PromptUsage,
started: bool,
finished: bool,
}
#[derive(Debug, PartialEq, Eq)]
enum MessagesContentBlockKind {
Text,
Thinking,
ToolUse,
}
impl MessagesChunkGenerator {
/// The caller supplies a unique response ID and the public response model name.
/// Tool IDs are derived from that response ID and the per-response tool index.
/// The default opaque thinking signature is the response ID.
pub fn new(id: impl Into<String>, model: impl Into<String>, thinking_mode: bool) -> Self {
let id = id.into();
Self {
signature: id.clone(),
id,
model: model.into(),
thinking_mode,
content_index: 0,
tool_call_index: 0,
active: None,
prompt_usage: PromptUsage::default(),
started: false,
finished: false,
}
}
/// Set the signature emitted at the end of a thinking block.
pub fn with_signature(mut self, signature: impl Into<String>) -> Self {
self.signature = signature.into();
self
}
fn close_block(&mut self, events: &mut Vec<MessagesStreamEvent>) {
if let Some(kind) = self.active.take() {
if kind == MessagesContentBlockKind::Thinking {
events.push(MessagesStreamEvent::ContentBlockDelta {
index: self.content_index,
delta: MessagesDelta::SignatureDelta {
signature: self.signature.clone(),
},
});
}
events.push(MessagesStreamEvent::ContentBlockStop {
index: self.content_index,
});
self.content_index += 1;
}
}
fn open_block(
&mut self,
kind: MessagesContentBlockKind,
content_block: MessagesResponseContent,
events: &mut Vec<MessagesStreamEvent>,
) {
self.close_block(events);
events.push(MessagesStreamEvent::ContentBlockStart {
index: self.content_index,
content_block,
});
if self.content_index == 0 {
events.push(MessagesStreamEvent::Ping);
}
self.active = Some(kind);
}
fn delta(&self, delta: MessagesDelta) -> MessagesStreamEvent {
MessagesStreamEvent::ContentBlockDelta {
index: self.content_index,
delta,
}
}
}
impl ChunkGenerator for MessagesChunkGenerator {
type Chunk = MessagesStreamEvent;
async fn generate(&mut self, chunk: OutputChunk) -> Vec<Self::Chunk> {
if self.finished {
return Vec::new();
}
let mut events = Vec::new();
match chunk {
OutputChunk::Start { usage, .. } => {
if self.started {
return events;
}
self.started = true;
self.prompt_usage = usage;
events.push(MessagesStreamEvent::MessageStart {
message: MessagesResponse::new(
self.id.clone(),
self.model.clone(),
0,
usage.prompt_tokens,
usage.prompt_cache_hit_tokens,
),
});
if self.thinking_mode {
self.open_block(
MessagesContentBlockKind::Thinking,
MessagesResponseContent::Thinking {
thinking: String::new(),
signature: String::new(),
},
&mut events,
);
}
}
OutputChunk::Raw { content } => {
if self.active != Some(MessagesContentBlockKind::Text) {
self.open_block(
MessagesContentBlockKind::Text,
MessagesResponseContent::Text {
text: String::new(),
},
&mut events,
);
}
events.push(self.delta(MessagesDelta::TextDelta { text: content }));
}
OutputChunk::Reasoning { content } => {
if self.active != Some(MessagesContentBlockKind::Thinking) {
self.open_block(
MessagesContentBlockKind::Thinking,
MessagesResponseContent::Thinking {
thinking: String::new(),
signature: String::new(),
},
&mut events,
);
}
events.push(self.delta(MessagesDelta::ThinkingDelta { thinking: content }));
}
OutputChunk::ToolCall {
tool_name,
arguments,
} => {
let id = format!("toolu_{}_{}", self.id, self.tool_call_index);
self.tool_call_index += 1;
let input = if arguments.is_empty() {
serde_json::json!({})
} else {
serde_json::from_str(&arguments).unwrap_or_else(|_| serde_json::json!({}))
};
self.open_block(
MessagesContentBlockKind::ToolUse,
MessagesResponseContent::ToolUse {
id,
name: tool_name,
input,
},
&mut events,
);
}
OutputChunk::ToolArgumentsDelta { content } => {
if self.active == Some(MessagesContentBlockKind::ToolUse) {
events.push(self.delta(MessagesDelta::InputJsonDelta {
partial_json: content,
}));
}
}
OutputChunk::Finish {
reason,
stop_sequence,
usage,
} => {
self.close_block(&mut events);
events.push(MessagesStreamEvent::MessageDelta {
delta: MessagesStopInfo {
stop_reason: Some(reason.into()),
stop_sequence,
},
usage: MessagesUsage::from((self.prompt_usage, usage)),
});
events.push(MessagesStreamEvent::MessageStop);
self.finished = true;
}
OutputChunk::ToolCallBegin => {}
}
events
}
}

5
deepseek-recipe/src/protocol/anthropic/messages/response/mod.rs Normal file
View File

@ -0,0 +1,5 @@
pub use chunk_generator::*;
pub use schema::*;
mod chunk_generator;
mod schema;

262
deepseek-recipe/src/protocol/anthropic/messages/response/schema.rs Normal file
View File

@ -0,0 +1,262 @@
use std::collections::BTreeMap;
use serde::Serialize;
use crate::response::ProtocolResponse;
use crate::stream::{CompletionUsage, FinishReason, PromptUsage};
use crate::util::append_delta::AppendDelta;
use super::chunk_generator::MessagesChunkGenerator;
/// A complete Messages response, also usable as an accumulator for generated
/// events.
///
/// Tool argument fragments are parsed when their content block stops. Invalid
/// or truncated JSON produces `{}`. Events with invalid indices or incompatible
/// delta types leave the response unchanged.
#[derive(Debug, Serialize)]
pub struct MessagesResponse {
pub id: String,
#[serde(rename = "type")]
pub response_type: MessagesResponseType,
pub role: MessagesRole,
pub model: String,
pub content: Vec<MessagesResponseContent>,
pub stop_reason: Option<MessagesStopReason>,
pub stop_sequence: Option<String>,
pub usage: MessagesUsage,
#[serde(skip)]
partial_tool_inputs: BTreeMap<usize, String>,
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum MessagesResponseType {
Message,
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum MessagesRole {
Assistant,
}
/// A complete content block, distinct from the delta applied to it.
#[derive(Debug, Serialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum MessagesResponseContent {
Text {
text: String,
},
Thinking {
thinking: String,
signature: String,
},
ToolUse {
id: String,
name: String,
input: serde_json::Value,
},
}
#[derive(Debug, Serialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum MessagesDelta {
TextDelta { text: String },
ThinkingDelta { thinking: String },
SignatureDelta { signature: String },
InputJsonDelta { partial_json: String },
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum MessagesStopReason {
EndTurn,
MaxTokens,
StopSequence,
ToolUse,
Refusal,
}
impl From<FinishReason> for MessagesStopReason {
fn from(reason: FinishReason) -> Self {
match reason {
FinishReason::Stop | FinishReason::EndOfStream => Self::EndTurn,
FinishReason::Length => Self::MaxTokens,
FinishReason::StopSequence => Self::StopSequence,
FinishReason::ToolCalls => Self::ToolUse,
FinishReason::ContentFilter => Self::Refusal,
}
}
}
#[derive(Debug, Serialize)]
pub struct MessagesStopInfo {
pub stop_reason: Option<MessagesStopReason>,
pub stop_sequence: Option<String>,
}
/// Protocol representation of caller-supplied accounting.
#[derive(Debug, Serialize)]
pub struct MessagesUsage {
pub input_tokens: usize,
pub cache_creation_input_tokens: usize,
pub cache_read_input_tokens: usize,
pub output_tokens: usize,
pub service_tier: MessagesServiceTier,
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum MessagesServiceTier {
Standard,
}
impl From<(PromptUsage, CompletionUsage)> for MessagesUsage {
fn from((prompt, completion): (PromptUsage, CompletionUsage)) -> Self {
Self {
input_tokens: prompt
.prompt_tokens
.saturating_sub(prompt.prompt_cache_hit_tokens),
cache_creation_input_tokens: 0,
cache_read_input_tokens: prompt.prompt_cache_hit_tokens,
output_tokens: completion.completion_tokens,
service_tier: MessagesServiceTier::Standard,
}
}
}
/// A Messages stream event. Serialize it as JSON and obtain its SSE event name
/// with [`Self::event_type`]. The caller supplies SSE framing and transport.
#[derive(Debug, Serialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum MessagesStreamEvent {
MessageStart {
message: MessagesResponse,
},
ContentBlockStart {
index: usize,
content_block: MessagesResponseContent,
},
ContentBlockDelta {
index: usize,
delta: MessagesDelta,
},
ContentBlockStop {
index: usize,
},
MessageDelta {
delta: MessagesStopInfo,
usage: MessagesUsage,
},
MessageStop,
Ping,
}
impl MessagesStreamEvent {
/// Event name used by the SSE transport.
pub fn event_type(&self) -> &'static str {
match self {
Self::MessageStart { .. } => "message_start",
Self::ContentBlockStart { .. } => "content_block_start",
Self::ContentBlockDelta { .. } => "content_block_delta",
Self::ContentBlockStop { .. } => "content_block_stop",
Self::MessageDelta { .. } => "message_delta",
Self::MessageStop => "message_stop",
Self::Ping => "ping",
}
}
}
impl ProtocolResponse for MessagesResponse {
type ChunkGenerator = MessagesChunkGenerator;
fn new(
id: String,
model: String,
_created: u64,
prompt_tokens: usize,
prompt_cache_hit_tokens: usize,
) -> Self {
Self {
id,
response_type: MessagesResponseType::Message,
role: MessagesRole::Assistant,
model,
content: Vec::new(),
stop_reason: None,
stop_sequence: None,
usage: MessagesUsage::from((
PromptUsage {
prompt_tokens,
prompt_cache_hit_tokens,
},
CompletionUsage::default(),
)),
partial_tool_inputs: BTreeMap::new(),
}
}
fn chunk_event_type(chunk: &MessagesStreamEvent) -> Option<&'static str> {
Some(chunk.event_type())
}
}
impl AppendDelta<MessagesStreamEvent> for MessagesResponse {
fn append(&mut self, event: MessagesStreamEvent) {
match event {
MessagesStreamEvent::MessageStart { message } => *self = message,
MessagesStreamEvent::ContentBlockStart {
index,
content_block,
} => {
if index == self.content.len() {
self.content.push(content_block);
}
}
MessagesStreamEvent::ContentBlockDelta { index, delta } => {
let Some(block) = self.content.get_mut(index) else {
return;
};
match (block, delta) {
(
MessagesResponseContent::Text { text },
MessagesDelta::TextDelta { text: value },
) => text.push_str(&value),
(
MessagesResponseContent::Thinking { thinking, .. },
MessagesDelta::ThinkingDelta { thinking: value },
) => thinking.push_str(&value),
(
MessagesResponseContent::Thinking { signature, .. },
MessagesDelta::SignatureDelta { signature: value },
) => signature.push_str(&value),
(
MessagesResponseContent::ToolUse { .. },
MessagesDelta::InputJsonDelta { partial_json },
) => {
self.partial_tool_inputs
.entry(index)
.or_default()
.push_str(&partial_json);
}
_ => {}
}
}
MessagesStreamEvent::ContentBlockStop { index } => {
if let Some(json) = self.partial_tool_inputs.remove(&index)
&& let Some(MessagesResponseContent::ToolUse { input, .. }) =
self.content.get_mut(index)
{
*input = serde_json::from_str(&json).unwrap_or_else(|_| serde_json::json!({}));
}
}
MessagesStreamEvent::MessageDelta { delta, usage } => {
self.stop_reason = delta.stop_reason;
self.stop_sequence = delta.stop_sequence;
self.usage = usage;
}
MessagesStreamEvent::MessageStop | MessagesStreamEvent::Ping => {}
}
}
}

6
deepseek-recipe/src/protocol/anthropic/mod.rs Normal file
View File

@ -0,0 +1,6 @@
//! Anthropic-compatible protocol types and conversion.
pub use messages::request::MessagesRequest;
pub use messages::response::{MessagesResponse, MessagesStreamEvent};
pub mod messages;

139
deepseek-recipe/src/protocol/image.rs Normal file
View File

@ -0,0 +1,139 @@
//! Image input validation shared by the protocol adapters.
use base64::Engine;
use base64::engine::general_purpose::STANDARD;
use deepseek_recipe_core::multimodal::{
ImageDetail, ImageMediaType, ImageSource, MAX_IMAGE_URL_LEN, is_http_url,
};
use crate::request::ConversionError;
/// Text substituted for document blocks by the Messages and Responses adapters.
/// Document contents are not read or forwarded to the backend.
pub const UNSUPPORTED_DOCUMENT_PLACEHOLDER: &str = "[Unsupported Document]";
/// Error message for an image whose bytes are missing or unsupported.
pub const UNSUPPORTED_IMAGE_ERR: &str = "You have uploaded an unsupported image. Please make sure your image is valid and has one of the following formats: webp, png, jpeg, and gif.";
/// Convert an external image URL into an image source.
///
/// Checks the case-insensitive `http` prefix and a UTF-8 byte length no greater
/// than [`MAX_IMAGE_URL_LEN`]. Full URL validation belongs to the fetcher.
pub(super) fn external_image_source(
url: String,
detail: Option<ImageDetail>,
path: &str,
) -> Result<ImageSource, ConversionError> {
if !is_http_url(&url) {
return Err(ConversionError::bad_request(format!(
"{path}: invalid url: {url:?}"
)));
}
if url.len() > MAX_IMAGE_URL_LEN {
return Err(ConversionError::bad_request(format!(
"{path}: external link length {} too long, max link length {MAX_IMAGE_URL_LEN}. Consider passing file data directly with \
base64 data url instead.",
url.len()
)));
}
Ok(ImageSource::Url {
url,
detail: detail.unwrap_or_default(),
})
}
/// Convert a data URL into an image source.
///
/// The body is not inspected. The resolver accepts both a base64 body and a
/// percent-encoded body.
pub(super) fn data_url_image_source(data_url: String, detail: Option<ImageDetail>) -> ImageSource {
ImageSource::DataUrl {
data_url,
detail: detail.unwrap_or_default(),
}
}
/// Convert an `image_url` value into an image source.
///
/// Values with a case-insensitive `http` prefix are external. Other values must
/// contain the literal `base64,` marker. Data URL syntax, media type, and body
/// decoding are checked later by the image resolver.
pub(super) fn image_url_source(
url: String,
detail: Option<ImageDetail>,
path: &str,
) -> Result<ImageSource, ConversionError> {
if is_http_url(&url) {
external_image_source(url, detail, path)
} else if url.contains("base64,") {
Ok(data_url_image_source(url, detail))
} else {
Err(ConversionError::bad_request(format!(
"{path}: Unsupported image_url format"
)))
}
}
/// Build a base64 data URL from a declared media type and base64 payload.
///
/// Normalizes trailing padding and validates the standard base64 encoding.
/// Empty payloads and invalid image contents are checked by the resolver and
/// preprocessor, respectively.
pub(super) fn base64_image_source(
media_type: &str,
data: String,
path: &str,
) -> Result<ImageSource, ConversionError> {
let media_type =
ImageMediaType::from_mime(media_type).ok_or_else(|| unsupported_image(path))?;
let data = normalize_base64_padding(data);
if STANDARD.decode(&data).is_err() {
return Err(ConversionError::bad_request(format!(
"{path}: base64 decode error"
)));
}
Ok(ImageSource::DataUrl {
data_url: format!("data:{};base64,{data}", media_type.mime()),
detail: ImageDetail::default(),
})
}
/// Return the error for an unsupported image.
pub(super) fn unsupported_image(path: &str) -> ConversionError {
ConversionError::bad_request(format!("{path}: {UNSUPPORTED_IMAGE_ERR}"))
}
/// Return the error for an image in a message that does not accept images.
pub(super) fn image_not_allowed(role: &str) -> ConversionError {
ConversionError::bad_request(format!("Image in {role} message is unsupported"))
}
/// Return the error for a file identifier, which requires a file service this
/// library does not provide.
pub(super) fn file_id_unsupported(path: &str) -> ConversionError {
ConversionError::bad_request(format!(
"{path}: file_id is not supported. Pass the image as a base64 data url instead."
))
}
/// Normalize the padding of a base64 payload.
///
/// A caller may supply the wrong number of trailing `=` characters. A payload
/// carrying at least one trailing `=` has its padding dropped and replaced with
/// the canonical amount. A payload with no trailing `=` is returned unchanged,
/// because a truncated payload and an unpadded payload are indistinguishable.
/// A payload whose unpadded length leaves a single symbol is also returned
/// unchanged, so the decoder reports the invalid payload.
fn normalize_base64_padding(mut data: String) -> String {
if !data.ends_with('=') {
return data;
}
let unpadded_len = data.trim_end_matches('=').len();
let remainder = unpadded_len % 4;
if remainder == 1 {
return data;
}
data.truncate(unpadded_len);
data.push_str(&"=".repeat((4 - remainder) % 4));
data
}

5
deepseek-recipe/src/protocol/mod.rs Normal file
View File

@ -0,0 +1,5 @@
pub mod anthropic;
pub mod openai;
mod image;
mod validation;

2
deepseek-recipe/src/protocol/openai/chat_completion/mod.rs Normal file
View File

@ -0,0 +1,2 @@
pub mod request;
pub mod response;

504
deepseek-recipe/src/protocol/openai/chat_completion/request/convert.rs Normal file
View File

@ -0,0 +1,504 @@
use std::collections::HashSet;
use deepseek_recipe_core::conversation::{Conversation, ReasoningEffort, ResponseFormat};
use deepseek_recipe_core::messages::{InputMessage, ToolCall};
use deepseek_recipe_core::multimodal::{IMAGE_SPECIAL_TOKEN, ImageSource};
use deepseek_recipe_core::tools::{ToolChoice, ToolDefinition};
use crate::protocol::image::{
data_url_image_source, file_id_unsupported, image_not_allowed, image_url_source,
};
use crate::protocol::validation::{
contains_json_instruction, image_special_token_not_allowed, validate_max_tokens,
validate_sampling, validate_tool_name, validate_tool_parameters, validate_tool_text,
};
use crate::request::{
ConversationRequest, ConversionError, ConversionOptions, InferenceOptions, ProtocolRequest,
};
use crate::stream::state_machine::{ParsingOptions, ReasoningStage};
use super::super::response::{ChatCompletionChunkGenerator, ChatCompletionResponse};
use super::schema::{
ChatCompletionReasoningEffort, ChatCompletionRequest, ChatCompletionRequestContent,
ChatCompletionRequestContentBlock, ChatCompletionRequestMessage, ChatCompletionResponseFormat,
ChatCompletionStopSequences, ChatCompletionStreamOptions, ChatCompletionThinkingType,
ChatCompletionTool, ChatCompletionToolChoiceMode, ChatCompletionToolChoiceOption,
};
/// Maximum number of stop sequences accepted by the adapter.
const MAX_STOP_SEQUENCES: usize = 16;
impl ProtocolRequest for ChatCompletionRequest {
type Response = ChatCompletionResponse;
fn convert(self, options: ConversionOptions) -> Result<ConversationRequest, ConversionError> {
let (thinking_mode, reasoning_effort) = self.resolve_thinking(options);
let mut parsing_options = self.parsing_options(thinking_mode);
let inference_options = self.inference_options()?;
validate_api_params(
self.stop.as_ref(),
self.stream,
self.stream_options.as_ref(),
)?;
let (tools, tool_choice) = convert_tools(self.tools, self.tool_choice, thinking_mode)?;
parsing_options.parse_tool_calls = !tools.is_empty() && tool_choice != ToolChoice::None;
parsing_options.tool_call_initial_stage = tool_choice == ToolChoice::Required;
let messages = transform_messages(self.messages)?;
validate_tool_text(&tools, &messages)?;
let response_format = convert_response_format(&messages, self.response_format)?;
parsing_options.parse_json_output = matches!(response_format, ResponseFormat::JsonObject);
Ok(ConversationRequest {
conversation: Conversation {
messages,
thinking_mode,
tools,
tool_choice,
reasoning_effort,
response_format,
},
inference_options,
parsing_options,
model: Some(self.model),
stream: self.stream.unwrap_or(false),
})
}
fn chunk_generator(
request: &ConversationRequest,
id: String,
model: String,
) -> ChatCompletionChunkGenerator {
ChatCompletionChunkGenerator::new(
id,
model,
!request.stream,
request.conversation.thinking_mode,
)
}
}
impl ChatCompletionRequest {
fn resolve_thinking(&self, options: ConversionOptions) -> (bool, Option<ReasoningEffort>) {
let effort = self
.reasoning_effort
.and_then(ChatCompletionReasoningEffort::to_reasoning_effort);
let thinking = match self.thinking.as_ref().map(|thinking| thinking.kind) {
Some(ChatCompletionThinkingType::Enabled) => true,
Some(ChatCompletionThinkingType::Disabled) => false,
None => match self.reasoning_effort {
Some(ChatCompletionReasoningEffort::None) => false,
Some(_) => true,
None => options.default_thinking_mode,
},
};
(
thinking,
thinking.then_some(effort.unwrap_or(ReasoningEffort::High)),
)
}
fn inference_options(&self) -> Result<InferenceOptions, ConversionError> {
if self.n.is_some_and(|value| value != 1) {
return Err(ConversionError::bad_request(
"Invalid n value (currently only n = 1 is supported)",
));
}
for (name, value) in [
("frequency_penalty", self.frequency_penalty),
("presence_penalty", self.presence_penalty),
] {
if value.is_some_and(|value| !(-2.0..=2.0).contains(&value)) {
return Err(ConversionError::bad_request(format!(
"{name} must be in [-2, 2]"
)));
}
}
validate_max_tokens(self.max_tokens, "max_tokens")?;
validate_sampling(self.temperature, self.top_p)?;
if self.seed.is_some_and(|value| value >= 1u64 << 63) {
return Err(ConversionError::bad_request("seed must be in [0, 2^63)"));
}
Ok(InferenceOptions {
max_tokens: self.max_tokens,
temperature: self.temperature,
top_p: self.top_p,
..InferenceOptions::default()
})
}
fn parsing_options(&self, thinking: bool) -> ParsingOptions {
ParsingOptions {
reasoning_initial_stage: thinking.then_some(ReasoningStage::Start),
stop_sequences: self
.stop
.clone()
.map(Vec::<String>::from)
.unwrap_or_default(),
..ParsingOptions::default()
}
}
/// Return the setting for usage inclusion in response chunks.
///
/// Read this before consuming the request with `convert`, then pass the value
/// to `ChatCompletionChunkGenerator::with_include_usage`.
pub fn include_usage(&self) -> bool {
self.stream != Some(true)
|| self
.stream_options
.as_ref()
.and_then(|options| options.include_usage)
.unwrap_or(false)
}
}
fn convert_tools(
tools: Option<Vec<ChatCompletionTool>>,
choice: Option<ChatCompletionToolChoiceOption>,
thinking: bool,
) -> Result<(Vec<ToolDefinition>, ToolChoice), ConversionError> {
let tools = tools.unwrap_or_default();
for (index, tool) in tools.iter().enumerate() {
validate_tool_name(&tool.function.name, &format!("tools.{index}.function.name"))?;
}
if matches!(
choice,
Some(ChatCompletionToolChoiceOption::Mode(
ChatCompletionToolChoiceMode::None
))
) {
return Ok((Vec::new(), ToolChoice::None));
}
if tools.is_empty() {
return Ok((Vec::new(), ToolChoice::Auto));
}
let mut names = HashSet::new();
let mut definitions = Vec::new();
for (index, tool) in tools.into_iter().enumerate() {
let tool = tool.function;
if !names.insert(tool.name.clone()) {
return Err(ConversionError::bad_request("Tool names must be unique"));
}
if let Some(parameters) = &tool.parameters {
validate_tool_parameters(parameters, &format!("tools.{index}.function.parameters"))?;
}
definitions.push(ToolDefinition {
name: tool.name,
description: tool.description,
parameters: tool.parameters.unwrap_or_else(|| serde_json::json!({})),
strict: tool.strict,
});
}
let choice = match choice {
Some(ChatCompletionToolChoiceOption::Named(named)) => {
let name = named.function.name;
if !names.contains(&name) {
return Err(ConversionError::bad_request(format!(
"tool_choice: no tool named '{name}' was specified"
)));
}
definitions.retain(|tool| tool.name == name);
ToolChoice::Required
}
Some(ChatCompletionToolChoiceOption::Mode(ChatCompletionToolChoiceMode::Required)) => {
ToolChoice::Required
}
_ => ToolChoice::Auto,
};
if choice == ToolChoice::Required && thinking {
return Err(ConversionError::bad_request(
"Thinking mode does not support this tool_choice",
));
}
Ok((definitions, choice))
}
impl ChatCompletionReasoningEffort {
fn to_reasoning_effort(self) -> Option<ReasoningEffort> {
match self {
Self::None => None,
Self::Minimal | Self::Low => Some(ReasoningEffort::Low),
Self::Medium | Self::High => Some(ReasoningEffort::High),
Self::Xhigh => Some(ReasoningEffort::Xhigh),
Self::Max => Some(ReasoningEffort::Max),
}
}
}
impl ChatCompletionRequestContent {
/// Return whether text carries [`IMAGE_SPECIAL_TOKEN`].
fn contains_image_special_token(&self) -> bool {
match self {
Self::String(content) => content.contains(IMAGE_SPECIAL_TOKEN),
Self::List(blocks) => blocks.iter().any(|block| match block {
ChatCompletionRequestContentBlock::Text { text } => {
text.contains(IMAGE_SPECIAL_TOKEN)
}
_ => false,
}),
}
}
/// Convert text and image blocks. Each image block contributes one
/// [`IMAGE_SPECIAL_TOKEN`] placeholder in the returned text.
fn into_content(self, path: &str) -> Result<(String, Vec<ImageSource>), ConversionError> {
match self {
Self::String(content) => Ok((content, Vec::new())),
Self::List(blocks) => {
let mut texts = Vec::new();
let mut image_sources = Vec::new();
for (block_index, block) in blocks.into_iter().enumerate() {
match block {
ChatCompletionRequestContentBlock::Text { text } => texts.push(text),
ChatCompletionRequestContentBlock::ImageUrl { image_url } => {
let block_path = format!("{path}.content[{block_index}].image_url.url");
image_sources.push(image_url_source(
image_url.url,
image_url.detail,
&block_path,
)?);
texts.push(IMAGE_SPECIAL_TOKEN.to_string());
}
ChatCompletionRequestContentBlock::File {
file_id,
file_data,
filename: _,
} => {
if let Some(_file_id) = file_id {
return Err(file_id_unsupported(&format!(
"{path}.content[{block_index}]"
)));
}
let Some(file_data) = file_data else {
return Err(ConversionError::bad_request(format!(
"{path}.content[{block_index}]: file must have file_data"
)));
};
image_sources.push(data_url_image_source(file_data, None));
texts.push(IMAGE_SPECIAL_TOKEN.to_string());
}
}
}
Ok((texts.join("\n\n"), image_sources))
}
}
}
/// Convert text blocks and reject image blocks.
fn into_text(self, path: &str, role: &str) -> Result<String, ConversionError> {
let (content, image_sources) = self.into_content(path)?;
if !image_sources.is_empty() {
return Err(image_not_allowed(role));
}
Ok(content)
}
}
/// Return whether any message carries [`IMAGE_SPECIAL_TOKEN`] in text that
/// reaches the prompt.
///
/// Message content and assistant reasoning content are checked. Tool call names
/// and arguments are checked after conversion by [`validate_tool_text`].
fn messages_contain_image_special_token(messages: &[ChatCompletionRequestMessage]) -> bool {
messages.iter().any(|message| match message {
ChatCompletionRequestMessage::System { content }
| ChatCompletionRequestMessage::User { content }
| ChatCompletionRequestMessage::Tool { content, .. } => {
content.contains_image_special_token()
}
ChatCompletionRequestMessage::Assistant {
content,
reasoning_content,
..
} => {
content
.as_ref()
.is_some_and(ChatCompletionRequestContent::contains_image_special_token)
|| reasoning_content
.as_ref()
.is_some_and(|text| text.contains(IMAGE_SPECIAL_TOKEN))
}
ChatCompletionRequestMessage::LatestReminder { content } => {
content.contains(IMAGE_SPECIAL_TOKEN)
}
})
}
fn transform_messages(
messages: Vec<ChatCompletionRequestMessage>,
) -> Result<Vec<InputMessage>, ConversionError> {
if messages.is_empty() {
return Err(ConversionError::bad_request("Empty input messages"));
}
if messages_contain_image_special_token(&messages) {
return Err(image_special_token_not_allowed());
}
let mut result = Vec::new();
let mut message_iter = messages.into_iter().enumerate();
while let Some((index, message)) = message_iter.next() {
let path = format!("messages[{index}]");
match message {
ChatCompletionRequestMessage::System { content } => {
let content = content.into_text(&path, "system")?;
if !content.is_empty() {
result.push(InputMessage::System { content });
}
}
ChatCompletionRequestMessage::User { content } => {
let (content, image_sources) = content.into_content(&path)?;
result.push(InputMessage::User {
content,
image_sources,
});
}
ChatCompletionRequestMessage::Assistant {
content,
reasoning_content,
tool_calls,
} => {
if content.is_none() && tool_calls.is_none() {
return Err(ConversionError::bad_request(
"Invalid assistant message: content or tool_calls must be set",
));
}
let content = content
.map(|content| content.into_text(&path, "assistant"))
.transpose()?
.unwrap_or_default();
let mut tool_results = Vec::new();
let mut input_tool_calls = None;
if let Some(tool_calls) = &tool_calls {
let count = tool_calls.len();
if count == 0 {
return Err(ConversionError::bad_request(format!(
"Invalid '{path}.tool_calls': empty array. Expected an array with minimum length 1, but got an empty \
array instead."
)));
}
let mut tool_ids: HashSet<String> = HashSet::new();
for tool_call in tool_calls {
if tool_ids.contains(&tool_call.id) {
return Err(ConversionError::bad_request(format!(
"Duplicate value for 'tool_call_id' of {} in message[{}]",
tool_call.id, index
)));
}
tool_ids.insert(tool_call.id.clone());
}
// The assistant message is followed by one tool message per call.
let mut checked_tool_call_ids: HashSet<String> = HashSet::new();
for _ in 0..count {
let Some((
tool_result_index,
ChatCompletionRequestMessage::Tool {
content,
tool_call_id,
},
)) = message_iter.next()
else {
return Err(ConversionError::bad_request(
"An assistant message with 'tool_calls' must be followed by tool messages responding to each \
'tool_call_id'. (insufficient tool messages following tool_calls message)",
));
};
if !tool_ids.contains(&tool_call_id) {
return Err(ConversionError::bad_request(format!(
"An assistant message with 'tool_calls' must be followed by tool messages responding to each \
'tool_call_id'. Unexpected tool_call_id: {tool_call_id}"
)));
}
if checked_tool_call_ids.contains(&tool_call_id) {
return Err(ConversionError::bad_request(format!(
"Duplicate value for 'tool_call_id' of {tool_call_id} in message[{tool_result_index}]",
)));
}
checked_tool_call_ids.insert(tool_call_id.clone());
let (content, image_sources) = content
.into_content(&format!("messages[{tool_result_index}].content"))?;
tool_results.push(InputMessage::Tool {
content,
image_sources,
tool_call_id,
});
}
input_tool_calls = Some(
tool_calls
.iter()
.map(|tool_call| ToolCall {
id: tool_call.id.clone(),
name: tool_call.function.name.clone(),
arguments: tool_call.function.arguments.clone(),
})
.collect(),
);
}
result.push(InputMessage::Assistant {
content,
reasoning_content,
tool_calls: input_tool_calls,
});
result.extend(tool_results);
}
ChatCompletionRequestMessage::Tool { .. } => {
// Tool results must follow an assistant tool call.
return Err(ConversionError::bad_request(
"Messages with role 'tool' must be a response to a preceding message with 'tool_calls'",
));
}
ChatCompletionRequestMessage::LatestReminder { content } => {
result.push(InputMessage::LatestReminder { content });
}
}
}
Ok(result)
}
fn convert_response_format(
messages: &[InputMessage],
response_format: Option<ChatCompletionResponseFormat>,
) -> Result<ResponseFormat, ConversionError> {
match response_format {
Some(ChatCompletionResponseFormat::JsonObject) => {
if !contains_json_instruction(messages) {
return Err(ConversionError::bad_request(
"Prompt must contain the word 'json' in some form to use 'response_format' of type 'json_object'.",
));
}
Ok(ResponseFormat::JsonObject)
}
Some(ChatCompletionResponseFormat::Regex { .. }) => Err(ConversionError::bad_request(
"response_format: only text and json_object are supported",
)),
Some(ChatCompletionResponseFormat::Text | ChatCompletionResponseFormat::JsonSchema)
| None => Ok(ResponseFormat::Text),
}
}
/// Validate stop sequences and streaming options.
fn validate_api_params(
stop: Option<&ChatCompletionStopSequences>,
stream: Option<bool>,
stream_options: Option<&ChatCompletionStreamOptions>,
) -> Result<(), ConversionError> {
if let Some(ChatCompletionStopSequences::Array(stop_sequences)) = stop
&& stop_sequences.len() > MAX_STOP_SEQUENCES
{
return Err(ConversionError::bad_request(format!(
"ChatCompletionStopSequences string array too long: {}",
stop_sequences.len()
)));
}
if stream != Some(true) && stream_options.is_some() {
return Err(ConversionError::bad_request(
"stream_options should be set along with stream = true",
));
}
Ok(())
}

24
deepseek-recipe/src/protocol/openai/chat_completion/request/mod.rs Normal file
View File

@ -0,0 +1,24 @@
//! Chat Completions request deserialization and conversion.
//!
//! [`ChatCompletionRequest`] accepts messages, sampling parameters, client
//! tools, and JSON object output. An explicit `thinking` setting overrides
//! `reasoning_effort`, which overrides the caller's thinking default. Sampling
//! values are validated and passed through to the caller.
//!
//! Text, images, and historical tool calls are supported. A tool result must
//! follow the assistant message that requested it, and tool names must be
//! unique. Named or required tool choices require thinking to be disabled when
//! tools are present. JSON Schema output, `logprobs`, and `top_logprobs` are
//! accepted and ignored. Function-tool `strict` flags are passed through for the
//! caller to enforce. Model resolution and tool execution belong to the caller.
//!
//! Input text must not spell out the image placeholder. Every placeholder in a
//! prompt corresponds to one image source, and the adapter inserts them for
//! image blocks only, so message content, assistant reasoning content, tool
//! definitions, and historical tool calls that carry the placeholder spelling
//! are rejected.
pub use schema::*;
mod convert;
mod schema;

235
deepseek-recipe/src/protocol/openai/chat_completion/request/schema.rs Normal file
View File

@ -0,0 +1,235 @@
use serde::Deserialize;
use deepseek_recipe_core::multimodal::ImageDetail;
/// A Chat Completions request. Unknown fields are ignored.
///
/// Model resolution and inference execution belong to the caller. Client tools,
/// historical tool calls, and image blocks are supported. Server tools are
/// rejected by this adapter. Constrained decoding and model-specific sampling
/// belong to the caller. JSON Schema output, `logprobs`, and `top_logprobs` are
/// accepted and ignored.
#[derive(Debug, Deserialize)]
pub struct ChatCompletionRequest {
pub messages: Vec<ChatCompletionRequestMessage>,
pub model: String,
/// Validated in [-2, 2], then omitted from the converted request. Extract
/// before conversion if the backend supports it.
pub frequency_penalty: Option<f32>,
pub max_tokens: Option<u32>,
/// Only one completion per request is supported.
pub n: Option<u8>,
/// Validated in [-2, 2], then omitted from the converted request. Extract
/// before conversion if the backend supports it.
pub presence_penalty: Option<f32>,
/// Validated in [0, 2^63), then omitted from the converted request. Extract
/// before conversion if the backend supports it.
pub seed: Option<u64>,
pub stop: Option<ChatCompletionStopSequences>,
pub stream: Option<bool>,
pub stream_options: Option<ChatCompletionStreamOptions>,
pub temperature: Option<f32>,
pub top_p: Option<f32>,
/// Text or JSON object output. JSON Schema settings are accepted and ignored.
pub response_format: Option<ChatCompletionResponseFormat>,
pub tools: Option<Vec<ChatCompletionTool>>,
/// With tools present, required and named choices require thinking off.
pub tool_choice: Option<ChatCompletionToolChoiceOption>,
pub reasoning_effort: Option<ChatCompletionReasoningEffort>,
/// Explicit thinking overrides reasoning effort and the conversion default.
pub thinking: Option<ChatCompletionThinking>,
/// Deserialized for the caller; conversion leaves this setting unused.
pub parallel_tool_calls: Option<bool>,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "role", rename_all = "snake_case")]
pub enum ChatCompletionRequestMessage {
System {
content: ChatCompletionRequestContent,
},
User {
content: ChatCompletionRequestContent,
},
Assistant {
content: Option<ChatCompletionRequestContent>,
reasoning_content: Option<String>,
tool_calls: Option<Vec<ChatCompletionRequestToolCall>>,
},
Tool {
content: ChatCompletionRequestContent,
tool_call_id: String,
},
LatestReminder {
content: String,
},
}
#[derive(Debug, Deserialize)]
pub struct ChatCompletionRequestToolCall {
pub id: String,
pub function: ChatCompletionRequestFunctionCall,
}
#[derive(Debug, Deserialize)]
pub struct ChatCompletionRequestFunctionCall {
pub name: String,
pub arguments: String,
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
pub enum ChatCompletionRequestContent {
String(String),
List(Vec<ChatCompletionRequestContentBlock>),
}
#[derive(Debug, Deserialize)]
pub struct ChatCompletionImageUrl {
/// HTTP URL or base64 data URL. A value that does not start with `http`
/// must carry a `base64` body.
pub url: String,
#[serde(default)]
pub detail: Option<ImageDetail>,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ChatCompletionRequestContentBlock {
Text {
text: String,
},
ImageUrl {
image_url: ChatCompletionImageUrl,
},
/// One image carried as a data URL. The resolver accepts base64 and
/// percent-encoded bodies. `file_id` is rejected during conversion; the
/// filename is accepted and ignored.
File {
file_id: Option<String>,
file_data: Option<String>,
filename: Option<String>,
},
}
/// One stop sequence or a list of stop sequences.
#[derive(Debug, Clone, Deserialize)]
#[serde(untagged)]
pub enum ChatCompletionStopSequences {
Array(Vec<String>),
String(String),
}
impl From<ChatCompletionStopSequences> for Vec<String> {
fn from(value: ChatCompletionStopSequences) -> Self {
match value {
ChatCompletionStopSequences::Array(values) => values,
ChatCompletionStopSequences::String(value) => vec![value],
}
}
}
#[derive(Debug, Deserialize)]
pub struct ChatCompletionStreamOptions {
/// Include a null usage field before the final chunk. The final chunk
/// carries usage regardless of this setting.
pub include_usage: Option<bool>,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ChatCompletionResponseFormat {
JsonObject,
/// Accepted and ignored, including nested schema fields.
JsonSchema,
Regex {
regex: String,
},
Text,
}
#[derive(Debug, Deserialize)]
pub struct ChatCompletionTool {
pub r#type: ChatCompletionToolType,
pub function: ChatCompletionFunctionDefinition,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ChatCompletionToolType {
Function,
}
#[derive(Debug, Deserialize)]
pub struct ChatCompletionFunctionDefinition {
pub name: String,
pub description: Option<String>,
/// A JSON Schema of type object. Omission renders an empty schema.
pub parameters: Option<serde_json::Value>,
/// Passed through for the caller to enforce during inference.
pub strict: Option<bool>,
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
pub enum ChatCompletionToolChoiceOption {
/// One of `"none"`, `"auto"`, or `"required"`.
Mode(ChatCompletionToolChoiceMode),
Named(ChatCompletionNamedToolChoice),
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ChatCompletionToolChoiceMode {
None,
Auto,
Required,
}
#[derive(Debug, Deserialize)]
pub struct ChatCompletionNamedToolChoice {
pub r#type: ChatCompletionToolType,
pub function: ChatCompletionToolChoiceFunction,
}
#[derive(Debug, Deserialize)]
pub struct ChatCompletionToolChoiceFunction {
pub name: String,
}
/// Optional thinking control for DeepSeek-compatible requests.
#[derive(Debug, Deserialize)]
pub struct ChatCompletionThinking {
#[serde(rename = "type")]
pub kind: ChatCompletionThinkingType,
/// Deserialized for the caller; conversion leaves this setting unused.
pub budget_tokens: Option<u64>,
}
#[derive(Debug, Clone, Copy, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ChatCompletionThinkingType {
#[serde(alias = "adaptive")]
Enabled,
Disabled,
}
/// Reasoning effort accepted by this adapter. Unless overridden by `thinking`,
/// `none` disables thinking and other values enable it. Enabled thinking uses
/// high effort by default. Supplied values map to the shared reasoning levels.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ChatCompletionReasoningEffort {
None,
Minimal,
Low,
Medium,
High,
Xhigh,
Max,
}

169
deepseek-recipe/src/protocol/openai/chat_completion/response/chunk_generator.rs Normal file
View File

@ -0,0 +1,169 @@
use std::time::{SystemTime, UNIX_EPOCH};
use crate::stream::{ChunkGenerator, OutputChunk, PromptUsage};
use super::schema::{
CHAT_COMPLETION_CHUNK_OBJECT, ChatCompletionChunk, ChatCompletionChunkChoice,
ChatCompletionFinishReason, ChatCompletionFunctionArgumentsDelta, ChatCompletionMessageDelta,
ChatCompletionResponseFunctionCall, ChatCompletionResponseToolCall, ChatCompletionRole,
ChatCompletionToolCallDelta, ChatCompletionToolCallType, ChatCompletionUsage,
};
/// Converts parsed output into Chat Completions streaming chunks.
///
/// The caller supplies an ordered stream for one inference. Tool argument
/// deltas follow the tool call whose arguments they extend.
#[derive(Debug)]
pub struct ChatCompletionChunkGenerator {
id: String,
model: String,
created: u64,
thinking_mode: bool,
include_usage: bool,
tool_call_index: u32,
prompt_usage: PromptUsage,
system_fingerprint: Option<String>,
}
impl ChatCompletionChunkGenerator {
/// The caller supplies a unique response ID and public model name. Tool call
/// IDs combine the response ID and per-response tool index.
pub fn new(id: String, model: String, include_usage: bool, thinking_mode: bool) -> Self {
Self {
id,
model,
created: SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|elapsed| elapsed.as_secs())
.unwrap_or(0),
thinking_mode,
include_usage,
tool_call_index: 0,
prompt_usage: PromptUsage::default(),
system_fingerprint: None,
}
}
/// Apply usage settings extracted from the original Chat Completions request.
pub fn with_include_usage(mut self, include_usage: bool) -> Self {
self.include_usage = include_usage;
self
}
fn new_chunk(
&self,
delta: ChatCompletionMessageDelta,
finish_reason: Option<ChatCompletionFinishReason>,
usage: Option<ChatCompletionUsage>,
) -> Vec<ChatCompletionChunk> {
// Include null usage before the final snapshot when requested. Otherwise the
// final chunk includes usage.
let usage = if self.include_usage {
Some(usage)
} else {
usage.map(Some)
};
vec![ChatCompletionChunk {
id: self.id.clone(),
object: CHAT_COMPLETION_CHUNK_OBJECT,
created: self.created,
model: self.model.clone(),
system_fingerprint: self.system_fingerprint.clone(),
choices: vec![ChatCompletionChunkChoice {
index: 0,
delta,
logprobs: None,
finish_reason,
}],
usage,
}]
}
}
impl ChunkGenerator for ChatCompletionChunkGenerator {
type Chunk = ChatCompletionChunk;
async fn generate(&mut self, chunk: OutputChunk) -> Vec<Self::Chunk> {
match chunk {
OutputChunk::Start {
system_fingerprint,
usage,
} => {
self.prompt_usage = usage;
self.system_fingerprint = system_fingerprint;
let delta = ChatCompletionMessageDelta {
role: Some(ChatCompletionRole::Assistant),
content: if self.thinking_mode {
Some(None)
} else {
Some(Some(String::new()))
},
reasoning_content: self.thinking_mode.then(|| Some(String::new())),
tool_calls: None,
};
self.new_chunk(delta, None, None)
}
OutputChunk::Raw { content } => {
let delta = ChatCompletionMessageDelta {
content: Some(Some(content)),
reasoning_content: self.thinking_mode.then_some(None),
..Default::default()
};
self.new_chunk(delta, None, None)
}
OutputChunk::Reasoning { content } => {
let delta = ChatCompletionMessageDelta {
content: Some(None),
reasoning_content: Some(Some(content)),
..Default::default()
};
self.new_chunk(delta, None, None)
}
OutputChunk::ToolCallBegin => vec![],
OutputChunk::ToolCall {
tool_name,
arguments,
} => {
let index = self.tool_call_index;
self.tool_call_index += 1;
let delta = ChatCompletionMessageDelta {
tool_calls: Some(vec![ChatCompletionToolCallDelta::Start {
index,
call: ChatCompletionResponseToolCall {
id: format!("call_{}_{index}", self.id),
r#type: ChatCompletionToolCallType::Function,
function: ChatCompletionResponseFunctionCall {
name: tool_name,
arguments,
},
},
}]),
..Default::default()
};
self.new_chunk(delta, None, None)
}
OutputChunk::ToolArgumentsDelta { content } => {
let delta = ChatCompletionMessageDelta {
tool_calls: Some(vec![ChatCompletionToolCallDelta::Arguments {
index: self.tool_call_index - 1,
function: ChatCompletionFunctionArgumentsDelta { arguments: content },
}]),
..Default::default()
};
self.new_chunk(delta, None, None)
}
OutputChunk::Finish { reason, usage, .. } => {
let delta = ChatCompletionMessageDelta {
content: Some(Some(String::new())),
reasoning_content: self.thinking_mode.then_some(None),
..Default::default()
};
self.new_chunk(
delta,
Some(reason.into()),
Some(ChatCompletionUsage::from((self.prompt_usage, usage))),
)
}
}
}
}

5
deepseek-recipe/src/protocol/openai/chat_completion/response/mod.rs Normal file
View File

@ -0,0 +1,5 @@
pub use chunk_generator::*;
pub use schema::*;
mod chunk_generator;
mod schema;

309
deepseek-recipe/src/protocol/openai/chat_completion/response/schema.rs Normal file
View File

@ -0,0 +1,309 @@
use serde::Serialize;
use crate::response::ProtocolResponse;
use crate::stream::{CompletionUsage, FinishReason, PromptUsage};
use crate::util::append_delta::AppendDelta;
use super::chunk_generator::ChatCompletionChunkGenerator;
const CHAT_COMPLETION_OBJECT: &str = "chat.completion";
pub(crate) const CHAT_COMPLETION_CHUNK_OBJECT: &str = "chat.completion.chunk";
/// A complete response, also usable as an accumulator for generated chunks.
/// Chunks are accumulated in inference order. Tool argument deltas extend the
/// tool call identified by their index.
#[derive(Debug, Serialize)]
pub struct ChatCompletionResponse {
pub id: String,
pub object: &'static str,
pub created: u64,
pub model: String,
pub system_fingerprint: Option<String>,
pub choices: Vec<ChatCompletionChoice>,
#[serde(skip_serializing_if = "Option::is_none")]
pub usage: Option<ChatCompletionUsage>,
}
#[derive(Debug, Serialize)]
pub struct ChatCompletionChoice {
pub index: u32,
pub message: ChatCompletionResponseMessage,
pub logprobs: Option<ChatCompletionLogprobs>,
pub finish_reason: Option<ChatCompletionFinishReason>,
}
#[derive(Debug, Serialize)]
pub struct ChatCompletionChunk {
pub id: String,
pub object: &'static str,
pub created: u64,
pub model: String,
pub system_fingerprint: Option<String>,
pub choices: Vec<ChatCompletionChunkChoice>,
#[serde(skip_serializing_if = "Option::is_none")]
pub usage: Option<Option<ChatCompletionUsage>>,
}
#[derive(Debug, Serialize)]
pub struct ChatCompletionChunkChoice {
pub index: u32,
pub delta: ChatCompletionMessageDelta,
pub logprobs: Option<ChatCompletionLogprobs>,
pub finish_reason: Option<ChatCompletionFinishReason>,
}
/// A complete assistant message accumulated from response deltas.
#[derive(Debug, Serialize)]
pub struct ChatCompletionResponseMessage {
pub role: ChatCompletionRole,
pub content: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub reasoning_content: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tool_calls: Option<Vec<ChatCompletionResponseToolCall>>,
}
/// An incremental update to an assistant message.
#[derive(Debug, Default, Serialize)]
pub struct ChatCompletionMessageDelta {
#[serde(skip_serializing_if = "Option::is_none")]
pub role: Option<ChatCompletionRole>,
/// `None` omits the field; `Some(None)` serializes an explicit null.
#[serde(skip_serializing_if = "Option::is_none")]
pub content: Option<Option<String>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub reasoning_content: Option<Option<String>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tool_calls: Option<Vec<ChatCompletionToolCallDelta>>,
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ChatCompletionRole {
Assistant,
}
/// A complete tool call in an assistant response message.
#[derive(Debug, Serialize)]
pub struct ChatCompletionResponseToolCall {
pub id: String,
pub r#type: ChatCompletionToolCallType,
pub function: ChatCompletionResponseFunctionCall,
}
#[derive(Debug, Serialize)]
pub struct ChatCompletionResponseFunctionCall {
pub name: String,
pub arguments: String,
}
/// An initial tool call or an update to its arguments.
#[derive(Debug, Serialize)]
#[serde(untagged)]
pub enum ChatCompletionToolCallDelta {
Start {
index: u32,
#[serde(flatten)]
call: ChatCompletionResponseToolCall,
},
Arguments {
index: u32,
function: ChatCompletionFunctionArgumentsDelta,
},
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ChatCompletionToolCallType {
Function,
}
#[derive(Debug, Serialize)]
pub struct ChatCompletionFunctionArgumentsDelta {
pub arguments: String,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ChatCompletionFinishReason {
Stop,
Length,
ContentFilter,
InsufficientSystemResource,
ToolCalls,
Aborted,
}
impl From<FinishReason> for ChatCompletionFinishReason {
fn from(value: FinishReason) -> Self {
match value {
FinishReason::Stop | FinishReason::StopSequence | FinishReason::EndOfStream => {
Self::Stop
}
FinishReason::ToolCalls => Self::ToolCalls,
FinishReason::ContentFilter => Self::ContentFilter,
FinishReason::Length => Self::Length,
}
}
}
/// Log probabilities are not supplied by the current inference contract.
#[derive(Debug, Serialize)]
pub struct ChatCompletionLogprobs {
#[serde(skip_serializing_if = "Option::is_none")]
pub content: Option<Vec<ChatCompletionTokenLogprob>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub reasoning_content: Option<Vec<ChatCompletionTokenLogprob>>,
}
impl AppendDelta<ChatCompletionLogprobs> for ChatCompletionLogprobs {
fn append(&mut self, delta: ChatCompletionLogprobs) {
self.content.append(delta.content);
self.reasoning_content.append(delta.reasoning_content);
}
}
#[derive(Debug, Serialize)]
pub struct ChatCompletionTokenLogprob {
pub token: String,
pub logprob: f32,
pub bytes: Option<Vec<u8>>,
pub top_logprobs: Vec<ChatCompletionTopLogprob>,
}
#[derive(Debug, Serialize)]
pub struct ChatCompletionTopLogprob {
pub token: String,
pub logprob: f32,
pub bytes: Option<Vec<u8>>,
}
#[derive(Debug, Serialize)]
pub struct ChatCompletionUsage {
pub prompt_tokens: usize,
pub completion_tokens: usize,
pub total_tokens: usize,
pub prompt_tokens_details: ChatCompletionInputTokenUsage,
#[serde(skip_serializing_if = "Option::is_none")]
pub completion_tokens_details: Option<ChatCompletionOutputTokenUsage>,
pub prompt_cache_hit_tokens: usize,
pub prompt_cache_miss_tokens: usize,
}
#[derive(Debug, Serialize)]
pub struct ChatCompletionInputTokenUsage {
pub cached_tokens: usize,
}
#[derive(Debug, Serialize)]
pub struct ChatCompletionOutputTokenUsage {
pub reasoning_tokens: usize,
}
impl From<(PromptUsage, CompletionUsage)> for ChatCompletionUsage {
fn from((prompt, completion): (PromptUsage, CompletionUsage)) -> Self {
Self {
prompt_tokens: prompt.prompt_tokens,
completion_tokens: completion.completion_tokens,
total_tokens: prompt.prompt_tokens + completion.completion_tokens,
prompt_tokens_details: ChatCompletionInputTokenUsage {
cached_tokens: prompt.prompt_cache_hit_tokens,
},
completion_tokens_details: None,
prompt_cache_hit_tokens: prompt.prompt_cache_hit_tokens,
prompt_cache_miss_tokens: prompt
.prompt_tokens
.saturating_sub(prompt.prompt_cache_hit_tokens),
}
}
}
impl ProtocolResponse for ChatCompletionResponse {
type ChunkGenerator = ChatCompletionChunkGenerator;
fn new(
id: String,
model: String,
created: u64,
_prompt_tokens: usize,
_prompt_cache_hit_tokens: usize,
) -> Self {
Self {
id,
object: CHAT_COMPLETION_OBJECT,
created,
model,
system_fingerprint: None,
choices: vec![],
usage: None,
}
}
fn done_message() -> Option<String> {
Some("[DONE]".to_string())
}
}
impl AppendDelta<ChatCompletionChunk> for ChatCompletionResponse {
fn append(&mut self, chunk: ChatCompletionChunk) {
if let Some(Some(usage)) = chunk.usage {
self.usage = Some(usage);
}
if self.system_fingerprint.is_none() {
self.system_fingerprint = chunk.system_fingerprint;
}
for chunk_choice in chunk.choices {
let choice = match self
.choices
.iter_mut()
.find(|choice| choice.index == chunk_choice.index)
{
Some(choice) => choice,
None => {
self.choices.push(ChatCompletionChoice {
index: chunk_choice.index,
message: ChatCompletionResponseMessage {
role: ChatCompletionRole::Assistant,
content: None,
reasoning_content: None,
tool_calls: None,
},
logprobs: None,
finish_reason: None,
});
self.choices.last_mut().expect("the choice was inserted")
}
};
choice.message.append(chunk_choice.delta);
choice.logprobs.append(chunk_choice.logprobs);
if chunk_choice.finish_reason.is_some() {
choice.finish_reason = chunk_choice.finish_reason;
}
}
}
}
impl AppendDelta<ChatCompletionMessageDelta> for ChatCompletionResponseMessage {
fn append(&mut self, delta: ChatCompletionMessageDelta) {
if let Some(role) = delta.role {
self.role = role;
}
self.content.append(delta.content.flatten());
self.reasoning_content
.append(delta.reasoning_content.flatten());
if let Some(tool_deltas) = delta.tool_calls {
let tool_calls = self.tool_calls.get_or_insert_with(Vec::new);
for tool_delta in tool_deltas {
match tool_delta {
ChatCompletionToolCallDelta::Start { call, .. } => tool_calls.push(call),
ChatCompletionToolCallDelta::Arguments { index, function } => {
if let Some(call) = tool_calls.get_mut(index as usize) {
call.function.arguments.push_str(&function.arguments);
}
}
}
}
}
}
}

5
deepseek-recipe/src/protocol/openai/mod.rs Normal file
View File

@ -0,0 +1,5 @@
pub use chat_completion::request::ChatCompletionRequest;
pub use responses::request::ResponsesRequest;
pub mod chat_completion;
pub mod responses;

2
deepseek-recipe/src/protocol/openai/responses/mod.rs Normal file
View File

@ -0,0 +1,2 @@
pub mod request;
pub mod response;

667
deepseek-recipe/src/protocol/openai/responses/request/convert.rs Normal file
View File

@ -0,0 +1,667 @@
use std::collections::{HashMap, HashSet};
use deepseek_recipe_core::conversation::{Conversation, ReasoningEffort, ResponseFormat};
use deepseek_recipe_core::messages::{InputMessage, ToolCall};
use deepseek_recipe_core::multimodal::{IMAGE_SPECIAL_TOKEN, ImageSource};
use deepseek_recipe_core::tools::ToolChoice;
use itertools::Itertools;
use crate::protocol::image::{
UNSUPPORTED_DOCUMENT_PLACEHOLDER, file_id_unsupported, image_not_allowed, image_url_source,
};
use crate::protocol::openai::responses::response::{ResponsesChunkGenerator, ResponsesResponse};
use crate::protocol::validation::{
contains_json_instruction, image_special_token_not_allowed, validate_max_tokens,
validate_sampling, validate_tool_text,
};
use crate::request::{
ConversationRequest, ConversionError, ConversionOptions, InferenceOptions, ProtocolRequest,
};
use crate::stream::state_machine::{ParsingOptions, ReasoningStage};
use super::schema::{
ResponsesContentBlock, ResponsesInput, ResponsesInputImage, ResponsesInputItem,
ResponsesInputMessage, ResponsesMessageContent, ResponsesMessageRole, ResponsesReasoningEffort,
ResponsesReasoningItem, ResponsesRequest, ResponsesTextConfig, ResponsesTextFormat,
ResponsesTool, ResponsesToolCallOutputContent, ResponsesToolCallOutputItem,
ResponsesTypedInputItem,
};
use super::tools::ResponsesToolSet;
impl ResponsesMessageContent {
/// Convert text, substitute document placeholders, and collect images with
/// one [`IMAGE_SPECIAL_TOKEN`] placeholder per image.
fn into_content(self, path: &str) -> Result<(String, Vec<ImageSource>), ConversionError> {
match self {
ResponsesMessageContent::String(s) => Ok((s, Vec::new())),
ResponsesMessageContent::List(blocks) => {
let mut texts = Vec::new();
let mut image_sources = Vec::new();
for (block_index, block) in blocks.into_iter().enumerate() {
let block_path = format!("{path}.content[{block_index}]");
match block {
ResponsesContentBlock::InputText { text }
| ResponsesContentBlock::OutputText { text } => texts.push(text),
ResponsesContentBlock::InputImage(image) => {
image_sources.push(input_image_source(image, &block_path)?);
texts.push(IMAGE_SPECIAL_TOKEN.to_string());
}
ResponsesContentBlock::InputFile {} => {
texts.push(UNSUPPORTED_DOCUMENT_PLACEHOLDER.to_string());
}
ResponsesContentBlock::Unsupported => return Err(unsupported_content()),
}
}
Ok((texts.join("\n\n"), image_sources))
}
}
}
/// Convert text and document placeholders, rejecting images.
fn into_text(self, path: &str, role: &str) -> Result<String, ConversionError> {
let (content, image_sources) = self.into_content(path)?;
if !image_sources.is_empty() {
return Err(image_not_allowed(role));
}
Ok(content)
}
}
impl ResponsesToolCallOutputContent {
/// Convert text and document placeholders, collecting image sources.
fn into_content(self, path: &str) -> Result<(String, Vec<ImageSource>), ConversionError> {
match self {
ResponsesToolCallOutputContent::String(s) => Ok((s, Vec::new())),
ResponsesToolCallOutputContent::List(items) => {
let mut texts = Vec::new();
let mut image_sources = Vec::new();
for (item_index, item) in items.into_iter().enumerate() {
let item_path = format!("{path}[{item_index}]");
match item {
ResponsesToolCallOutputItem::InputText { text } => texts.push(text),
ResponsesToolCallOutputItem::InputImage(image) => {
image_sources.push(input_image_source(image, &item_path)?);
texts.push(IMAGE_SPECIAL_TOKEN.to_string());
}
ResponsesToolCallOutputItem::InputFile {} => {
texts.push(UNSUPPORTED_DOCUMENT_PLACEHOLDER.to_string());
}
ResponsesToolCallOutputItem::Unsupported => {
return Err(unsupported_content());
}
}
}
Ok((texts.join("\n\n"), image_sources))
}
}
}
}
/// Return whether the input or instructions carry [`IMAGE_SPECIAL_TOKEN`].
///
/// Message text, tool output text, and reasoning text are checked. Tool call
/// names and arguments are checked after conversion by [`validate_tool_text`].
fn contains_image_special_token(
input: Option<&ResponsesInput>,
instructions: Option<&str>,
) -> bool {
if instructions.is_some_and(|text| text.contains(IMAGE_SPECIAL_TOKEN)) {
return true;
}
match input {
Some(ResponsesInput::String(content)) => content.contains(IMAGE_SPECIAL_TOKEN),
Some(ResponsesInput::Items(items)) => items.iter().any(item_contains_image_special_token),
None => false,
}
}
/// Return whether one input item carries [`IMAGE_SPECIAL_TOKEN`].
fn item_contains_image_special_token(item: &ResponsesInputItem) -> bool {
match item {
ResponsesInputItem::Message(message) => {
message_content_contains_image_special_token(&message.content)
}
ResponsesInputItem::Typed(typed) => match typed {
ResponsesTypedInputItem::Message(message) => {
message_content_contains_image_special_token(&message.content)
}
ResponsesTypedInputItem::FunctionCallOutput(output) => {
output_content_contains_image_special_token(&output.output)
}
ResponsesTypedInputItem::CustomToolCallOutput(output) => output
.output
.as_ref()
.is_some_and(output_content_contains_image_special_token),
ResponsesTypedInputItem::Reasoning(reasoning) => {
reasoning.content.as_ref().is_some_and(|blocks| {
blocks
.iter()
.any(|block| block.text.contains(IMAGE_SPECIAL_TOKEN))
})
}
_ => false,
},
}
}
/// Return whether one message content carries [`IMAGE_SPECIAL_TOKEN`].
fn message_content_contains_image_special_token(content: &ResponsesMessageContent) -> bool {
match content {
ResponsesMessageContent::String(content) => content.contains(IMAGE_SPECIAL_TOKEN),
ResponsesMessageContent::List(blocks) => blocks.iter().any(|block| match block {
ResponsesContentBlock::InputText { text }
| ResponsesContentBlock::OutputText { text } => text.contains(IMAGE_SPECIAL_TOKEN),
_ => false,
}),
}
}
/// Return whether one tool call output carries [`IMAGE_SPECIAL_TOKEN`].
fn output_content_contains_image_special_token(content: &ResponsesToolCallOutputContent) -> bool {
match content {
ResponsesToolCallOutputContent::String(content) => content.contains(IMAGE_SPECIAL_TOKEN),
ResponsesToolCallOutputContent::List(items) => items.iter().any(|item| match item {
ResponsesToolCallOutputItem::InputText { text } => text.contains(IMAGE_SPECIAL_TOKEN),
_ => false,
}),
}
}
/// Convert one Responses image reference.
fn input_image_source(
image: ResponsesInputImage,
path: &str,
) -> Result<ImageSource, ConversionError> {
let image_url = image.image_url.filter(|url| !url.is_empty());
match (image_url, image.file_id) {
(None, None) => Err(ConversionError::bad_request(format!(
"{path}: input_image must have image_url or file_id"
))),
(Some(_), Some(_)) => Err(ConversionError::bad_request(format!(
"{path}: input_image cannot have both image_url and file_id"
))),
(None, Some(_)) => Err(file_id_unsupported(path)),
(Some(url), None) => image_url_source(url, image.detail, &format!("{path}.image_url")),
}
}
fn check_tool_call_id(
item_idx: usize,
call_id: &str,
pending: &HashSet<String>,
resolved: &HashSet<String>,
) -> Result<(), ConversionError> {
if call_id.is_empty() {
return Err(ConversionError::bad_request(format!(
"Invalid 'input[{item_idx}].call_id': empty string. Expected a string with minimum length 1, but got an empty string \
instead."
)));
}
if pending.contains(call_id) || resolved.contains(call_id) {
return Err(ConversionError::bad_request(format!(
"Duplicate 'call_id': {call_id}."
)));
}
Ok(())
}
fn check_tool_output_id(
item_idx: usize,
call_id: &str,
pending: &mut HashSet<String>,
resolved: &mut HashSet<String>,
) -> Result<(), ConversionError> {
if call_id.is_empty() {
return Err(ConversionError::bad_request(format!(
"Invalid 'input[{item_idx}].call_id': empty string. Expected a string with minimum length 1, but got an empty string \
instead."
)));
}
if !pending.contains(call_id) {
if resolved.contains(call_id) {
return Err(ConversionError::bad_request(format!(
"Duplicate tool output for call_id: {call_id}."
)));
}
return Err(ConversionError::bad_request(format!(
"No tool call found for tool output with call_id {call_id}."
)));
}
pending.remove(call_id);
resolved.insert(call_id.to_string());
Ok(())
}
fn check_pending_calls(pending_call_ids: &HashSet<String>) -> Result<(), ConversionError> {
if let Some(call_id) = pending_call_ids.iter().next() {
return Err(ConversionError::bad_request(format!(
"No tool output found for tool call {call_id}."
)));
}
Ok(())
}
#[derive(Default)]
struct ResponsesToolResult {
content: String,
image_sources: Vec<ImageSource>,
}
#[derive(Default)]
struct ResponsesToolCallGroup {
calls: Vec<ToolCall>,
outputs: HashMap<String, ResponsesToolResult>,
}
impl ResponsesToolCallGroup {
fn is_complete(&self) -> bool {
self.outputs.len() >= self.calls.len()
}
fn append_messages(self, messages: &mut Vec<InputMessage>) {
if self.calls.is_empty() {
return;
}
let mut calls = self.calls;
calls.sort_by(|a, b| a.id.cmp(&b.id));
let mut outputs = self.outputs;
let tool_results: Vec<InputMessage> = calls
.iter()
.filter_map(|call| {
outputs.remove(&call.id).map(|output| InputMessage::Tool {
content: output.content,
image_sources: output.image_sources,
tool_call_id: call.id.clone(),
})
})
.collect();
if !matches!(messages.last(), Some(InputMessage::Assistant { .. })) {
messages.push(InputMessage::Assistant {
content: String::new(),
reasoning_content: None,
tool_calls: None,
});
}
if let Some(InputMessage::Assistant { tool_calls, .. }) = messages.last_mut() {
match tool_calls {
Some(existing) => existing.extend(calls),
None => *tool_calls = Some(calls),
}
}
messages.extend(tool_results);
}
}
fn push_call_to_group(pending_group: &mut Option<ResponsesToolCallGroup>, tool_call: ToolCall) {
pending_group
.get_or_insert_with(ResponsesToolCallGroup::default)
.calls
.push(tool_call);
}
fn push_output_to_group(
pending_group: &mut Option<ResponsesToolCallGroup>,
call_id: String,
output: ResponsesToolResult,
messages: &mut Vec<InputMessage>,
) {
let Some(group) = pending_group.as_mut() else {
messages.push(InputMessage::Tool {
content: output.content,
image_sources: output.image_sources,
tool_call_id: call_id,
});
return;
};
group.outputs.insert(call_id, output);
if group.is_complete() {
append_tool_call_group(pending_group, messages);
}
}
fn append_tool_call_group(
pending_group: &mut Option<ResponsesToolCallGroup>,
messages: &mut Vec<InputMessage>,
) {
if let Some(group) = pending_group.take() {
group.append_messages(messages);
}
}
fn push_message(
msg: ResponsesInputMessage,
item_idx: usize,
messages: &mut Vec<InputMessage>,
) -> Result<(), ConversionError> {
let path = format!("input[{item_idx}]");
match msg.role {
ResponsesMessageRole::System => {
let content = msg.content.into_text(&path, "system")?;
messages.push(InputMessage::System { content });
}
ResponsesMessageRole::User | ResponsesMessageRole::Developer => {
let (content, image_sources) = msg.content.into_content(&path)?;
messages.push(InputMessage::User {
content,
image_sources,
});
}
ResponsesMessageRole::Assistant => {
let content = msg.content.into_text(&path, "assistant")?;
if let Some(InputMessage::Assistant {
content: last_content,
tool_calls: None,
..
}) = messages.last_mut()
{
if !content.is_empty() {
if !last_content.is_empty() {
last_content.push_str("\n\n");
}
last_content.push_str(&content);
}
} else {
messages.push(InputMessage::Assistant {
content,
reasoning_content: None,
tool_calls: None,
});
}
}
}
Ok(())
}
fn push_reasoning(reasoning: ResponsesReasoningItem, messages: &mut Vec<InputMessage>) {
let text = reasoning
.content
.unwrap_or_default()
.into_iter()
.map(|c| c.text)
.join("\n");
if text.is_empty() {
return;
}
if !matches!(
messages.last(),
Some(InputMessage::Assistant { content, .. }) if content.is_empty()
) {
messages.push(InputMessage::Assistant {
content: String::new(),
reasoning_content: None,
tool_calls: None,
});
}
if let Some(InputMessage::Assistant {
reasoning_content, ..
}) = messages.last_mut()
{
match reasoning_content {
Some(existing) => {
existing.push('\n');
existing.push_str(&text);
}
None => *reasoning_content = Some(text),
}
}
}
fn transform_input_items(
items: Vec<ResponsesInputItem>,
messages: &mut Vec<InputMessage>,
tools: &ResponsesToolSet,
) -> Result<(), ConversionError> {
if items.is_empty() {
return Err(ConversionError::bad_request(
"Input items array must not be empty",
));
}
let mut pending_call_ids: HashSet<String> = HashSet::new();
let mut resolved_call_ids: HashSet<String> = HashSet::new();
let mut pending_group: Option<ResponsesToolCallGroup> = None;
let mut historical_names = HashMap::new();
for (item_idx, item) in items.into_iter().enumerate() {
match item {
ResponsesInputItem::Typed(typed) => match typed {
ResponsesTypedInputItem::Message(msg) => {
check_pending_calls(&pending_call_ids)?;
append_tool_call_group(&mut pending_group, messages);
push_message(msg, item_idx, messages)?;
}
ResponsesTypedInputItem::Reasoning(reasoning) => {
check_pending_calls(&pending_call_ids)?;
append_tool_call_group(&mut pending_group, messages);
push_reasoning(reasoning, messages);
}
ResponsesTypedInputItem::FunctionCall(fc) => {
check_tool_call_id(
item_idx,
&fc.call_id,
&pending_call_ids,
&resolved_call_ids,
)?;
pending_call_ids.insert(fc.call_id.clone());
push_call_to_group(
&mut pending_group,
tools.function_call(fc, &mut historical_names),
);
}
ResponsesTypedInputItem::FunctionCallOutput(fco) => {
check_tool_output_id(
item_idx,
&fco.call_id,
&mut pending_call_ids,
&mut resolved_call_ids,
)?;
let (content, image_sources) = fco
.output
.into_content(&format!("input[{item_idx}].output"))?;
push_output_to_group(
&mut pending_group,
fco.call_id,
ResponsesToolResult {
content,
image_sources,
},
messages,
);
}
ResponsesTypedInputItem::CustomToolCall(call) => {
check_tool_call_id(
item_idx,
&call.call_id,
&pending_call_ids,
&resolved_call_ids,
)?;
pending_call_ids.insert(call.call_id.clone());
push_call_to_group(
&mut pending_group,
ToolCall {
id: call.call_id,
name: call.name,
arguments: serde_json::json!({"input": call.input}).to_string(),
},
);
}
ResponsesTypedInputItem::CustomToolCallOutput(output) => {
check_tool_output_id(
item_idx,
&output.call_id,
&mut pending_call_ids,
&mut resolved_call_ids,
)?;
let (content, image_sources) = output
.output
.map(|content| content.into_content(&format!("input[{item_idx}].output")))
.transpose()?
.unwrap_or_default();
push_output_to_group(
&mut pending_group,
output.call_id,
ResponsesToolResult {
content,
image_sources,
},
messages,
);
}
ResponsesTypedInputItem::Unsupported => {}
},
ResponsesInputItem::Message(msg) => {
check_pending_calls(&pending_call_ids)?;
append_tool_call_group(&mut pending_group, messages);
push_message(msg, item_idx, messages)?;
}
}
}
append_tool_call_group(&mut pending_group, messages);
check_pending_calls(&pending_call_ids)?;
Ok(())
}
impl ProtocolRequest for ResponsesRequest {
type Response = ResponsesResponse;
fn convert(self, options: ConversionOptions) -> Result<ConversationRequest, ConversionError> {
if contains_image_special_token(self.input.as_ref(), self.instructions.as_deref()) {
return Err(image_special_token_not_allowed());
}
let (thinking_mode, reasoning_effort) = self.resolve_thinking(options);
let inference_options = self.inference_options()?;
let converted_tools = ResponsesToolSet::convert(self.tools, &options)?;
let mut messages: Vec<InputMessage> = Vec::new();
if let Some(instructions) = self.instructions
&& !instructions.is_empty()
{
messages.push(InputMessage::System {
content: instructions,
});
}
match self.input {
Some(ResponsesInput::String(s)) => messages.push(InputMessage::User {
content: s,
image_sources: Vec::new(),
}),
Some(ResponsesInput::Items(items)) => {
transform_input_items(items, &mut messages, &converted_tools)?
}
None if messages.is_empty() => {
return Err(ConversionError::bad_request(
"Either input or instructions must be provided",
));
}
None => {}
}
let (tools, tool_choice) =
converted_tools.select(self.tool_choice, thinking_mode, &options)?;
validate_tool_text(&tools, &messages)?;
let response_format = convert_text_format(self.text, &messages)?;
let parsing_options = ParsingOptions {
parse_tool_calls: !tools.is_empty(),
tool_call_initial_stage: tool_choice == ToolChoice::Required,
parse_json_output: matches!(response_format, ResponseFormat::JsonObject),
reasoning_initial_stage: thinking_mode.then_some(ReasoningStage::Start),
..ParsingOptions::default()
};
Ok(ConversationRequest {
conversation: Conversation {
messages,
thinking_mode,
reasoning_effort,
tools,
tool_choice,
response_format,
},
inference_options,
parsing_options,
model: Some(self.model),
stream: self.stream.unwrap_or(false),
})
}
fn chunk_generator(
_request: &ConversationRequest,
id: String,
model: String,
) -> ResponsesChunkGenerator {
ResponsesChunkGenerator::new(id, model)
}
}
impl ResponsesRequest {
/// Return the custom tool names declared in `tools`.
/// Read before `convert` and pass to
/// [`ResponsesChunkGenerator::with_custom_tool_names`].
pub fn custom_tool_names(&self) -> HashSet<String> {
self.tools
.iter()
.flatten()
.filter_map(|tool| match tool {
ResponsesTool::Custom { name } => Some(name.clone()),
_ => None,
})
.collect()
}
fn resolve_thinking(&self, options: ConversionOptions) -> (bool, Option<ReasoningEffort>) {
let effort = self
.reasoning
.as_ref()
.and_then(|reasoning| reasoning.effort);
let effort = match effort {
Some(ResponsesReasoningEffort::None) => return (false, None),
Some(ResponsesReasoningEffort::Minimal | ResponsesReasoningEffort::Low) => {
ReasoningEffort::Low
}
Some(ResponsesReasoningEffort::Medium | ResponsesReasoningEffort::High) => {
ReasoningEffort::High
}
Some(ResponsesReasoningEffort::Xhigh) => ReasoningEffort::Xhigh,
Some(ResponsesReasoningEffort::Max) => ReasoningEffort::Max,
None if !options.default_thinking_mode => return (false, None),
None => ReasoningEffort::High,
};
(true, Some(effort))
}
fn inference_options(&self) -> Result<InferenceOptions, ConversionError> {
validate_max_tokens(self.max_output_tokens, "max_output_tokens")?;
validate_sampling(self.temperature, self.top_p)?;
Ok(InferenceOptions {
max_tokens: self.max_output_tokens,
temperature: self.temperature,
top_p: self.top_p,
..InferenceOptions::default()
})
}
}
fn convert_text_format(
text: Option<ResponsesTextConfig>,
messages: &[InputMessage],
) -> Result<ResponseFormat, ConversionError> {
match text.map(|text| text.format).unwrap_or_default() {
ResponsesTextFormat::Text | ResponsesTextFormat::JsonSchema => Ok(ResponseFormat::Text),
ResponsesTextFormat::JsonObject => {
if !contains_json_instruction(messages) {
return Err(ConversionError::bad_request(
"Response input messages must contain the word 'json' to use text.format of type json_object",
));
}
Ok(ResponseFormat::JsonObject)
}
}
}
fn unsupported_content() -> ConversionError {
ConversionError::bad_request("Unsupported content block")
}

26
deepseek-recipe/src/protocol/openai/responses/request/mod.rs Normal file
View File

@ -0,0 +1,26 @@
//! Responses request deserialization and conversion.
//!
//! [`ResponsesRequest`] supports text, images, plaintext reasoning history,
//! instructions, sampling parameters, reasoning effort, and JSON object output.
//! Client tools include functions, namespaces, and the `apply_patch` custom tool.
//!
//! Document content, encrypted reasoning, conversation storage, and hosted tool
//! execution are outside the supported scope. Web-search declarations and
//! choices are ignored by default; [`crate::request::WebSearchBehavior::Reject`]
//! rejects them instead.
//!
//! Input text must not spell out the image placeholder. Every placeholder in a
//! prompt corresponds to one image source, and the adapter inserts them for
//! image blocks only, so text input, instructions, message text, tool output
//! text, reasoning text, tool definitions, and historical tool calls that carry
//! the placeholder spelling are rejected.
//!
//! JSON Schema output, `logprobs`, and `top_logprobs` are accepted and ignored.
//! Function-tool `strict` settings are passed through for the caller to enforce.
//! The caller supplies model inference, tool execution, and HTTP transport.
pub use schema::*;
mod convert;
mod schema;
mod tools;

303
deepseek-recipe/src/protocol/openai/responses/request/schema.rs Normal file
View File

@ -0,0 +1,303 @@
use serde::{Deserialize, Deserializer};
use deepseek_recipe_core::multimodal::ImageDetail;
/// A Responses request with text and image input and client tools.
///
/// Unknown top-level fields, including `logprobs` and `top_logprobs`, are
/// ignored. Model resolution, server tools, and encrypted reasoning recovery
/// belong to the caller.
#[derive(Debug, Deserialize)]
pub struct ResponsesRequest {
pub model: String,
pub input: Option<ResponsesInput>,
pub instructions: Option<String>,
pub max_output_tokens: Option<u32>,
pub reasoning: Option<ResponsesReasoningConfig>,
pub stream: Option<bool>,
pub temperature: Option<f32>,
pub text: Option<ResponsesTextConfig>,
pub tool_choice: Option<ResponsesToolChoice>,
pub tools: Option<Vec<ResponsesTool>>,
pub top_p: Option<f32>,
/// Caller metadata. Extract this before consuming the request.
pub user: Option<String>,
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
pub enum ResponsesInput {
String(String),
Items(Vec<ResponsesInputItem>),
}
/// An explicitly typed item or a message without a `type` field.
#[derive(Debug)]
pub enum ResponsesInputItem {
Typed(ResponsesTypedInputItem),
Message(ResponsesInputMessage),
}
impl<'de> Deserialize<'de> for ResponsesInputItem {
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
let value = serde_json::Value::deserialize(deserializer)?;
// The presence of `type` selects typed-item deserialization.
if value.get("type").is_some() {
serde_json::from_value(value)
.map(Self::Typed)
.map_err(serde::de::Error::custom)
} else {
serde_json::from_value(value)
.map(Self::Message)
.map_err(serde::de::Error::custom)
}
}
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ResponsesTypedInputItem {
Message(ResponsesInputMessage),
FunctionCall(ResponsesFunctionCall),
FunctionCallOutput(ResponsesFunctionCallOutput),
Reasoning(ResponsesReasoningItem),
CustomToolCall(ResponsesCustomToolCall),
CustomToolCallOutput(ResponsesCustomToolCallOutput),
/// Unsupported item types, including `web_search_call`, are ignored.
#[serde(other)]
Unsupported,
}
#[derive(Debug, Deserialize)]
pub struct ResponsesInputMessage {
pub role: ResponsesMessageRole,
pub content: ResponsesMessageContent,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ResponsesMessageRole {
User,
Assistant,
System,
Developer,
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
pub enum ResponsesMessageContent {
String(String),
List(Vec<ResponsesContentBlock>),
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ResponsesContentBlock {
InputText {
text: String,
},
OutputText {
text: String,
},
InputImage(ResponsesInputImage),
/// Documents are rendered as an unsupported-document placeholder. Every
/// field is accepted and ignored.
InputFile {},
/// Content block types not listed above are rejected during conversion.
#[serde(other)]
Unsupported,
}
/// One image referenced by URL or by a file identifier. A file identifier
/// requires a file service and is rejected during conversion.
#[derive(Debug, Deserialize)]
pub struct ResponsesInputImage {
/// HTTP URL or base64 data URL. An empty string is treated as absent, and a
/// value that does not start with `http` must carry a `base64` body.
pub image_url: Option<String>,
#[serde(default)]
pub detail: Option<ImageDetail>,
pub file_id: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct ResponsesFunctionCall {
pub arguments: String,
pub call_id: String,
pub name: String,
pub namespace: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct ResponsesFunctionCallOutput {
pub call_id: String,
pub output: ResponsesToolCallOutputContent,
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
pub enum ResponsesToolCallOutputContent {
String(String),
List(Vec<ResponsesToolCallOutputItem>),
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ResponsesToolCallOutputItem {
InputText {
text: String,
},
InputImage(ResponsesInputImage),
/// Documents are rendered as an unsupported-document placeholder. Every
/// field is accepted and ignored.
InputFile {},
/// Content block types not listed above are rejected during conversion.
#[serde(other)]
Unsupported,
}
#[derive(Debug, Deserialize)]
pub struct ResponsesReasoningItem {
pub content: Option<Vec<ResponsesReasoningContent>>,
/// Encrypted reasoning is ignored; plaintext reasoning is preserved.
pub encrypted_content: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct ResponsesReasoningContent {
pub text: String,
}
#[derive(Debug, Deserialize)]
pub struct ResponsesCustomToolCall {
pub call_id: String,
pub name: String,
pub input: String,
}
#[derive(Debug, Deserialize)]
pub struct ResponsesCustomToolCallOutput {
pub call_id: String,
pub output: Option<ResponsesToolCallOutputContent>,
}
#[derive(Debug, Deserialize)]
pub struct ResponsesReasoningConfig {
pub effort: Option<ResponsesReasoningEffort>,
/// Summary setting available to the caller before conversion.
pub summary: Option<ResponsesReasoningSummary>,
/// Alternate summary setting available to the caller before conversion.
pub generate_summary: Option<ResponsesReasoningSummary>,
}
#[derive(Debug, Clone, Copy, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ResponsesReasoningEffort {
None,
Minimal,
Low,
Medium,
High,
Xhigh,
Max,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ResponsesReasoningSummary {
Auto,
Concise,
Detailed,
}
#[derive(Debug, Deserialize)]
pub struct ResponsesTextConfig {
#[serde(default)]
pub format: ResponsesTextFormat,
/// Verbosity setting available to the caller before conversion.
pub verbosity: Option<ResponsesVerbosity>,
}
#[derive(Debug, Default, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ResponsesTextFormat {
#[default]
Text,
JsonObject,
/// Accepted and ignored, including all schema fields. Uses plain text output.
JsonSchema,
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ResponsesVerbosity {
Low,
Medium,
High,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ResponsesTool {
Function(ResponsesFunctionTool),
Namespace {
name: String,
description: Option<String>,
tools: Vec<ResponsesNamespaceTool>,
},
/// The `apply_patch` custom tool is converted to a function tool.
Custom {
name: String,
},
#[serde(alias = "web_search_2025_08_26")]
WebSearch,
#[serde(other)]
Unsupported,
}
#[derive(Debug, Deserialize)]
pub struct ResponsesFunctionTool {
pub name: String,
pub description: Option<String>,
pub parameters: Option<serde_json::Value>,
/// Passed through; backend enforcement belongs to the caller.
pub strict: Option<bool>,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ResponsesNamespaceTool {
Function(ResponsesFunctionTool),
/// Custom tools inside a namespace are rejected during conversion.
Custom {
name: String,
},
}
#[derive(Debug, Deserialize)]
#[serde(untagged)]
pub enum ResponsesToolChoice {
Mode(ResponsesToolChoiceMode),
Named(ResponsesNamedToolChoice),
}
#[derive(Debug, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ResponsesToolChoiceMode {
None,
Auto,
Required,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ResponsesNamedToolChoice {
Function {
name: String,
},
Custom {
name: String,
},
#[serde(alias = "web_search_2025_08_26")]
WebSearch,
}

291
deepseek-recipe/src/protocol/openai/responses/request/tools.rs Normal file
View File

@ -0,0 +1,291 @@
use std::collections::{HashMap, HashSet};
use deepseek_recipe_core::messages::ToolCall;
use deepseek_recipe_core::tools::{ToolChoice, ToolDefinition};
use crate::protocol::validation::{validate_tool_name, validate_tool_parameters};
use crate::request::{ConversionError, ConversionOptions, WebSearchBehavior};
use super::schema::{
ResponsesFunctionCall, ResponsesFunctionTool, ResponsesNamedToolChoice, ResponsesNamespaceTool,
ResponsesTool, ResponsesToolChoice, ResponsesToolChoiceMode,
};
pub(super) struct ResponsesToolSet {
definitions: Vec<ResponsesFunctionTool>,
namespace_names: HashMap<(String, String), String>,
top_level_names: HashSet<String>,
}
impl ResponsesToolSet {
pub(super) fn convert(
tools: Option<Vec<ResponsesTool>>,
options: &ConversionOptions,
) -> Result<Self, ConversionError> {
let tools = tools.unwrap_or_default();
let mut converted = Self {
definitions: Vec::new(),
namespace_names: HashMap::new(),
top_level_names: tools
.iter()
.filter_map(|tool| match tool {
ResponsesTool::Function(function) => Some(function.name.clone()),
_ => None,
})
.collect(),
};
let mut namespaces = HashSet::new();
for (index, tool) in tools.into_iter().enumerate() {
let path = format!("tools[{index}]");
match tool {
ResponsesTool::Function(function) => {
validate_tool_name(&function.name, &format!("{path}.name"))?;
converted.definitions.push(function);
}
ResponsesTool::Namespace {
name,
description,
tools,
} => {
validate_tool_name(&name, &format!("{path}.name"))?;
if !namespaces.insert(name.clone()) {
return Err(ConversionError::bad_request(format!(
"{path}: duplicate namespace name '{name}'"
)));
}
let mut inner_names = HashSet::new();
for (inner_index, tool) in tools.into_iter().enumerate() {
let inner_path = format!("{path}.tools[{inner_index}]");
let ResponsesNamespaceTool::Function(mut function) = tool else {
return Err(ConversionError::bad_request(format!(
"{inner_path}: custom tools are not supported inside a namespace"
)));
};
validate_tool_name(&function.name, &format!("{inner_path}.name"))?;
if !inner_names.insert(function.name.clone()) {
return Err(ConversionError::bad_request(format!(
"{inner_path}: tool names within a namespace must be unique"
)));
}
if let Some(namespace_description) = description
.as_deref()
.filter(|description| !description.is_empty())
{
let function_description =
function.description.as_deref().unwrap_or_default();
function.description =
Some(format!("{namespace_description}\n{function_description}"));
}
let function_name =
unique_name(join_namespace(&name, &function.name), |candidate| {
converted.is_name_taken(candidate)
});
converted
.namespace_names
.insert((name.clone(), function.name), function_name.clone());
function.name = function_name;
converted.definitions.push(function);
}
}
ResponsesTool::Custom { name } => {
if name != "apply_patch" {
return Err(ConversionError::bad_request(format!(
"Unsupported custom tool: '{name}'. Only 'apply_patch' is supported"
)));
}
converted.definitions.push(apply_patch_tool());
}
ResponsesTool::WebSearch => match options.responses_web_search {
WebSearchBehavior::Ignore => {}
WebSearchBehavior::Reject => {
return Err(ConversionError::bad_request(
"Server tools are not supported",
));
}
},
ResponsesTool::Unsupported => {}
}
}
Ok(converted)
}
pub(super) fn function_call(
&self,
call: ResponsesFunctionCall,
historical_names: &mut HashMap<(String, String), String>,
) -> ToolCall {
let name = match call.namespace {
Some(namespace) => {
let key = (namespace, call.name);
if let Some(name) = self.namespace_names.get(&key) {
name.clone()
} else if let Some(name) = historical_names.get(&key) {
name.clone()
} else {
// Each historical namespace/name pair uses one unique name.
let name = unique_name(join_namespace(&key.0, &key.1), |candidate| {
self.is_name_taken(candidate)
|| historical_names.values().any(|name| name == candidate)
});
historical_names.insert(key, name.clone());
name
}
}
None => call.name,
};
ToolCall {
id: call.call_id,
name,
arguments: call.arguments,
}
}
pub(super) fn select(
self,
choice: Option<ResponsesToolChoice>,
thinking: bool,
options: &ConversionOptions,
) -> Result<(Vec<ToolDefinition>, ToolChoice), ConversionError> {
// Dropping the web_search tool also drops a choice naming it.
let choice = match choice {
Some(ResponsesToolChoice::Named(ResponsesNamedToolChoice::WebSearch)) => {
match options.responses_web_search {
WebSearchBehavior::Ignore => None,
WebSearchBehavior::Reject => {
return Err(ConversionError::bad_request(
"Server tools are not supported",
));
}
}
}
other => other,
};
// Disabling tools skips parameter-schema and final-name checks.
// Definition conversion has already handled tool names and kinds.
if matches!(
choice,
Some(ResponsesToolChoice::Mode(ResponsesToolChoiceMode::None))
) {
return Ok((Vec::new(), ToolChoice::None));
}
let mut tools = self.definitions;
if tools.is_empty() {
return Ok((Vec::new(), ToolChoice::Auto));
}
let mut names = HashSet::new();
for tool in &tools {
if !names.insert(&tool.name) {
return Err(ConversionError::bad_request("Tool names must be unique"));
}
if let Some(parameters) = &tool.parameters {
validate_tool_parameters(parameters, &format!("Tool '{}' parameters", tool.name))?;
}
}
let required = match choice {
Some(ResponsesToolChoice::Named(
ResponsesNamedToolChoice::Function { name }
| ResponsesNamedToolChoice::Custom { name },
)) => {
if !names.contains(&name) {
return Err(ConversionError::bad_request(format!(
"tool_choice: no tool named '{name}' was specified"
)));
}
tools.retain(|tool| tool.name == name);
true
}
Some(ResponsesToolChoice::Mode(ResponsesToolChoiceMode::Required)) => true,
_ => false,
};
if required && thinking {
return Err(ConversionError::bad_request(
"Thinking mode does not support this tool_choice",
));
}
Ok((
tools.into_iter().map(Into::into).collect(),
if required {
ToolChoice::Required
} else {
ToolChoice::Auto
},
))
}
fn is_name_taken(&self, candidate: &str) -> bool {
self.top_level_names.contains(candidate)
|| self.namespace_names.values().any(|name| name == candidate)
}
}
impl From<ResponsesFunctionTool> for ToolDefinition {
fn from(function: ResponsesFunctionTool) -> Self {
Self {
name: function.name,
description: function.description,
parameters: function.parameters.unwrap_or_else(|| serde_json::json!({})),
strict: function.strict,
}
}
}
fn join_namespace(namespace: &str, name: &str) -> String {
format!("{namespace}::{name}")
}
fn unique_name(base: String, is_taken: impl Fn(&str) -> bool) -> String {
if !is_taken(&base) {
return base;
}
for index in 1.. {
let candidate = format!("{base}_{index}");
if !is_taken(&candidate) {
return candidate;
}
}
unreachable!()
}
fn apply_patch_tool() -> ResponsesFunctionTool {
ResponsesFunctionTool {
name: "apply_patch".to_string(),
description: Some(APPLY_PATCH_DESCRIPTION.to_string()),
parameters: Some(serde_json::json!({
"type": "object",
"additionalProperties": false,
"properties": {
"input": {
"type": "string",
"description": "The entire contents of the apply_patch command (the *** Begin Patch ... *** End Patch envelope)."
}
},
"required": ["input"]
})),
strict: None,
}
}
const APPLY_PATCH_DESCRIPTION: &str = "Use the `apply_patch` tool to edit files.\nYour patch language is a stripped‑down, file‑oriented diff format designed to be easy \
to parse and safe to apply. You can think of it as a high‑level envelope:\n\n*** Begin Patch\n[ one or more file sections ]\n*** \
End Patch\n\nWithin that envelope, you get a sequence of file operations.\nYou MUST include a header to specify the action you \
are taking.\nEach operation starts with one of three headers:\n\n*** Add File: <path> - create a new file. Every following line \
is a + line (the initial contents).\n*** Delete File: <path> - remove an existing file. Nothing follows.\n*** Update File: <path> \
- patch an existing file in place (optionally with a rename).\n\nMay be immediately followed by *** Move to: <new path> if you \
want to rename the file.\nThen one or more “hunks”, each introduced by @@ (optionally followed by a hunk header).\nWithin a hunk \
each line starts with:\n\nFor instructions on [context_before] and [context_after]:\n- By default, show 3 lines of code \
immediately above and 3 lines immediately below each change. If a change is within 3 lines of a previous change, do NOT duplicate \
the first change’s [context_after] lines in the second change’s [context_before] lines.\n- If 3 lines of context is insufficient \
to uniquely identify the snippet of code within the file, use the @@ operator to indicate the class or function to which the \
snippet belongs. For instance, we might have:\n@@ class BaseClass\n[3 lines of pre-context]\n- [old_code]\n+ [new_code]\n[3 lines \
of post-context]\n\n- If a code block is repeated so many times in a class or function such that even a single `@@` statement and \
3 lines of context cannot uniquely identify the snippet of code, you can use multiple `@@` statements to jump to the right \
context. For instance:\n\n@@ class BaseClass\n@@ \t def method():\n[3 lines of pre-context]\n- [old_code]\n+ [new_code]\n[3 lines \
of post-context]\n\nThe full grammar definition is below:\nPatch := Begin { FileOp } End\nBegin := \"*** Begin Patch\" \
NEWLINE\nEnd := \"*** End Patch\" NEWLINE\nFileOp := AddFile | DeleteFile | UpdateFile\nAddFile := \"*** Add File: \" path \
NEWLINE { \"+\" line NEWLINE }\nDeleteFile := \"*** Delete File: \" path NEWLINE\nUpdateFile := \"*** Update File: \" path \
NEWLINE [ MoveTo ] { Hunk }\nMoveTo := \"*** Move to: \" newPath NEWLINE\nHunk := \"@@\" [ header ] NEWLINE { HunkLine } [ \"*** \
End of File\" NEWLINE ]\nHunkLine := (\" \" | \"-\" | \"+\") text NEWLINE\n\nA full patch can combine several operations:\n\n*** \
Begin Patch\n*** Add File: hello.txt\n+Hello world\n*** Update File: src/app.py\n*** Move to: src/main.py\n@@ def \
greet():\n-print(\"Hi\")\n+print(\"Hello, world!\")\n*** Delete File: obsolete.txt\n*** End Patch\n\nIt is important to \
remember:\n\n- You must include a header with your intended action (Add/Delete/Update)\n- You must prefix new lines with `+` even \
when creating a new file\n- File references can only be relative, NEVER ABSOLUTE.\n";

225
deepseek-recipe/src/protocol/openai/responses/response/accumulator.rs Normal file
View File

@ -0,0 +1,225 @@
use crate::util::append_delta::AppendDelta;
use super::schema::{
ResponsesContentPart, ResponsesOutputItem, ResponsesResponse, ResponsesStreamEvent,
};
/// Apply events in generation order.
impl AppendDelta<ResponsesStreamEvent> for ResponsesResponse {
fn append(&mut self, event: ResponsesStreamEvent) {
match event {
ResponsesStreamEvent::Created { response, .. }
| ResponsesStreamEvent::InProgress { response, .. }
| ResponsesStreamEvent::Completed { response, .. }
| ResponsesStreamEvent::Incomplete { response, .. } => *self = response,
ResponsesStreamEvent::OutputItemAdded {
item, output_index, ..
} => {
if output_index == self.output.len() && valid_item_content(&item) {
self.output.push(item);
}
}
ResponsesStreamEvent::OutputItemDone {
item, output_index, ..
} => {
if let Some(current) = self.output.get_mut(output_index)
&& item_id(current) == item_id(&item)
&& std::mem::discriminant(current) == std::mem::discriminant(&item)
&& valid_item_content(&item)
{
*current = item;
}
}
ResponsesStreamEvent::ContentPartAdded {
content_index,
item_id,
output_index,
part,
..
} => {
if let Some(item) = self.matching_item_mut(output_index, &item_id)
&& let Some(content) = matching_content_mut(item, &part)
&& content_index == content.len()
{
content.push(part);
}
}
ResponsesStreamEvent::ContentPartDone {
content_index,
item_id,
output_index,
part,
..
} => {
if let Some(item) = self.matching_item_mut(output_index, &item_id)
&& let Some(content) = matching_content_mut(item, &part)
&& let Some(current) = content.get_mut(content_index)
&& std::mem::discriminant(current) == std::mem::discriminant(&part)
{
*current = part;
}
}
ResponsesStreamEvent::OutputTextDelta {
content_index,
delta,
item_id,
logprobs: delta_logprobs,
output_index,
..
} => {
if let Some(ResponsesOutputItem::Message { content, .. }) =
self.matching_item_mut(output_index, &item_id)
&& let Some(ResponsesContentPart::OutputText { text, logprobs, .. }) =
content.get_mut(content_index)
{
text.push_str(&delta);
logprobs.extend(delta_logprobs);
}
}
ResponsesStreamEvent::OutputTextDone {
content_index,
item_id,
logprobs: final_logprobs,
output_index,
text: final_text,
..
} => {
if let Some(ResponsesOutputItem::Message { content, .. }) =
self.matching_item_mut(output_index, &item_id)
&& let Some(ResponsesContentPart::OutputText { text, logprobs, .. }) =
content.get_mut(content_index)
{
*text = final_text;
*logprobs = final_logprobs;
}
}
ResponsesStreamEvent::ReasoningTextDelta {
content_index,
delta,
item_id,
output_index,
..
} => {
if let Some(ResponsesOutputItem::Reasoning { content, .. }) =
self.matching_item_mut(output_index, &item_id)
&& let Some(ResponsesContentPart::ReasoningText { text }) =
content.get_mut(content_index)
{
text.push_str(&delta);
}
}
ResponsesStreamEvent::ReasoningTextDone {
content_index,
item_id,
output_index,
text: final_text,
..
} => {
if let Some(ResponsesOutputItem::Reasoning { content, .. }) =
self.matching_item_mut(output_index, &item_id)
&& let Some(ResponsesContentPart::ReasoningText { text }) =
content.get_mut(content_index)
{
*text = final_text;
}
}
ResponsesStreamEvent::FunctionCallArgumentsDelta {
delta,
item_id,
output_index,
..
} => {
if let Some(ResponsesOutputItem::FunctionCall { arguments, .. }) =
self.matching_item_mut(output_index, &item_id)
{
arguments.push_str(&delta);
}
}
ResponsesStreamEvent::FunctionCallArgumentsDone {
arguments: final_arguments,
item_id,
output_index,
..
} => {
if let Some(ResponsesOutputItem::FunctionCall { arguments, .. }) =
self.matching_item_mut(output_index, &item_id)
{
*arguments = final_arguments;
}
}
ResponsesStreamEvent::CustomToolCallInputDelta {
delta,
item_id,
output_index,
..
} => {
if let Some(ResponsesOutputItem::CustomToolCall { input, .. }) =
self.matching_item_mut(output_index, &item_id)
{
input.push_str(&delta);
}
}
ResponsesStreamEvent::CustomToolCallInputDone {
input: final_input,
item_id,
output_index,
..
} => {
if let Some(ResponsesOutputItem::CustomToolCall { input, .. }) =
self.matching_item_mut(output_index, &item_id)
{
*input = final_input;
}
}
}
}
}
impl ResponsesResponse {
fn matching_item_mut(
&mut self,
output_index: usize,
expected_id: &str,
) -> Option<&mut ResponsesOutputItem> {
self.output
.get_mut(output_index)
.filter(|item| item_id(item) == expected_id)
}
}
fn item_id(item: &ResponsesOutputItem) -> &str {
match item {
ResponsesOutputItem::Message { id, .. }
| ResponsesOutputItem::Reasoning { id, .. }
| ResponsesOutputItem::FunctionCall { id, .. }
| ResponsesOutputItem::CustomToolCall { id, .. } => id,
}
}
fn valid_item_content(item: &ResponsesOutputItem) -> bool {
match item {
ResponsesOutputItem::Message { content, .. } => content
.iter()
.all(|part| matches!(part, ResponsesContentPart::OutputText { .. })),
ResponsesOutputItem::Reasoning { content, .. } => content
.iter()
.all(|part| matches!(part, ResponsesContentPart::ReasoningText { .. })),
ResponsesOutputItem::FunctionCall { .. } | ResponsesOutputItem::CustomToolCall { .. } => {
true
}
}
}
fn matching_content_mut<'a>(
item: &'a mut ResponsesOutputItem,
part: &ResponsesContentPart,
) -> Option<&'a mut Vec<ResponsesContentPart>> {
match (item, part) {
(ResponsesOutputItem::Message { content, .. }, ResponsesContentPart::OutputText { .. })
| (
ResponsesOutputItem::Reasoning { content, .. },
ResponsesContentPart::ReasoningText { .. },
) => Some(content),
_ => None,
}
}

420
deepseek-recipe/src/protocol/openai/responses/response/chunk_generator.rs Normal file
View File

@ -0,0 +1,420 @@
use std::collections::HashSet;
use std::time::{SystemTime, UNIX_EPOCH};
use crate::response::ProtocolResponse;
use crate::stream::{ChunkGenerator, FinishReason, OutputChunk, PromptUsage};
use super::custom_tool_input::ResponsesCustomToolInputParser;
use super::schema::{
ResponsesContentPart, ResponsesIncompleteDetails, ResponsesIncompleteReason,
ResponsesOutputItem, ResponsesPhase, ResponsesResponse, ResponsesRole, ResponsesStatus,
ResponsesStreamEvent, ResponsesUsage,
};
/// Generate Responses events from one `Start` to `Finish` sequence.
/// Repeated starts and chunks outside this sequence are ignored.
/// Argument deltas extend only the active tool call.
/// Custom tool output requires `with_custom_tool_names`.
/// Other tools retain their model names, including flattened namespaces.
#[derive(Debug)]
pub struct ResponsesChunkGenerator {
response: ResponsesResponse,
active: Option<usize>,
sequence_number: usize,
tool_call_index: usize,
custom_tool_names: HashSet<String>,
custom_tool_input: Option<ResponsesCustomToolInputParser>,
prompt_usage: PromptUsage,
started: bool,
finished: bool,
}
enum ResponsesTextKind {
Output,
Reasoning,
}
impl ResponsesChunkGenerator {
/// Use a unique response ID and public model name.
/// Item and call IDs derive from the response ID and indices.
pub fn new(id: impl Into<String>, model: impl Into<String>) -> Self {
Self {
response: ResponsesResponse::new(id.into(), model.into(), timestamp(), 0, 0),
active: None,
sequence_number: 0,
tool_call_index: 0,
custom_tool_names: HashSet::new(),
custom_tool_input: None,
prompt_usage: PromptUsage::default(),
started: false,
finished: false,
}
}
/// Set declared custom tool names whose JSON `input` strings are decoded.
/// Defaults to an empty set; other tools retain their function arguments.
pub fn with_custom_tool_names(mut self, custom_tool_names: HashSet<String>) -> Self {
self.custom_tool_names = custom_tool_names;
self
}
fn next_sequence_number(&mut self) -> usize {
let sequence_number = self.sequence_number;
self.sequence_number += 1;
sequence_number
}
fn open_item(&mut self, mut item: ResponsesOutputItem, events: &mut Vec<ResponsesStreamEvent>) {
let output_index = self.response.output.len();
self.response.output.push(item.clone());
let content = match &mut item {
ResponsesOutputItem::Message { id, content, .. }
| ResponsesOutputItem::Reasoning { id, content, .. } => {
content.pop().map(|part| (id.clone(), part))
}
ResponsesOutputItem::FunctionCall { .. }
| ResponsesOutputItem::CustomToolCall { .. } => None,
};
self.active = Some(output_index);
events.push(ResponsesStreamEvent::OutputItemAdded {
item,
output_index,
sequence_number: self.next_sequence_number(),
});
if let Some((item_id, part)) = content {
events.push(ResponsesStreamEvent::ContentPartAdded {
content_index: 0,
item_id,
output_index,
part,
sequence_number: self.next_sequence_number(),
});
}
}
fn close_item(&mut self, status: ResponsesStatus, events: &mut Vec<ResponsesStreamEvent>) {
let Some(output_index) = self.active.take() else {
return;
};
if let Some(mut parser) = self.custom_tool_input.take() {
self.append_custom_input(output_index, parser.finish(), events);
}
let item = &mut self.response.output[output_index];
match item {
ResponsesOutputItem::Message {
status: item_status,
..
}
| ResponsesOutputItem::Reasoning {
status: item_status,
..
}
| ResponsesOutputItem::FunctionCall {
status: item_status,
..
}
| ResponsesOutputItem::CustomToolCall {
status: item_status,
..
} => *item_status = status,
}
let item = item.clone();
match &item {
ResponsesOutputItem::Message { id, content, .. }
| ResponsesOutputItem::Reasoning { id, content, .. } => {
for (content_index, part) in content.iter().enumerate() {
let sequence_number = self.next_sequence_number();
events.push(match part {
ResponsesContentPart::OutputText { text, logprobs, .. } => {
ResponsesStreamEvent::OutputTextDone {
content_index,
item_id: id.clone(),
output_index,
text: text.clone(),
logprobs: logprobs.clone(),
sequence_number,
}
}
ResponsesContentPart::ReasoningText { text } => {
ResponsesStreamEvent::ReasoningTextDone {
content_index,
item_id: id.clone(),
output_index,
text: text.clone(),
sequence_number,
}
}
});
events.push(ResponsesStreamEvent::ContentPartDone {
content_index,
item_id: id.clone(),
output_index,
part: part.clone(),
sequence_number: self.next_sequence_number(),
});
}
}
ResponsesOutputItem::FunctionCall { id, arguments, .. } => {
events.push(ResponsesStreamEvent::FunctionCallArgumentsDone {
arguments: arguments.clone(),
item_id: id.clone(),
output_index,
sequence_number: self.next_sequence_number(),
});
}
ResponsesOutputItem::CustomToolCall { id, input, .. } => {
events.push(ResponsesStreamEvent::CustomToolCallInputDone {
input: input.clone(),
item_id: id.clone(),
output_index,
sequence_number: self.next_sequence_number(),
});
}
}
events.push(ResponsesStreamEvent::OutputItemDone {
item,
output_index,
sequence_number: self.next_sequence_number(),
});
}
fn append_text(
&mut self,
delta: String,
kind: ResponsesTextKind,
events: &mut Vec<ResponsesStreamEvent>,
) {
let reasoning = matches!(kind, ResponsesTextKind::Reasoning);
let current = self
.active
.and_then(|index| self.response.output.get(index));
let continues_item = matches!(
(reasoning, current),
(true, Some(ResponsesOutputItem::Reasoning { .. }))
| (false, Some(ResponsesOutputItem::Message { .. }))
);
if !continues_item {
self.close_item(ResponsesStatus::Completed, events);
let output_index = self.response.output.len();
let item = if reasoning {
ResponsesOutputItem::Reasoning {
id: format!("rs_{}_{output_index}", self.response.id),
status: ResponsesStatus::InProgress,
content: vec![ResponsesContentPart::ReasoningText {
text: String::new(),
}],
summary: Vec::new(),
}
} else {
ResponsesOutputItem::Message {
id: format!("msg_{}_{output_index}", self.response.id),
status: ResponsesStatus::InProgress,
role: ResponsesRole::Assistant,
phase: ResponsesPhase::FinalAnswer,
content: vec![ResponsesContentPart::OutputText {
annotations: Vec::new(),
logprobs: Vec::new(),
text: String::new(),
}],
}
};
self.open_item(item, events);
}
let Some(output_index) = self.active else {
return;
};
let (item_id, content) = match &mut self.response.output[output_index] {
ResponsesOutputItem::Message { id, content, .. }
| ResponsesOutputItem::Reasoning { id, content, .. } => (id.clone(), content),
ResponsesOutputItem::FunctionCall { .. }
| ResponsesOutputItem::CustomToolCall { .. } => return,
};
let Some(part) = content.first_mut() else {
return;
};
match part {
ResponsesContentPart::OutputText { text, .. }
| ResponsesContentPart::ReasoningText { text } => text.push_str(&delta),
}
let sequence_number = self.next_sequence_number();
events.push(if reasoning {
ResponsesStreamEvent::ReasoningTextDelta {
content_index: 0,
delta,
item_id,
output_index,
sequence_number,
}
} else {
ResponsesStreamEvent::OutputTextDelta {
content_index: 0,
delta,
item_id,
logprobs: Vec::new(),
output_index,
sequence_number,
}
});
}
fn append_custom_input(
&mut self,
output_index: usize,
delta: String,
events: &mut Vec<ResponsesStreamEvent>,
) {
if delta.is_empty() {
return;
}
if let ResponsesOutputItem::CustomToolCall { id, input, .. } =
&mut self.response.output[output_index]
{
input.push_str(&delta);
let item_id = id.clone();
events.push(ResponsesStreamEvent::CustomToolCallInputDelta {
delta,
item_id,
output_index,
sequence_number: self.next_sequence_number(),
});
}
}
}
impl ChunkGenerator for ResponsesChunkGenerator {
type Chunk = ResponsesStreamEvent;
async fn generate(&mut self, chunk: OutputChunk) -> Vec<Self::Chunk> {
if self.finished || (!self.started && !matches!(chunk, OutputChunk::Start { .. })) {
return Vec::new();
}
let mut events = Vec::new();
match chunk {
OutputChunk::Start { usage, .. } => {
if self.started {
return events;
}
self.started = true;
self.prompt_usage = usage;
events.push(ResponsesStreamEvent::Created {
response: self.response.clone(),
sequence_number: self.next_sequence_number(),
});
events.push(ResponsesStreamEvent::InProgress {
response: self.response.clone(),
sequence_number: self.next_sequence_number(),
});
}
OutputChunk::Raw { content } => {
self.append_text(content, ResponsesTextKind::Output, &mut events);
}
OutputChunk::Reasoning { content } => {
self.append_text(content, ResponsesTextKind::Reasoning, &mut events);
}
OutputChunk::ToolCall {
tool_name,
arguments,
} => {
if let Some(output_index) = self.active
&& let ResponsesOutputItem::Message { phase, .. } =
&mut self.response.output[output_index]
{
*phase = ResponsesPhase::Commentary;
}
self.close_item(ResponsesStatus::Completed, &mut events);
let output_index = self.response.output.len();
let call_id = format!("call_{}_{}", self.response.id, self.tool_call_index);
let item = if self.custom_tool_names.contains(&tool_name) {
let mut parser = ResponsesCustomToolInputParser::default();
let input = parser.feed(&arguments);
self.custom_tool_input = Some(parser);
ResponsesOutputItem::CustomToolCall {
id: format!("ctc_{}_{output_index}", self.response.id),
status: ResponsesStatus::InProgress,
call_id,
name: tool_name,
input,
}
} else {
let (name, namespace) = match tool_name.split_once("::") {
Some((namespace, name)) => (name.to_string(), Some(namespace.to_string())),
None => (tool_name, None),
};
ResponsesOutputItem::FunctionCall {
id: format!("fc_{}_{output_index}", self.response.id),
status: ResponsesStatus::InProgress,
call_id,
name,
namespace,
arguments,
}
};
self.tool_call_index += 1;
self.open_item(item, &mut events);
}
OutputChunk::ToolArgumentsDelta { content } => {
if let Some(output_index) = self.active
&& let ResponsesOutputItem::FunctionCall { id, arguments, .. } =
&mut self.response.output[output_index]
{
arguments.push_str(&content);
let item_id = id.clone();
events.push(ResponsesStreamEvent::FunctionCallArgumentsDelta {
delta: content,
item_id,
output_index,
sequence_number: self.next_sequence_number(),
});
} else if let Some(output_index) = self.active
&& let Some(parser) = &mut self.custom_tool_input
{
let delta = parser.feed(&content);
self.append_custom_input(output_index, delta, &mut events);
}
}
OutputChunk::Finish { reason, usage, .. } => {
let incomplete_reason = match reason {
FinishReason::Length => Some(ResponsesIncompleteReason::MaxOutputTokens),
FinishReason::ContentFilter => Some(ResponsesIncompleteReason::ContentFilter),
FinishReason::Stop
| FinishReason::ToolCalls
| FinishReason::StopSequence
| FinishReason::EndOfStream => None,
};
self.response.incomplete_details =
incomplete_reason.map(|reason| ResponsesIncompleteDetails { reason });
self.response.status = if self.response.incomplete_details.is_some() {
ResponsesStatus::Incomplete
} else {
ResponsesStatus::Completed
};
self.response.usage = Some(ResponsesUsage::from((self.prompt_usage, usage)));
self.response.completed_at = Some(timestamp());
self.close_item(self.response.status, &mut events);
let response = self.response.clone();
let sequence_number = self.next_sequence_number();
events.push(if self.response.status == ResponsesStatus::Incomplete {
ResponsesStreamEvent::Incomplete {
response,
sequence_number,
}
} else {
ResponsesStreamEvent::Completed {
response,
sequence_number,
}
});
self.finished = true;
}
OutputChunk::ToolCallBegin => {}
}
events
}
}
fn timestamp() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|elapsed| elapsed.as_secs())
.unwrap_or(0)
}

116
deepseek-recipe/src/protocol/openai/responses/response/custom_tool_input.rs Normal file
View File

@ -0,0 +1,116 @@
/// Decode the leading JSON `input` string, preserving nonmatching arguments.
/// Content after the closing quote is ignored.
#[derive(Debug, Default)]
pub(super) struct ResponsesCustomToolInputParser {
state: ResponsesCustomToolInputParserState,
prefix: String,
prefix_part: usize,
prefix_offset: usize,
escape: String,
}
#[derive(Debug, Default)]
enum ResponsesCustomToolInputParserState {
#[default]
Prefix,
Value,
Passthrough,
Done,
}
const PREFIX_PARTS: [&[u8]; 4] = [b"{", b"\"input\"", b":", b"\""];
impl ResponsesCustomToolInputParser {
/// Decode available value text, retaining incomplete prefixes and escapes.
pub(super) fn feed(&mut self, chunk: &str) -> String {
let mut output = String::new();
for ch in chunk.chars() {
match self.state {
ResponsesCustomToolInputParserState::Prefix => self.feed_prefix(ch, &mut output),
ResponsesCustomToolInputParserState::Value => self.feed_value(ch, &mut output),
ResponsesCustomToolInputParserState::Passthrough => output.push(ch),
ResponsesCustomToolInputParserState::Done => break,
}
}
output
}
/// End parsing and return any unfinished prefix or escape unchanged.
/// Repeated calls and subsequent input produce no further output.
pub(super) fn finish(&mut self) -> String {
self.state = ResponsesCustomToolInputParserState::Done;
let mut output = std::mem::take(&mut self.prefix);
output.push_str(&std::mem::take(&mut self.escape));
output
}
fn feed_prefix(&mut self, ch: char, output: &mut String) {
self.prefix.push(ch);
let part = PREFIX_PARTS[self.prefix_part];
if ch == char::from(part[self.prefix_offset]) {
self.prefix_offset += 1;
if self.prefix_offset == part.len() {
self.prefix_part += 1;
self.prefix_offset = 0;
if self.prefix_part == PREFIX_PARTS.len() {
self.prefix.clear();
self.state = ResponsesCustomToolInputParserState::Value;
}
}
} else if self.prefix_offset != 0 || !ch.is_ascii_whitespace() {
output.push_str(&self.prefix);
self.prefix.clear();
self.state = ResponsesCustomToolInputParserState::Passthrough;
}
}
fn feed_value(&mut self, ch: char, output: &mut String) {
if self.escape.is_empty() {
match ch {
'\\' => self.escape.push(ch),
'"' => self.state = ResponsesCustomToolInputParserState::Done,
_ => output.push(ch),
}
return;
}
if ch == '"' && !self.escape.ends_with('\\') {
output.push_str(&std::mem::take(&mut self.escape));
self.state = ResponsesCustomToolInputParserState::Done;
return;
}
self.escape.push(ch);
if incomplete_escape(&self.escape) {
return;
}
let raw = std::mem::take(&mut self.escape);
match serde_json::from_str::<String>(&format!("\"{raw}\"")) {
Ok(decoded) => output.push_str(&decoded),
Err(_) => output.push_str(&raw),
}
}
}
fn incomplete_escape(raw: &str) -> bool {
if raw.len() < 6 {
return partial_unicode_escape(raw);
}
// Validate the first ASCII escape before inspecting a surrogate pair.
// Invalid escapes can contain multibyte characters at any position.
let Some(first) = raw.get(..6) else {
return false;
};
if !partial_unicode_escape(first) {
return false;
}
let high_surrogate =
u16::from_str_radix(&first[2..], 16).is_ok_and(|value| (0xD800..=0xDBFF).contains(&value));
high_surrogate && raw.len() < 12 && partial_unicode_escape(&raw[6..])
}
fn partial_unicode_escape(raw: &str) -> bool {
raw.len() <= 6
&& ("\\u".starts_with(raw)
|| raw
.strip_prefix("\\u")
.is_some_and(|digits| digits.chars().all(|ch| ch.is_ascii_hexdigit())))
}

7
deepseek-recipe/src/protocol/openai/responses/response/mod.rs Normal file
View File

@ -0,0 +1,7 @@
pub use chunk_generator::*;
pub use schema::*;
mod accumulator;
mod chunk_generator;
mod custom_tool_input;
mod schema;

321
deepseek-recipe/src/protocol/openai/responses/response/schema.rs Normal file
View File

@ -0,0 +1,321 @@
use serde::Serialize;
use strum_macros::IntoStaticStr;
use crate::response::ProtocolResponse;
use crate::stream::{CompletionUsage, PromptUsage};
use super::chunk_generator::ResponsesChunkGenerator;
/// A complete Responses response with event accumulation.
/// Deltas append, done events replace values, and full responses replace state.
/// Incremental events with mismatched indices, IDs, or types are ignored.
#[derive(Debug, Clone, Serialize)]
pub struct ResponsesResponse {
pub id: String,
pub object: &'static str,
pub created_at: u64,
pub model: String,
pub status: ResponsesStatus,
pub completed_at: Option<u64>,
pub error: Option<ResponsesError>,
pub incomplete_details: Option<ResponsesIncompleteDetails>,
pub output: Vec<ResponsesOutputItem>,
/// Usage is null until `Finish`; constructor token counts are ignored.
pub usage: Option<ResponsesUsage>,
}
/// Response and output item states generated from parsed output chunks.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ResponsesStatus {
InProgress,
Completed,
Incomplete,
}
/// Error fields for a complete response. Parsed output chunks contain no error
/// information, so generated responses set `error` to null.
#[derive(Debug, Clone, Serialize)]
pub struct ResponsesError {
pub code: String,
pub message: String,
}
#[derive(Debug, Clone, Serialize)]
pub struct ResponsesIncompleteDetails {
pub reason: ResponsesIncompleteReason,
}
#[derive(Debug, Clone, Copy, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ResponsesIncompleteReason {
MaxOutputTokens,
ContentFilter,
}
/// Output items retain their position in the response throughout streaming.
/// Item IDs identify output items in events. Call IDs associate subsequent
/// tool results with their calls.
#[derive(Debug, Clone, Serialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ResponsesOutputItem {
Message {
id: String,
status: ResponsesStatus,
role: ResponsesRole,
phase: ResponsesPhase,
content: Vec<ResponsesContentPart>,
},
Reasoning {
id: String,
status: ResponsesStatus,
content: Vec<ResponsesContentPart>,
summary: Vec<()>,
},
FunctionCall {
id: String,
status: ResponsesStatus,
call_id: String,
name: String,
#[serde(skip_serializing_if = "Option::is_none")]
namespace: Option<String>,
arguments: String,
},
CustomToolCall {
id: String,
status: ResponsesStatus,
call_id: String,
name: String,
input: String,
},
}
/// Content produced from parsed model output.
/// Annotations and log probabilities are emitted as empty arrays.
#[derive(Debug, Clone, Serialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ResponsesContentPart {
OutputText {
annotations: Vec<()>,
logprobs: Vec<()>,
text: String,
},
ReasoningText {
text: String,
},
}
#[derive(Debug, Clone, Copy, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ResponsesRole {
Assistant,
}
/// Answer content immediately before a tool call uses `commentary` in the
/// completed output item.
#[derive(Debug, Clone, Copy, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ResponsesPhase {
Commentary,
FinalAnswer,
}
/// Backend prompt counts combined with completion counts accumulated by the
/// stream processor.
#[derive(Debug, Clone, Copy, Serialize)]
pub struct ResponsesUsage {
pub input_tokens: usize,
pub input_tokens_details: ResponsesInputTokensDetails,
pub output_tokens: usize,
pub output_tokens_details: ResponsesOutputTokensDetails,
pub total_tokens: usize,
}
#[derive(Debug, Clone, Copy, Serialize)]
pub struct ResponsesInputTokensDetails {
pub cached_tokens: usize,
}
#[derive(Debug, Clone, Copy, Serialize)]
pub struct ResponsesOutputTokensDetails {
/// Always zero because the backend reports no reasoning token count.
pub reasoning_tokens: usize,
}
impl From<(PromptUsage, CompletionUsage)> for ResponsesUsage {
fn from((prompt, completion): (PromptUsage, CompletionUsage)) -> Self {
Self {
input_tokens: prompt.prompt_tokens,
input_tokens_details: ResponsesInputTokensDetails {
cached_tokens: prompt.prompt_cache_hit_tokens,
},
output_tokens: completion.completion_tokens,
output_tokens_details: ResponsesOutputTokensDetails {
reasoning_tokens: 0,
},
total_tokens: prompt.prompt_tokens + completion.completion_tokens,
}
}
}
/// A typed Responses SSE event. The `type` field is also the SSE event name.
/// Sequence numbers increase across all event kinds within one response.
#[derive(Debug, Clone, Serialize, IntoStaticStr)]
#[serde(tag = "type")]
pub enum ResponsesStreamEvent {
#[serde(rename = "response.created")]
#[strum(serialize = "response.created")]
Created {
response: ResponsesResponse,
sequence_number: usize,
},
#[serde(rename = "response.in_progress")]
#[strum(serialize = "response.in_progress")]
InProgress {
response: ResponsesResponse,
sequence_number: usize,
},
#[serde(rename = "response.output_item.added")]
#[strum(serialize = "response.output_item.added")]
OutputItemAdded {
item: ResponsesOutputItem,
output_index: usize,
sequence_number: usize,
},
#[serde(rename = "response.content_part.added")]
#[strum(serialize = "response.content_part.added")]
ContentPartAdded {
content_index: usize,
item_id: String,
output_index: usize,
part: ResponsesContentPart,
sequence_number: usize,
},
#[serde(rename = "response.output_text.delta")]
#[strum(serialize = "response.output_text.delta")]
OutputTextDelta {
content_index: usize,
delta: String,
item_id: String,
logprobs: Vec<()>,
output_index: usize,
sequence_number: usize,
},
#[serde(rename = "response.output_text.done")]
#[strum(serialize = "response.output_text.done")]
OutputTextDone {
content_index: usize,
item_id: String,
logprobs: Vec<()>,
output_index: usize,
sequence_number: usize,
text: String,
},
#[serde(rename = "response.reasoning_text.delta")]
#[strum(serialize = "response.reasoning_text.delta")]
ReasoningTextDelta {
content_index: usize,
delta: String,
item_id: String,
output_index: usize,
sequence_number: usize,
},
#[serde(rename = "response.reasoning_text.done")]
#[strum(serialize = "response.reasoning_text.done")]
ReasoningTextDone {
content_index: usize,
item_id: String,
output_index: usize,
sequence_number: usize,
text: String,
},
#[serde(rename = "response.function_call_arguments.delta")]
#[strum(serialize = "response.function_call_arguments.delta")]
FunctionCallArgumentsDelta {
delta: String,
item_id: String,
output_index: usize,
sequence_number: usize,
},
#[serde(rename = "response.function_call_arguments.done")]
#[strum(serialize = "response.function_call_arguments.done")]
FunctionCallArgumentsDone {
arguments: String,
item_id: String,
output_index: usize,
sequence_number: usize,
},
#[serde(rename = "response.custom_tool_call_input.delta")]
#[strum(serialize = "response.custom_tool_call_input.delta")]
CustomToolCallInputDelta {
delta: String,
item_id: String,
output_index: usize,
sequence_number: usize,
},
#[serde(rename = "response.custom_tool_call_input.done")]
#[strum(serialize = "response.custom_tool_call_input.done")]
CustomToolCallInputDone {
input: String,
item_id: String,
output_index: usize,
sequence_number: usize,
},
#[serde(rename = "response.content_part.done")]
#[strum(serialize = "response.content_part.done")]
ContentPartDone {
content_index: usize,
item_id: String,
output_index: usize,
part: ResponsesContentPart,
sequence_number: usize,
},
#[serde(rename = "response.output_item.done")]
#[strum(serialize = "response.output_item.done")]
OutputItemDone {
item: ResponsesOutputItem,
output_index: usize,
sequence_number: usize,
},
#[serde(rename = "response.completed")]
#[strum(serialize = "response.completed")]
Completed {
response: ResponsesResponse,
sequence_number: usize,
},
#[serde(rename = "response.incomplete")]
#[strum(serialize = "response.incomplete")]
Incomplete {
response: ResponsesResponse,
sequence_number: usize,
},
}
impl ProtocolResponse for ResponsesResponse {
type ChunkGenerator = ResponsesChunkGenerator;
fn new(
id: String,
model: String,
created: u64,
_prompt_tokens: usize,
_prompt_cache_hit_tokens: usize,
) -> Self {
Self {
id,
object: "response",
created_at: created,
model,
status: ResponsesStatus::InProgress,
completed_at: None,
error: None,
incomplete_details: None,
output: vec![],
usage: None,
}
}
fn chunk_event_type(chunk: &ResponsesStreamEvent) -> Option<&'static str> {
Some(chunk.into())
}
}

131
deepseek-recipe/src/protocol/validation.rs Normal file
View File

@ -0,0 +1,131 @@
use deepseek_recipe_core::messages::InputMessage;
use deepseek_recipe_core::multimodal::IMAGE_SPECIAL_TOKEN;
use deepseek_recipe_core::tools::ToolDefinition;
use deepseek_recipe_core::util::json_formatter::stringify_python_style;
use crate::request::ConversionError;
/// Return the error for input text carrying [`IMAGE_SPECIAL_TOKEN`].
///
/// An adapter inserts one placeholder per image source. A placeholder in input
/// text would reach the prompt without an image source, so the adapters reject
/// input text that spells it out.
pub(super) fn image_special_token_not_allowed() -> ConversionError {
ConversionError::bad_request(format!(
"The sub-string \"{IMAGE_SPECIAL_TOKEN}\" in your input is not allowed for this API. Please remove \
\"{IMAGE_SPECIAL_TOKEN}\" from your input and try again."
))
}
/// Return the error for converted tool text carrying [`IMAGE_SPECIAL_TOKEN`].
///
/// A rendered tool prompt contains the name, description, and parameter schema
/// of every definition, and the tool calls of a historical assistant turn
/// contribute their names and arguments. Declared tool names are already
/// restricted by [`validate_tool_name`]; historical call names and arguments are
/// not.
pub(super) fn validate_tool_text(
tools: &[ToolDefinition],
messages: &[InputMessage],
) -> Result<(), ConversionError> {
if tools
.iter()
.any(tool_definition_contains_image_special_token)
|| messages.iter().any(tool_calls_contain_image_special_token)
{
return Err(image_special_token_not_allowed());
}
Ok(())
}
/// Return whether a tool definition carries [`IMAGE_SPECIAL_TOKEN`].
fn tool_definition_contains_image_special_token(tool: &ToolDefinition) -> bool {
tool.name.contains(IMAGE_SPECIAL_TOKEN)
|| tool
.description
.as_deref()
.is_some_and(|description| description.contains(IMAGE_SPECIAL_TOKEN))
|| stringify_python_style(&tool.parameters).contains(IMAGE_SPECIAL_TOKEN)
}
/// Return whether the tool calls of one message carry [`IMAGE_SPECIAL_TOKEN`].
fn tool_calls_contain_image_special_token(message: &InputMessage) -> bool {
let InputMessage::Assistant { tool_calls, .. } = message else {
return false;
};
tool_calls.as_ref().is_some_and(|tool_calls| {
tool_calls.iter().any(|tool_call| {
tool_call.name.contains(IMAGE_SPECIAL_TOKEN)
|| tool_call.arguments.contains(IMAGE_SPECIAL_TOKEN)
})
})
}
pub(super) fn validate_max_tokens(value: Option<u32>, field: &str) -> Result<(), ConversionError> {
if value == Some(0) {
return Err(ConversionError::bad_request(format!(
"{field} must be greater than zero"
)));
}
Ok(())
}
pub(super) fn validate_sampling(
temperature: Option<f32>,
top_p: Option<f32>,
) -> Result<(), ConversionError> {
if temperature.is_some_and(|value| !(0.0..=2.0).contains(&value)) {
return Err(ConversionError::bad_request(
"temperature must be in [0, 2]",
));
}
if top_p.is_some_and(|value| !value.is_finite() || value <= 0.0 || value > 1.0) {
return Err(ConversionError::bad_request("top_p must be in (0, 1]"));
}
Ok(())
}
pub(super) fn validate_tool_name(name: &str, path: &str) -> Result<(), ConversionError> {
if name.is_empty()
|| name.len() > 128
|| !name
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_' || byte == b'-')
{
return Err(ConversionError::bad_request(format!(
"{path}: expected 1-128 ASCII letters, digits, underscores or hyphens"
)));
}
Ok(())
}
pub(super) fn validate_tool_parameters(
schema: &serde_json::Value,
path: &str,
) -> Result<(), ConversionError> {
if schema.get("type").and_then(serde_json::Value::as_str) != Some("object") {
return Err(ConversionError::bad_request(format!(
"{path} must be a JSON Schema of type object"
)));
}
// Schema validation uses local data with HTTP and file resolution disabled.
jsonschema::JSONSchema::compile(schema)
.map(|_| ())
.map_err(|error| ConversionError::bad_request(format!("{path}: {error}")))
}
pub(super) fn contains_json_instruction(messages: &[InputMessage]) -> bool {
messages.iter().any(|message| {
let content = match message {
InputMessage::System { content }
| InputMessage::User { content, .. }
| InputMessage::Assistant { content, .. }
| InputMessage::Tool { content, .. }
| InputMessage::LatestReminder { content } => content,
};
content
.as_bytes()
.windows(4)
.any(|part| part.eq_ignore_ascii_case(b"json"))
})
}

109
deepseek-recipe/src/request/mod.rs Normal file
View File

@ -0,0 +1,109 @@
//! Shared request data and the conversion contract for protocol adapters.
use deepseek_recipe_core::conversation::Conversation;
use crate::response::ProtocolResponse;
use crate::stream::state_machine::ParsingOptions;
pub use options::{ConversionOptions, WebSearchBehavior};
mod options;
/// Inference parameters extracted from a protocol request.
///
/// Unspecified parameters remain `None`; model-specific defaults belong to the
/// caller.
#[derive(Debug, Default)]
pub struct InferenceOptions {
pub max_tokens: Option<u32>,
pub temperature: Option<f32>,
pub top_p: Option<f32>,
/// Passed through for applications that support a thinking budget.
pub thinking_budget_tokens: Option<u64>,
/// Passed through for the caller to control parallel tool execution.
pub disable_parallel_tool_use: Option<bool>,
}
/// A conversation with inference parameters and output parsing options.
///
/// Protocol adapters produce this representation. The caller renders the
/// conversation, tokenizes the prompt, and constructs the inference request.
/// Protocol-specific metadata is available on the protocol request before
/// conversion.
#[derive(Debug)]
pub struct ConversationRequest {
pub conversation: Conversation,
pub inference_options: InferenceOptions,
pub parsing_options: ParsingOptions,
/// Original model name, when exposed by the adapter. The caller resolves it.
pub model: Option<String>,
pub stream: bool,
}
impl ConversationRequest {
/// Construct a request with default inference, parsing, and transport options.
pub fn new(conversation: Conversation) -> Self {
Self {
conversation,
inference_options: InferenceOptions::default(),
parsing_options: ParsingOptions::default(),
model: None,
stream: false,
}
}
}
/// Convert protocol input into a shared conversation request.
pub trait ProtocolRequest: 'static + Send + Sync {
type Response: ProtocolResponse;
/// Validate and convert the protocol's supported request fields.
///
/// Extract any protocol-specific metadata needed by the application first.
///
/// # Errors
///
/// Returns [`ConversionError::BadRequest`] for invalid or unsupported input, or
/// [`ConversionError::Internal`] if an internal conversion step fails.
fn convert(self, options: ConversionOptions) -> Result<ConversationRequest, ConversionError>;
/// Construct protocol events using the converted request's resolved settings.
fn chunk_generator(
request: &ConversationRequest,
id: String,
model: String,
) -> <Self::Response as ProtocolResponse>::ChunkGenerator;
}
/// A failure while converting protocol input into a conversation request.
#[derive(Debug)]
pub enum ConversionError {
/// The input is invalid or uses an unsupported feature.
BadRequest { detail: String },
/// An internal conversion operation failed.
Internal { detail: String },
}
impl ConversionError {
pub fn bad_request(detail: impl Into<String>) -> Self {
ConversionError::BadRequest {
detail: detail.into(),
}
}
pub fn internal(detail: impl Into<String>) -> Self {
ConversionError::Internal {
detail: detail.into(),
}
}
}
impl std::fmt::Display for ConversionError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::BadRequest { detail } | Self::Internal { detail } => f.write_str(detail),
}
}
}
impl std::error::Error for ConversionError {}

70
deepseek-recipe/src/request/options.rs Normal file
View File

@ -0,0 +1,70 @@
/// How an adapter handles supported `web_search` declarations and tool choices.
/// Messages also applies this setting to assistant server-tool content blocks.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
#[non_exhaustive]
pub enum WebSearchBehavior {
/// Drop matching declarations and choices, plus Messages server-tool blocks.
#[default]
Ignore,
/// Reject matching declarations and choices, plus Messages server-tool blocks.
Reject,
}
/// Defaults for conversion from protocol requests.
#[derive(Debug, Clone, Copy)]
#[non_exhaustive]
pub struct ConversionOptions {
/// Default thinking mode for requests with an unspecified thinking mode.
///
/// Defaults to `true`.
pub default_thinking_mode: bool,
/// Behavior for Responses `web_search` tool declarations and tool choices.
/// Historical `web_search_call` input items are ignored under either setting.
///
/// Defaults to [`WebSearchBehavior::Ignore`].
pub responses_web_search: WebSearchBehavior,
/// Behavior for the Messages `web_search` server tool, its named tool
/// choice, and its `server_tool_use` / `web_search_tool_result` blocks.
///
/// Defaults to [`WebSearchBehavior::Reject`].
pub messages_web_search: WebSearchBehavior,
}
impl ConversionOptions {
/// Construct options that enable thinking for requests that do not
/// specify a thinking mode.
pub const fn new() -> Self {
Self {
default_thinking_mode: true,
responses_web_search: WebSearchBehavior::Ignore,
messages_web_search: WebSearchBehavior::Reject,
}
}
/// Set the default thinking mode, consuming and returning `self`.
#[must_use]
pub fn with_default_thinking_mode(mut self, default_thinking_mode: bool) -> Self {
self.default_thinking_mode = default_thinking_mode;
self
}
/// Set the Responses `web_search` behavior, consuming and returning `self`.
#[must_use]
pub fn with_responses_web_search(mut self, responses_web_search: WebSearchBehavior) -> Self {
self.responses_web_search = responses_web_search;
self
}
/// Set the Messages `web_search` behavior, consuming and returning `self`.
#[must_use]
pub fn with_messages_web_search(mut self, messages_web_search: WebSearchBehavior) -> Self {
self.messages_web_search = messages_web_search;
self
}
}
impl Default for ConversionOptions {
fn default() -> Self {
Self::new()
}
}

35
deepseek-recipe/src/response.rs Normal file
View File

@ -0,0 +1,35 @@
//! Protocol response construction and event accumulation.
use serde::Serialize;
use crate::stream::ChunkGenerator;
use crate::util::append_delta::AppendDelta;
/// A protocol response that can accumulate events from its chunk generator.
pub trait ProtocolResponse:
'static + Send + Serialize + AppendDelta<<Self::ChunkGenerator as ChunkGenerator>::Chunk>
{
/// Generator for this protocol's response events.
type ChunkGenerator: ChunkGenerator;
/// Construct a response with caller-supplied identity and input token usage.
fn new(
id: String,
model: String,
created: u64,
prompt_tokens: usize,
prompt_cache_hit_tokens: usize,
) -> Self;
/// Return the SSE event name, or `None` for an unnamed data event.
fn chunk_event_type(
_chunk: &<Self::ChunkGenerator as ChunkGenerator>::Chunk,
) -> Option<&'static str> {
None
}
/// Return a final transport sentinel if the protocol requires one.
fn done_message() -> Option<String> {
None
}
}

71
deepseek-recipe/src/stream/decoder.rs Normal file
View File

@ -0,0 +1,71 @@
//! Decode token ids into text for the stream processor.
use std::sync::Arc;
use tokenizers::Tokenizer;
use super::StreamError;
/// Replacement character emitted while a multi-token character is incomplete.
const REPLACEMENT_CHARACTER: char = '\u{FFFD}';
/// Decode token ids into text.
///
/// Implemented for [`Tokenizer`]; other backends can implement it to feed a
/// stream processor.
pub trait TokenizerDecoder: Send {
/// Decode `ids` into text, skipping special tokens when requested.
fn decode_ids(&self, ids: &[u32], skip_special_tokens: bool) -> Result<String, String>;
}
impl TokenizerDecoder for Tokenizer {
fn decode_ids(&self, ids: &[u32], skip_special_tokens: bool) -> Result<String, String> {
self.decode(ids, skip_special_tokens)
.map_err(|error| error.to_string())
}
}
impl<T> TokenizerDecoder for Arc<T>
where
T: TokenizerDecoder + Sync,
{
fn decode_ids(&self, ids: &[u32], skip_special_tokens: bool) -> Result<String, String> {
(**self).decode_ids(ids, skip_special_tokens)
}
}
/// Buffer token ids until they decode into text.
pub struct StreamDecoder {
decoder: Box<dyn TokenizerDecoder>,
ids: Vec<u32>,
pending: usize,
}
impl StreamDecoder {
pub fn new(decoder: Box<dyn TokenizerDecoder>) -> Self {
Self {
decoder,
ids: Vec::new(),
pending: 0,
}
}
/// Return the text contributed by the buffered token ids and the number of
/// ids it accounts for, or `None` while no text is available yet.
///
/// Special tokens drive the parser's state machine, so they stay in the
/// decoded text. A decoder failure is returned as a stream error.
pub fn decode(&mut self, token_id: u32) -> Result<Option<(String, usize)>, StreamError> {
self.pending += 1;
self.ids.push(token_id);
let content = self
.decoder
.decode_ids(&self.ids, false)
.map_err(|detail| StreamError::Decode { detail })?;
if content.is_empty() || content.ends_with(REPLACEMENT_CHARACTER) {
return Ok(None);
}
self.ids.clear();
Ok(Some((content, std::mem::take(&mut self.pending))))
}
}

61
deepseek-recipe/src/stream/inference.rs Normal file
View File

@ -0,0 +1,61 @@
/// Prompt-side token counts supplied by the inference backend. Unspecified
/// counts default to zero.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct PromptUsage {
/// Total input tokens, including tokens read from the prompt cache.
pub prompt_tokens: usize,
/// Input tokens read from the prompt cache.
pub prompt_cache_hit_tokens: usize,
}
/// Completion-side token counts accumulated by the stream processor from
/// `Text.content_tokens` and token IDs that decode into text.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct CompletionUsage {
/// Tokens accounted for by processed chunks, including discarded markup
/// and the entire chunk containing a local stop sequence. Undecoded trailing
/// token IDs and chunks after the stop are excluded.
pub completion_tokens: usize,
}
/// Text and metadata received from an inference backend.
#[derive(Debug)]
pub enum InferenceChunk {
/// Initial metadata reported before the model produces output.
Ready {
system_fingerprint: Option<String>,
prompt_usage: PromptUsage,
},
/// Text produced by the model, and the tokens that text accounts for.
Text {
content: String,
content_tokens: usize,
},
/// One generated token id, decoded by the processor's tokenizer.
Token { token_id: u32 },
/// Terminal metadata reported after the model stops.
Finish {
finish_reason: InferenceFinishReason,
},
}
/// The finish reason reported by the inference backend.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum InferenceFinishReason {
Stop,
Length,
ContentFilter,
}
/// Final result after applying the stream parser's tool and stop-sequence
/// rules.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum FinishReason {
Stop,
Length,
ToolCalls,
StopSequence,
ContentFilter,
/// The source ended without an explicit finish reason.
EndOfStream,
}

74
deepseek-recipe/src/stream/mod.rs Normal file
View File

@ -0,0 +1,74 @@
//! Parse model output and turn it into protocol-specific response events.
pub use decoder::TokenizerDecoder;
pub use inference::{
CompletionUsage, FinishReason, InferenceChunk, InferenceFinishReason, PromptUsage,
};
pub use processor::StreamProcessor;
mod decoder;
mod inference;
mod processor;
pub mod state_machine;
/// An error that ends stream processing before normal completion.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum StreamError {
/// A token-id chunk arrived without an attached tokenizer.
MissingTokenizer,
/// The tokenizer failed to decode buffered token ids.
Decode {
/// The decoder's failure message.
detail: String,
},
}
impl std::fmt::Display for StreamError {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::MissingTokenizer => {
formatter.write_str("token chunk received without a tokenizer")
}
Self::Decode { detail } => write!(formatter, "failed to decode token ids: {detail}"),
}
}
}
impl std::error::Error for StreamError {}
/// A parsed piece of model output, before protocol-specific event conversion.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum OutputChunk {
Start {
system_fingerprint: Option<String>,
usage: PromptUsage,
},
Raw {
content: String,
},
Reasoning {
content: String,
},
ToolCallBegin,
ToolCall {
tool_name: String,
arguments: String,
},
ToolArgumentsDelta {
content: String,
},
Finish {
reason: FinishReason,
stop_sequence: Option<String>,
usage: CompletionUsage,
},
}
/// Convert parsed output into a protocol's response events.
pub trait ChunkGenerator: Send + 'static {
/// Event type emitted by this protocol.
type Chunk;
/// Produce zero or more events for one parsed output chunk.
fn generate(&mut self, chunk: OutputChunk) -> impl Future<Output = Vec<Self::Chunk>> + Send;
}

369
deepseek-recipe/src/stream/processor.rs Normal file
View File

@ -0,0 +1,369 @@
use std::collections::VecDeque;
use std::pin::pin;
use async_stream::stream;
use tokio_stream::{Stream, StreamExt};
use super::decoder::{StreamDecoder, TokenizerDecoder};
use super::inference::{
CompletionUsage, FinishReason, InferenceChunk, InferenceFinishReason, PromptUsage,
};
use super::state_machine::{OutputAction, OutputActionSegment, ParsingOptions, StateMachine};
use super::{ChunkGenerator, OutputChunk, StreamError};
/// Parse backend inference chunks and produce protocol response events.
pub struct StreamProcessor<G> {
generator: G,
options: ParsingOptions,
decoder: Option<StreamDecoder>,
}
impl<G> StreamProcessor<G>
where
G: ChunkGenerator,
{
/// Combine a protocol event generator with output parsing options.
pub fn new(generator: G, options: ParsingOptions) -> Self {
Self {
generator,
options,
decoder: None,
}
}
/// Decode `InferenceChunk::Token` ids with the supplied decoder.
///
/// Ids that contribute text count as completion tokens, including the ids
/// buffered while a multi-token character was incomplete. IDs still buffered
/// at the end of input contribute neither text nor completion usage.
/// Without a decoder, a token chunk fails the stream with
/// [`StreamError::MissingTokenizer`].
pub fn with_tokenizer(mut self, decoder: impl TokenizerDecoder + 'static) -> Self {
self.decoder = Some(StreamDecoder::new(Box::new(decoder)));
self
}
/// Consume inference chunks until a finish chunk, matched stop sequence, or EOF.
///
/// A ready chunk received before output starts supplies the initial prompt
/// usage and fingerprint. Without it, the start event uses zero prompt usage
/// and no fingerprint. Later ready chunks do not update emitted metadata.
/// Completion usage accumulates each processed chunk's token count, including
/// the entire chunk containing a stop sequence; later chunks are not read.
/// Missing tokenizers and decoder failures yield an error and end the stream
/// without a normal finish event.
pub fn process(
self,
inference: impl Stream<Item = InferenceChunk> + Send,
) -> impl Stream<Item = Result<G::Chunk, StreamError>> + Send
where
G::Chunk: Send,
{
let mut generator = self.generator;
let options = self.options;
let mut decoder = self.decoder;
stream! {
let mut inference = pin!(inference);
let mut state_machine = StateMachine::new(options);
let mut stashed = StashedChunks::new();
let mut started = false;
let mut prompt_usage = PromptUsage::default();
let mut completion_usage = CompletionUsage::default();
let mut backend_finish = None;
let mut stop_sequence = None;
while let Some(chunk) = inference.next().await {
let (content, content_tokens) = match chunk {
InferenceChunk::Ready {
system_fingerprint,
prompt_usage: ready_usage,
} => {
prompt_usage = ready_usage;
if !started {
for out in generator.generate(OutputChunk::Start { system_fingerprint, usage: prompt_usage }).await {
yield Ok(out);
}
started = true;
}
continue;
}
InferenceChunk::Finish { finish_reason } => {
if !started {
for out in generator.generate(OutputChunk::Start { system_fingerprint: None, usage: prompt_usage }).await {
yield Ok(out);
}
started = true;
}
backend_finish = Some(finish_reason);
break;
}
InferenceChunk::Text {
content,
content_tokens,
} => (content, content_tokens),
InferenceChunk::Token { token_id } => {
let Some(decoder) = decoder.as_mut() else {
yield Err(StreamError::MissingTokenizer);
return;
};
match decoder.decode(token_id) {
Ok(Some(decoded)) => decoded,
Ok(None) => continue,
Err(error) => {
yield Err(error);
return;
}
}
}
};
if !started {
for out in generator.generate(OutputChunk::Start { system_fingerprint: None, usage: prompt_usage }).await {
yield Ok(out);
}
started = true;
}
completion_usage.completion_tokens += content_tokens;
let actions = state_machine.feed(&content);
stashed.push(Some(content));
for out in stashed.apply_actions(actions, &mut generator).await {
yield Ok(out);
}
stop_sequence = stashed.take_stop_sequence();
if stop_sequence.is_some() {
break;
}
}
if !started {
for out in generator.generate(OutputChunk::Start { system_fingerprint: None, usage: prompt_usage }).await {
yield Ok(out);
}
}
let actions = state_machine.finish();
for out in stashed.apply_actions(actions, &mut generator).await {
yield Ok(out);
}
// Combine the parts of a stop sequence matched across source chunks.
if let Some(tail) = stashed.take_stop_sequence() {
stop_sequence.get_or_insert_with(String::new).push_str(&tail);
}
let reason = if stop_sequence.is_some() {
FinishReason::StopSequence
} else {
match backend_finish {
Some(InferenceFinishReason::Stop) if stashed.has_tool_calls => {
FinishReason::ToolCalls
}
Some(InferenceFinishReason::Stop) => FinishReason::Stop,
Some(InferenceFinishReason::Length) => FinishReason::Length,
Some(InferenceFinishReason::ContentFilter) => FinishReason::ContentFilter,
None => FinishReason::EndOfStream,
}
};
let finish = OutputChunk::Finish {
reason,
stop_sequence,
usage: completion_usage,
};
for out in generator.generate(finish).await {
yield Ok(out);
}
}
}
}
struct StashedChunks {
chunks: VecDeque<Option<String>>,
last_stashed_action: Option<OutputActionSegment>,
stashed_tool_name: String,
last_pop_action: OutputAction,
stop_sequence: Option<String>,
has_tool_calls: bool,
}
impl StashedChunks {
fn new() -> Self {
Self {
chunks: VecDeque::new(),
last_stashed_action: None,
stashed_tool_name: String::new(),
last_pop_action: OutputAction::Skip,
stop_sequence: None,
has_tool_calls: false,
}
}
fn push(&mut self, content: Option<String>) {
self.chunks.push_back(content);
}
fn take_stop_sequence(&mut self) -> Option<String> {
self.stop_sequence.take()
}
fn pop(&mut self, action: OutputActionSegment) -> Option<OutputChunk> {
let mut front_chunk = self.chunks.pop_front()?;
let front_len = front_chunk
.as_ref()
.map(|content| content.len())
.unwrap_or(0);
if action.len < front_len {
let remaining = front_chunk
.as_mut()
.map(|content| content.split_off(action.len));
self.chunks.push_front(remaining);
}
let last_pop_action = self.last_pop_action;
self.last_pop_action = action.action;
match action.action {
OutputAction::Raw => non_empty(front_chunk).map(|content| OutputChunk::Raw { content }),
OutputAction::Reasoning => {
non_empty(front_chunk).map(|content| OutputChunk::Reasoning { content })
}
OutputAction::ToSpace => Some(OutputChunk::Raw {
content: " ".to_string(),
}),
OutputAction::Skip | OutputAction::SkipInvalid { .. } => None,
OutputAction::StopSequence => {
let content = front_chunk.unwrap_or_default();
self.stop_sequence
.get_or_insert_with(String::new)
.push_str(&content);
None
}
OutputAction::ToolCallBegin => {
if last_pop_action != OutputAction::ToolCallBegin {
Some(OutputChunk::ToolCallBegin)
} else {
None
}
}
OutputAction::ToolName => {
if let Some(content) = front_chunk {
self.stashed_tool_name.push_str(&content);
}
None
}
OutputAction::ToolNameEnd => {
if last_pop_action != OutputAction::ToolNameEnd {
self.has_tool_calls = true;
let tool_name = std::mem::take(&mut self.stashed_tool_name);
Some(OutputChunk::ToolCall {
tool_name,
arguments: String::new(),
})
} else {
None
}
}
OutputAction::LabelToolCallArguments { label, .. } => {
if last_pop_action != action.action {
Some(OutputChunk::ToolArgumentsDelta {
content: label.to_string(),
})
} else {
None
}
}
OutputAction::RawToolCallArguments { string } => {
let content = if string {
escape_json_string(front_chunk.unwrap_or_default())
} else {
front_chunk.unwrap_or_default()
};
Some(OutputChunk::ToolArgumentsDelta { content })
}
OutputAction::ToolCallArgumentsEnd { output } => {
if last_pop_action == action.action {
return None;
}
output.map(|output| OutputChunk::ToolArgumentsDelta {
content: output.to_string(),
})
}
OutputAction::Label(label) => {
if last_pop_action != action.action {
Some(OutputChunk::Raw {
content: label.to_string(),
})
} else {
None
}
}
}
}
fn apply_whole_chunks(
&mut self,
action: &mut OutputActionSegment,
outputs: &mut Vec<OutputChunk>,
) {
loop {
let front_len = self
.chunks
.front()
.map(|content| content.as_ref().map(|content| content.len()).unwrap_or(0));
if front_len.is_some_and(|front_len| front_len <= action.len) {
let front_len = front_len.unwrap();
if let Some(chunk) = self.pop(OutputActionSegment::new(action.action, front_len)) {
outputs.push(chunk);
}
action.len -= front_len;
} else {
return;
}
}
}
async fn apply_actions<G>(
&mut self,
actions: Vec<OutputActionSegment>,
generator: &mut G,
) -> Vec<G::Chunk>
where
G: ChunkGenerator,
{
let mut outputs = Vec::new();
let mut last_stashed_action = self.last_stashed_action.take();
for action in actions {
if let Some(last_stashed_action) = &mut last_stashed_action {
if action.action == last_stashed_action.action {
last_stashed_action.len += action.len;
} else {
self.apply_whole_chunks(last_stashed_action, &mut outputs);
if last_stashed_action.len > 0
&& let Some(chunk) = self.pop(last_stashed_action.clone())
{
outputs.push(chunk);
}
*last_stashed_action = action;
}
} else {
last_stashed_action = Some(action);
}
}
if let Some(last_stashed_action) = &mut last_stashed_action {
self.apply_whole_chunks(last_stashed_action, &mut outputs);
}
self.last_stashed_action = last_stashed_action;
let mut chunks = Vec::new();
for output in outputs {
chunks.extend(generator.generate(output).await);
}
chunks
}
}
fn non_empty(content: Option<String>) -> Option<String> {
content.filter(|content| !content.is_empty())
}
fn escape_json_string(content: String) -> String {
match serde_json::to_string(&content) {
Ok(escaped) if escaped.len() > 2 => escaped[1..escaped.len() - 1].to_string(),
_ => String::new(),
}
}

577
deepseek-recipe/src/stream/state_machine.rs Normal file
View File

@ -0,0 +1,577 @@
//! Incrementally recognize reasoning, tool-call markup, and stop sequences.
const JSON_BEGIN_LABELS: [&str; 2] = ["```json\n", "```\n"];
/// Position within the reasoning section when parsing begins.
#[derive(Debug, Clone, Copy)]
pub enum ReasoningStage {
/// Begin reasoning while discarding leading newlines.
Start,
/// Continue reasoning while preserving leading newlines.
Reasoning,
/// Continue answer content while preserving leading newlines.
Content,
}
impl ReasoningStage {
pub fn start_from_reasoning(&self) -> bool {
matches!(self, Self::Start | Self::Reasoning)
}
}
const RAW_JSON_BEGIN_LABELS: [&str; 2] = ["{", "["];
const JSON_END_LABEL: &str = "\n```";
const DSML_BEGIN_LABEL: &str = "<｜DSML｜";
const DSML_END_LABEL: &str = "</｜DSML｜";
const DSML_TOOL_CALLS_END_LABELS: [&str; 2] = ["</｜DSML｜tool_calls>", "</｜DSML｜ calls>"];
pub const REASONING_END_LABEL: &str = "</think>";
/// Settings for parsing inference output.
#[derive(Debug, Clone)]
pub struct ParsingOptions {
/// Parse model-specific tool-call markup.
pub parse_tool_calls: bool,
/// Begin parsing inside the tool-call block specified by the prompt.
/// Takes precedence over `reasoning_initial_stage`.
pub tool_call_initial_stage: bool,
/// Recognize JSON fences or a raw `{` or `[` prefix and replace surrounding
/// text with spaces. This does not validate JSON syntax.
pub parse_json_output: bool,
/// Initial reasoning or answer stage. `None` starts answer content with
/// leading-newline removal.
pub reasoning_initial_stage: Option<ReasoningStage>,
/// Match stop sequences in ordinary answer text and detected JSON output.
/// Reasoning, tool-call markup, and surrounding JSON-mode text are excluded.
/// The state machine discards empty sequences during construction.
pub stop_sequences: Vec<String>,
}
impl Default for ParsingOptions {
fn default() -> Self {
Self {
parse_tool_calls: true,
tool_call_initial_stage: false,
parse_json_output: false,
reasoning_initial_stage: None,
stop_sequences: Vec::new(),
}
}
}
/// An incremental parser whose output segments refer to input byte lengths.
pub struct StateMachine {
options: ParsingOptions,
state: State,
}
impl StateMachine {
pub fn new(mut options: ParsingOptions) -> Self {
options
.stop_sequences
.retain(|sequence| !sequence.is_empty());
let stage = if options.tool_call_initial_stage {
Stage::ToolCalls
} else if let Some(reasoning_initial_stage) = &options.reasoning_initial_stage {
match reasoning_initial_stage {
ReasoningStage::Start => Stage::Reasoning { is_leading: true },
ReasoningStage::Reasoning => Stage::Reasoning { is_leading: false },
ReasoningStage::Content => Stage::Common { is_leading: false },
}
} else {
Stage::Common { is_leading: true }
};
let state = State::new(stage, &options);
Self { options, state }
}
/// Parse the next source chunk, retaining incomplete markers for later input.
pub fn feed(&mut self, content: &str) -> Vec<OutputActionSegment> {
self.state.feed(content, &self.options)
}
/// Process buffered input when the source ends.
pub fn finish(self) -> Vec<OutputActionSegment> {
self.state.finish().1
}
}
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum InvalidTokenKind {
ExtraEndOfThinking,
ContentAfterFinished,
}
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum OutputAction {
Raw,
ToSpace,
SkipInvalid { kind: InvalidTokenKind },
Skip,
StopSequence,
ToolCallBegin,
ToolName,
ToolNameEnd,
LabelToolCallArguments { label: &'static str, id: u32 },
RawToolCallArguments { string: bool },
ToolCallArgumentsEnd { output: Option<&'static str> },
Reasoning,
Label(&'static str),
}
#[derive(Debug, Clone)]
pub struct OutputActionSegment {
pub action: OutputAction,
pub len: usize,
}
impl OutputActionSegment {
pub fn new(action: OutputAction, len: usize) -> Self {
Self { action, len }
}
}
#[derive(Debug, Clone, Copy)]
enum Stage {
Common { is_leading: bool },
Json,
MatchedJson,
ToolCalls,
ToolName,
ToolCallArguments { is_leading: bool },
ToolCallParamName,
ToolCallParamType,
ToolCallParamValue { string: bool },
Reasoning { is_leading: bool },
Finished,
}
#[derive(Debug)]
struct MatchBranch {
pub state: MatchState,
pub next_stage: Stage,
pub action_on_matched: ActionOnMatched,
}
#[derive(Debug)]
struct MatchBranches(Vec<MatchBranch>);
impl MatchBranches {
fn new() -> Self {
Self(Vec::new())
}
fn new_branch(
&mut self,
pattern: MatchPattern,
next_stage: Stage,
action_on_matched: ActionOnMatched,
) {
self.0.push(MatchBranch {
state: pattern.new_state(),
next_stage,
action_on_matched,
});
}
fn new_string_branch<T: ToString + ?Sized>(
&mut self,
label: &T,
next_stage: Stage,
action_on_matched: ActionOnMatched,
) {
self.new_branch(
MatchPattern::String(label.to_string()),
next_stage,
action_on_matched,
);
}
fn new_strings_branch<T: ToString>(
&mut self,
labels: &[T],
next_stage: Stage,
action_on_matched: ActionOnMatched,
) {
labels
.iter()
.for_each(|label| self.new_string_branch(label, next_stage, action_on_matched));
}
fn add_tool_call_begin_branches(&mut self) {
self.new_branch(
MatchPattern::NewlinesAndString(DSML_BEGIN_LABEL.to_string()),
Stage::ToolCalls,
ActionOnMatched::Skip,
);
}
}
#[derive(Debug)]
struct State {
stage: Stage,
match_branches: MatchBranches,
action_on_unmatched: ActionOnUnmatched,
stashed_size: usize,
}
type ActionOnMatched = OutputAction;
type ActionOnUnmatched = OutputAction;
impl State {
fn new(stage: Stage, options: &ParsingOptions) -> Self {
let mut match_branches = MatchBranches::new();
let mut add_stop_sequence_branches = || {
match_branches.new_strings_branch(
&options.stop_sequences,
Stage::Finished,
ActionOnMatched::StopSequence,
);
};
let action_on_unmatched = match stage {
Stage::Common { is_leading } => {
if options.parse_json_output {
match_branches.new_strings_branch(
&JSON_BEGIN_LABELS,
Stage::Json,
ActionOnMatched::Skip,
);
for raw_json_label in RAW_JSON_BEGIN_LABELS {
match_branches.new_string_branch(
raw_json_label,
Stage::Json,
ActionOnMatched::Label(raw_json_label),
);
}
} else {
add_stop_sequence_branches();
}
if options.parse_tool_calls {
match_branches.add_tool_call_begin_branches();
}
if is_leading {
match_branches.new_branch(
MatchPattern::LeadingNewline,
Stage::Common { is_leading: true },
ActionOnMatched::Skip,
);
}
match_branches.new_string_branch(
REASONING_END_LABEL,
Stage::Common { is_leading: false },
ActionOnMatched::SkipInvalid {
kind: InvalidTokenKind::ExtraEndOfThinking,
},
);
if options.parse_json_output {
ActionOnUnmatched::ToSpace
} else {
ActionOnUnmatched::Raw
}
}
Stage::Json => {
add_stop_sequence_branches();
match_branches.new_string_branch(
JSON_END_LABEL,
Stage::MatchedJson,
ActionOnMatched::Skip,
);
ActionOnUnmatched::Raw
}
Stage::MatchedJson => {
if options.parse_tool_calls {
match_branches.add_tool_call_begin_branches();
}
ActionOnUnmatched::ToSpace
}
Stage::ToolCalls => {
match_branches.new_string_branch(
"invoke name=\"",
Stage::ToolName,
ActionOnMatched::ToolCallBegin,
);
match_branches.new_strings_branch(
&DSML_TOOL_CALLS_END_LABELS,
Stage::Finished,
ActionOnMatched::Skip,
);
ActionOnUnmatched::Skip
}
Stage::ToolName => {
match_branches.new_string_branch(
"\"",
Stage::ToolCallArguments { is_leading: true },
ActionOnMatched::ToolNameEnd,
);
ActionOnUnmatched::ToolName
}
Stage::ToolCallArguments { is_leading } => {
match_branches.new_string_branch(
"parameter name=",
Stage::ToolCallParamName,
if is_leading {
ActionOnMatched::LabelToolCallArguments { label: "{", id: 0 }
} else {
ActionOnMatched::LabelToolCallArguments { label: ", ", id: 1 }
},
);
match_branches.new_string_branch(
DSML_END_LABEL,
Stage::ToolCalls,
if is_leading {
ActionOnMatched::ToolCallArgumentsEnd { output: Some("{}") }
} else {
ActionOnMatched::ToolCallArgumentsEnd { output: Some("}") }
},
);
ActionOnUnmatched::Skip
}
Stage::ToolCallParamName => {
match_branches.new_string_branch(
" ",
Stage::ToolCallParamType,
ActionOnMatched::LabelToolCallArguments { label: ": ", id: 2 },
);
ActionOnUnmatched::RawToolCallArguments { string: false }
}
Stage::ToolCallParamType => {
match_branches.new_string_branch(
"true\">",
Stage::ToolCallParamValue { string: true },
ActionOnMatched::LabelToolCallArguments { label: "\"", id: 3 },
);
match_branches.new_string_branch(
"false\">",
Stage::ToolCallParamValue { string: false },
ActionOnMatched::Skip,
);
ActionOnUnmatched::Skip
}
Stage::ToolCallParamValue { string } => {
match_branches.new_string_branch(
DSML_END_LABEL,
Stage::ToolCallArguments { is_leading: false },
if string {
ActionOnMatched::LabelToolCallArguments { label: "\"", id: 4 }
} else {
ActionOnMatched::Skip
},
);
ActionOnUnmatched::RawToolCallArguments { string }
}
Stage::Reasoning { is_leading } => {
if options.parse_tool_calls {
match_branches.add_tool_call_begin_branches();
}
if is_leading {
match_branches.new_branch(
MatchPattern::LeadingNewline,
Stage::Reasoning { is_leading },
ActionOnMatched::Skip,
);
}
match_branches.new_branch(
MatchPattern::NewlinesAndString(REASONING_END_LABEL.to_string()),
Stage::Common { is_leading: true },
ActionOnMatched::Skip,
);
ActionOnUnmatched::Reasoning
}
Stage::Finished => ActionOnUnmatched::SkipInvalid {
kind: InvalidTokenKind::ContentAfterFinished,
},
};
Self {
stage,
match_branches,
action_on_unmatched,
stashed_size: 0,
}
}
fn feed(&mut self, content: &str, options: &ParsingOptions) -> Vec<OutputActionSegment> {
let mut output_actions = Vec::new();
for byte in content.bytes() {
self.stashed_size += 1;
let mut matched_branch = None;
for branch in self.match_branches.0.iter_mut() {
if branch.state.feed(byte) {
matched_branch = Some((
branch.state.matching_len(),
branch.next_stage,
branch.action_on_matched,
));
break;
}
}
if let Some((matching_len, next_stage, action_on_matched)) = matched_branch {
if matching_len < self.stashed_size {
output_actions.push(OutputActionSegment::new(
self.action_on_unmatched,
self.stashed_size - matching_len,
));
}
if matching_len > 0 {
output_actions.push(OutputActionSegment::new(action_on_matched, matching_len));
}
*self = State::new(next_stage, options);
}
}
let matching_len = self
.match_branches
.0
.iter()
.map(|branch| branch.state.matching_len())
.max()
.unwrap_or(0);
if matching_len < self.stashed_size {
output_actions.push(OutputActionSegment::new(
self.action_on_unmatched,
self.stashed_size - matching_len,
));
self.stashed_size = matching_len;
}
output_actions
}
fn finish(self) -> (Stage, Vec<OutputActionSegment>) {
let mut output_actions = Vec::new();
if self.stashed_size > 0 {
output_actions.push(OutputActionSegment::new(
self.action_on_unmatched,
self.stashed_size,
));
}
(self.stage, output_actions)
}
}
enum MatchPattern {
String(String),
NewlinesAndString(String),
LeadingNewline,
}
impl MatchPattern {
fn new_state(self) -> MatchState {
match self {
Self::String(s) => {
let kmp_table = kmp_table(s.as_bytes());
MatchState::String {
s,
kmp_table,
len: 0,
}
}
Self::NewlinesAndString(s) => {
let kmp_table = kmp_table(s.as_bytes());
MatchState::NewlinesAndString {
s,
kmp_table,
len: 0,
newline_count: 0,
}
}
Self::LeadingNewline => MatchState::LeadingNewline {
leading: true,
matched: false,
},
}
}
}
type KMPTable = Vec<usize>;
fn kmp_table(pattern: &[u8]) -> KMPTable {
let mut table = vec![0; pattern.len()];
let mut len = 0;
let mut i = 1;
while i < pattern.len() {
if pattern[i] == pattern[len] {
len += 1;
table[i] = len;
i += 1;
} else if len != 0 {
len = table[len - 1];
} else {
table[i] = 0;
i += 1;
}
}
table
}
#[derive(Debug)]
enum MatchState {
String {
s: String,
kmp_table: KMPTable,
len: usize,
},
NewlinesAndString {
s: String,
kmp_table: KMPTable,
len: usize,
newline_count: usize,
},
LeadingNewline {
leading: bool,
matched: bool,
},
}
fn kmp_next(s: &str, kmp_table: &[usize], mut len: usize, byte: u8) -> usize {
while len > 0 && s.as_bytes().get(len) != Some(&byte) {
len = kmp_table[len - 1];
}
if s.as_bytes().get(len) == Some(&byte) {
len += 1;
}
len
}
impl MatchState {
fn feed(&mut self, byte: u8) -> bool {
match self {
MatchState::String { s, kmp_table, len } => {
*len = kmp_next(s, kmp_table, *len, byte);
*len == s.len()
}
MatchState::NewlinesAndString {
s,
kmp_table,
len,
newline_count,
} => {
if *len == 0 && byte == b'\n' {
*newline_count += 1;
} else {
let next = kmp_next(s, kmp_table, *len, byte);
if next <= *len {
*newline_count = 0
}
*len = next;
}
*len == s.len()
}
MatchState::LeadingNewline { leading, matched } => {
if *leading {
*matched = byte == b'\n';
}
*leading = false;
*matched
}
}
}
fn matching_len(&self) -> usize {
match self {
MatchState::String { len, .. } => *len,
MatchState::NewlinesAndString {
len, newline_count, ..
} => *len + *newline_count,
MatchState::LeadingNewline { matched, .. } => {
if *matched {
1
} else {
0
}
}
}
}
}

39
deepseek-recipe/src/util/append_delta.rs Normal file
View File

@ -0,0 +1,39 @@
/// Accumulate an incremental update into an existing value.
pub trait AppendDelta<D> {
/// Apply one update in arrival order.
fn append(&mut self, delta: D);
}
impl<T> AppendDelta<Option<T>> for Option<T>
where
T: AppendDelta<T>,
{
fn append(&mut self, delta: Option<T>) {
let Some(delta) = delta else {
return;
};
if let Some(inner) = self {
inner.append(delta);
} else {
*self = Some(delta);
}
}
}
impl<T> AppendDelta<Vec<T>> for Vec<T> {
fn append(&mut self, delta: Vec<T>) {
self.extend(delta);
}
}
impl AppendDelta<String> for String {
fn append(&mut self, delta: String) {
self.push_str(&delta);
}
}
impl AppendDelta<&str> for String {
fn append(&mut self, delta: &str) {
self.push_str(delta);
}
}

1
deepseek-recipe/src/util/mod.rs Normal file
View File

@ -0,0 +1 @@
pub mod append_delta;

212
docs/development.md Normal file
View File

@ -0,0 +1,212 @@
# Development guide
Run commands from the repository root. Use the Rust toolchain pinned in
[rust-toolchain.toml](../rust-toolchain.toml); rustup selects it automatically
inside this checkout. Python bindings require Python 3.10 or later.
## Native dependencies
The workspace requires a C/C++ compiler. OpenCV development headers and
libraries, Clang, and libclang are also required by the image crate's default
`opencv-preprocess` feature, by the Rust example server, by the inference demo,
and by the Python bindings. An application that supplies its own image
preprocessor builds the image crate with `--no-default-features` and without
OpenCV.
For Debian 12 (bookworm):
```sh
sudo apt update
sudo apt install -y build-essential pkg-config clang libclang-dev libopencv-dev
```
For Python development on Debian, also install `python3-dev` and `python3-venv`.
Use OpenCV 4.x for this workspace. The pinned Rust `opencv` 0.93 bindings support
OpenCV 3.4 and 4.x, not OpenCV 5.x. For other systems or a custom installation,
follow the binding version's [installation guide][opencv-install] and
[troubleshooting guide][opencv-troubleshooting]. Configure `PKG_CONFIG_PATH`,
`OpenCV_DIR`, or `LIBCLANG_PATH` as appropriate for your installation.
To check an OpenCV 4 installation discovered through pkg-config:
```sh
pkg-config --modversion opencv4
```
## Rust source installation
To use an unreleased checkout, create an application beside this repository:
```sh
cargo new --edition 2024 ../recipe-example
```
Add these entries under `[dependencies]` in `../recipe-example/Cargo.toml`.
Adjust the paths if the checkout directory has a different name.
```toml
deepseek-recipe = { path = "../deepseek-recipe/deepseek-recipe" }
deepseek-recipe-encoding = { path = "../deepseek-recipe/deepseek-recipe-encoding" }
serde_json = "1"
```
Save the Rust quick-start example from the [README](../README.md#quick-start)
as `../recipe-example/src/main.rs`. Run it from the repository root to use the
pinned toolchain:
```sh
cargo run --manifest-path ../recipe-example/Cargo.toml
```
## Rust checks
Run the full checks when changing Rust code, public APIs, or dependencies:
```sh
cargo fmt --all -- --check
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo test --workspace --locked
RUSTDOCFLAGS='-D warnings' cargo doc --workspace --no-deps --locked
```
The last command uses POSIX shell syntax. On PowerShell, set
`$env:RUSTDOCFLAGS = '-D warnings'` before running `cargo doc`.
For protocol or prompt work without OpenCV, check the relevant packages:
```sh
cargo test -p deepseek-recipe-core -p deepseek-recipe -p deepseek-recipe-encoding -p encoding-decoding-demo --locked
```
The image crate compiles without OpenCV when its default features are disabled:
```sh
cargo check -p deepseek-recipe-image --no-default-features --locked
```
Cargo downloads dependencies on the first build.
## Python development and tests
After installing the native dependencies, create a virtual environment:
```sh
python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install 'maturin>=1.9,<2' 'pytest>=8,<10' httpx
maturin develop --manifest-path deepseek-recipe-python/Cargo.toml --locked
python3 -m pytest deepseek-recipe-python/tests server-py/tests
```
On Windows PowerShell, activate with `.venv\Scripts\Activate.ps1` instead.
Run `maturin develop` again after changing Rust bindings. The tests exercise
request conversion, prompt rendering, response processing, and the FastAPI
example with mock inference; no model weights or API credentials are needed.
For an installation without editable development, use:
```sh
python3 -m pip install ./deepseek-recipe-python
```
## Wheel packaging
The extension links OpenCV dynamically, so a wheel is only installable on a
machine with a matching OpenCV until the libraries are copied into it:
`delocate` on macOS, `auditwheel` on Linux. The libraries of the build machine
become part of the wheel, so the wheel is built against an OpenCV that is small
enough to bundle.
Build that OpenCV from source, with only the modules the extension uses. OpenCL,
TBB, OpenEXR, and TIFF are left out because each of them links libraries of its
own that the wheel would then have to carry.
```sh
git clone --depth 1 --branch 4.12.0 https://github.com/opencv/opencv.git
cmake -S opencv -B opencv/build \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="$PWD/target/opencv-min" \
-DCMAKE_OSX_DEPLOYMENT_TARGET=11.0 \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_LIST=core,imgproc,imgcodecs \
-DWITH_OPENCL=OFF -DWITH_TBB=OFF -DWITH_OPENEXR=OFF -DWITH_TIFF=OFF \
-DWITH_JPEG=ON -DWITH_PNG=ON -DWITH_WEBP=ON
cmake --build opencv/build --parallel
cmake --install opencv/build
```
Keep `CMAKE_OSX_DEPLOYMENT_TARGET` at the oldest release the wheel should support:
the installed libraries decide the platform tag of the wheel.
Then build the wheel against that OpenCV, and copy its libraries into it:
```sh
cd deepseek-recipe-python
export OPENCV_INCLUDE_PATHS="$PWD/../target/opencv-min/include/opencv4"
export OPENCV_LINK_PATHS="$PWD/../target/opencv-min/lib"
export OPENCV_LINK_LIBS=opencv_core,opencv_imgproc,opencv_imgcodecs
maturin build --release
delocate-wheel -w dist-repaired dist/*.whl # `auditwheel repair` on Linux
twine check dist-repaired/*
```
The `OPENCV_*` variables take precedence over pkg-config, so an OpenCV installed
elsewhere on the build machine cannot take over. The linker still resolves
`-lopencv_core` in the order of its search paths, so a build machine that carries
another OpenCV in its flags has to search this one first:
```sh
export RUSTFLAGS="-L$PWD/../target/opencv-min/lib ${RUSTFLAGS:-}"
```
On macOS this produces a wheel of about 8 MB; bundling an OpenCV installed by a
package manager produces a much larger one, because that build links every
library of its own dependencies.
Upload the repaired artifacts after checking them against TestPyPI:
```sh
twine upload --repository testpypi deepseek-recipe-python/dist-repaired/*
twine upload deepseek-recipe-python/dist-repaired/*
```
An upload authenticates with an API token: `__token__` as the user name and the
token as the password, in `~/.pypirc` or in `TWINE_USERNAME` and
`TWINE_PASSWORD`.
The wheel bundles OpenCV and the codecs that its `imgcodecs` module compiles in,
so it also carries their license texts:
`deepseek-recipe-python/THIRD_PARTY_LICENSES` reproduces them and the package
declares the file as a license file.
## API documentation
Build and open the Rust API documentation locally:
```sh
cargo doc --workspace --no-deps --locked --open
```
The Python package includes a `py.typed` marker and
[type stubs](../deepseek-recipe-python/python/deepseek_recipe/_native.pyi).
## Example servers
[server-rs](../server-rs/README.md) and [server-py](../server-py/README.md) listen on
`127.0.0.1:7777` by default. Run one at a time or choose different ports. Both
default to mock inference and provide JSON and SSE examples for the three protocols.
[encoding-decoding-demo](../encoding-decoding-demo/README.md) listens on
`127.0.0.1:7778`. It encodes prompts, displays special tokens, and decodes complete
model output into Chat Completions, Responses, or Messages without executing a model.
The examples have no authentication. Keep them on loopback for local use.
Applications exposed to other users need authentication, request limits, and
appropriate network access controls. The default `ReqwestImageFetcher` follows
redirects and does not filter private, loopback, or link-local destinations.
When accepting untrusted image URLs, enforce destination restrictions on the
initial request and every redirect, using a custom fetcher or network policy.
Image byte quotas do not provide this destination filtering.
[opencv-install]: https://github.com/twistedfall/opencv-rust/blob/v0.93.7/INSTALL.md
[opencv-troubleshooting]: https://github.com/twistedfall/opencv-rust/blob/v0.93.7/TROUBLESHOOTING.md

111
docs/streaming.md Normal file
View File

@ -0,0 +1,111 @@
# Response streaming
**English** | [中文](streaming.zh.md)
`StreamProcessor` converts backend output into Messages, Chat Completions, or
Responses events. The examples below use mock inference and print Chat
Completions chunks. The application supplies the backend and HTTP transport.
## Rust
Add the dependencies to your application:
```sh
cargo add deepseek-recipe@0.1 serde_json@1 tokio-stream@0.1
cargo add tokio@1 --features macros,rt-multi-thread
```
Save this as `src/main.rs` and run `cargo run`:
```rust
use deepseek_recipe::openai::ChatCompletionRequest;
use deepseek_recipe::request::{ConversionOptions, ProtocolRequest};
use deepseek_recipe::stream::{
InferenceChunk, InferenceFinishReason, PromptUsage, StreamProcessor,
};
use serde_json::json;
use tokio_stream::{StreamExt, iter};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let body = json!({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
"thinking": {"type": "disabled"},
});
// The chunk generator uses the converted request's resolved settings.
let request: ChatCompletionRequest = serde_json::from_value(body)?;
let converted = request.convert(ConversionOptions::default())?;
let generator =
ChatCompletionRequest::chunk_generator(&converted, "id-1".into(), "deepseek-flash".into());
let processor = StreamProcessor::new(generator, converted.parsing_options);
// Mock the backend's inference chunks.
let inference = iter(vec![
InferenceChunk::Ready {
system_fingerprint: Some("fp-1".into()),
prompt_usage: PromptUsage {
prompt_tokens: 4,
prompt_cache_hit_tokens: 0,
},
},
InferenceChunk::Text {
content: "Hello!".into(),
content_tokens: 2,
},
InferenceChunk::Finish {
finish_reason: InferenceFinishReason::Stop,
},
]);
// Convert them into protocol chunks.
let mut chunks = std::pin::pin!(processor.process(inference));
while let Some(chunk) = chunks.next().await {
println!("{}", serde_json::to_string(&chunk?)?);
}
Ok(())
}
```
## Python
Install the package with `python3 -m pip install deepseek-recipe`.
`push` processes one backend chunk; `finish` completes response processing.
```python
from deepseek_recipe import (
ChatCompletionRequest,
ConversionOptions,
InferenceChunk,
InferenceFinishReason,
PromptUsage,
StreamProcessor,
)
request = ChatCompletionRequest({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
"thinking": {"type": "disabled"},
})
converted = request.convert(ConversionOptions())
generator = ChatCompletionRequest.chunk_generator(converted, "id-1", "deepseek-flash")
processor = StreamProcessor(generator, converted.parsing_options)
# Mock the backend's inference chunks.
chunks = []
for chunk in [
InferenceChunk.ready(system_fingerprint="fp-1", prompt_usage=PromptUsage(prompt_tokens=4)),
InferenceChunk.text("Hello!", content_tokens=2),
InferenceChunk.finish(InferenceFinishReason.Stop),
]:
chunks.extend(processor.push(chunk))
chunks.extend(processor.finish())
for chunk in chunks:
print(chunk.to_json())
processor.close()
```
See the [project README](../README.md) for supported scope and the
[Rust API documentation](https://docs.rs/deepseek-recipe) for API contracts.

108
docs/streaming.zh.md Normal file
View File

@ -0,0 +1,108 @@
# 流式响应
[English](streaming.md) | **中文**
`StreamProcessor` 把后端输出转换为 Messages、Chat Completions 或 Responses 事件。以下示例使用 mock 推理并打印 Chat Completions chunk。后端与 HTTP 传输由应用提供。
## Rust
为你的应用添加依赖：
```sh
cargo add deepseek-recipe@0.1 serde_json@1 tokio-stream@0.1
cargo add tokio@1 --features macros,rt-multi-thread
```
将以下代码保存为 `src/main.rs`，然后运行 `cargo run`：
```rust
use deepseek_recipe::openai::ChatCompletionRequest;
use deepseek_recipe::request::{ConversionOptions, ProtocolRequest};
use deepseek_recipe::stream::{
InferenceChunk, InferenceFinishReason, PromptUsage, StreamProcessor,
};
use serde_json::json;
use tokio_stream::{StreamExt, iter};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let body = json!({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
"thinking": {"type": "disabled"},
});
// chunk 生成器使用转换后请求的最终设置。
let request: ChatCompletionRequest = serde_json::from_value(body)?;
let converted = request.convert(ConversionOptions::default())?;
let generator =
ChatCompletionRequest::chunk_generator(&converted, "id-1".into(), "deepseek-flash".into());
let processor = StreamProcessor::new(generator, converted.parsing_options);
// 模拟后端的推理 chunk。
let inference = iter(vec![
InferenceChunk::Ready {
system_fingerprint: Some("fp-1".into()),
prompt_usage: PromptUsage {
prompt_tokens: 4,
prompt_cache_hit_tokens: 0,
},
},
InferenceChunk::Text {
content: "Hello!".into(),
content_tokens: 2,
},
InferenceChunk::Finish {
finish_reason: InferenceFinishReason::Stop,
},
]);
// 将其转换为协议 chunk。
let mut chunks = std::pin::pin!(processor.process(inference));
while let Some(chunk) = chunks.next().await {
println!("{}", serde_json::to_string(&chunk?)?);
}
Ok(())
}
```
## Python
安装包：`python3 -m pip install deepseek-recipe`。
`push` 处理一个后端 chunk；`finish` 完成响应处理。
```python
from deepseek_recipe import (
ChatCompletionRequest,
ConversionOptions,
InferenceChunk,
InferenceFinishReason,
PromptUsage,
StreamProcessor,
)
request = ChatCompletionRequest({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
"thinking": {"type": "disabled"},
})
converted = request.convert(ConversionOptions())
generator = ChatCompletionRequest.chunk_generator(converted, "id-1", "deepseek-flash")
processor = StreamProcessor(generator, converted.parsing_options)
# 模拟后端的推理 chunk。
chunks = []
for chunk in [
InferenceChunk.ready(system_fingerprint="fp-1", prompt_usage=PromptUsage(prompt_tokens=4)),
InferenceChunk.text("Hello!", content_tokens=2),
InferenceChunk.finish(InferenceFinishReason.Stop),
]:
chunks.extend(processor.push(chunk))
chunks.extend(processor.finish())
for chunk in chunks:
print(chunk.to_json())
processor.close()
```
支持范围见[项目 README](../README.zh.md)，API 约定见 [Rust API 文档](https://docs.rs/deepseek-recipe)。

186
docs/tokenizer.md Normal file
View File

@ -0,0 +1,186 @@
# Use with tokenizer
**English** | [中文](tokenizer.zh.md)
With a tokenizer attached, a `PromptEncoding` encodes the prompt into model
token IDs, and a `StreamProcessor` accepts token IDs. Neither loads a tokenizer
on its own: pass one with `with_tokenizer`, then call the corresponding method.
Using `encode` or passing token IDs without an attached tokenizer returns an
error.
Use the tokenizer that matches your model. This repository bundles copies under
[`static/tokenizers/`](../static/tokenizers/README.md) for the V4 and V4.1
prompt templates; point the loader at `tokenizer.json` in the matching
directory. A tokenizer from the Hugging Face Hub works too, but its special
token spelling must match the encoding's constants, and its image token must
resolve to the same ID.
## Rust
Add the dependencies to your application:
```sh
cargo add deepseek-recipe@0.1 deepseek-recipe-encoding@0.1 serde_json@1 tokenizers@0.23
```
Save this as `src/main.rs` and run `cargo run`. The tokenizer path below points
at this repository's bundled file; use your own `tokenizer.json` otherwise:
```rust
use deepseek_recipe::openai::ChatCompletionRequest;
use deepseek_recipe::request::{ConversionOptions, ProtocolRequest};
use deepseek_recipe_encoding::PromptEncoding;
use deepseek_recipe_encoding::v4::dsv41::DeepseekV41Encoding;
use serde_json::json;
use tokenizers::Tokenizer;
fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let request: ChatCompletionRequest = serde_json::from_value(json!({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
}))?;
let converted = request.convert(ConversionOptions::default())?;
// Attach the tokenizer, then encode the conversation's tokens.
let tokenizer = Tokenizer::from_file("static/tokenizers/v41/tokenizer.json")?;
let encoding = DeepseekV41Encoding::new().with_tokenizer(tokenizer);
let token_ids = encoding.encode(&converted.conversation)?;
println!("{token_ids:?}");
Ok(())
}
```
`Tokenizer` implements the `TokenizerEncoder` trait used by `with_tokenizer`.
Any other type implementing that trait can be attached instead. Encoding is a
two-step operation: `render_conversation` returns the prompt text and its image
placeholders, and `encode` tokenizes that prompt with added special tokens
disabled, because the prompt already contains the special token text.
### Decoding
A backend can return token IDs instead of text. Attach the same tokenizer to the
stream processor to decode those chunks. The example also needs `tokio-stream`,
plus `tokio` with the `macros` and `rt-multi-thread` features, as listed in
[response streaming](streaming.md):
```rust
use deepseek_recipe::openai::ChatCompletionRequest;
use deepseek_recipe::request::{ConversionOptions, ProtocolRequest};
use deepseek_recipe::stream::{InferenceChunk, InferenceFinishReason, StreamProcessor};
use serde_json::json;
use tokenizers::Tokenizer;
use tokio_stream::{StreamExt, iter};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let request: ChatCompletionRequest = serde_json::from_value(json!({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
"thinking": {"type": "disabled"},
}))?;
let converted = request.convert(ConversionOptions::default())?;
let generator =
ChatCompletionRequest::chunk_generator(&converted, "id-1".into(), "deepseek-flash".into());
// The backend answers with token IDs; the attached tokenizer decodes them.
let tokenizer = Tokenizer::from_file("static/tokenizers/v41/tokenizer.json")?;
let token_ids = tokenizer.encode("Hello there!", false)?.get_ids().to_vec();
let processor =
StreamProcessor::new(generator, converted.parsing_options).with_tokenizer(tokenizer);
let inference = iter(
token_ids
.into_iter()
.map(|token_id| InferenceChunk::Token { token_id })
.chain([InferenceChunk::Finish {
finish_reason: InferenceFinishReason::Stop,
}]),
);
let mut chunks = std::pin::pin!(processor.process(inference));
while let Some(chunk) = chunks.next().await {
println!("{}", serde_json::to_string(&chunk?)?);
}
Ok(())
}
```
The processor decodes ids incrementally and buffers them while a multi-token
character is incomplete; ids that produced text count as completion tokens.
Special tokens stay in the decoded text because they drive output parsing. A
`Token` chunk without an attached tokenizer fails the stream with
`StreamError::MissingTokenizer`. `Tokenizer` implements the `TokenizerDecoder`
trait used by `with_tokenizer`.
## Python
Install the package with `python3 -m pip install deepseek-recipe`:
```python
from deepseek_recipe import (
ChatCompletionRequest,
ConversionOptions,
DeepseekV41Encoding,
Tokenizer,
)
request = ChatCompletionRequest({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
})
converted = request.convert(ConversionOptions())
tokenizer = Tokenizer.from_file("static/tokenizers/v41/tokenizer.json")
encoding = DeepseekV41Encoding().with_tokenizer(tokenizer)
print(encoding.encode(converted.conversation))
```
`Tokenizer.from_pretrained` downloads and caches a `tokenizer.json` from the
Hugging Face Hub, and `Tokenizer.from_str` loads the file contents directly.
`render_conversation` and `encode` return the same results as in Rust.
### Decoding
Pass the tokenizer as the processor's third argument. Token-id chunks are
decoded with the same behavior as in Rust; without a tokenizer, a token chunk
raises `RuntimeError` and closes the processor.
```python
from deepseek_recipe import (
ChatCompletionRequest,
ConversionOptions,
InferenceChunk,
InferenceFinishReason,
StreamProcessor,
Tokenizer,
)
converted = ChatCompletionRequest({
"model": "deepseek-flash",
"messages": [{"role": "user", "content": "Hello"}],
"thinking": {"type": "disabled"},
}).convert(ConversionOptions())
generator = ChatCompletionRequest.chunk_generator(converted, "id-1", "deepseek-flash")
# The backend answers with token IDs; the tokenizer passed to the processor
# decodes them.
tokenizer = Tokenizer.from_file("static/tokenizers/v41/tokenizer.json")
token_ids = tokenizer.encode("Hello there!")
processor = StreamProcessor(generator, converted.parsing_options, tokenizer)
chunks = []
for chunk in (
*(InferenceChunk.token(token_id) for token_id in token_ids),
InferenceChunk.finish(InferenceFinishReason.Stop),
):
chunks.extend(processor.push(chunk))
chunks.extend(processor.finish())
for chunk in chunks:
print(chunk.to_json())
processor.close()
```
See the [project README](../README.md) for supported scope and the
[response streaming](streaming.md) guide for the full chunk sequence.

Some files were not shown because too many files have changed in this diff Show More