Crate sokuto_runtime 

sokuto-runtime

高速トークナイズおよび ONNX Runtime による超低遅延推論パイプラインを提供するクレート。

概要

• Hugging Face tokenizers を用いたトークナイズ処理。
• ONNX Runtime (ort) を用いた単一フォワードパス決定ヘッド推論。
• 候補数が多い場合の粗密探索(Coarse-to-Fine)やバッチ推論。

Re-exports

pub use dag::DagDefinition;

pub use dag::DagError;

pub use dag::DagExecutionResult;

pub use dag::DagNode;

pub use dag::DagNodeType;

pub use dag::InProcessDagExecutor;

pub use dag::LowLatencyResourcePool;

pub use dag::LowLatencySessionPool;

pub use dag::NodeCondition;

pub use dag::NodeConditionOp;

pub use dag::PooledResource;

pub use dag::PooledSession;

pub use dag::StepExecutionResult;

pub use dag::validate_dag;

pub use engine::AlignedBuffer64;

pub use engine::BatchScratchpad;

pub use engine::CacheCompressionMode;

pub use engine::CachedStateFeatures;

pub use engine::CachedTensorPayload;

pub use engine::CandidateEmbeddingCache;

pub use engine::CoarseScorer;

pub use engine::CoarseToFineConfig;

pub use engine::DEFAULT_COARSE_THRESHOLD;

pub use engine::DEFAULT_DUAL_MAX_BATCH_CHUNK_SIZE;

pub use engine::DEFAULT_MAX_BATCH_CHUNK_SIZE;

pub use engine::DEFAULT_NEGATIVE_KEYS;

pub use engine::DEFAULT_TOP_M_CANDIDATES;

pub use engine::DualSessionEngine;

pub use engine::EmbeddingCoarseScorer;

pub use engine::ExecutionProvider;

pub use engine::FilteredCandidates;

pub use engine::HierarchicalCoarseScorer;

pub use engine::InferenceEngine;

pub use engine::LexicalCoarseScorer;

pub use engine::ModelDrivenCoarseScorer;

pub use engine::OptimizationLevel;

pub use engine::SessionConfig;

pub use engine::StateCacheManager;

pub use engine::filter_top_candidates;

pub use engine::is_aligned_64;

pub use engine::is_negative_candidate;

pub use engine::reconstruct_probabilities;

pub use error::Result;

pub use error::RuntimeError;

pub use hierarchical::CoarseToFineRouter;

pub use hierarchical::DEFAULT_BEAM_MARGIN_THRESHOLD;

pub use hierarchical::DEFAULT_CATCHALL_THRESHOLD;

pub use hierarchical::DEFAULT_COARSE_POWER_ALPHA;

pub use hierarchical::DEFAULT_ENTROPY_TEMP_GAMMA;

pub use hierarchical::DEFAULT_ESCALATE_CONFIDENCE_THRESHOLD;

pub use hierarchical::DirichletCalibrator;

pub use hierarchical::HierarchicalEngine;

pub use hierarchical::HierarchicalExecutionTrace;

pub use hierarchical::HierarchicalMapping;

pub use hierarchical::HierarchicalMappingBuilder;

pub use hierarchical::HierarchicalResult;

pub use hierarchical::HierarchicalRouterConfig;

pub use hierarchical::combine_and_reconstruct_probabilities;

pub use hierarchical::compute_fine_temperature;

pub use hierarchical::compute_normalized_entropy;

pub use hierarchical::softmax_f64;

pub use hierarchical::temperature_scaled_softmax;

pub use tokenizer::BatchTokenizedQuestions;

pub use tokenizer::JevTokenizer;

pub use tokenizer::TokenizedQuestion;

pub use sokuto_core as core;

Modules

dag

インプロセス DAG 実行器モジュール

engine

推論エンジンモジュール

error

ランタイムエラー型定義モジュール

hierarchical

粗密二段階階層ルーティング (Coarse-to-Fine Hierarchical Routing)

tokenizer

高速トークナイザー統合モジュール

Functions

version

ランタイムの初期化確認用関数。