Skip to content

vllm.logprobs

PromptLogprobs module-attribute 

PromptLogprobs = list[Optional[dict[int, Logprob]]]

SampleLogprobs module-attribute 

SampleLogprobs = list[dict[int, Logprob]]

Logprob dataclass

Infos for supporting OpenAI compatible logprobs and token ranks.

Attributes:

Name Type Description
logprob float

The logprob of chosen token
rank Optional[int]

The vocab rank of chosen token (>=1)
decoded_token Optional[str]

The decoded chosen token index
Source code in vllm/logprobs.py 
10
11
12
13
14
15
16
17
18
19
20
21
@dataclass
class Logprob:
    """Infos for supporting OpenAI compatible logprobs and token ranks.

    Attributes:
        logprob: The logprob of chosen token
        rank: The vocab rank of chosen token (>=1)
        decoded_token: The decoded chosen token index
    """
    logprob: float
    rank: Optional[int] = None
    decoded_token: Optional[str] = None

decoded_token class-attribute instance-attribute 

decoded_token: Optional[str] = None

logprob instance-attribute 

logprob: float

rank class-attribute instance-attribute 

rank: Optional[int] = None

__init__ 

__init__(
    logprob: float,
    rank: Optional[int] = None,
    decoded_token: Optional[str] = None,
) -> None