Skip to content

FP8 Inference Error #20

@D-Octopus

Description

@D-Octopus

Description

We encountered an error while attempting to run FP8 inference using the script run_server_fp8.sh.

Error Message

AttributeError: '_OpNamespace' '_C' object has no attribute 'cutlass_scaled_mm_supports_fp8'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions