Thanks to visit codestin.com
Credit goes to github.com

Skip to content

how to run internvl1.5-8bit with nvidia v100 #144

@StevenBanama

Description

@StevenBanama

ERROR about flash_attr, can u help to provide version for these old nv card?


out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = flash_attn_cuda.fwd(
^^^^^^^^^^^^^^^^^^^^
RuntimeError: FlashAttention only supports Ampere GPUs or newer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions