support musa backend in FlagEmbedding

[qiyulei-mt]

What does this PR do?
It supports utilization of Moore Threads GPU in FlagEmbedding. The input of devices can be 0, [0], ["musa:0"], "musa:0". For example, the case below, you can set devices as 0, [0], ["musa:0"], "musa:0".

import os
from FlagEmbedding import FlagModel

model = FlagModel(
    'BAAI/bge-small-en-v1.5',
    query_instruction_for_retrieval="Represent this sentence for searching relevant passages: ",
    query_instruction_format="{}{}",
    devices="musa:3",
    pooling_method='cls',
    cache_dir=os.getenv('HF_HUB_CACHE', None),
)

queries = [
    "What is the capital of France?",
    "What is the population of China?",
] * 100
passages = [
    "Paris is the capital of France.",
    "The population of China is over 1.4 billion people."
] * 100

queries_embeddings = model.encode_queries(queries)
passages_embeddings = model.encode_corpus(passages)

cos_scores = queries_embeddings @ passages_embeddings.T
print(cos_scores[:2, :2])

After running the code, you can get corresponding results.

[caizhi-mt]

@hanhainebula please take a look at this PR

[hanhainebula]

Looks great! However, a few modifications may be necessary. Before calling torch.musa.is_available(), it would be prudent to check if torch has the musa attribute. Otherwise, it may raise an AttributeError for other backends. A possible workable modification could be:

...
if hasattr(torch, 'musa') and torch.musa.is_available():
...

Hello, @qiyulei-mt, could you please update all the corresponding code? Once that's done, I will merge this PR. Thanks!

[hanhainebula]

Hello, @qiyulei-mt and @caizhi-mt. I just directly modified the corresponding code in qiyulei-mt:musa_support. Now this PR will be merged. Thanks for your submission.

[caizhi-mt]

Hello, @qiyulei-mt and @caizhi-mt. I just directly modified the corresponding code in qiyulei-mt:musa_support. Now this PR will be merged. Thanks for your submission.

Thank you very much for your work. We will continue to contribute code to this repository.