Woonggi
complete
Added ability to select safetensors PR via revision selection on dedicated endpoints.
knuts
if we request a model what are the chances it gets support?
Junu Kwon
knuts: Hello, thank you for reaching out and submitting a request.
Currently, Dedicated Endpoints can only launch models saved in
safetensors
format. The model you are attempting to use appears to be saved as a pickle file. Regrettably, we do not have any plans to extend our support to include other file types at this time. Still, you may try converting the model to safetensors
, as the model's architecture (MistralForCausalLM
, can be seen in config.json
) is on our support list.We appreciate your understanding and look forward to providing you with optimal support. If you have any questions or concerns, please do not hesitate to ask, and we will be happy to assist you further.
Woonggi
Hello knuts, we are excited to announce that this model is now available on Friendli Dedicated endpoints.
We added ability to run models using specific huggingface commits, and models can be run using PR commits that include safetensors.
For the model in question, it is as follows:
hf_model_name: NeverSleep/Lumimaid-v0.2-123B
hf_commit_revision: 6fb83f2532aa3b6b97fba2642ec4abdaeb9c2b8e
Enter the model ID and if it says the model is not supported, select the revision next to it and enter the provided commit.