PixtralMistral models can now 👀 !pip install --upgrade mistral_inference # >= 1.4.0Download:from huggingface_hub import snapshot_downloadfrom pathlib import Pathmistral_models_path = Path.home().joinpath('mistral_models', 'Pixtral')mistral_models_path.mkdir(parents=True, exist_ok=True)snapshot_download(repo_id="mistralai/Pixtral-12B-2409", allow_patterns=["params.json", "consolidated.safetensors", "tekken.json"], local_dir=mistral_models_path)CLI example:mistral-chat $HOME/mistral_models/Pixtral --instruct --max_tokens 256 --temperature 0.35E.g. Try out something like:Text prompt: What can you see on the following picture?[You can input zero, one or more images now.]Image path or url [Leave empty and press enter to finish image input]: https://picsum.photos/id/237/200/300Image path or url [Leave empty and press enter to finish image input]:I see a black dog lying on a wooden surface. The dog appears to be looking up, and its eyes are clearly visible.Python:Load the modelfrom mistral_inference.transformer import Transformerfrom mistral_inference.generate import generatefrom mistral_common.tokens.tokenizers.mistral import MistralTokenizerfrom mistral_common.protocol.instruct.messages import UserMessage, TextChunk, ImageURLChunkfrom mistral_common.protocol.instruct.request import ChatCompletionRequesttokenizer = MistralTokenizer.from_file(f"{mistral_models_path}/tekken.json")model = Transformer.from_folder(mistral_models_path)Run:url = "https://huggingface.co/datasets/patrickvonplaten/random_img/resolve/main/yosemite.png"prompt = "Describe the image."completion_request = ChatCompletionRequest(messages=[UserMessage(content=[ImageURLChunk(image_url=url), TextChunk(text=prompt)])])encoded = tokenizer.encode_chat_completion(completion_request)images = encoded.imagestokens = encoded.tokensout_tokens, _ = generate([tokens], model, images=[images], max_tokens=256, temperature=0.35, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)result = tokenizer.decode(out_tokens[0])print(result)