Hey, I made multiple upgrades to my fork of localllama on Vulkan, and OpenVino. On some paths ingestion was double the speed and inference 60% faster or more. Also significantly reduced ram cost for initial load and bug fixes fixing some broken inference paths. It was tested on integrated intel graphics 13th gen and on B10. Designed to get good performance on laptops along with the CPU with A303B and other usable models. How does an opensource release like this usually get traction? I wont merge it into localllama due to their backlog. Also, did someone already solve this for open source Intel? And should i release it as a localllama fork or my alternative custom inference engine that i made beside it that runs independently? And id any of you are interested we can talk about collaboration. We need intel inference to work better to have more options. Hopefully I can help with that.   submitted by   /u/Fun-Aardvark-1143 [link]   [comments]