I am making a project for myself and the first step of it is document analysis and OCR of Educational content/books, curriculums like STEM, English, and some Arabic mixed in the middle are the mainly parsed documents so having table, formula, figure and diagram extraction are a must, and I have about 500 labeled pages for the question banks ready for export as I heard that the layout detector could be finetuned. My hardware is a Lenovo laptop (i7 14700HX, RTX 5060 8 GB of VRAM, 24GB RAM) and running windows 11. I tried using PaddleOCR-VL-1.6, It's a 0.9B parameters model, and documented to use about 4GB or VRAM. but when I used it It was occupying the whole GPU and spilling about 6GB of RAM, it was taking 13~70 sec./page which is obviously slow. I was using the paddlepaddle framework with the correct CUDA version for my GPU, tried limiting VRAM usage by flagging system resources (ai idea) but got "not enough VRAM" message when I parsed more than 1 page in a folder, 1 page worked fine (the warm up of the VLM took a bit of time though), but when I put more than 1 page in that folder and ran the program again I got that error message. I read through Hugging Face and found out that vLLM was the recommended path but that would require Linux. So I wanted confirmation from someone with similar specs as me that might have gone through a similar issue and found a solution. because vLLM require a dual boot to Linux or WSL2 which I don't have enough storage for. I could buy a another SSD for my laptop (would cost a 2 month salary in my country ffs), so I need confirmation first before committing. tldr; Is there any hope of running the title or should I keep this idea in a trash bin?   submitted by   /u/Mezrotix [link]   [comments]