Loading…
3
A guy on r/LocalLLaMA told me my RAG setup was just expensive autocomplete, so I rebuilt it
His exact words were "you're burning tokens on vibes, not retrieval" and it stung because he was right about my chunking. I switched from fixed 1000 character chunks to splitting on headers first, then dropping anything under 200 characters, and my answers stopped pulling random garbage from page 4 of the PDFs. Anyway has anyone else had a stranger on the internet hand them the fix they needed? Curious what your chunking setup looks like now.
1 comments
Log in to join the discussion
Log In1 Comment
morgan_lewis1823d ago
Ha, that stings because I did the exact same thing, but heads up: dropping chunks under 200 chars will bite you when a short header or table row is the only place your answer lives. I keep them if they have a number or name in them, otherwise yeah, toss them.
5