Is it possible to fine tune LLMs on data you want them to remember?

Basically I want to take large codebases and transfer train an LLM on them so that I don't have to have to insert the code into prompt and deal with context windows and token limits.

Is this doable or is there a better way of handling large code bases in terms of being able to ask LLM questions about them?