Skip to main content

Using LM Studio With Roo Code

Roo Code supports running models locally using LM Studio. LM Studio provides a user-friendly interface for downloading, configuring, and running local language models. It also includes a built-in local inference server that emulates the OpenAI API, making it easy to integrate with Roo Code.

Website: https://lmstudio.ai/

Setting Up LM Studio​

  1. Download and Install LM Studio: Download LM Studio from the LM Studio website.
  2. Download a Model: Use the LM Studio interface to search for and download a model in GGUF format. Browse all available models in the LM Studio search interface or at Hugging Face.
  3. Start the Local Server:
    • Open LM Studio.
    • Click the "Local Server" tab (the icon looks like <->).
    • Select the model you downloaded.
    • Click "Start Server".

Configuration in Roo Code​

  1. Open Roo Code Settings: Click the gear icon () in the Roo Code panel.
  2. Select Provider: Choose "LM Studio" from the "API Provider" dropdown.
  3. Enter Model ID: Enter the file name of the model you loaded in LM Studio (e.g., codellama-7b.Q4_0.gguf). You can find this in the LM Studio "Local Server" tab.
  4. (Optional) Base URL: By default, Roo Code will connect to LM Studio at http://localhost:1234. If you've configured LM Studio to use a different address or port, enter the full URL here.

Tips and Notes​

  • Resource Requirements: Running large language models locally can be resource-intensive. Make sure your computer meets the minimum requirements for the model you choose.
  • Model Selection: LM Studio provides a wide range of models. Experiment to find the one that best suits your needs.
  • Local Server: The LM Studio local server must be running for Roo Code to connect to it.
  • LM Studio Documentation: Refer to the LM Studio documentation for more information.
  • Troubleshooting: If you see a "Please check the LM Studio developer logs to debug what went wrong" error, you may need to adjust the context length settings in LM Studio.
  • Token Tracking: Roo Code tracks token usage for models run via LM Studio, helping you monitor consumption.
  • Reasoning Support: For models that support it, Roo Code can parse "think" tags or similar reasoning indicators in LM Studio responses, offering more insight into the model's process.