Troubleshooting
Troubleshooting
This section covers common issues encountered while using Moltext to compile documentation and provides solutions for resolving them.
Authentication and API Keys
Error: API Key is required for OpenAI
This occurs when you attempt to use the default LLM processing without providing an OpenAI API key.
- Solution 1: Export your key as an environment variable:
export OPENAI_API_KEY='sk-...'. - Solution 2: Pass the key directly using the
-kor--keyflag. - Solution 3: Use
--rawmode to skip AI processing entirely. No key is required for raw Markdown extraction. - Solution 4: If using a local inference server (like Ollama), ensure you provide the
--base-urlpointing to your local endpoint.
Crawling and Extraction Issues
Empty or Minimal Output in context.md
If the resulting file contains only headers but no content, the crawler may be struggling with the site structure.
- JavaScript-Heavy Sites: Moltext uses
cheeriofor fast, static HTML parsing. It does not execute client-side JavaScript. If the documentation is a Single Page Application (SPA) that renders content via JS after load, Moltext may only see a blank shell. - Aggressive Filtering: Moltext automatically strips
<nav>,<footer>, and elements with classes like.sidebaror.navto reduce noise. If the target site happens to wrap its primary content in these tags, the content will be lost.- Fix: Use the
--rawflag to see if the underlying HTML is being captured before LLM normalization.
- Fix: Use the
- Domain Mismatch: The crawler is restricted to the domain of the initial URL. If the documentation links to a sub-domain (e.g., from
example.comtodocs.example.com), it will not follow those links unless you start the crawl at the specific sub-domain.
Timeouts or Connection Refused
- Network Timeout: Moltext has a default request timeout of 10 seconds. On slow documentation servers, this may trigger a failure.
- Bot Protection: Some sites block the default
User-Agent. Moltext identifies asMoltext/1.0. If you are being blocked, ensure your IP isn't rate-limited by the target host.
LLM Processing Failures
Rate Limiting (HTTP 429)
When processing large documentation sets (e.g., --limit 500), you may hit OpenAI rate limits.
- Solution: Use a local model via Ollama or LM Studio.
moltext https://docs.example.com --base-url http://localhost:11434/v1 --model llama3 - Fallback: If the LLM process fails for a specific page, Moltext is designed to gracefully fall back to the raw Markdown version of that page rather than crashing the entire compilation.
Inaccurate Summaries / Hallucinations The LLM may occasionally strip technical details if it perceives them as "filler."
- Solution: If you require 100% deterministic fidelity (e.g., for complex API signatures), use the
--rawflag. This bypasses the LLM "compression" layer and provides the direct Markdown conversion of the HTML.
CLI and Environment
Command Not Found
If moltext is not recognized after installation:
- Solution: Ensure your global
npmbin directory is in your$PATH. You can also run the tool directly via npx:npx moltext https://docs.example.com
Output Path Permissions
- Error:
EACCES: permission denied, open 'context.md' - Solution: Ensure you have write permissions in the directory where you are running the command, or specify a different output path using the
-oflag (e.g.,-o /tmp/context.md).