Troubleshooting

This section covers common issues encountered while using Moltext to compile documentation and provides solutions for resolving them.

Authentication and API Keys

Error: API Key is required for OpenAI This occurs when you attempt to use the default LLM processing without providing an OpenAI API key.

Solution 1: Export your key as an environment variable: export OPENAI_API_KEY='sk-...'.
Solution 2: Pass the key directly using the -k or --key flag.
Solution 3: Use --raw mode to skip AI processing entirely. No key is required for raw Markdown extraction.
Solution 4: If using a local inference server (like Ollama), ensure you provide the --base-url pointing to your local endpoint.

Crawling and Extraction Issues

Empty or Minimal Output in context.md If the resulting file contains only headers but no content, the crawler may be struggling with the site structure.

JavaScript-Heavy Sites: Moltext uses cheerio for fast, static HTML parsing. It does not execute client-side JavaScript. If the documentation is a Single Page Application (SPA) that renders content via JS after load, Moltext may only see a blank shell.
Aggressive Filtering: Moltext automatically strips <nav>, <footer>, and elements with classes like .sidebar or .nav to reduce noise. If the target site happens to wrap its primary content in these tags, the content will be lost.
- Fix: Use the --raw flag to see if the underlying HTML is being captured before LLM normalization.
Domain Mismatch: The crawler is restricted to the domain of the initial URL. If the documentation links to a sub-domain (e.g., from example.com to docs.example.com), it will not follow those links unless you start the crawl at the specific sub-domain.

Timeouts or Connection Refused

Network Timeout: Moltext has a default request timeout of 10 seconds. On slow documentation servers, this may trigger a failure.
Bot Protection: Some sites block the default User-Agent. Moltext identifies as Moltext/1.0. If you are being blocked, ensure your IP isn't rate-limited by the target host.

LLM Processing Failures

Rate Limiting (HTTP 429) When processing large documentation sets (e.g., --limit 500), you may hit OpenAI rate limits.

Solution: Use a local model via Ollama or LM Studio.

moltext https://docs.example.com --base-url http://localhost:11434/v1 --model llama3

Fallback: If the LLM process fails for a specific page, Moltext is designed to gracefully fall back to the raw Markdown version of that page rather than crashing the entire compilation.

Inaccurate Summaries / Hallucinations The LLM may occasionally strip technical details if it perceives them as "filler."

Solution: If you require 100% deterministic fidelity (e.g., for complex API signatures), use the --raw flag. This bypasses the LLM "compression" layer and provides the direct Markdown conversion of the HTML.

CLI and Environment

Command Not Found If moltext is not recognized after installation:

Solution: Ensure your global npm bin directory is in your $PATH. You can also run the tool directly via npx:
```
npx moltext https://docs.example.com
```

Output Path Permissions

Error: EACCES: permission denied, open 'context.md'
Solution: Ensure you have write permissions in the directory where you are running the command, or specify a different output path using the -o flag (e.g., -o /tmp/context.md).