LLM-translator/AGENTS.md

LLM Translator — Agent Guide

Project Overview

A 4-phase translation pipeline web app. Users input source text, then sequentially execute:

1. Phase 1 (Rough Translation) — LLM translates + extracts proper nouns, summary, style
2. Phase 2 (Proper Noun Review) — User reviews/edits extracted proper nouns and style
3. Phase 3 (Re-translation) — LLM re-translates with proper noun/style constraints applied
4. Phase 4 (Polish) — LLM polishes for readability while preserving proper nouns

Architecture: FastAPI backend + Vue 3 SPA (Vite build, served as static files by FastAPI). Single-server deployment: uvicorn backend.main:app serves both API and frontend.

Tech Stack

Layer	Technology
Backend	Python 3.14+, FastAPI, Uvicorn
Auth	JWT (PyJWT), bcrypt (direct via bcrypt module)
LLM Client	OpenAI SDK (openai) — OpenAI-compatible API format
Frontend	Vue 3 + Composition API <script setup>, TypeScript, Pinia
Styling	Tailwind CSS v3 with darkMode: 'class' strategy
Build	Vite (frontend), no bundler for backend

Directory Structure

LLM-translator/
├── backend/                   # FastAPI application
│   ├── main.py                # App entry: routes, CORS, static file mounting
│   ├── auth.py                # JWT auth + bcrypt password verification
│   ├── llm_client.py          # OpenAI-compatible client (async)
│   ├── models.py              # Pydantic schemas for all API I/O
│   ├── prompts.py             # Phase 1–4 prompt templates
│   └── sessions.py            # In-memory session store (TTL 24h, UUID keys)
├── frontend/                  # Vue 3 SPA source
│   ├── src/
│   │   ├── main.ts            # App entry + dark mode pre-mount class application
│   │   ├── App.vue            # Root: LoginView | TranslatorView based on auth state
│   │   ├── views/
│   │   │   ├── LoginView.vue  # ID/PW login form → localStorage JWT
│   │   │   └── TranslatorView.vue  # Main UI: model selectors, phase buttons, comparison
│   │   ├── components/
│   │   │   ├── LLMSelector.vue        # Model dropdown (dark mode aware)
│   │   │   ├── ProperNounsEditor.vue  # Proper noun table + style input
│   │   │   └── ComparisonView.vue     # 3-column Phase 1/3/4 result comparison
│   │   ├── stores/
│   │   │   ├── auth.ts          # JWT state (check/login/logout)
│   │   │   ├── translation.ts   # Pipeline state + localStorage persistence
│   │   │   └── theme.ts         # Dark mode toggle + localStorage + system preference detection
│   │   ├── api.ts               # Axios client with JWT interceptor
│   │   └── types.ts             # TypeScript interfaces mirroring backend models
│   ├── tailwind.config.js       # darkMode: 'class' configured here
│   ├── vite.config.ts           # Dev proxy to /api → localhost:8000; build output → dist/
│   └── package.json
├── config/
│   ├── llms.json                # LLM server configs [{alias, base_url, api_key, model, context_size}]
│   └── users.json               # Users [{id, password}] — password_plain is migrated on startup
├── requirements.txt             # Python deps: fastapi, uvicorn[standard], pyjwt, bcrypt, openai, python-dotenv
├── .env.example                 # Environment variables template
├── tests/                       # Backend API and session regression tests
└── README.md                    # Setup and operation guide

Key Patterns & Conventions

Backend

• No passlib — use bcrypt module directly (bcrypt.hashpw, bcrypt.checkpw). The project uses Python 3.14 which has compatibility issues with passlib's bcrypt wrapper.
• LLM client caching — llm_client.py caches an AsyncOpenAI client by alias. Clients are lazily instantiated on first use.
• Session store — process-local in-memory dict (sessions.py) scoped by authenticated owner. Key is UUID hex (16 chars). TTL is 24h from last access; there is no persistence beyond process lifetime.
• JSON extraction — LLM responses may be wrapped in markdown code blocks. _extract_json() strips ```json` ... wrappers before json.loads().
• Long-text chunking — phases 1, 3, and 4 process text sequentially near sentence boundaries. Defaults are 1,500 characters per chunk and a 180-second timeout per LLM call; both are configurable by environment variables.
• Progress feedback — LLM calls stream response deltas into transient session progress. The frontend polls /api/sessions/{session_id}/progress and displays the current chunk and a rolling preview.
• Malformed Phase 1 JSON — parsing tolerates fences, surrounding prose, alternate translation keys, missing noun fields, and trailing commas. Remaining failures trigger one JSON-repair call and then a plain-translation fallback.

Frontend

• All Vue components use <script setup lang="ts"> with Composition API. No Options API.
• State management via Pinia stores — auth, translation, theme. Stores persist model selections and history to localStorage.
• Dark mode is class-based (<html class="dark">). Applied in main.ts before mount to prevent flash, then toggled by useThemeStore.toggle(). Every UI component has corresponding dark: Tailwind classes.
• API calls go through /api proxy during dev (vite.config.ts). In production the same origin serves both API and static files.

LLM Configuration

Models are defined in config/llms.json. Each entry:

{
  "alias": "Qwen3.6-35B",      // Display name, used by LLM phases 1, 3, and 4
  "base_url": "http://...",     // OpenAI-compatible base URL (must end with /v1)
  "api_key": "...",             // API key for this server
  "model": "hx370/qwen3.6-35b-a3b",  // Exact model name as used by the LLM provider
  "context_size": 65536         // Token context window size (for length warnings)
}

Translation Pipeline Data Flow

source_text ──► Phase1(LLM) ──► {translated, proper_nouns[], summary, style}
                                    │
                                    ▼ user edits in ProperNounsEditor
                              phase2_proper_nouns[], phase2_style
                                    │
                                    ▼ Phase3(LLM): re-translate with constraints
                              phase3_result
                                    │
                                    ▼ Phase4(LLM): polish for readability
                              phase4_result (final)

Phase dependencies are enforced: Phase 2 and 3 require Phase 1, and Phase 4 requires Phase 3. Running Phase 3 automatically submits the current Phase 2 edits.

Theme / Dark Mode

• theme.ts store manages state in Pinia + localStorage. On init, checks saved value or falls back to system preference (prefers-color-scheme).
• Every component with background/border/text colors must include matching dark: variants.
• Default light theme: white/slate backgrounds, blue accents. Dark theme: gray-800/900 backgrounds, adjusted borders, preserved accent colors (blue, purple, emerald remain readable).

Running the Project

# 1. Install dependencies
pip install -r requirements.txt
cd frontend && npm install && cd ..

# 2. Build frontend (required before serving in production mode)
cd frontend && npm run build && cd ..

# 3. Configure LLM servers and users
#    Edit config/llms.json and config/users.json directly

# 4. Start server
uvicorn backend.main:app --host 0.0.0.0 --port 8000
# Or with env vars for custom paths:
JWT_SECRET_KEY=your-secret uvicorn backend.main:app --reload

# 5. Open http://localhost:8000

Default login: admin / changeme123. First startup auto-hashes plain-text passwords in users.json and saves the file.

Adding a New Phase or Modifying Prompts

1. Edit backend/prompts.py: add new prompt templates using f-strings with source data parameters
2. Add API endpoint in backend/main.py following the existing pattern:

   @app.post("/api/translate/{session_id}/phaseN")
   async def run_phaseN(session_id: str, user_id: str = Depends(get_current_user)):
       session = session_store.get(session_id, user_id)
       # ... call chat_complete(alias, system_prompt, user_content)
       updated = session_store.update(
           session_id, {"phaseN_result": result_text}, user_id
       )
       return SessionResponse(session_id=session_id, data=updated).model_dump()
   

3. Add frontend component and button in TranslatorView.vue
4. Update translation.ts store with new execution method + state

File I/O Notes

• config/llms.json — loaded once at startup by llm_client.py._load_llm_configs(). Not re-read on requests. Changes require server restart.
• config/users.json — loaded once at startup by auth.py._load_users_file(). Plain-text passwords are auto-migrated to bcrypt hashes and saved back on first run only.

Testing Against LLM Server

# Quick connectivity test (use exact model name from llms.json)
curl -s http://btuna.net:45455/v1/chat/completions \
  -H "Authorization: Bearer btuna-public-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"hx370/qwen3.6-35b-a3b","messages":[{"role":"user","content":"test"}],"max_tokens":10}'

If you get PermissionDeniedError, the API key lacks access to that model — try a different model or contact server admin for correct permissions.