Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ https://github.com/user-attachments/assets/bd5d38b9-9309-40b9-93ca-918dfa4f3fd4
- [Table of Contents](#table-of-contents)
- [Getting Started](#getting-started)
- [Prerequisites](#prerequisites)
- [Security](#security)
- [Installation](#installation)
- [Docker Compose](#docker-compose)
- [Manual Setup](#manual-setup)
Expand Down Expand Up @@ -133,6 +134,12 @@ https://github.com/user-attachments/assets/bd5d38b9-9309-40b9-93ca-918dfa4f3fd4
- **OpenAI**: An API key with models like `gpt-4o` or `gpt-3.5-turbo`.
- **Ollama**: A running Ollama server with models like `qwen3:8b`.

### Security

**paperless-gpt has no built-in authentication.** Its web UI and `/api/*` endpoints are open to anyone who can reach the port — by default it listens on all interfaces (`LISTEN_INTERFACE` defaults to `:8080`), so a plain `-p 8080:8080` (as in the example below) exposes it to your whole LAN/VPN, not just `localhost`. Anyone who can reach it can rewrite documents in your connected paperless-ngx instance, trigger LLM/OCR jobs against your API keys, and change settings — with zero credentials required.

Do not expose it directly to the internet or an untrusted network. Put it behind a reverse proxy that adds authentication (e.g. Authelia, Authentik, a Basic Auth layer), restrict it to a VPN/Tailscale network, or otherwise limit who can reach the port.

### Installation

#### Docker Compose
Expand Down Expand Up @@ -414,6 +421,7 @@ paperless-gpt offers different methods for processing documents, giving you flex
- **Best for**: Providers that handle multi-page documents efficiently, reduced API calls
- **Configuration**: `OCR_PROCESS_MODE: "whole_pdf"`
- **Note**: Processing large PDFs may cause you to hit the API limit of your OCR provider. If you encounter problems with large documents, consider switching to `pdf` mode, which processes pages individually.
- **Note**: `OCR_LIMIT_PAGES` does **not** apply in this mode — the whole point of `whole_pdf` is to hand the OCR provider the entire document in one shot, so it always processes every page regardless of that setting. Use `pdf` or `image` mode if you need a page cap.

### Provider Compatibility

Expand Down Expand Up @@ -607,7 +615,7 @@ For best results with the enhanced OCR features:
| `PDF_OCR_COMPLETE_TAG` | Tag used to mark documents as OCR-processed. | No | paperless-gpt-ocr-complete |
| `PDF_SKIP_EXISTING_OCR` | Whether to skip OCR processing for PDFs that already have OCR. Works with `pdf` and `whole_pdf` processing modes (`OCR_PROCESS_MODE`). | No | false |
| `AUTO_OCR_TAG` | Tag for automatically processing docs with OCR. | No | paperless-gpt-ocr-auto |
| `OCR_LIMIT_PAGES` | Limit the number of pages for OCR. Set to `0` for no limit. | No | 5 |
| `OCR_LIMIT_PAGES` | Limit the number of pages for OCR. Set to `0` for no limit. Not applied in `whole_pdf` mode (see [Whole PDF Mode](#whole-pdf-mode)), which always processes the entire document. | No | 5 |
| `LOG_LEVEL` | Application log level (`info`, `debug`, `warn`, `error`). | No | info |
| `LISTEN_INTERFACE` | Network interface to listen on. | No | 8080 |
| `AUTO_GENERATE_TITLE` | Generate titles automatically if `paperless-gpt-auto` is used. | No | true |
Expand Down
26 changes: 24 additions & 2 deletions paperless.go
Original file line number Diff line number Diff line change
Expand Up @@ -1253,8 +1253,30 @@ func (client *PaperlessClient) DownloadDocumentAsPDF(ctx context.Context, docume
}
}

// Use pdfcpu to split the PDF
err = api.SplitFile(originalPDFPath, docDir, 1, nil)
// Use pdfcpu to split the PDF. When a page limit applies, trim the
// source down to just the pages we need first so an oversized document
// doesn't cost the same split work regardless of OCR_LIMIT_PAGES - the
// unlimited split extracted (and wrote to disk) every page up front and
// only used the first pagesToProcess afterward.
splitSourcePath := originalPDFPath
if pagesToProcess < totalPages {
trimDir, err := os.MkdirTemp("", "pgpt-trim-*")
if err != nil {
return nil, nil, 0, fmt.Errorf("error creating temp dir for page-limited trim: %w", err)
}
defer os.RemoveAll(trimDir)

// Keep the "original.pdf" basename so pdfcpu's split output naming
// (derived from the input file's basename) still produces
// original_1.pdf, original_2.pdf, ... in docDir below.
splitSourcePath = filepath.Join(trimDir, "original.pdf")
selection := []string{fmt.Sprintf("1-%d", pagesToProcess)}
if err := api.TrimFile(originalPDFPath, splitSourcePath, selection, nil); err != nil {
return nil, nil, 0, fmt.Errorf("error trimming PDF to page limit: %w", err)
}
}

err = api.SplitFile(splitSourcePath, docDir, 1, nil)
if err != nil {
return nil, nil, 0, fmt.Errorf("error splitting PDF: %w", err)
}
Expand Down
53 changes: 53 additions & 0 deletions paperless_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import (
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"testing"

"github.com/stretchr/testify/assert"
Expand Down Expand Up @@ -757,6 +759,57 @@ func TestDownloadDocumentAsPDF(t *testing.T) {
// Testing with splitting=true would be more complex so we'll skip that for simplicity
}

// TestDownloadDocumentAsPDF_SplitWithPageLimit verifies that when a page
// limit is set, the split step only produces (and pdfcpu only has to work
// through) the limited number of pages - not every page in the source PDF.
func TestDownloadDocumentAsPDF_SplitWithPageLimit(t *testing.T) {
env := newTestEnv(t)
defer env.teardown()

documentID := 456

// tests/pdf/five-pager.pdf has 5 pages.
pdfFile := "tests/pdf/five-pager.pdf"
pdfContent, err := os.ReadFile(pdfFile)
require.NoError(t, err)

downloadPath := fmt.Sprintf("/api/documents/%d/download/", documentID)
env.setMockResponse(downloadPath, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
w.Write(pdfContent)
})

ctx := context.Background()
env.client.CacheFolder = "tests/tmp"
os.RemoveAll(env.client.CacheFolder)
defer os.RemoveAll(env.client.CacheFolder)

limitPages := 2
pdfPaths, _, totalPages, err := env.client.DownloadDocumentAsPDF(ctx, documentID, limitPages, true)
require.NoError(t, err)
assert.Equal(t, 5, totalPages, "the source document has 5 pages")
assert.Len(t, pdfPaths, limitPages, "only the page-limited count of split files should be returned")

for _, p := range pdfPaths {
_, err := os.Stat(p)
assert.NoError(t, err, "each returned split path should exist on disk")
}

// Confirm no split output beyond the limit was written to docDir either -
// this is the actual bug being guarded against: pdfcpu used to split
// every page up front regardless of limitPages.
docDir := filepath.Join(env.client.CacheFolder, fmt.Sprintf("document-%d-pdf", documentID))
entries, err := os.ReadDir(docDir)
require.NoError(t, err)
var splitFileCount int
for _, e := range entries {
if strings.HasPrefix(e.Name(), "original_") && strings.HasSuffix(e.Name(), ".pdf") {
splitFileCount++
}
}
assert.Equal(t, limitPages, splitFileCount, "no more than the page-limited count of split files should exist on disk")
}

func TestParsePaperlessValidationErrors(t *testing.T) {
t.Run("real-world response with created_date + one custom_field", func(t *testing.T) {
body := []byte(`{"created_date":["Date has wrong format. Use one of these formats instead: YYYY-MM-DD."],"custom_fields":[{},{},{},{},{},{},{},{"non_field_errors":["Date has wrong format. Use one of these formats instead: YYYY-MM-DD."]}]}`)
Expand Down
Loading