Compare commits

..

1 Commits

Author SHA1 Message Date
Tim Baek fe6783c166 Merge pull request #19030 from open-webui/dev
0.6.37
2025-11-23 22:10:05 -05:00
937 changed files with 102346 additions and 221802 deletions
+1 -10
View File
@@ -19,13 +19,4 @@ FORWARDED_ALLOW_IPS='*'
# DO NOT TRACK
SCARF_NO_ANALYTICS=true
DO_NOT_TRACK=true
ANONYMIZED_TELEMETRY=false
# Valkey Vector Store (requires VECTOR_DB=valkey)
# VALKEY_URL='valkey://localhost:6379'
# VALKEY_COLLECTION_PREFIX='open_webui'
# VALKEY_INDEX_TYPE='HNSW'
# VALKEY_DISTANCE_METRIC='COSINE'
# VALKEY_HNSW_M='16'
# VALKEY_HNSW_EF_CONSTRUCTION='200'
# VALKEY_HNSW_EF_RUNTIME='10'
ANONYMIZED_TELEMETRY=false
+36 -30
View File
@@ -1,76 +1,82 @@
name: Feature Request
description: Suggest a new feature or improvement
description: Suggest an idea for this project
title: 'feat: '
labels: ['triage']
body:
- type: markdown
attributes:
value: |
## Before Submitting
## Important Notes
### Before submitting
Please check the **open AND closed** [Issues](https://github.com/open-webui/open-webui/issues) AND [Discussions](https://github.com/open-webui/open-webui/discussions) to see if a similar request has been posted.
It's likely we're already tracking it! If youre unsure, start a discussion post first.
Please check **open AND closed** [Issues](https://github.com/open-webui/open-webui/issues) and [Discussions](https://github.com/open-webui/open-webui/discussions) for similar requests. If you find one, add your input there instead.
#### Scope
### Scope Guidelines
If your feature request is likely to take more than a quick coding session to implement, test and verify, then open it in the **Ideas** section of the [Discussions](https://github.com/open-webui/open-webui/discussions) instead.
**We will close and force move your feature request to the Ideas section, if we believe your feature request is not trivial/quick to implement.**
This is to ensure the issues tab is used only for issues, quickly addressable feature requests and tracking tickets by the maintainers.
Other feature requests belong in the **Ideas** section of the [Discussions](https://github.com/open-webui/open-webui/discussions).
If your feature request might impact others in the community, definitely open a discussion instead and evaluate whether and how to implement it.
This will help us efficiently focus on improving the project.
### Collaborate respectfully
We value a **constructive attitude**, so please be mindful of your communication. If negativity is part of your approach, our capacity to engage may be limited. We're here to help if you're **open to learning** and **communicating positively**.
Feature requests that require significant implementation effort should be posted in the **Ideas** section of [Discussions](https://github.com/open-webui/open-webui/discussions) instead. We will move oversized feature requests to Discussions to keep the Issues tab focused on actionable items.
Remember:
- Open WebUI is a **volunteer-driven project**
- It's managed by a **single maintainer**
- It's supported by contributors who also have **full-time jobs**
If your request might impact the broader community, please open a Discussion first so others can weigh in on the design.
### Be Respectful
Open WebUI is a volunteer-driven project maintained by a small team. We value constructive, positive communication. Please be mindful of maintainers' time and energy.
We appreciate your time and ask that you **respect ours**.
### Contributing
If you encounter an issue, we highly encourage you to submit a pull request or fork the project. We actively work to prevent contributor burnout to maintain the quality and continuity of Open WebUI.
If you encounter an issue, we encourage you to submit a pull request or fork the project. We actively work to prevent contributor burnout and maintain project quality.
### Reproducibility
If a bug cannot be reproduced with a `:main` or `:dev` Docker setup, or a `pip install` with Python 3.11, it may be moved to the "Issues" section in Discussions for community assistance.
### Bug reproducibility
If a bug cannot be reproduced with a `:main` or `:dev` Docker setup, or a `pip install` with Python 3.11, it may require additional help from the community. In such cases, we will move it to the "[issues](https://github.com/open-webui/open-webui/discussions/categories/issues)" Discussions section due to our limited resources. We encourage the community to assist with these issues. Remember, its not that the issue doesnt exist; we need your help!
- type: checkboxes
id: existing-issue
attributes:
label: Check Existing Issues
description: Confirm you have searched for similar requests.
description: Please confirm that you've checked for existing similar requests
options:
- label: I have searched all existing **open AND closed** issues and discussions and found none comparable to my request.
- label: I have searched for all existing **open AND closed** issues and discussions for similar requests. I have found none that is comparable to my request.
required: true
- type: checkboxes
id: feature-scope
attributes:
label: Verify Feature Scope
description: Confirm this request belongs in Issues rather than Discussions.
description: Please confirm the feature's scope is within the described scope
options:
- label: I believe this feature request is appropriately scoped for the Issues section as described above.
- label: I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions.
required: true
- type: textarea
id: problem-description
attributes:
label: Problem Description
description: Is this related to a problem? Describe the pain point clearly.
placeholder: "e.g., I'm frustrated when..."
description: Is your feature request related to a problem? Please provide a clear and concise description of what the problem is.
placeholder: "Ex. I'm always frustrated when... / Not related to a problem"
validations:
required: true
- type: textarea
id: solution-description
attributes:
label: Proposed Solution
description: Describe what you would like to happen.
label: Desired Solution you'd like
description: Clearly describe what you want to happen.
validations:
required: true
- type: textarea
id: alternatives-considered
attributes:
label: Alternatives Considered
description: Describe any alternative solutions or workarounds you have considered.
description: A clear and concise description of any alternative solutions or features you've considered.
- type: textarea
id: additional-context
attributes:
label: Additional Context
description: Add any other context, mockups, or screenshots about the feature request.
description: Add any other context or screenshots about the feature request here.
+5 -10
View File
@@ -4,22 +4,17 @@ updates:
directory: '/'
schedule:
interval: monthly
target-branch: dev
target-branch: 'dev'
- package-ecosystem: pip
directory: '/backend'
schedule:
interval: monthly
target-branch: dev
target-branch: 'dev'
- package-ecosystem: github-actions
- package-ecosystem: 'github-actions'
directory: '/'
schedule:
# Check for updates to GitHub Actions every week
interval: monthly
target-branch: dev
- package-ecosystem: npm
directory: '/'
schedule:
interval: monthly
target-branch: dev
target-branch: 'dev'
+33 -55
View File
@@ -1,104 +1,82 @@
<!--
⚠️ CRITICAL CHECKS FOR CONTRIBUTORS (READ, DON'T DELETE) ⚠️
1. Target the `dev` branch. PRs targeting `main` will be automatically closed.
2. Do NOT delete the CLA section at the bottom. It is required for the bot to accept your PR.
-->
# Pull Request Checklist
### Note to first-time contributors: Please open a discussion post in [Discussions](https://github.com/open-webui/open-webui/discussions) to discuss your idea/fix with the community before creating a pull request, and describe your changes before submitting a pull request.
This is to ensure large feature PRs are discussed with the community first, before starting work on it. If the community does not want this feature or it is not relevant for Open WebUI as a project, it can be identified in the discussion before working on the feature and submitting the PR.
<!--
### ⚠️ Important: Your PR is a contribution, not a guarantee of merge.
The most impactful way to contribute to Open WebUI is through well-written bug reports, detailed feature discussions, and thoughtful ideas. These directly shape the project. If you do open a pull request, please know that Open WebUI is held to the highest standard of code quality, consistency, and architectural coherence, and every line merged becomes something the core team must own, maintain, and support indefinitely. Submitted code may be refactored, rewritten, or used as inspiration for a different implementation. This is not a reflection of your work's quality. It is how we ensure that a small team can deeply understand and evolve every part of the codebase.
-->
**Before submitting, make sure you've checked the following:**
- [ ] **Linked Issue/Discussion:** This PR references an existing [Issue](https://github.com/open-webui/open-webui/issues) or [Discussion](https://github.com/open-webui/open-webui/discussions) — `Closes #___` / `Relates to #___`. If one does not exist, create one first. PRs without a linked issue or discussion may be closed without review.
- [ ] **Target branch:** The pull request targets the `dev` branch. **PRs targeting `main` will be immediately closed.**
- [ ] **Description:** A concise description of the changes is provided below.
- [ ] **Changelog:** A changelog entry following [Keep a Changelog](https://keepachangelog.com/) format is included at the bottom.
- [ ] **Documentation:** Relevant documentation has been added or updated in the [Open WebUI Docs Repository](https://github.com/open-webui/docs).
- [ ] **Dependencies:** Any new or updated dependencies are explained, tested, and documented.
- [ ] **Testing:** Manual tests have been performed to verify the fix/feature works correctly and does not introduce regressions. Screenshots or recordings are included where applicable.
- [ ] **No Unchecked AI Code:** This PR is either human-written or has undergone thorough human review AND manual testing. Unreviewed AI-generated PRs may be closed immediately.
- [ ] **Self-Review:** A self-review of the code has been performed, ensuring adherence to project coding standards.
- [ ] **Architecture:** Smart defaults are preferred over new settings. Local state is used for ephemeral UI logic. Major architectural or UX changes have been discussed first.
- [ ] **Git Hygiene:** The PR is atomic (one logical change), rebased on `dev`, and contains no unrelated commits.
- [ ] **Title Prefix:** The PR title uses one of the following prefixes:
- **BREAKING CHANGE**: Changes affecting backward compatibility
- **build**: Build system or dependency changes
- **ci**: CI/CD workflow changes
- **chore**: Refactoring, cleanup, or non-functional changes
- **docs**: Documentation additions or updates
- **feat**: New features or enhancements
- **fix**: Bug fixes or corrections
- [ ] **Target branch:** Verify that the pull request targets the `dev` branch. **Not targeting the `dev` branch will lead to immediate closure of the PR.**
- [ ] **Description:** Provide a concise description of the changes made in this pull request down below.
- [ ] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description.
- [ ] **Documentation:** If necessary, update relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs) like environment variables, the tutorials, or other documentation sources.
- [ ] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation?
- [ ] **Testing:** Perform manual tests to **verify the implemented fix/feature works as intended AND does not break any other functionality**. Take this as an opportunity to **make screenshots of the feature/fix and include it in the PR description**.
- [ ] **Agentic AI Code:** Confirm this Pull Request is **not written by any AI Agent** or has at least **gone through additional human review AND manual testing**. If any AI Agent is the co-author of this PR, it may lead to immediate closure of the PR.
- [ ] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
- [ ] **Title Prefix:** To clearly categorize this pull request, prefix the pull request title using one of the following:
- **BREAKING CHANGE**: Significant changes that may affect compatibility
- **build**: Changes that affect the build system or external dependencies
- **ci**: Changes to our continuous integration processes or workflows
- **chore**: Refactor, cleanup, or other non-functional code changes
- **docs**: Documentation update or addition
- **feat**: Introduces a new feature or enhancement to the codebase
- **fix**: Bug fix or error correction
- **i18n**: Internationalization or localization changes
- **perf**: Performance improvements
- **refactor**: Code restructuring
- **style**: Formatting changes (whitespace, semicolons, etc.)
- **test**: Test additions or corrections
- **WIP**: Work in progress
- **perf**: Performance improvement
- **refactor**: Code restructuring for better maintainability, readability, or scalability
- **style**: Changes that do not affect the meaning of the code (white space, formatting, missing semi-colons, etc.)
- **test**: Adding missing tests or correcting existing tests
- **WIP**: Work in progress, a temporary label for incomplete or ongoing work
# Changelog Entry
### Description
- [Describe the changes, including motivation and impact]
- [Concisely describe the changes made in this pull request, including any relevant motivation and impact (e.g., fixing a bug, adding a feature, or improving performance)]
### Added
- [New features, functionalities, or additions]
- [List any new features, functionalities, or additions]
### Changed
- [Changes, updates, refactorings, or optimizations]
- [List any changes, updates, refactorings, or optimizations]
### Deprecated
- [Deprecated functionality or features]
- [List any deprecated functionality or features that have been removed]
### Removed
- [Removed features, files, or functionalities]
- [List any removed features, files, or functionalities]
### Fixed
- [Bug fixes or corrections]
- [List any fixes, corrections, or bug fixes]
### Security
- [Security-related changes or vulnerability fixes]
- [List any new or updated security-related changes, including vulnerability fixes]
### Breaking Changes
- **BREAKING CHANGE**: [Changes affecting compatibility or functionality]
- **BREAKING CHANGE**: [List any breaking changes affecting compatibility or functionality]
---
### Additional Information
- [Any additional context, notes, or references to related issues/commits]
- [Insert any additional context, notes, or explanations for the changes]
- [Reference any related issues, commits, or other relevant information]
### Screenshots or Videos
- [Attach relevant screenshots or videos demonstrating the changes]
- [Attach any relevant screenshots or videos demonstrating the changes]
### Contributor License Agreement
<!--
🚨 DO NOT DELETE THE TEXT BELOW 🚨
Keep the "Contributor License Agreement" confirmation text intact.
Deleting it will trigger the CLA-Bot to INVALIDATE your PR.
Your PR will NOT be reviewed or merged until you check the box below confirming that you have read and agree to the terms of the CLA.
-->
- [ ] By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms.
By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms.
> [!NOTE]
> Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
-40
View File
@@ -1,40 +0,0 @@
# ─────────────────────────────────────────────────────────────────────────────
# Backend CI — Python formatting checks via Ruff
# Runs on pushes and PRs to main/dev when backend files change
# ─────────────────────────────────────────────────────────────────────────────
name: Python CI
on:
push:
branches: [main, dev]
paths: ['backend/**', 'pyproject.toml', 'uv.lock']
pull_request:
branches: [main, dev]
paths: ['backend/**', 'pyproject.toml', 'uv.lock']
concurrency:
group: backend-${{ github.ref }}
cancel-in-progress: true
jobs:
# ── Ruff format check across supported Python versions ───────────────────
format-check:
name: Ruff Format (${{ matrix.python-version }})
runs-on: ubuntu-latest
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
python-version: ['3.11', '3.12']
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- name: Install formatter
run: pip install "ruff>=0.15.5"
- name: Verify formatting
run: ruff format --check . --exclude .venv --exclude venv
+72
View File
@@ -0,0 +1,72 @@
name: Release
on:
push:
branches:
- main # or whatever branch you want to use
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v5
- name: Check for changes in package.json
run: |
git diff --cached --diff-filter=d package.json || {
echo "No changes to package.json"
exit 1
}
- name: Get version number from package.json
id: get_version
run: |
VERSION=$(jq -r '.version' package.json)
echo "::set-output name=version::$VERSION"
- name: Extract latest CHANGELOG entry
id: changelog
run: |
CHANGELOG_CONTENT=$(awk 'BEGIN {print_section=0;} /^## \[/ {if (print_section == 0) {print_section=1;} else {exit;}} print_section {print;}' CHANGELOG.md)
CHANGELOG_ESCAPED=$(echo "$CHANGELOG_CONTENT" | sed ':a;N;$!ba;s/\n/%0A/g')
echo "Extracted latest release notes from CHANGELOG.md:"
echo -e "$CHANGELOG_CONTENT"
echo "::set-output name=content::$CHANGELOG_ESCAPED"
- name: Create GitHub release
uses: actions/github-script@v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const changelog = `${{ steps.changelog.outputs.content }}`;
const release = await github.rest.repos.createRelease({
owner: context.repo.owner,
repo: context.repo.repo,
tag_name: `v${{ steps.get_version.outputs.version }}`,
name: `v${{ steps.get_version.outputs.version }}`,
body: changelog,
})
console.log(`Created release ${release.data.html_url}`)
- name: Upload package to GitHub release
uses: actions/upload-artifact@v4
with:
name: package
path: |
.
!.git
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Trigger Docker build workflow
uses: actions/github-script@v8
with:
script: |
github.rest.actions.createWorkflowDispatch({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: 'docker-build.yaml',
ref: 'v${{ steps.get_version.outputs.version }}',
})
+63
View File
@@ -0,0 +1,63 @@
name: Deploy to HuggingFace Spaces
on:
push:
branches:
- dev
- main
workflow_dispatch:
jobs:
check-secret:
runs-on: ubuntu-latest
outputs:
token-set: ${{ steps.check-key.outputs.defined }}
steps:
- id: check-key
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
if: "${{ env.HF_TOKEN != '' }}"
run: echo "defined=true" >> $GITHUB_OUTPUT
deploy:
runs-on: ubuntu-latest
needs: [check-secret]
if: needs.check-secret.outputs.token-set == 'true'
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
steps:
- name: Checkout repository
uses: actions/checkout@v5
with:
lfs: true
- name: Remove git history
run: rm -rf .git
- name: Prepend YAML front matter to README.md
run: |
echo "---" > temp_readme.md
echo "title: Open WebUI" >> temp_readme.md
echo "emoji: 🐳" >> temp_readme.md
echo "colorFrom: purple" >> temp_readme.md
echo "colorTo: gray" >> temp_readme.md
echo "sdk: docker" >> temp_readme.md
echo "app_port: 8080" >> temp_readme.md
echo "---" >> temp_readme.md
cat README.md >> temp_readme.md
mv temp_readme.md README.md
- name: Configure git
run: |
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config --global user.name "github-actions[bot]"
- name: Set up Git and push to Space
run: |
git init --initial-branch=main
git lfs install
git lfs track "*.ttf"
git lfs track "*.jpg"
rm demo.gif
git add .
git commit -m "GitHub deploy: ${{ github.sha }}"
git push --force https://open-webui:${HF_TOKEN}@huggingface.co/spaces/open-webui/open-webui main
+800
View File
@@ -0,0 +1,800 @@
name: Create and publish Docker images with specific build args
on:
workflow_dispatch:
push:
branches:
- main
- dev
tags:
- v*
env:
REGISTRY: ghcr.io
jobs:
build-main-image:
runs-on: ${{ matrix.runner }}
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
- platform: linux/arm64
runner: ubuntu-24.04-arm
steps:
# GitHub Packages requires the entire repository name to be in lowercase
# although the repository owner has a lowercase username, this prevents some people from running actions after forking
- name: Set repository and image name to lowercase
run: |
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}
echo "FULL_IMAGE_NAME=ghcr.io/${IMAGE_NAME,,}" >>${GITHUB_ENV}
env:
IMAGE_NAME: '${{ github.repository }}'
- name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Checkout repository
uses: actions/checkout@v5
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images (default latest tag)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
- name: Extract metadata for Docker cache
id: cache-meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
${{ github.ref_type == 'tag' && 'type=raw,value=main' || '' }}
flavor: |
prefix=cache-${{ matrix.platform }}-
latest=false
- name: Build Docker image (latest)
uses: docker/build-push-action@v5
id: build
with:
context: .
push: true
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.FULL_IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=registry,ref=${{ steps.cache-meta.outputs.tags }}
cache-to: type=registry,ref=${{ steps.cache-meta.outputs.tags }},mode=max
build-args: |
BUILD_HASH=${{ github.sha }}
- name: Export digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-main-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
build-cuda-image:
runs-on: ${{ matrix.runner }}
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
- platform: linux/arm64
runner: ubuntu-24.04-arm
steps:
# GitHub Packages requires the entire repository name to be in lowercase
# although the repository owner has a lowercase username, this prevents some people from running actions after forking
- name: Set repository and image name to lowercase
run: |
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}
echo "FULL_IMAGE_NAME=ghcr.io/${IMAGE_NAME,,}" >>${GITHUB_ENV}
env:
IMAGE_NAME: '${{ github.repository }}'
- name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Checkout repository
uses: actions/checkout@v5
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images (cuda tag)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,enable=${{ github.ref == 'refs/heads/main' }},prefix=,suffix=,value=cuda
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
suffix=-cuda,onlatest=true
- name: Extract metadata for Docker cache
id: cache-meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
${{ github.ref_type == 'tag' && 'type=raw,value=main' || '' }}
flavor: |
prefix=cache-cuda-${{ matrix.platform }}-
latest=false
- name: Build Docker image (cuda)
uses: docker/build-push-action@v5
id: build
with:
context: .
push: true
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.FULL_IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=registry,ref=${{ steps.cache-meta.outputs.tags }}
cache-to: type=registry,ref=${{ steps.cache-meta.outputs.tags }},mode=max
build-args: |
BUILD_HASH=${{ github.sha }}
USE_CUDA=true
- name: Export digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-cuda-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
build-cuda126-image:
runs-on: ${{ matrix.runner }}
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
- platform: linux/arm64
runner: ubuntu-24.04-arm
steps:
# GitHub Packages requires the entire repository name to be in lowercase
# although the repository owner has a lowercase username, this prevents some people from running actions after forking
- name: Set repository and image name to lowercase
run: |
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}
echo "FULL_IMAGE_NAME=ghcr.io/${IMAGE_NAME,,}" >>${GITHUB_ENV}
env:
IMAGE_NAME: '${{ github.repository }}'
- name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Checkout repository
uses: actions/checkout@v5
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images (cuda126 tag)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,enable=${{ github.ref == 'refs/heads/main' }},prefix=,suffix=,value=cuda126
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
suffix=-cuda126,onlatest=true
- name: Extract metadata for Docker cache
id: cache-meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
${{ github.ref_type == 'tag' && 'type=raw,value=main' || '' }}
flavor: |
prefix=cache-cuda126-${{ matrix.platform }}-
latest=false
- name: Build Docker image (cuda126)
uses: docker/build-push-action@v5
id: build
with:
context: .
push: true
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.FULL_IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=registry,ref=${{ steps.cache-meta.outputs.tags }}
cache-to: type=registry,ref=${{ steps.cache-meta.outputs.tags }},mode=max
build-args: |
BUILD_HASH=${{ github.sha }}
USE_CUDA=true
USE_CUDA_VER=cu126
- name: Export digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-cuda126-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
build-ollama-image:
runs-on: ${{ matrix.runner }}
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
- platform: linux/arm64
runner: ubuntu-24.04-arm
steps:
# GitHub Packages requires the entire repository name to be in lowercase
# although the repository owner has a lowercase username, this prevents some people from running actions after forking
- name: Set repository and image name to lowercase
run: |
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}
echo "FULL_IMAGE_NAME=ghcr.io/${IMAGE_NAME,,}" >>${GITHUB_ENV}
env:
IMAGE_NAME: '${{ github.repository }}'
- name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Checkout repository
uses: actions/checkout@v5
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images (ollama tag)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,enable=${{ github.ref == 'refs/heads/main' }},prefix=,suffix=,value=ollama
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
suffix=-ollama,onlatest=true
- name: Extract metadata for Docker cache
id: cache-meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
${{ github.ref_type == 'tag' && 'type=raw,value=main' || '' }}
flavor: |
prefix=cache-ollama-${{ matrix.platform }}-
latest=false
- name: Build Docker image (ollama)
uses: docker/build-push-action@v5
id: build
with:
context: .
push: true
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.FULL_IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=registry,ref=${{ steps.cache-meta.outputs.tags }}
cache-to: type=registry,ref=${{ steps.cache-meta.outputs.tags }},mode=max
build-args: |
BUILD_HASH=${{ github.sha }}
USE_OLLAMA=true
- name: Export digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-ollama-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
build-slim-image:
runs-on: ${{ matrix.runner }}
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
- platform: linux/arm64
runner: ubuntu-24.04-arm
steps:
# GitHub Packages requires the entire repository name to be in lowercase
# although the repository owner has a lowercase username, this prevents some people from running actions after forking
- name: Set repository and image name to lowercase
run: |
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}
echo "FULL_IMAGE_NAME=ghcr.io/${IMAGE_NAME,,}" >>${GITHUB_ENV}
env:
IMAGE_NAME: '${{ github.repository }}'
- name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Checkout repository
uses: actions/checkout@v5
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images (slim tag)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,enable=${{ github.ref == 'refs/heads/main' }},prefix=,suffix=,value=slim
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
suffix=-slim,onlatest=true
- name: Extract metadata for Docker cache
id: cache-meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
${{ github.ref_type == 'tag' && 'type=raw,value=main' || '' }}
flavor: |
prefix=cache-slim-${{ matrix.platform }}-
latest=false
- name: Build Docker image (slim)
uses: docker/build-push-action@v5
id: build
with:
context: .
push: true
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.FULL_IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=registry,ref=${{ steps.cache-meta.outputs.tags }}
cache-to: type=registry,ref=${{ steps.cache-meta.outputs.tags }},mode=max
build-args: |
BUILD_HASH=${{ github.sha }}
USE_SLIM=true
- name: Export digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-slim-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
merge-main-images:
runs-on: ubuntu-latest
needs: [build-main-image]
steps:
# GitHub Packages requires the entire repository name to be in lowercase
# although the repository owner has a lowercase username, this prevents some people from running actions after forking
- name: Set repository and image name to lowercase
run: |
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}
echo "FULL_IMAGE_NAME=ghcr.io/${IMAGE_NAME,,}" >>${GITHUB_ENV}
env:
IMAGE_NAME: '${{ github.repository }}'
- name: Download digests
uses: actions/download-artifact@v5
with:
pattern: digests-main-*
path: /tmp/digests
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images (default latest tag)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.FULL_IMAGE_NAME }}@sha256:%s ' *)
- name: Inspect image
run: |
docker buildx imagetools inspect ${{ env.FULL_IMAGE_NAME }}:${{ steps.meta.outputs.version }}
merge-cuda-images:
runs-on: ubuntu-latest
needs: [build-cuda-image]
steps:
# GitHub Packages requires the entire repository name to be in lowercase
# although the repository owner has a lowercase username, this prevents some people from running actions after forking
- name: Set repository and image name to lowercase
run: |
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}
echo "FULL_IMAGE_NAME=ghcr.io/${IMAGE_NAME,,}" >>${GITHUB_ENV}
env:
IMAGE_NAME: '${{ github.repository }}'
- name: Download digests
uses: actions/download-artifact@v5
with:
pattern: digests-cuda-*
path: /tmp/digests
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images (default latest tag)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,enable=${{ github.ref == 'refs/heads/main' }},prefix=,suffix=,value=cuda
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
suffix=-cuda,onlatest=true
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.FULL_IMAGE_NAME }}@sha256:%s ' *)
- name: Inspect image
run: |
docker buildx imagetools inspect ${{ env.FULL_IMAGE_NAME }}:${{ steps.meta.outputs.version }}
merge-cuda126-images:
runs-on: ubuntu-latest
needs: [build-cuda126-image]
steps:
# GitHub Packages requires the entire repository name to be in lowercase
# although the repository owner has a lowercase username, this prevents some people from running actions after forking
- name: Set repository and image name to lowercase
run: |
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}
echo "FULL_IMAGE_NAME=ghcr.io/${IMAGE_NAME,,}" >>${GITHUB_ENV}
env:
IMAGE_NAME: '${{ github.repository }}'
- name: Download digests
uses: actions/download-artifact@v5
with:
pattern: digests-cuda126-*
path: /tmp/digests
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images (default latest tag)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,enable=${{ github.ref == 'refs/heads/main' }},prefix=,suffix=,value=cuda126
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
suffix=-cuda126,onlatest=true
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.FULL_IMAGE_NAME }}@sha256:%s ' *)
- name: Inspect image
run: |
docker buildx imagetools inspect ${{ env.FULL_IMAGE_NAME }}:${{ steps.meta.outputs.version }}
merge-ollama-images:
runs-on: ubuntu-latest
needs: [build-ollama-image]
steps:
# GitHub Packages requires the entire repository name to be in lowercase
# although the repository owner has a lowercase username, this prevents some people from running actions after forking
- name: Set repository and image name to lowercase
run: |
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}
echo "FULL_IMAGE_NAME=ghcr.io/${IMAGE_NAME,,}" >>${GITHUB_ENV}
env:
IMAGE_NAME: '${{ github.repository }}'
- name: Download digests
uses: actions/download-artifact@v5
with:
pattern: digests-ollama-*
path: /tmp/digests
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images (default ollama tag)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,enable=${{ github.ref == 'refs/heads/main' }},prefix=,suffix=,value=ollama
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
suffix=-ollama,onlatest=true
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.FULL_IMAGE_NAME }}@sha256:%s ' *)
- name: Inspect image
run: |
docker buildx imagetools inspect ${{ env.FULL_IMAGE_NAME }}:${{ steps.meta.outputs.version }}
merge-slim-images:
runs-on: ubuntu-latest
needs: [build-slim-image]
steps:
# GitHub Packages requires the entire repository name to be in lowercase
# although the repository owner has a lowercase username, this prevents some people from running actions after forking
- name: Set repository and image name to lowercase
run: |
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}
echo "FULL_IMAGE_NAME=ghcr.io/${IMAGE_NAME,,}" >>${GITHUB_ENV}
env:
IMAGE_NAME: '${{ github.repository }}'
- name: Download digests
uses: actions/download-artifact@v5
with:
pattern: digests-slim-*
path: /tmp/digests
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images (default slim tag)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,enable=${{ github.ref == 'refs/heads/main' }},prefix=,suffix=,value=slim
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
suffix=-slim,onlatest=true
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.FULL_IMAGE_NAME }}@sha256:%s ' *)
- name: Inspect image
run: |
docker buildx imagetools inspect ${{ env.FULL_IMAGE_NAME }}:${{ steps.meta.outputs.version }}
-331
View File
@@ -1,331 +0,0 @@
name: Create and publish Docker images with specific build args
on:
workflow_dispatch:
push:
branches:
- main
- dev
tags:
- v*
concurrency:
group: docker-${{ github.ref }}
cancel-in-progress: true
env:
REGISTRY: ghcr.io
jobs:
build:
runs-on: ${{ matrix.platform.runner }}
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
platform:
- arch: linux/amd64
runner: ubuntu-latest
- arch: linux/arm64
runner: ubuntu-24.04-arm
variant:
- name: main
suffix: ""
build_args: ""
free_disk: false
- name: cuda
suffix: "-cuda"
build_args: "USE_CUDA=true"
free_disk: true
- name: cuda126
suffix: "-cuda126"
build_args: |
USE_CUDA=true
USE_CUDA_VER=cu126
free_disk: true
- name: ollama
suffix: "-ollama"
build_args: "USE_OLLAMA=true"
free_disk: false
- name: slim
suffix: "-slim"
build_args: "USE_SLIM=true"
free_disk: false
steps:
- name: Prepare environment
run: |
echo "IMAGE_NAME=${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
echo "FULL_IMAGE_NAME=${REGISTRY}/${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
platform=${{ matrix.platform.arch }}
echo "PLATFORM_PAIR=${platform//\//-}" >> ${GITHUB_ENV}
- name: Free disk space
if: matrix.variant.free_disk
run: rm -rf /opt/hostedtoolcache
- name: Checkout repository
uses: actions/checkout@v5
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
${{ matrix.variant.suffix != '' && format('type=raw,enable={0},prefix=,suffix=,value={1}', github.ref == 'refs/heads/main', matrix.variant.name) || '' }}
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
${{ matrix.variant.suffix != '' && format('suffix={0},onlatest=true', matrix.variant.suffix) || '' }}
- name: Extract metadata for Docker cache
id: cache-meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
${{ github.ref_type == 'tag' && 'type=raw,value=main' || '' }}
flavor: |
prefix=cache-${{ matrix.variant.name }}-${{ matrix.platform.arch }}-
latest=false
- name: Build Docker image
uses: docker/build-push-action@v5
id: build
with:
context: .
push: true
platforms: ${{ matrix.platform.arch }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.FULL_IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=registry,ref=${{ steps.cache-meta.outputs.tags }}
cache-to: type=registry,ref=${{ steps.cache-meta.outputs.tags }},mode=max
sbom: true
build-args: |
BUILD_HASH=${{ github.sha }}
${{ matrix.variant.build_args }}
- name: Export digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-${{ matrix.variant.name }}-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
merge:
runs-on: ubuntu-latest
needs: [build]
if: ${{ !cancelled() }}
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
variant:
- name: main
suffix: ""
- name: cuda
suffix: "-cuda"
- name: cuda126
suffix: "-cuda126"
- name: ollama
suffix: "-ollama"
- name: slim
suffix: "-slim"
steps:
- name: Prepare environment
run: |
echo "IMAGE_NAME=${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
echo "FULL_IMAGE_NAME=${REGISTRY}/${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
- name: Download digests
id: download
uses: actions/download-artifact@v5
with:
pattern: digests-${{ matrix.variant.name }}-*
path: /tmp/digests
merge-multiple: true
continue-on-error: true
- name: Check digests
id: check
run: |
count=$(find /tmp/digests -type f 2>/dev/null | wc -l | tr -d ' ')
echo "digest_count=$count" >> $GITHUB_OUTPUT
if [ "$count" -lt 2 ]; then
echo "::warning::${{ matrix.variant.name }}: found $count digest(s), need 2 (one per arch). Skipping merge."
echo "skip=true" >> $GITHUB_OUTPUT
else
echo "skip=false" >> $GITHUB_OUTPUT
fi
- name: Set up Docker Buildx
if: steps.check.outputs.skip != 'true'
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
if: steps.check.outputs.skip != 'true'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker images
if: steps.check.outputs.skip != 'true'
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.FULL_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=tag
type=sha,prefix=git-
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
${{ matrix.variant.suffix != '' && format('type=raw,enable={0},prefix=,suffix=,value={1}', github.ref == 'refs/heads/main', matrix.variant.name) || '' }}
flavor: |
latest=${{ github.ref == 'refs/heads/main' }}
${{ matrix.variant.suffix != '' && format('suffix={0},onlatest=true', matrix.variant.suffix) || '' }}
- name: Create manifest list and push
if: steps.check.outputs.skip != 'true'
working-directory: /tmp/digests
run: |
docker buildx imagetools create \
$(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.FULL_IMAGE_NAME }}@sha256:%s ' *)
- name: Inspect image
if: steps.check.outputs.skip != 'true'
run: |
docker buildx imagetools inspect ${{ env.FULL_IMAGE_NAME }}:${{ steps.meta.outputs.version }}
copy-to-dockerhub:
runs-on: ubuntu-latest
if: ${{ !cancelled() && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')) }}
needs: [merge]
continue-on-error: true
strategy:
fail-fast: false
matrix:
include:
- variant: main
suffix: ""
- variant: cuda
suffix: "-cuda"
- variant: cuda126
suffix: "-cuda126"
- variant: ollama
suffix: "-ollama"
- variant: slim
suffix: "-slim"
steps:
- name: Prepare environment
run: |
echo "IMAGE_NAME=${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
echo "FULL_IMAGE_NAME=${REGISTRY}/${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Log in to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Determine source and destination tags
id: tags
run: |
DOCKERHUB_IMAGE="openwebui/open-webui"
SUFFIX="${{ matrix.suffix }}"
if [[ "${{ github.ref }}" == refs/tags/v* ]]; then
VERSION="${{ github.ref_name }}"
VERSION="${VERSION#v}"
MAJOR_MINOR="${VERSION%.*}"
echo "tags<<EOF" >> $GITHUB_OUTPUT
echo "${VERSION}${SUFFIX}" >> $GITHUB_OUTPUT
echo "${MAJOR_MINOR}${SUFFIX}" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
else
if [ -z "$SUFFIX" ]; then
echo "tags=latest" >> $GITHUB_OUTPUT
else
VARIANT_NAME="${SUFFIX#-}"
echo "tags<<EOF" >> $GITHUB_OUTPUT
echo "latest${SUFFIX}" >> $GITHUB_OUTPUT
echo "${VARIANT_NAME}" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
fi
fi
echo "dockerhub_image=${DOCKERHUB_IMAGE}" >> $GITHUB_OUTPUT
- name: Copy images from GHCR to Docker Hub
run: |
DOCKERHUB_IMAGE="${{ steps.tags.outputs.dockerhub_image }}"
SUFFIX="${{ matrix.suffix }}"
if [[ "${{ github.ref }}" == refs/tags/v* ]]; then
VERSION="${{ github.ref_name }}"
VERSION="${VERSION#v}"
SOURCE_TAG="${VERSION}${SUFFIX}"
else
if [ -z "$SUFFIX" ]; then
SOURCE_TAG="latest"
else
SOURCE_TAG="latest${SUFFIX}"
fi
fi
SOURCE="${{ env.FULL_IMAGE_NAME }}:${SOURCE_TAG}"
echo "Copying from ${SOURCE} to Docker Hub..."
while IFS= read -r TAG; do
[ -z "$TAG" ] && continue
DEST="${DOCKERHUB_IMAGE}:${TAG}"
echo " -> ${DEST}"
docker buildx imagetools create -t "${DEST}" "${SOURCE}"
done <<< "${{ steps.tags.outputs.tags }}"
+49
View File
@@ -0,0 +1,49 @@
name: Python CI
on:
push:
branches:
- main
- dev
paths:
- 'backend/**'
- 'pyproject.toml'
- 'uv.lock'
pull_request:
branches:
- main
- dev
paths:
- 'backend/**'
- 'pyproject.toml'
- 'uv.lock'
jobs:
build:
name: 'Format Backend'
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- 3.11.x
- 3.12.x
steps:
- uses: actions/checkout@v5
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '${{ matrix.python-version }}'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install black
- name: Format backend
run: npm run format:backend
- name: Check for changes after format
run: git diff --exit-code
@@ -0,0 +1,65 @@
name: Frontend Build
on:
push:
branches:
- main
- dev
paths-ignore:
- 'backend/**'
- 'pyproject.toml'
- 'uv.lock'
pull_request:
branches:
- main
- dev
paths-ignore:
- 'backend/**'
- 'pyproject.toml'
- 'uv.lock'
jobs:
build:
name: 'Format & Build Frontend'
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v5
- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: '22'
- name: Install Dependencies
run: npm install --force
- name: Format Frontend
run: npm run format
- name: Run i18next
run: npm run i18n:parse
- name: Check for Changes After Format
run: git diff --exit-code
- name: Build Frontend
run: npm run build
test-frontend:
name: 'Frontend Unit Tests'
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v5
- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: '22'
- name: Install Dependencies
run: npm ci --force
- name: Run vitest
run: npm run test:frontend
-63
View File
@@ -1,63 +0,0 @@
# ─────────────────────────────────────────────────────────────────────────────
# Frontend CI — Lint, format check, build, and unit tests
# Runs on pushes and PRs to main/dev, skipping backend-only changes
# ─────────────────────────────────────────────────────────────────────────────
name: Frontend Build
on:
push:
branches: [main, dev]
paths-ignore: ['backend/**', 'pyproject.toml', 'uv.lock']
pull_request:
branches: [main, dev]
paths-ignore: ['backend/**', 'pyproject.toml', 'uv.lock']
concurrency:
group: frontend-${{ github.ref }}
cancel-in-progress: true
jobs:
# ── Format, i18n, and production build ────────────────────────────────────
format-and-build:
name: Format & Build
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version: '22'
- name: Install dependencies
run: npm install --force
- name: Verify code formatting
run: npm run format
- name: Verify i18n strings
run: npm run i18n:parse
- name: Ensure working tree is clean
run: git diff --exit-code
- name: Production build
run: npm run build
# ── Vitest unit tests ────────────────────────────────────────────────────
unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version: '22'
- name: Install dependencies (frozen lockfile)
run: npm ci --force
- name: Execute test suite
run: npm run test:frontend
+255
View File
@@ -0,0 +1,255 @@
name: Integration Test
on:
push:
branches:
- main
- dev
pull_request:
branches:
- main
- dev
jobs:
cypress-run:
name: Run Cypress Integration Tests
runs-on: ubuntu-latest
steps:
- name: Maximize build space
uses: AdityaGarg8/remove-unwanted-software@v4.1
with:
remove-android: 'true'
remove-haskell: 'true'
remove-codeql: 'true'
- name: Checkout Repository
uses: actions/checkout@v4
- name: Build and run Compose Stack
run: |
docker compose \
--file docker-compose.yaml \
--file docker-compose.api.yaml \
--file docker-compose.a1111-test.yaml \
up --detach --build
- name: Delete Docker build cache
run: |
docker builder prune --all --force
- name: Wait for Ollama to be up
timeout-minutes: 5
run: |
until curl --output /dev/null --silent --fail http://localhost:11434; do
printf '.'
sleep 1
done
echo "Service is up!"
- name: Preload Ollama model
run: |
docker exec ollama ollama pull qwen:0.5b-chat-v1.5-q2_K
- name: Cypress run
uses: cypress-io/github-action@v6
env:
LIBGL_ALWAYS_SOFTWARE: 1
with:
browser: chrome
wait-on: 'http://localhost:3000'
config: baseUrl=http://localhost:3000
- uses: actions/upload-artifact@v4
if: always()
name: Upload Cypress videos
with:
name: cypress-videos
path: cypress/videos
if-no-files-found: ignore
- name: Extract Compose logs
if: always()
run: |
docker compose logs > compose-logs.txt
- uses: actions/upload-artifact@v4
if: always()
name: Upload Compose logs
with:
name: compose-logs
path: compose-logs.txt
if-no-files-found: ignore
# pytest:
# name: Run Backend Tests
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v4
# - name: Set up Python
# uses: actions/setup-python@v5
# with:
# python-version: ${{ matrix.python-version }}
# - name: Install dependencies
# run: |
# python -m pip install --upgrade pip
# pip install -r backend/requirements.txt
# - name: pytest run
# run: |
# ls -al
# cd backend
# PYTHONPATH=. pytest . -o log_cli=true -o log_cli_level=INFO
migration_test:
name: Run Migration Tests
runs-on: ubuntu-latest
services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
# mysql:
# image: mysql
# env:
# MYSQL_ROOT_PASSWORD: mysql
# MYSQL_DATABASE: mysql
# options: >-
# --health-cmd "mysqladmin ping -h localhost"
# --health-interval 10s
# --health-timeout 5s
# --health-retries 5
# ports:
# - 3306:3306
steps:
- name: Checkout Repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Set up uv
uses: yezz123/setup-uv@v4
with:
uv-venv: venv
- name: Activate virtualenv
run: |
. venv/bin/activate
echo PATH=$PATH >> $GITHUB_ENV
- name: Install dependencies
run: |
uv pip install -r backend/requirements.txt
- name: Test backend with SQLite
id: sqlite
env:
WEBUI_SECRET_KEY: secret-key
GLOBAL_LOG_LEVEL: debug
run: |
cd backend
uvicorn open_webui.main:app --port "8080" --forwarded-allow-ips '*' &
UVICORN_PID=$!
# Wait up to 40 seconds for the server to start
for i in {1..40}; do
curl -s http://localhost:8080/api/config > /dev/null && break
sleep 1
if [ $i -eq 40 ]; then
echo "Server failed to start"
kill -9 $UVICORN_PID
exit 1
fi
done
# Check that the server is still running after 5 seconds
sleep 5
if ! kill -0 $UVICORN_PID; then
echo "Server has stopped"
exit 1
fi
- name: Test backend with Postgres
if: success() || steps.sqlite.conclusion == 'failure'
env:
WEBUI_SECRET_KEY: secret-key
GLOBAL_LOG_LEVEL: debug
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
DATABASE_POOL_SIZE: 10
DATABASE_POOL_MAX_OVERFLOW: 10
DATABASE_POOL_TIMEOUT: 30
run: |
cd backend
uvicorn open_webui.main:app --port "8081" --forwarded-allow-ips '*' &
UVICORN_PID=$!
# Wait up to 20 seconds for the server to start
for i in {1..20}; do
curl -s http://localhost:8081/api/config > /dev/null && break
sleep 1
if [ $i -eq 20 ]; then
echo "Server failed to start"
kill -9 $UVICORN_PID
exit 1
fi
done
# Check that the server is still running after 5 seconds
sleep 5
if ! kill -0 $UVICORN_PID; then
echo "Server has stopped"
exit 1
fi
# Check that service will reconnect to postgres when connection will be closed
status_code=$(curl --write-out %{http_code} -s --output /dev/null http://localhost:8081/health/db)
if [[ "$status_code" -ne 200 ]] ; then
echo "Server has failed before postgres reconnect check"
exit 1
fi
echo "Terminating all connections to postgres..."
python -c "import os, psycopg2 as pg2; \
conn = pg2.connect(dsn=os.environ['DATABASE_URL'].replace('+pool', '')); \
cur = conn.cursor(); \
cur.execute('SELECT pg_terminate_backend(psa.pid) FROM pg_stat_activity psa WHERE datname = current_database() AND pid <> pg_backend_pid();')"
status_code=$(curl --write-out %{http_code} -s --output /dev/null http://localhost:8081/health/db)
if [[ "$status_code" -ne 200 ]] ; then
echo "Server has not reconnected to postgres after connection was closed: returned status $status_code"
exit 1
fi
# - name: Test backend with MySQL
# if: success() || steps.sqlite.conclusion == 'failure' || steps.postgres.conclusion == 'failure'
# env:
# WEBUI_SECRET_KEY: secret-key
# GLOBAL_LOG_LEVEL: debug
# DATABASE_URL: mysql://root:mysql@localhost:3306/mysql
# run: |
# cd backend
# uvicorn open_webui.main:app --port "8083" --forwarded-allow-ips '*' &
# UVICORN_PID=$!
# # Wait up to 20 seconds for the server to start
# for i in {1..20}; do
# curl -s http://localhost:8083/api/config > /dev/null && break
# sleep 1
# if [ $i -eq 20 ]; then
# echo "Server failed to start"
# kill -9 $UVICORN_PID
# exit 1
# fi
# done
# # Check that the server is still running after 5 seconds
# sleep 5
# if ! kill -0 $UVICORN_PID; then
# echo "Server has stopped"
# exit 1
# fi
-74
View File
@@ -1,74 +0,0 @@
# ─────────────────────────────────────────────────────────────────────────────
# Release — Create GitHub release from CHANGELOG, trigger Docker builds
# Runs on pushes to main when package.json version changes
# ─────────────────────────────────────────────────────────────────────────────
name: Release
on:
push:
branches: [main]
concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false
jobs:
# ── Create release and trigger downstream workflows ──────────────────────
publish:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: write
actions: write
steps:
- uses: actions/checkout@v5
- name: Abort if package.json unchanged
run: |
git diff --cached --diff-filter=d package.json || {
echo "package.json not modified — skipping release"
exit 1
}
- name: Read version
id: pkg
run: echo "version=$(jq -r '.version' package.json)" >> $GITHUB_OUTPUT
- name: Extract release notes from CHANGELOG
run: |
VER="${{ steps.pkg.outputs.version }}"
awk "/^## \[${VER}\]/{found=1; next} /^## \[/{if(found) exit} found{print}" \
CHANGELOG.md > /tmp/release-notes.md
- name: Publish GitHub release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
if gh release view "v${{ steps.pkg.outputs.version }}" &>/dev/null; then
echo "Release v${{ steps.pkg.outputs.version }} already exists — skipping creation"
else
gh release create "v${{ steps.pkg.outputs.version }}" \
--title "v${{ steps.pkg.outputs.version }}" \
--notes-file /tmp/release-notes.md
fi
- name: Archive source
uses: actions/upload-artifact@v4
with:
name: release-archive
path: |
.
!.git
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Trigger Docker build
uses: actions/github-script@v8
with:
script: |
github.rest.actions.createWorkflowDispatch({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: 'docker.yaml',
ref: 'v${{ steps.pkg.outputs.version }}',
})
-1
View File
@@ -310,4 +310,3 @@ dist
cypress/videos
cypress/screenshots
.vscode/settings.json
.cptr
-8
View File
@@ -1,8 +0,0 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.5
hooks:
- id: ruff
args: [--fix, backend]
- id: ruff-format
args: [backend]
+2
View File
@@ -3,6 +3,8 @@ pnpm-lock.yaml
package-lock.json
yarn.lock
kubernetes/
# Copy of .gitignore
.DS_Store
node_modules
+1 -1611
View File
File diff suppressed because it is too large Load Diff
+3 -3
View File
@@ -1,7 +1,7 @@
# Contributor License Agreement
# Open WebUI Contributor License Agreement
By submitting my contributions to this repository in any form, I grant Open WebUI Inc. a perpetual, worldwide, irrevocable, royalty-free license, under copyright and patent, to use, modify, distribute, sublicense, and commercialize my work under any terms they choose, both now and in the future.
By submitting my contributions to Open WebUI, I grant Open WebUI full freedom to use my work in any way they choose, under any terms they like, both now and in the future. This approach helps ensure the project remains unified, flexible, and easy to maintain, while empowering Open WebUI to respond quickly to the needs of its users and the wider community.
I represent that my contributions are my original work (or that I have sufficient rights to grant this license) and that I have the authority to enter into this agreement.
Taking part in this process means my work can be seamlessly integrated and combined with others, ensuring longevity and adaptability for everyone who benefits from the Open WebUI project. This collaborative approach strengthens the projects future and helps guarantee that improvements can always be shared and distributed in the most effective way possible.
**_To the fullest extent permitted by law, my contributions are provided on an “as is” basis, with no warranties or guarantees of any kind, and I disclaim any liability for any issues or damages arising from their use or incorporation into the project, regardless of the type of legal claim._**
+10 -27
View File
@@ -13,7 +13,6 @@ ARG USE_CUDA_VER=cu128
# IMPORTANT: If you change the embedding model (sentence-transformers/all-MiniLM-L6-v2) and vice versa, you aren't able to use RAG Chat with your previous documents loaded in the WebUI! You need to re-embed them.
ARG USE_EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2
ARG USE_RERANKING_MODEL=""
ARG USE_AUXILIARY_EMBEDDING_MODEL=TaylorAI/bge-micro-v2
# Tiktoken encoding name; models to use can be found at https://huggingface.co/models?library=tiktoken
ARG USE_TIKTOKEN_ENCODING_NAME="cl100k_base"
@@ -53,13 +52,9 @@ ARG USE_SLIM
ARG USE_PERMISSION_HARDENING
ARG USE_EMBEDDING_MODEL
ARG USE_RERANKING_MODEL
ARG USE_AUXILIARY_EMBEDDING_MODEL
ARG UID
ARG GID
# Python settings
ENV PYTHONUNBUFFERED=1
## Basis ##
ENV ENV=prod \
PORT=8080 \
@@ -69,8 +64,7 @@ ENV ENV=prod \
USE_SLIM_DOCKER=${USE_SLIM} \
USE_CUDA_DOCKER_VER=${USE_CUDA_VER} \
USE_EMBEDDING_MODEL_DOCKER=${USE_EMBEDDING_MODEL} \
USE_RERANKING_MODEL_DOCKER=${USE_RERANKING_MODEL} \
USE_AUXILIARY_EMBEDDING_MODEL_DOCKER=${USE_AUXILIARY_EMBEDDING_MODEL}
USE_RERANKING_MODEL_DOCKER=${USE_RERANKING_MODEL}
## Basis URL Config ##
ENV OLLAMA_BASE_URL="/ollama" \
@@ -91,7 +85,6 @@ ENV WHISPER_MODEL="base" \
## RAG Embedding model settings ##
ENV RAG_EMBEDDING_MODEL="$USE_EMBEDDING_MODEL_DOCKER" \
RAG_RERANKING_MODEL="$USE_RERANKING_MODEL_DOCKER" \
AUXILIARY_EMBEDDING_MODEL="$USE_AUXILIARY_EMBEDDING_MODEL_DOCKER" \
SENTENCE_TRANSFORMERS_HOME="/app/backend/data/cache/embedding/models"
## Tiktoken model settings ##
@@ -127,41 +120,31 @@ RUN chown -R $UID:$GID /app $HOME
RUN apt-get update && \
apt-get install -y --no-install-recommends \
git build-essential pandoc gcc netcat-openbsd curl jq \
libmariadb-dev \
python3-dev \
ffmpeg libsm6 libxext6 zstd \
ffmpeg libsm6 libxext6 \
&& rm -rf /var/lib/apt/lists/*
# install python dependencies
COPY --chown=$UID:$GID ./backend/requirements.txt ./requirements.txt
# Set UV_LINK_MODE to copy to prevent 0-byte file corruption in QEMU arm64 cross-builds
ENV UV_LINK_MODE=copy
RUN set -e; \
pip3 install --no-cache-dir uv; \
RUN pip3 install --no-cache-dir uv && \
if [ "$USE_CUDA" = "true" ]; then \
# If you use CUDA the whisper and embedding model will be downloaded on first use
# fix: pin torch<=2.9.1 - torch 2.10.0 aarch64 wheels cause SIGILL on ARM devices (RPi 4 Cortex-A72) #21349
pip3 install 'torch<=2.9.1' torchvision torchaudio --index-url https://download.pytorch.org/whl/$USE_CUDA_DOCKER_VER --no-cache-dir; \
uv pip install --system -r requirements.txt --no-cache-dir; \
python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ['RAG_EMBEDDING_MODEL'], device='cpu')"; \
python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ.get('AUXILIARY_EMBEDDING_MODEL', 'TaylorAI/bge-micro-v2'), device='cpu')"; \
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/$USE_CUDA_DOCKER_VER --no-cache-dir && \
uv pip install --system -r requirements.txt --no-cache-dir && \
python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ['RAG_EMBEDDING_MODEL'], device='cpu')" && \
python -c "import os; from faster_whisper import WhisperModel; WhisperModel(os.environ['WHISPER_MODEL'], device='cpu', compute_type='int8', download_root=os.environ['WHISPER_MODEL_DIR'])"; \
python -c "import os; import tiktoken; tiktoken.get_encoding(os.environ['TIKTOKEN_ENCODING_NAME'])"; \
python -c "import nltk; nltk.download('punkt_tab')"; \
else \
pip3 install 'torch<=2.9.1' torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu --no-cache-dir; \
uv pip install --system -r requirements.txt --no-cache-dir; \
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu --no-cache-dir && \
uv pip install --system -r requirements.txt --no-cache-dir && \
if [ "$USE_SLIM" != "true" ]; then \
python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ['RAG_EMBEDDING_MODEL'], device='cpu')"; \
python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ.get('AUXILIARY_EMBEDDING_MODEL', 'TaylorAI/bge-micro-v2'), device='cpu')"; \
python -c "import os; from sentence_transformers import SentenceTransformer; SentenceTransformer(os.environ['RAG_EMBEDDING_MODEL'], device='cpu')" && \
python -c "import os; from faster_whisper import WhisperModel; WhisperModel(os.environ['WHISPER_MODEL'], device='cpu', compute_type='int8', download_root=os.environ['WHISPER_MODEL_DIR'])"; \
python -c "import os; import tiktoken; tiktoken.get_encoding(os.environ['TIKTOKEN_ENCODING_NAME'])"; \
python -c "import nltk; nltk.download('punkt_tab')"; \
fi; \
fi; \
mkdir -p /app/backend/data; chown -R $UID:$GID /app/backend/data/; \
mkdir -p /app/backend/data && chown -R $UID:$GID /app/backend/data/ && \
rm -rf /var/lib/apt/lists/*;
# Install Ollama if requested
+35
View File
@@ -0,0 +1,35 @@
### Installing Both Ollama and Open WebUI Using Kustomize
For cpu-only pod
```bash
kubectl apply -f ./kubernetes/manifest/base
```
For gpu-enabled pod
```bash
kubectl apply -k ./kubernetes/manifest
```
### Installing Both Ollama and Open WebUI Using Helm
Package Helm file first
```bash
helm package ./kubernetes/helm/
```
For cpu-only pod
```bash
helm install ollama-webui ./ollama-webui-*.tgz
```
For gpu-enabled pod
```bash
helm install ollama-webui ./ollama-webui-*.tgz --set ollama.resources.limits.nvidia.com/gpu="1"
```
Check the `kubernetes/helm/values.yaml` file to know which parameters are available for customization
+4 -22
View File
@@ -1,6 +1,4 @@
Open WebUI License
Copyright (c) 2023- Open WebUI Inc. [Created by Timothy Jaeryang Baek]
Copyright (c) 2023-2025 Timothy Jaeryang Baek (Open WebUI)
All rights reserved.
Redistribution and use in source and binary forms, with or without
@@ -17,27 +15,11 @@ modification, are permitted provided that the following conditions are met:
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
4. Notwithstanding any other provision of this License, and as a material
condition of the rights granted herein, licensees are strictly prohibited
from altering, removing, obscuring, or replacing any "Open WebUI"
branding, including but not limited to the name, logo, or any visual,
textual, or symbolic identifiers that distinguish the software and its
interfaces, in any deployment or distribution, except in the following
circumstances: (i) deployments or distributions where the total number
of end users (defined as individual natural persons with direct access
to the application) does not exceed fifty (50) within any rolling
thirty (30) day period; (ii) the licensee has obtained specific prior
written permission from the copyright holder; or (iii) where the
licensee has obtained a duly executed enterprise license expressly
permitting such modification. For all other cases, any removal or
alteration of the "Open WebUI" branding shall constitute a material
breach of license.
4. Notwithstanding any other provision of this License, and as a material condition of the rights granted herein, licensees are strictly prohibited from altering, removing, obscuring, or replacing any "Open WebUI" branding, including but not limited to the name, logo, or any visual, textual, or symbolic identifiers that distinguish the software and its interfaces, in any deployment or distribution, regardless of the number of users, except as explicitly set forth in Clauses 5 and 6 below.
Materials governed by prior licenses retain those original license
terms, as specified in LICENSE_HISTORY.
5. The branding restriction enumerated in Clause 4 shall not apply in the following limited circumstances: (i) deployments or distributions where the total number of end users (defined as individual natural persons with direct access to the application) does not exceed fifty (50) within any rolling thirty (30) day period; (ii) cases in which the licensee is an official contributor to the codebase—with a substantive code change successfully merged into the main branch of the official codebase maintained by the copyright holder—who has obtained specific prior written permission for branding adjustment from the copyright holder; or (iii) where the licensee has obtained a duly executed enterprise license expressly permitting such modification. For all other cases, any removal or alteration of the "Open WebUI" branding shall constitute a material breach of license.
By contributing to this project, you agree to the project's Contributor
License Agreement (CONTRIBUTOR_LICENSE_AGREEMENT).
6. All code, modifications, or derivative works incorporated into this project prior to the incorporation of this branding clause remain licensed under the BSD 3-Clause License, and prior contributors retain all BSD-3 rights therein; if any such contributor requests the removal of their BSD-3-licensed code, the copyright holder will do so, and any replacement code will be licensed under the project's primary license then in effect. By contributing after this clause's adoption, you agree to the project's Contributor License Agreement (CLA) and to these updated terms for all new contributions.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+37 -59
View File
@@ -10,13 +10,11 @@
[![Discord](https://img.shields.io/badge/Discord-Open_WebUI-blue?logo=discord&logoColor=white)](https://discord.gg/5rJgQTnV4s)
[![](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/tjbck)
![Open WebUI Banner](./banner.png)
**Open WebUI is an [extensible](https://docs.openwebui.com/features/extensibility/plugin), feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline.** It supports various LLM runners like **Ollama** and **OpenAI-compatible APIs**, with **built-in inference engine** for RAG, making it a **powerful AI deployment solution**.
**Open WebUI is an [extensible](https://docs.openwebui.com/features/plugin/), feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline.** It supports various LLM runners like **Ollama** and **OpenAI-compatible APIs**, with **built-in inference engine** for RAG, making it a **powerful AI deployment solution**.
Passionate about open-source AI? [Join our team →](https://careers.openwebui.com/)
![Open WebUI Demo](./demo.png)
![Open WebUI Demo](./demo.gif)
> [!TIP]
> **Looking for an [Enterprise Plan](https://docs.openwebui.com/enterprise)?** **[Speak with Our Sales Team Today!](https://docs.openwebui.com/enterprise)**
@@ -27,84 +25,58 @@ For more information, be sure to check out our [Open WebUI Documentation](https:
## Key Features of Open WebUI ⭐
- 🚀 **Effortless Setup**: Install seamlessly via pip, uv, Docker, or Kubernetes (kubectl, kustomize, or helm), with `:ollama` and `:cuda` tagged images available for container deployments.
- 🚀 **Effortless Setup**: Install seamlessly using Docker or Kubernetes (kubectl, kustomize or helm) for a hassle-free experience with support for both `:ollama` and `:cuda` tagged images.
- 🤝 **Broad Model & API Integration**: Connect any OpenAI-compatible API alongside local Ollama models. Point the API URL at **LMStudio, GroqCloud, Mistral, OpenRouter, vLLM, and more** to mix and match providers freely.
- 🤝 **Ollama/OpenAI API Integration**: Effortlessly integrate OpenAI-compatible APIs for versatile conversations alongside Ollama models. Customize the OpenAI API URL to link with **LMStudio, GroqCloud, Mistral, OpenRouter, and more**.
- 🔐 **Granular RBAC & User Groups**: Administrators define detailed roles, groups, and permissions, giving each user exactly the access they need. Secure by default, with tailored experiences per group.
- 🛡️ **Granular Permissions and User Groups**: By allowing administrators to create detailed user roles and permissions, we ensure a secure user environment. This granularity not only enhances security but also allows for customized user experiences, fostering a sense of ownership and responsibility amongst users.
- 🧩 **Plugin Support**: Extend Open WebUI with **Filters**, **Actions**, **Pipes**, **Tools**, and **Skills**. Connect external services through **MCP**, **MCPO**, and **OpenAPI tool servers**. Build custom integrations, rate limits, approval flows, data connections, and more.
- 📱 **Responsive Design**: Enjoy a seamless experience across Desktop PC, Laptop, and Mobile devices.
- 🤖 **Models & Agents**: Wrap any base model with custom instructions, tools, and knowledge to build specialized agents. Supports dynamic variables, per-user/group access control, and community preset imports via [Open WebUI Community](https://openwebui.com/).
- 📱 **Progressive Web App (PWA) for Mobile**: Enjoy a native app-like experience on your mobile device with our PWA, providing offline access on localhost and a seamless user interface.
- 📝 **Notes**: A dedicated workspace for content outside conversations. Draft with a rich editor, use AI to rewrite selected text, and attach notes to any chat for full-context injection.
- ✒️🔢 **Full Markdown and LaTeX Support**: Elevate your LLM experience with comprehensive Markdown and LaTeX capabilities for enriched interaction.
- 📢 **Channels**: Real-time shared spaces where your team and AI models collaborate in one timeline. Tag models to draft or critique, with threads, reactions, pins, and access control.
- 🎤📹 **Hands-Free Voice/Video Call**: Experience seamless communication with integrated hands-free voice and video call features using multiple Speech-to-Text providers (Local Whisper, OpenAI, Deepgram, Azure) and Text-to-Speech engines (Azure, ElevenLabs, OpenAI, Transformers, WebAPI), allowing for dynamic and interactive chat environments.
- 🧠 **Persistent Memory**: The AI remembers facts about you across conversations, carrying context from one chat to the next.
- 🛠️ **Model Builder**: Easily create Ollama models via the Web UI. Create and add custom characters/agents, customize chat elements, and import models effortlessly through [Open WebUI Community](https://openwebui.com/) integration.
- **Live Workflow & Message Flow**: Watch the AI build and work through checklists in real time. Queue messages while the AI is still responding; they send automatically when it's ready.
- 🐍 **Native Python Function Calling Tool**: Enhance your LLMs with built-in code editor support in the tools workspace. Bring Your Own Function (BYOF) by simply adding your pure Python functions, enabling seamless integration with LLMs.
- 📅 **Calendar & AI Scheduling**: Built-in personal and shared calendars with month/week/day views, recurring events, color coding, attendees, and reminders. Models manage your schedule conversationally through native function calling.
- 💾 **Persistent Artifact Storage**: Built-in key-value storage API for artifacts, enabling features like journals, trackers, leaderboards, and collaborative tools with both personal and shared data scopes across sessions.
- ⏱️ **Automations**: Schedule prompts to run on recurring schedules, with runs surfaced on your calendar and each completed run linking back to the chat it produced.
- 📚 **Local RAG Integration**: Dive into the future of chat interactions with groundbreaking Retrieval Augmented Generation (RAG) support using your choice of 9 vector databases and multiple content extraction engines (Tika, Docling, Document Intelligence, Mistral OCR, External loaders). Load documents directly into chat or add files to your document library, effortlessly accessing them using the `#` command before a query.
- 📱 **Responsive Design & PWA**: Seamless experience across desktop, laptop, and mobile, with a Progressive Web App for native app-like feel and offline access on localhost.
- 🔍 **Web Search for RAG**: Perform web searches using 15+ providers including `SearXNG`, `Google PSE`, `Brave Search`, `Kagi`, `Mojeek`, `Tavily`, `Perplexity`, `serpstack`, `serper`, `Serply`, `DuckDuckGo`, `SearchApi`, `SerpApi`, `Bing`, `Jina`, `Exa`, `Sougou`, `Azure AI Search`, and `Ollama Cloud`, injecting results directly into your chat experience.
- ✒️🔢 **Full Markdown and LaTeX Support**: Comprehensive Markdown and LaTeX capabilities for enriched interaction.
- 🌐 **Web Browsing Capability**: Seamlessly integrate websites into your chat experience using the `#` command followed by a URL. This feature allows you to incorporate web content directly into your conversations, enhancing the richness and depth of your interactions.
- 🎤📹 **Hands-Free Voice/Video Call**: Integrated voice and video calls with multiple Speech-to-Text providers (Local Whisper, OpenAI, Deepgram, Azure) and Text-to-Speech engines (Azure, ElevenLabs, OpenAI, Transformers, WebAPI).
- 🎨 **Image Generation & Editing Integration**: Create and edit images using multiple engines including OpenAI's DALL-E, Gemini, ComfyUI (local), and AUTOMATIC1111 (local), with support for both generation and prompt-based editing workflows.
- 💾 **Persistent Artifact Storage**: Built-in key-value storage API for artifacts, enabling journals, trackers, leaderboards, and collaborative tools with personal and shared data scopes.
- ⚙️ **Many Models Conversations**: Effortlessly engage with various models simultaneously, harnessing their unique strengths for optimal responses. Enhance your experience by leveraging a diverse set of models in parallel.
- 📚 **Local RAG Integration**: Retrieval Augmented Generation backed by 9 vector databases and multiple content-extraction engines (Tika, Docling, Document Intelligence, Mistral OCR, PaddleOCR-vl, external loaders). Supports hybrid search (BM25 + vector) with reranking and full-context mode. Load documents into chat or pull them from your library with the `#` command.
- 🔐 **Role-Based Access Control (RBAC)**: Ensure secure access with restricted permissions; only authorized individuals can access your Ollama, and exclusive model creation/pulling rights are reserved for administrators.
- 🔍 **Web Search for RAG**: Search the web through dozens of providers including `SearXNG`, `Google PSE`, `Brave Search`, `Kagi`, `Mojeek`, `Tavily`, `Perplexity`, `Firecrawl`, `serpstack`, `serper`, `Serply`, `DuckDuckGo`, `SearchApi`, `SerpApi`, `Bing`, `Jina`, `Exa`, `Sougou`, `Azure AI Search`, and `Ollama Cloud`, injecting results directly into the conversation.
- 🗄️ **Flexible Database & Storage Options**: Choose from SQLite (with optional encryption), PostgreSQL, or configure cloud storage backends (S3, Google Cloud Storage, Azure Blob Storage) for scalable deployments.
- 🌐 **Web Browsing Capability**: Pull websites into chat with the `#` command followed by a URL, or let the model fetch them on its own when needed.
- 🔍 **Advanced Vector Database Support**: Select from 9 vector database options including ChromaDB, PGVector, Qdrant, Milvus, Elasticsearch, OpenSearch, Pinecone, S3Vector, and Oracle 23ai for optimal RAG performance.
- 🎨 **Image Generation & Editing**: Create and edit images with multiple engines including OpenAI DALL·E, Gemini, ComfyUI (local), and AUTOMATIC1111 (local), supporting both generation and prompt-based editing.
- 🔐 **Enterprise Authentication**: Full support for LDAP/Active Directory integration, SCIM 2.0 automated provisioning, and SSO via trusted headers alongside OAuth providers. Enterprise-grade user and group provisioning through SCIM 2.0 protocol, enabling seamless integration with identity providers like Okta, Azure AD, and Google Workspace for automated user lifecycle management.
- **Multi-Model Conversations**: Engage several models at once, harnessing their individual strengths in parallel for the best possible responses.
- **Cloud-Native Integration**: Native support for Google Drive and OneDrive/SharePoint file picking, enabling seamless document import from enterprise cloud storage.
- 📊 **Usage Analytics & Model Evaluation**: Admin dashboards track message volume, token consumption, and cost across users and models. Evaluate models with a built-in arena, A/B testing, and ELO-based leaderboards.
- 📊 **Production Observability**: Built-in OpenTelemetry support for traces, metrics, and logs, enabling comprehensive monitoring with your existing observability stack.
- 🗄 **Flexible Database & Storage**: Choose SQLite (with optional encryption) or PostgreSQL, and store files locally or on S3, Google Cloud Storage, or Azure Blob Storage.
- **Horizontal Scalability**: Redis-backed session management and WebSocket support for multi-worker and multi-node deployments behind load balancers.
- 🧬 **Advanced Vector Database Support**: Pick from 9 vector databases: ChromaDB, PGVector, Qdrant, Milvus, Elasticsearch, OpenSearch, Pinecone, S3Vector, and Oracle 23ai.
- 🌐🌍 **Multilingual Support**: Experience Open WebUI in your preferred language with our internationalization (i18n) support. Join us in expanding our supported languages! We're actively seeking contributors!
- 🪪 **Enterprise Authentication & Provisioning**: Full LDAP/Active Directory integration, SSO via trusted headers and OAuth providers, and SCIM 2.0 automated provisioning for identity providers like Okta, Azure AD, and Google Workspace.
- 🧩 **Pipelines, Open WebUI Plugin Support**: Seamlessly integrate custom logic and Python libraries into Open WebUI using [Pipelines Plugin Framework](https://github.com/open-webui/pipelines). Launch your Pipelines instance, set the OpenAI URL to the Pipelines URL, and explore endless possibilities. [Examples](https://github.com/open-webui/pipelines/tree/main/examples) include **Function Calling**, User **Rate Limiting** to control access, **Usage Monitoring** with tools like Langfuse, **Live Translation with LibreTranslate** for multilingual support, **Toxic Message Filtering** and much more.
- ☁️ **Cloud-Native File Integration**: Native Google Drive and OneDrive/SharePoint file picking for seamless document import from enterprise cloud storage.
- 🔭 **Production Observability**: Built-in OpenTelemetry support for traces, metrics, and logs, plugging into your existing monitoring stack.
- ⚖️ **Horizontal Scalability**: Redis-backed session management and WebSocket support for multi-worker, multi-node deployments behind load balancers.
- 🌐🌍 **Multilingual Support**: Use Open WebUI in your preferred language with i18n support. We're actively seeking contributors to expand language coverage!
- 🌟 **Continuous Updates**: We're committed to improving Open WebUI with regular updates, fixes, and new features.
- 🛡️ **Transparent Security Process**: Security reports are triaged, fixed, and published as open advisories through a documented responsible-disclosure process. See our [Security Policy](https://github.com/open-webui/open-webui/security).
- 🌟 **Continuous Updates**: We are committed to improving Open WebUI with regular updates, fixes, and new features.
Want to learn more about Open WebUI's features? Check out our [Open WebUI documentation](https://docs.openwebui.com/features) for a comprehensive overview!
## The Open WebUI Ecosystem 🌐
Open WebUI is the core, surrounded by companion apps and infrastructure that extend what your AI can do, where it can reach, and how you run it:
-**Open Terminal** ([open-webui/open-terminal](https://github.com/open-webui/open-terminal)): A self-hosted computing environment that plugs into Open WebUI, giving the AI a place to write code, run it, read output, fix errors, and iterate inside the chat.
- 🔒 **Terminals** · Enterprise ([open-webui/terminals](https://github.com/open-webui/terminals)): Per-user isolated containers with separate credentials, resource limits, and network rules. Automatic lifecycle management on Docker or Kubernetes.
- 💻 **cptr** ([open-webui/computer](https://github.com/open-webui/computer)): A standalone, mobile-first computer and coding agent that runs on the machine you own. Files, terminal, and git in a browser tab, reachable from your phone. Connect it into Open WebUI as a model, or reach it from Telegram, WhatsApp, and more.
- 🔄 **oikb** ([open-webui/oikb](https://github.com/open-webui/oikb)): Feed your Knowledge Bases from 45+ sources (GitHub, Confluence, ServiceNow, Salesforce, Jira, Slack, SharePoint, Notion, and more), keeping the tools your team already uses continuously in sync.
- 🖥️ **Native Desktop App** ([open-webui/desktop](https://github.com/open-webui/desktop)): Run Open WebUI as a native app on macOS, Windows, and Linux. System-wide Spotlight chat bar with screenshot capture, push-to-talk voice, and optional fully-local inference via a built-in llama.cpp engine.
Want to learn more? Check out our [Open WebUI documentation](https://docs.openwebui.com) for more details!
---
We are incredibly grateful for the generous support of our sponsors. Their contributions help us to maintain and improve our project, ensuring we can continue to deliver quality work to our community. Thank you!
@@ -198,6 +170,8 @@ After installation, you can access Open WebUI at [http://localhost:3000](http://
We offer various installation alternatives, including non-Docker native installation methods, Docker Compose, Kustomize, and Helm. Visit our [Open WebUI Documentation](https://docs.openwebui.com/getting-started/) or join our [Discord community](https://discord.gg/5rJgQTnV4s) for comprehensive guidance.
Look at the [Local Development Guide](https://docs.openwebui.com/getting-started/advanced-topics/development) for instructions on setting up a local development environment.
### Troubleshooting
Encountering connection issues? Our [Open WebUI Documentation](https://docs.openwebui.com/troubleshooting/) has got you covered. For further assistance and to join our vibrant community, visit the [Open WebUI Discord](https://discord.gg/5rJgQTnV4s).
@@ -214,6 +188,14 @@ docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=
### Keeping Your Docker Installation Up-to-Date
In case you want to update your local Docker installation to the latest version, you can do it with [Watchtower](https://containrrr.dev/watchtower/):
```bash
docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui
```
In the last part of the command, replace `open-webui` with your container name if it is different.
Check our Updating Guide available in our [Open WebUI Documentation](https://docs.openwebui.com/getting-started/updating).
### Using the Dev Branch 🌙
@@ -248,10 +230,6 @@ This project contains code under multiple licenses. The current codebase include
If you have any questions, suggestions, or need assistance, please open an issue or join our
[Open WebUI Discord community](https://discord.gg/5rJgQTnV4s) to connect with us! 🤝
## Security 🛡️
If you believe you've found a security vulnerability, or something that shouldn't be disclosed publicly, please [reach out confidentially through our responsible disclosure program on GitHub](https://github.com/open-webui/open-webui/security). We accept reports only through GitHub, not through any other platform. Thank you for helping us keep Open WebUI secure!
## Star History
<a href="https://star-history.com/#open-webui/open-webui&Date">
+1 -1
View File
@@ -1,3 +1,3 @@
export CORS_ALLOW_ORIGIN="http://localhost:5173;http://localhost:8080"
PORT="${PORT:-8080}"
uvicorn open_webui.main:app --port $PORT --host 0.0.0.0 --forwarded-allow-ips "${FORWARDED_ALLOW_IPS:-*}" --reload
uvicorn open_webui.main:app --port $PORT --host 0.0.0.0 --forwarded-allow-ips '*' --reload
+40 -44
View File
@@ -1,107 +1,103 @@
import base64
import os
import random
import sys
from pathlib import Path
from typing import Annotated
import typer
import uvicorn
from typing import Optional
from typing_extensions import Annotated
app = typer.Typer()
KEY_FILE = Path.cwd() / '.webui_secret_key'
DEFAULT_SECRET_KEY_LENGTH = 24
KEY_FILE = Path.cwd() / ".webui_secret_key"
def version_callback(value: bool) -> None:
def version_callback(value: bool):
if value:
from open_webui.env import VERSION
typer.echo(f'Open WebUI version: {VERSION}')
typer.echo(f"Open WebUI version: {VERSION}")
raise typer.Exit()
@app.command()
def main(
version: Annotated[bool | None, typer.Option('--version', callback=version_callback)] = None,
version: Annotated[
Optional[bool], typer.Option("--version", callback=version_callback)
] = None,
):
pass
@app.command()
def serve(
host: str = '0.0.0.0',
host: str = "0.0.0.0",
port: int = 8080,
):
os.environ['FROM_INIT_PY'] = 'true'
if os.getenv('WEBUI_SECRET_KEY') is None:
typer.echo('Loading WEBUI_SECRET_KEY from file, not provided as an environment variable.')
os.environ["FROM_INIT_PY"] = "true"
if os.getenv("WEBUI_SECRET_KEY") is None:
typer.echo(
"Loading WEBUI_SECRET_KEY from file, not provided as an environment variable."
)
if not KEY_FILE.exists():
key_length = int(os.getenv('WEBUI_SECRET_KEY_LENGTH', DEFAULT_SECRET_KEY_LENGTH))
if key_length < 1:
raise ValueError('WEBUI_SECRET_KEY_LENGTH must be a positive integer')
typer.echo(f'Generating a new secret key and saving it to {KEY_FILE}')
KEY_FILE.write_bytes(base64.b64encode(random.randbytes(key_length)))
typer.echo(f'Loading WEBUI_SECRET_KEY from {KEY_FILE}')
os.environ['WEBUI_SECRET_KEY'] = KEY_FILE.read_text()
typer.echo(f"Generating a new secret key and saving it to {KEY_FILE}")
KEY_FILE.write_bytes(base64.b64encode(random.randbytes(12)))
typer.echo(f"Loading WEBUI_SECRET_KEY from {KEY_FILE}")
os.environ["WEBUI_SECRET_KEY"] = KEY_FILE.read_text()
if os.getenv('USE_CUDA_DOCKER', 'false') == 'true':
typer.echo('CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries.')
LD_LIBRARY_PATH = os.getenv('LD_LIBRARY_PATH', '').split(':')
os.environ['LD_LIBRARY_PATH'] = ':'.join(
if os.getenv("USE_CUDA_DOCKER", "false") == "true":
typer.echo(
"CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries."
)
LD_LIBRARY_PATH = os.getenv("LD_LIBRARY_PATH", "").split(":")
os.environ["LD_LIBRARY_PATH"] = ":".join(
LD_LIBRARY_PATH
+ [
'/usr/local/lib/python3.11/site-packages/torch/lib',
'/usr/local/lib/python3.11/site-packages/nvidia/cudnn/lib',
"/usr/local/lib/python3.11/site-packages/torch/lib",
"/usr/local/lib/python3.11/site-packages/nvidia/cudnn/lib",
]
)
try:
import torch
assert torch.cuda.is_available(), 'CUDA not available'
typer.echo('CUDA seems to be working')
assert torch.cuda.is_available(), "CUDA not available"
typer.echo("CUDA seems to be working")
except Exception as e:
typer.echo(
'Error when testing CUDA but USE_CUDA_DOCKER is true. '
'Resetting USE_CUDA_DOCKER to false and removing '
f'LD_LIBRARY_PATH modifications: {e}'
"Error when testing CUDA but USE_CUDA_DOCKER is true. "
"Resetting USE_CUDA_DOCKER to false and removing "
f"LD_LIBRARY_PATH modifications: {e}"
)
os.environ['USE_CUDA_DOCKER'] = 'false'
os.environ['LD_LIBRARY_PATH'] = ':'.join(LD_LIBRARY_PATH)
os.environ["USE_CUDA_DOCKER"] = "false"
os.environ["LD_LIBRARY_PATH"] = ":".join(LD_LIBRARY_PATH)
import open_webui.main # noqa: F401
import open_webui.main # we need set environment variables before importing main
from open_webui.env import UVICORN_WORKERS # Import the workers setting
# On Windows, uvicorn's default loop factory hardcodes ProactorEventLoop,
# which is incompatible with psycopg v3 async. Setting loop='none' lets
# asyncio.run() respect the WindowsSelectorEventLoopPolicy set in db.py.
loop = 'none' if sys.platform == 'win32' else 'auto'
uvicorn.run(
'open_webui.main:app',
"open_webui.main:app",
host=host,
port=port,
forwarded_allow_ips='*',
forwarded_allow_ips="*",
workers=UVICORN_WORKERS,
loop=loop,
)
@app.command()
def dev(
host: str = '0.0.0.0',
host: str = "0.0.0.0",
port: int = 8080,
reload: bool = True,
):
uvicorn.run(
'open_webui.main:app',
"open_webui.main:app",
host=host,
port=port,
reload=reload,
forwarded_allow_ips='*',
forwarded_allow_ips="*",
)
if __name__ == '__main__':
if __name__ == "__main__":
app()
+3255 -2307
View File
File diff suppressed because it is too large Load Diff
+80 -75
View File
@@ -1,121 +1,126 @@
from __future__ import annotations
from enum import Enum
class MESSAGES(str, Enum):
DEFAULT = lambda msg='': f'{msg if msg else ""}'
MODEL_ADDED = lambda model='': f"The model '{model}' has been added successfully."
MODEL_DELETED = lambda model='': f"The model '{model}' has been deleted successfully."
DEFAULT = lambda msg="": f"{msg if msg else ''}"
MODEL_ADDED = lambda model="": f"The model '{model}' has been added successfully."
MODEL_DELETED = (
lambda model="": f"The model '{model}' has been deleted successfully."
)
class WEBHOOK_MESSAGES(str, Enum):
DEFAULT = lambda msg='': f'{msg if msg else ""}'
USER_SIGNUP = lambda username='': f'New user signed up: {username}' if username else 'New user signed up'
DEFAULT = lambda msg="": f"{msg if msg else ''}"
USER_SIGNUP = lambda username="": (
f"New user signed up: {username}" if username else "New user signed up"
)
class ERROR_MESSAGES(str, Enum):
def __str__(self) -> str:
return super().__str__()
DEFAULT = lambda err='': f'{"Something went wrong :/" if err == "" else "[ERROR: " + str(err) + "]"}'
ENV_VAR_NOT_FOUND = 'Required environment variable not found. Terminating now.'
CREATE_USER_ERROR = 'Oops! Something went wrong while creating your account. Please try again later. If the issue persists, contact support for assistance.'
DELETE_USER_ERROR = 'Oops! Something went wrong. We encountered an issue while trying to delete the user. Please give it another shot.'
EMAIL_MISMATCH = 'Uh-oh! This email does not match the email your provider is registered with. Please check your email and try again.'
EMAIL_TAKEN = 'Uh-oh! This email is already registered. Sign in with your existing account or choose another email to start anew.'
USERNAME_TAKEN = 'Uh-oh! This username is already registered. Please choose another username.'
PASSWORD_TOO_LONG = (
'Uh-oh! The password you entered is too long. Please make sure your password is less than 72 bytes long.'
DEFAULT = (
lambda err="": f'{"Something went wrong :/" if err == "" else "[ERROR: " + str(err) + "]"}'
)
COMMAND_TAKEN = 'Uh-oh! This command is already registered. Please choose another command string.'
FILE_EXISTS = 'Uh-oh! This file is already registered. Please choose another file.'
ENV_VAR_NOT_FOUND = "Required environment variable not found. Terminating now."
CREATE_USER_ERROR = "Oops! Something went wrong while creating your account. Please try again later. If the issue persists, contact support for assistance."
DELETE_USER_ERROR = "Oops! Something went wrong. We encountered an issue while trying to delete the user. Please give it another shot."
EMAIL_MISMATCH = "Uh-oh! This email does not match the email your provider is registered with. Please check your email and try again."
EMAIL_TAKEN = "Uh-oh! This email is already registered. Sign in with your existing account or choose another email to start anew."
USERNAME_TAKEN = (
"Uh-oh! This username is already registered. Please choose another username."
)
PASSWORD_TOO_LONG = "Uh-oh! The password you entered is too long. Please make sure your password is less than 72 bytes long."
COMMAND_TAKEN = "Uh-oh! This command is already registered. Please choose another command string."
FILE_EXISTS = "Uh-oh! This file is already registered. Please choose another file."
ID_TAKEN = 'Uh-oh! This id is already registered. Please choose another id string.'
MODEL_ID_TAKEN = 'Uh-oh! This model id is already registered. Please choose another model id string.'
NAME_TAG_TAKEN = 'Uh-oh! This name tag is already registered. Please choose another name tag string.'
MODEL_ID_TOO_LONG = 'The model id is too long. Please make sure your model id is less than 256 characters long.'
ID_TAKEN = "Uh-oh! This id is already registered. Please choose another id string."
MODEL_ID_TAKEN = "Uh-oh! This model id is already registered. Please choose another model id string."
NAME_TAG_TAKEN = "Uh-oh! This name tag is already registered. Please choose another name tag string."
MODEL_ID_TOO_LONG = "The model id is too long. Please make sure your model id is less than 256 characters long."
INVALID_TOKEN = 'Your session has expired or the token is invalid. Please sign in again.'
INVALID_CRED = 'The email or password provided is incorrect. Please check for typos and try logging in again.'
INVALID_TOKEN = (
"Your session has expired or the token is invalid. Please sign in again."
)
INVALID_CRED = "The email or password provided is incorrect. Please check for typos and try logging in again."
INVALID_EMAIL_FORMAT = "The email format you entered is invalid. Please double-check and make sure you're using a valid email address (e.g., yourname@example.com)."
INCORRECT_PASSWORD = 'The password provided is incorrect. Please check for typos and try again.'
INVALID_TRUSTED_HEADER = (
'Your provider has not provided a trusted header. Please contact your administrator for assistance.'
INCORRECT_PASSWORD = (
"The password provided is incorrect. Please check for typos and try again."
)
INVALID_TRUSTED_HEADER = "Your provider has not provided a trusted header. Please contact your administrator for assistance."
EXISTING_USERS = "You can't turn off authentication because there are existing users. If you want to disable WEBUI_AUTH, make sure your web interface doesn't have any existing users and is a fresh installation."
UNAUTHORIZED = '401 Unauthorized'
ACCESS_PROHIBITED = (
'You do not have permission to access this resource. Please contact your administrator for assistance.'
UNAUTHORIZED = "401 Unauthorized"
ACCESS_PROHIBITED = "You do not have permission to access this resource. Please contact your administrator for assistance."
ACTION_PROHIBITED = (
"The requested action has been restricted as a security measure."
)
ACTION_PROHIBITED = 'The requested action has been restricted as a security measure.'
FILE_NOT_SENT = 'FILE_NOT_SENT'
FILE_NOT_SENT = "FILE_NOT_SENT"
FILE_NOT_SUPPORTED = "Oops! It seems like the file format you're trying to upload is not supported. Please upload a file with a supported format and try again."
NOT_FOUND = "We could not find what you're looking for :/"
USER_NOT_FOUND = "We could not find what you're looking for :/"
API_KEY_NOT_FOUND = "Oops! It looks like there's a hiccup. The API key is missing. Please make sure to provide a valid API key to access this feature."
API_KEY_NOT_ALLOWED = 'Use of API key is not enabled in the environment.'
API_KEY_NOT_ALLOWED = "Use of API key is not enabled in the environment."
MALICIOUS = 'Unusual activities detected, please try again in a few minutes.'
MALICIOUS = "Unusual activities detected, please try again in a few minutes."
PANDOC_NOT_INSTALLED = 'Pandoc is not installed on the server. Please contact your administrator for assistance.'
INCORRECT_FORMAT = lambda err='': f'Invalid format. Please use the correct format{err}'
RATE_LIMIT_EXCEEDED = 'API rate limit exceeded'
PANDOC_NOT_INSTALLED = "Pandoc is not installed on the server. Please contact your administrator for assistance."
INCORRECT_FORMAT = (
lambda err="": f"Invalid format. Please use the correct format{err}"
)
RATE_LIMIT_EXCEEDED = "API rate limit exceeded"
MODEL_NOT_FOUND = lambda name='': f"Model '{name}' was not found"
OPENAI_NOT_FOUND = lambda name='': 'OpenAI API was not found'
OLLAMA_NOT_FOUND = 'WebUI could not connect to Ollama'
CREATE_API_KEY_ERROR = 'Oops! Something went wrong while creating your API key. Please try again later. If the issue persists, contact support for assistance.'
API_KEY_CREATION_NOT_ALLOWED = 'API key creation is not allowed in the environment.'
MODEL_NOT_FOUND = lambda name="": f"Model '{name}' was not found"
OPENAI_NOT_FOUND = lambda name="": "OpenAI API was not found"
OLLAMA_NOT_FOUND = "WebUI could not connect to Ollama"
CREATE_API_KEY_ERROR = "Oops! Something went wrong while creating your API key. Please try again later. If the issue persists, contact support for assistance."
API_KEY_CREATION_NOT_ALLOWED = "API key creation is not allowed in the environment."
EMPTY_CONTENT = 'The content provided is empty. Please ensure that there is text or data present before proceeding.'
EMPTY_CONTENT = "The content provided is empty. Please ensure that there is text or data present before proceeding."
DB_NOT_SQLITE = 'This feature is only available with SQLite databases.'
DB_NOT_SQLITE = "This feature is only available when running with SQLite databases."
INVALID_URL = 'The URL you provided is invalid. Please double-check and try again.'
WEB_SEARCH_ERROR = lambda err='': err if err else 'Something went wrong while searching the web.'
OLLAMA_API_DISABLED = 'The Ollama API is disabled. Please enable it to use this feature.'
FILE_TOO_LARGE = lambda size='': (
f"Oops! The file you're trying to upload is too large. Please upload a file that is less than {size}."
INVALID_URL = (
"Oops! The URL you provided is invalid. Please double-check and try again."
)
DUPLICATE_CONTENT = 'Duplicate content detected. Please provide unique content to proceed.'
FILE_NOT_PROCESSED = (
'Extracted content is not available for this file. Please ensure that the file is processed before proceeding.'
WEB_SEARCH_ERROR = (
lambda err="": f"{err if err else 'Oops! Something went wrong while searching the web.'}"
)
INVALID_PASSWORD = lambda err='': err if err else 'The password does not meet the required validation criteria.'
OLLAMA_API_DISABLED = (
"The Ollama API is disabled. Please enable it to use this feature."
)
AUTOMATION_LIMIT_EXCEEDED = lambda size='': f'Automation limit reached ({size})'
AUTOMATION_TOO_FREQUENT = lambda interval='': f'Schedule too frequent. Minimum interval is {interval} seconds.'
AUTOMATION_INVALID_RRULE = lambda err='': f'Invalid RRULE: {err}'
AUTOMATION_NO_FUTURE_RUNS = 'RRULE has no future occurrences'
FILE_TOO_LARGE = (
lambda size="": f"Oops! The file you're trying to upload is too large. Please upload a file that is less than {size}."
)
FEATURE_DISABLED = lambda name='': f'{name} is disabled'
INPUT_TOO_LONG = lambda size='': f'Input prompt exceeds maximum length of {size}'
SERVER_CONNECTION_ERROR = 'Open WebUI: Server Connection Error'
REQUIRED_FIELD_EMPTY = lambda name='': f'Required field {name} is empty'
OAUTH_NOT_CONFIGURED = lambda name='': f"Provider '{name}' is not configured"
DUPLICATE_CONTENT = (
"Duplicate content detected. Please provide unique content to proceed."
)
FILE_NOT_PROCESSED = "Extracted content is not available for this file. Please ensure that the file is processed before proceeding."
INVALID_PASSWORD = lambda err="": (
err if err else "The password does not meet the required validation criteria."
)
class TASKS(str, Enum):
def __str__(self) -> str:
return super().__str__()
DEFAULT = lambda task='': f'{task if task else "generation"}'
TITLE_GENERATION = 'title_generation'
FOLLOW_UP_GENERATION = 'follow_up_generation'
TAGS_GENERATION = 'tags_generation'
EMOJI_GENERATION = 'emoji_generation'
QUERY_GENERATION = 'query_generation'
IMAGE_PROMPT_GENERATION = 'image_prompt_generation'
AUTOCOMPLETE_GENERATION = 'autocomplete_generation'
FUNCTION_CALLING = 'function_calling'
MOA_RESPONSE_GENERATION = 'moa_response_generation'
DEFAULT = lambda task="": f"{task if task else 'generation'}"
TITLE_GENERATION = "title_generation"
FOLLOW_UP_GENERATION = "follow_up_generation"
TAGS_GENERATION = "tags_generation"
EMOJI_GENERATION = "emoji_generation"
QUERY_GENERATION = "query_generation"
IMAGE_PROMPT_GENERATION = "image_prompt_generation"
AUTOCOMPLETE_GENERATION = "autocomplete_generation"
FUNCTION_CALLING = "function_calling"
MOA_RESPONSE_GENERATION = "moa_response_generation"
+536 -722
View File
File diff suppressed because it is too large Load Diff
+139 -127
View File
@@ -1,10 +1,11 @@
import asyncio
import inspect
import json
import logging
import sys
from typing import AsyncGenerator, Generator, Iterator
import inspect
import json
import asyncio
from pydantic import BaseModel
from typing import AsyncGenerator, Generator, Iterator
from fastapi import (
Depends,
FastAPI,
@@ -15,52 +16,61 @@ from fastapi import (
UploadFile,
status,
)
from pydantic import BaseModel
from starlette.responses import Response, StreamingResponse
from open_webui.config import BYPASS_ADMIN_ACCESS_CONTROL
from open_webui.constants import ERROR_MESSAGES
from open_webui.env import BYPASS_MODEL_ACCESS_CONTROL, GLOBAL_LOG_LEVEL
from open_webui.models.functions import Functions
from open_webui.models.models import Models
from open_webui.models.users import UserModel
from open_webui.socket.main import (
get_event_call,
get_event_emitter,
)
from open_webui.utils.access_control import check_model_access
from open_webui.models.users import UserModel
from open_webui.models.functions import Functions
from open_webui.models.models import Models
from open_webui.utils.plugin import (
load_function_module_by_id,
get_function_module_from_cache,
)
from open_webui.utils.tools import get_tools
from open_webui.utils.access_control import has_access
from open_webui.env import SRC_LOG_LEVELS, GLOBAL_LOG_LEVEL
from open_webui.utils.misc import (
add_or_update_system_message,
get_last_user_message,
prepend_to_first_user_message_content,
openai_chat_chunk_message_template,
openai_chat_completion_message_template,
prepend_to_first_user_message_content,
)
from open_webui.utils.payload import (
apply_model_params_to_body_openai,
apply_system_prompt_to_body,
)
from open_webui.utils.plugin import (
get_function_module_from_cache,
load_function_module_by_id,
)
logging.basicConfig(stream=sys.stdout, level=GLOBAL_LOG_LEVEL)
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MAIN"])
async def get_function_module_by_id(request: Request, pipe_id: str):
function_module, _, _ = await get_function_module_from_cache(request, pipe_id)
def get_function_module_by_id(request: Request, pipe_id: str):
function_module, _, _ = get_function_module_from_cache(request, pipe_id)
if hasattr(function_module, 'valves') and hasattr(function_module, 'Valves'):
if hasattr(function_module, "valves") and hasattr(function_module, "Valves"):
Valves = function_module.Valves
valves = await Functions.get_function_valves_by_id(pipe_id)
valves = Functions.get_function_valves_by_id(pipe_id)
if valves:
try:
function_module.valves = Valves(**{k: v for k, v in valves.items() if v is not None})
function_module.valves = Valves(
**{k: v for k, v in valves.items() if v is not None}
)
except Exception as e:
log.exception(f'Error loading valves for function {pipe_id}: {e}')
log.exception(f"Error loading valves for function {pipe_id}: {e}")
raise e
else:
function_module.valves = Valves()
@@ -69,19 +79,19 @@ async def get_function_module_by_id(request: Request, pipe_id: str):
async def get_function_models(request):
pipes = await Functions.get_functions_by_type('pipe', active_only=True)
pipes = Functions.get_functions_by_type("pipe", active_only=True)
pipe_models = []
for pipe in pipes:
try:
function_module = await get_function_module_by_id(request, pipe.id)
function_module = get_function_module_by_id(request, pipe.id)
has_user_valves = False
if hasattr(function_module, 'UserValves'):
if hasattr(function_module, "UserValves"):
has_user_valves = True
# Check if function is a manifold
if hasattr(function_module, 'pipes'):
if hasattr(function_module, "pipes"):
sub_pipes = []
# Handle pipes being a list, sync function, or async function
@@ -97,30 +107,32 @@ async def get_function_models(request):
log.exception(e)
sub_pipes = []
log.debug(f"get_function_models: function '{pipe.id}' is a manifold of {sub_pipes}")
log.debug(
f"get_function_models: function '{pipe.id}' is a manifold of {sub_pipes}"
)
for p in sub_pipes:
sub_pipe_id = f'{pipe.id}.{p["id"]}'
sub_pipe_name = p['name']
sub_pipe_name = p["name"]
if hasattr(function_module, 'name'):
sub_pipe_name = f'{function_module.name}{sub_pipe_name}'
if hasattr(function_module, "name"):
sub_pipe_name = f"{function_module.name}{sub_pipe_name}"
pipe_flag = {'type': pipe.type}
pipe_flag = {"type": pipe.type}
pipe_models.append(
{
'id': sub_pipe_id,
'name': sub_pipe_name,
'object': 'model',
'created': pipe.created_at,
'owned_by': 'openai',
'pipe': pipe_flag,
'has_user_valves': has_user_valves,
"id": sub_pipe_id,
"name": sub_pipe_name,
"object": "model",
"created": pipe.created_at,
"owned_by": "openai",
"pipe": pipe_flag,
"has_user_valves": has_user_valves,
}
)
else:
pipe_flag = {'type': 'pipe'}
pipe_flag = {"type": "pipe"}
log.debug(
f"get_function_models: function '{pipe.id}' is a single pipe {{ 'id': {pipe.id}, 'name': {pipe.name} }}"
@@ -128,13 +140,13 @@ async def get_function_models(request):
pipe_models.append(
{
'id': pipe.id,
'name': pipe.name,
'object': 'model',
'created': pipe.created_at,
'owned_by': 'openai',
'pipe': pipe_flag,
'has_user_valves': has_user_valves,
"id": pipe.id,
"name": pipe.name,
"object": "model",
"created": pipe.created_at,
"owned_by": "openai",
"pipe": pipe_flag,
"has_user_valves": has_user_valves,
}
)
except Exception as e:
@@ -144,7 +156,9 @@ async def get_function_models(request):
return pipe_models
async def generate_function_chat_completion(request, form_data, user, models: dict = {}):
async def generate_function_chat_completion(
request, form_data, user, models: dict = {}
):
async def execute_pipe(pipe, params):
if inspect.iscoroutinefunction(pipe):
return await pipe(**params)
@@ -155,35 +169,35 @@ async def generate_function_chat_completion(request, form_data, user, models: di
if isinstance(res, str):
return res
if isinstance(res, Generator):
return ''.join(map(str, res))
return "".join(map(str, res))
if isinstance(res, AsyncGenerator):
return ''.join([str(stream) async for stream in res])
return "".join([str(stream) async for stream in res])
def process_line(form_data: dict, line):
if isinstance(line, BaseModel):
line = line.model_dump_json()
line = f'data: {line}'
line = f"data: {line}"
if isinstance(line, dict):
line = f'data: {json.dumps(line)}'
line = f"data: {json.dumps(line)}"
try:
line = line.decode('utf-8')
line = line.decode("utf-8")
except Exception:
pass
if line.startswith('data:'):
return f'{line}\n\n'
if line.startswith("data:"):
return f"{line}\n\n"
else:
line = openai_chat_chunk_message_template(form_data['model'], line)
return f'data: {json.dumps(line)}\n\n'
line = openai_chat_chunk_message_template(form_data["model"], line)
return f"data: {json.dumps(line)}\n\n"
def get_pipe_id(form_data: dict) -> str:
pipe_id = form_data['model']
if '.' in pipe_id:
pipe_id, _ = pipe_id.split('.', 1)
pipe_id = form_data["model"]
if "." in pipe_id:
pipe_id, _ = pipe_id.split(".", 1)
return pipe_id
async def get_function_params(function_module, form_data, user, extra_params=None):
def get_function_params(function_module, form_data, user, extra_params=None):
if extra_params is None:
extra_params = {}
@@ -191,25 +205,27 @@ async def generate_function_chat_completion(request, form_data, user, models: di
# Get the signature of the function
sig = inspect.signature(function_module.pipe)
params = {'body': form_data} | {k: v for k, v in extra_params.items() if k in sig.parameters}
params = {"body": form_data} | {
k: v for k, v in extra_params.items() if k in sig.parameters
}
if '__user__' in params and hasattr(function_module, 'UserValves'):
user_valves = await Functions.get_user_valves_by_id_and_user_id(pipe_id, user.id)
if "__user__" in params and hasattr(function_module, "UserValves"):
user_valves = Functions.get_user_valves_by_id_and_user_id(pipe_id, user.id)
try:
params['__user__']['valves'] = function_module.UserValves(**user_valves)
params["__user__"]["valves"] = function_module.UserValves(**user_valves)
except Exception as e:
log.exception(e)
params['__user__']['valves'] = function_module.UserValves()
params["__user__"]["valves"] = function_module.UserValves()
return params
model_id = form_data.get('model')
model_info = await Models.get_model_by_id(model_id)
model_id = form_data.get("model")
model_info = Models.get_model_by_id(model_id)
metadata = form_data.pop('metadata', {})
metadata = form_data.pop("metadata", {})
files = metadata.get('files', [])
tool_ids = metadata.get('tool_ids', [])
files = metadata.get("files", [])
tool_ids = metadata.get("tool_ids", [])
# Check if tool_ids is None
if tool_ids is None:
tool_ids = []
@@ -220,73 +236,66 @@ async def generate_function_chat_completion(request, form_data, user, models: di
__task_body__ = None
if metadata:
if all(k in metadata for k in ('session_id', 'chat_id', 'message_id')):
__event_emitter__ = await get_event_emitter(metadata)
__event_call__ = await get_event_call(metadata)
__task__ = metadata.get('task', None)
__task_body__ = metadata.get('task_body', None)
if all(k in metadata for k in ("session_id", "chat_id", "message_id")):
__event_emitter__ = get_event_emitter(metadata)
__event_call__ = get_event_call(metadata)
__task__ = metadata.get("task", None)
__task_body__ = metadata.get("task_body", None)
oauth_token = None
try:
oauth_session_id = request.cookies.get('oauth_session_id', None)
if oauth_session_id:
if request.cookies.get("oauth_session_id", None):
oauth_token = await request.app.state.oauth_manager.get_oauth_token(
user.id,
oauth_session_id,
request.cookies.get("oauth_session_id", None),
)
# Fallback: no cookie (automation, API key, etc.) — use most recent session
if oauth_token is None:
from open_webui.models.oauth_sessions import OAuthSessions
sessions = await OAuthSessions.get_sessions_by_user_id(user.id)
if sessions:
best = max(sessions, key=lambda s: s.updated_at)
oauth_token = await request.app.state.oauth_manager.get_oauth_token(
user.id,
best.id,
)
except Exception as e:
log.error(f'Error getting OAuth token: {e}')
log.error(f"Error getting OAuth token: {e}")
extra_params = {
'__event_emitter__': __event_emitter__,
'__event_call__': __event_call__,
'__chat_id__': metadata.get('chat_id', None),
'__session_id__': metadata.get('session_id', None),
'__message_id__': metadata.get('message_id', None),
'__task__': __task__,
'__task_body__': __task_body__,
'__files__': files,
'__user__': user.model_dump() if isinstance(user, UserModel) else {},
'__metadata__': metadata,
'__oauth_token__': oauth_token,
'__request__': request,
"__event_emitter__": __event_emitter__,
"__event_call__": __event_call__,
"__chat_id__": metadata.get("chat_id", None),
"__session_id__": metadata.get("session_id", None),
"__message_id__": metadata.get("message_id", None),
"__task__": __task__,
"__task_body__": __task_body__,
"__files__": files,
"__user__": user.model_dump() if isinstance(user, UserModel) else {},
"__metadata__": metadata,
"__oauth_token__": oauth_token,
"__request__": request,
}
extra_params['__tools__'] = metadata.get('tools', {})
extra_params["__tools__"] = await get_tools(
request,
tool_ids,
user,
{
**extra_params,
"__model__": models.get(form_data["model"], None),
"__messages__": form_data["messages"],
"__files__": files,
},
)
if model_info:
if model_info.base_model_id:
form_data['model'] = model_info.base_model_id
if not BYPASS_MODEL_ACCESS_CONTROL:
bypass = isinstance(user, UserModel) and user.role == 'admin' and BYPASS_ADMIN_ACCESS_CONTROL
await check_model_access(user if isinstance(user, UserModel) else UserModel(**user), model_info, bypass)
form_data["model"] = model_info.base_model_id
params = model_info.params.model_dump()
if params:
system = params.pop('system', None)
system = params.pop("system", None)
form_data = apply_model_params_to_body_openai(params, form_data)
form_data = await apply_system_prompt_to_body(system, form_data, metadata, user)
form_data = apply_system_prompt_to_body(system, form_data, metadata, user)
pipe_id = get_pipe_id(form_data)
function_module = await get_function_module_by_id(request, pipe_id)
function_module = get_function_module_by_id(request, pipe_id)
pipe = function_module.pipe
params = await get_function_params(function_module, form_data, user, extra_params)
params = get_function_params(function_module, form_data, user, extra_params)
if form_data.get('stream', False):
if form_data.get("stream", False):
async def stream_content():
try:
@@ -298,17 +307,17 @@ async def generate_function_chat_completion(request, form_data, user, models: di
yield data
return
if isinstance(res, dict):
yield f'data: {json.dumps(res)}\n\n'
yield f"data: {json.dumps(res)}\n\n"
return
except Exception as e:
log.error(f'Error: {e}')
yield f'data: {json.dumps({"error": {"detail": str(e)}})}\n\n'
log.error(f"Error: {e}")
yield f"data: {json.dumps({'error': {'detail':str(e)}})}\n\n"
return
if isinstance(res, str):
message = openai_chat_chunk_message_template(form_data['model'], res)
yield f'data: {json.dumps(message)}\n\n'
message = openai_chat_chunk_message_template(form_data["model"], res)
yield f"data: {json.dumps(message)}\n\n"
if isinstance(res, Iterator):
for line in res:
@@ -318,19 +327,22 @@ async def generate_function_chat_completion(request, form_data, user, models: di
async for line in res:
yield process_line(form_data, line)
finish_message = openai_chat_chunk_message_template(form_data['model'], '')
finish_message['choices'][0]['finish_reason'] = 'stop'
yield f'data: {json.dumps(finish_message)}\n\n'
yield 'data: [DONE]'
if isinstance(res, str) or isinstance(res, Generator):
finish_message = openai_chat_chunk_message_template(
form_data["model"], ""
)
finish_message["choices"][0]["finish_reason"] = "stop"
yield f"data: {json.dumps(finish_message)}\n\n"
yield "data: [DONE]"
return StreamingResponse(stream_content(), media_type='text/event-stream')
return StreamingResponse(stream_content(), media_type="text/event-stream")
else:
try:
res = await execute_pipe(pipe, params)
except Exception as e:
log.error(f'Error: {e}')
return {'error': {'detail': str(e)}}
log.error(f"Error: {e}")
return {"error": {"detail": str(e)}}
if isinstance(res, StreamingResponse) or isinstance(res, dict):
return res
@@ -338,4 +350,4 @@ async def generate_function_chat_completion(request, form_data, user, models: di
return res.model_dump()
message = await get_message_content(res)
return openai_chat_completion_message_template(form_data['model'], message)
return openai_chat_completion_message_template(form_data["model"], message)
+89 -308
View File
@@ -1,190 +1,99 @@
from __future__ import annotations
import os
import json
import logging
import os
import sys
from contextlib import asynccontextmanager, contextmanager
from contextlib import contextmanager
from typing import Any, Optional
from urllib.parse import parse_qs, urlencode, urlparse, urlunparse
from open_webui.internal.wrappers import register_connection
from open_webui.env import (
DATABASE_ENABLE_SESSION_SHARING,
DATABASE_ENABLE_SQLITE_WAL,
OPEN_WEBUI_DIR,
DATABASE_URL,
DATABASE_SCHEMA,
SRC_LOG_LEVELS,
DATABASE_POOL_MAX_OVERFLOW,
DATABASE_POOL_RECYCLE,
DATABASE_POOL_SIZE,
DATABASE_POOL_TIMEOUT,
DATABASE_SCHEMA,
DATABASE_SQLITE_PRAGMA_BUSY_TIMEOUT,
DATABASE_SQLITE_PRAGMA_CACHE_SIZE,
DATABASE_SQLITE_PRAGMA_JOURNAL_SIZE_LIMIT,
DATABASE_SQLITE_PRAGMA_MMAP_SIZE,
DATABASE_SQLITE_PRAGMA_SYNCHRONOUS,
DATABASE_SQLITE_PRAGMA_TEMP_STORE,
DATABASE_URL,
ENABLE_DB_MIGRATIONS,
OPEN_WEBUI_DIR,
DATABASE_ENABLE_SQLITE_WAL,
)
from sqlalchemy import Dialect, MetaData, create_engine, event, types
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from peewee_migrate import Router
from sqlalchemy import Dialect, create_engine, MetaData, event, types
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import Session, scoped_session, sessionmaker
from sqlalchemy.pool import NullPool, QueuePool
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.pool import QueuePool, NullPool
from sqlalchemy.sql.type_api import _T
from typing_extensions import Self
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["DB"])
# ── SSL URL normalization (used by sync engine & Alembic migrations) ─
#
# psycopg2 (sync) needs ``sslmode=`` in the connection string (it does
# not recognise the bare ``ssl=`` key that some ORMs emit). The helpers
# below strip all SSL-related query params, normalise them, and
# reattach them in the canonical libpq form.
#
# The **async** engine now uses psycopg (v3), which speaks libpq
# natively, so it needs no translation at all — the DATABASE_URL is
# passed through as-is.
# ─────────────────────────────────────────────────────────────────────
def _pop_first(params: dict[str, list[str]], key: str) -> str | None:
"""Pop a single-valued query param, returning ``None`` if absent."""
values = params.pop(key, None)
return values[0] if values else None
def _is_postgres_url(url: str) -> bool:
"""Return True if *url* looks like a PostgreSQL connection string."""
return bool(url) and any(url.startswith(p) for p in ('postgresql://', 'postgresql+', 'postgres://'))
def extract_ssl_params_from_url(url: str) -> tuple[str, dict[str, str]]:
"""Strip SSL query-string parameters from a PostgreSQL URL.
Returns ``(url_without_ssl, ssl_dict)`` where *ssl_dict* maps
canonical libpq key names (``sslmode``, ``sslrootcert``, …) to
their values. Non-PostgreSQL URLs are returned unchanged with an
empty dict.
"""
if not _is_postgres_url(url):
return url, {}
parsed = urlparse(url)
qp = parse_qs(parsed.query, keep_blank_values=True)
# Prefer sslmode (libpq canonical) over the bare ``ssl`` key.
sslmode_val = _pop_first(qp, 'sslmode')
ssl_val = _pop_first(qp, 'ssl')
ssl_mode = sslmode_val or ssl_val
ssl_dict: dict[str, str] = {}
if ssl_mode:
ssl_dict['sslmode'] = ssl_mode
for key in ('sslrootcert', 'sslcert', 'sslkey', 'sslcrl'):
val = _pop_first(qp, key)
if val:
ssl_dict[key] = val
if not ssl_dict:
return url, ssl_dict
cleaned_query = urlencode(qp, doseq=True)
return urlunparse(parsed._replace(query=cleaned_query)), ssl_dict
def reattach_ssl_params_to_url(url_without_ssl: str, ssl_dict: dict[str, str]) -> str:
"""Re-append SSL query-string parameters to a cleaned PostgreSQL URL.
Used for psycopg2/libpq consumers that expect ``sslmode`` and the
certificate-file keys in the connection string.
"""
if not ssl_dict:
return url_without_ssl
parts = [f'{k}={v}' for k, v in ssl_dict.items() if v]
if not parts:
return url_without_ssl
sep = '&' if '?' in url_without_ssl else '?'
return f'{url_without_ssl}{sep}{"&".join(parts)}'
# Backwards-compatible aliases for external callers.
extract_ssl_mode_from_url = extract_ssl_params_from_url
reattach_ssl_mode_to_url = reattach_ssl_params_to_url
class JSONField(types.TypeDecorator): # TEXT-backed JSON storage
"""Store arbitrary Python objects as JSON-encoded TEXT.
Used instead of native JSON columns for portability across SQLite and
PostgreSQL. Values are serialized with ``json.dumps`` on write and
deserialized with ``json.loads`` on read.
"""
impl = types.UnicodeText
class JSONField(types.TypeDecorator):
impl = types.Text
cache_ok = True
def process_bind_param(self, value: _T | None, dialect: Dialect) -> Any:
return json.dumps(value) if value is not None else None
def process_bind_param(self, value: Optional[_T], dialect: Dialect) -> Any:
return json.dumps(value)
def process_result_value(self, value: _T | None, dialect: Dialect) -> Any:
return json.loads(value) if value is not None else None
def process_result_value(self, value: Optional[_T], dialect: Dialect) -> Any:
if value is not None:
return json.loads(value)
def copy(self, **kwargs: Any) -> Self:
return JSONField(length=self.impl.length)
def copy(self, **kw: Any) -> Self:
return JSONField(self.impl.length)
def db_value(self, value):
return json.dumps(value)
def python_value(self, value):
if value is not None:
return json.loads(value)
# Normalize SSL params from the URL once; the sync engine needs them
# reattached in canonical libpq form for psycopg2.
_url_without_ssl, _ssl_dict = extract_ssl_params_from_url(DATABASE_URL)
# Workaround to handle the peewee migration
# This is required to ensure the peewee migration is handled before the alembic migration
def handle_peewee_migration(DATABASE_URL):
# db = None
try:
# Replace the postgresql:// with postgres:// to handle the peewee migration
db = register_connection(DATABASE_URL.replace("postgresql://", "postgres://"))
migrate_dir = OPEN_WEBUI_DIR / "internal" / "migrations"
router = Router(db, logger=log, migrate_dir=migrate_dir)
router.run()
db.close()
# For psycopg2 (sync engine), re-append sslmode + cert-file params.
SQLALCHEMY_DATABASE_URL = reattach_ssl_params_to_url(_url_without_ssl, _ssl_dict) if _ssl_dict else DATABASE_URL
def _make_async_url(url: str) -> str:
"""Convert a sync database URL to its async driver equivalent.
The async engine uses psycopg (v3) which speaks libpq natively,
so all standard connection-string parameters (``sslmode``,
``options``, ``target_session_attrs``, etc.) are passed through
without any translation.
"""
if url.startswith('sqlite+sqlcipher://'):
raise ValueError(
'sqlite+sqlcipher:// URLs are not supported with async engine. '
'Use standard sqlite:// or postgresql:// instead.'
except Exception as e:
log.error(f"Failed to initialize the database connection: {e}")
log.warning(
"Hint: If your database password contains special characters, you may need to URL-encode it."
)
if url.startswith('sqlite:///') or url.startswith('sqlite://'):
return url.replace('sqlite://', 'sqlite+aiosqlite://', 1)
# psycopg v3 — auto-selects async mode with create_async_engine
if url.startswith('postgresql+psycopg2://'):
return url.replace('postgresql+psycopg2://', 'postgresql+psycopg://', 1)
if url.startswith('postgresql://'):
return url.replace('postgresql://', 'postgresql+psycopg://', 1)
if url.startswith('postgres://'):
return url.replace('postgres://', 'postgresql+psycopg://', 1)
# For other dialects, return as-is and let SQLAlchemy handle it
return url
raise
finally:
# Properly closing the database connection
if db and not db.is_closed():
db.close()
# Assert if db connection has been closed
assert db.is_closed(), "Database connection is still open."
# ============================================================
# SYNC ENGINE (used only for: startup migrations, config loading,
# Alembic, peewee migration, health checks)
# ============================================================
handle_peewee_migration(DATABASE_URL)
SQLALCHEMY_DATABASE_URL = DATABASE_URL
# Handle SQLCipher URLs
if SQLALCHEMY_DATABASE_URL.startswith('sqlite+sqlcipher://'):
database_password = os.environ.get('DATABASE_PASSWORD')
if not database_password or database_password.strip() == '':
raise ValueError('DATABASE_PASSWORD is required when using sqlite+sqlcipher:// URLs')
if SQLALCHEMY_DATABASE_URL.startswith("sqlite+sqlcipher://"):
database_password = os.environ.get("DATABASE_PASSWORD")
if not database_password or database_password.strip() == "":
raise ValueError(
"DATABASE_PASSWORD is required when using sqlite+sqlcipher:// URLs"
)
# Extract database path from SQLCipher URL
db_path = SQLALCHEMY_DATABASE_URL.replace('sqlite+sqlcipher://', '')
db_path = SQLALCHEMY_DATABASE_URL.replace("sqlite+sqlcipher://", "")
if db_path.startswith("/"):
db_path = db_path[1:] # Remove leading slash for relative paths
# Create a custom creator function that uses sqlcipher3
def create_sqlcipher_connection():
@@ -194,63 +103,28 @@ if SQLALCHEMY_DATABASE_URL.startswith('sqlite+sqlcipher://'):
conn.execute(f"PRAGMA key = '{database_password}'")
return conn
# The dummy "sqlite://" URL would cause SQLAlchemy to auto-select
# SingletonThreadPool, which non-deterministically closes in-use
# connections when thread count exceeds pool_size, leading to segfaults
# in the native sqlcipher3 C library. Use NullPool by default for safety,
# or QueuePool if DATABASE_POOL_SIZE is explicitly configured.
if isinstance(DATABASE_POOL_SIZE, int) and DATABASE_POOL_SIZE > 0:
engine = create_engine(
'sqlite://',
creator=create_sqlcipher_connection,
pool_size=DATABASE_POOL_SIZE,
max_overflow=DATABASE_POOL_MAX_OVERFLOW,
pool_timeout=DATABASE_POOL_TIMEOUT,
pool_recycle=DATABASE_POOL_RECYCLE,
pool_pre_ping=True,
poolclass=QueuePool,
echo=False,
)
else:
engine = create_engine(
'sqlite://',
creator=create_sqlcipher_connection,
poolclass=NullPool,
echo=False,
)
engine = create_engine(
"sqlite://", # Dummy URL since we're using creator
creator=create_sqlcipher_connection,
echo=False,
)
log.info('Connected to encrypted SQLite database using SQLCipher')
log.info("Connected to encrypted SQLite database using SQLCipher")
elif 'sqlite' in SQLALCHEMY_DATABASE_URL:
engine = create_engine(SQLALCHEMY_DATABASE_URL, connect_args={'check_same_thread': False})
def _apply_sqlite_pragmas(dbapi_connection):
"""Apply all configured SQLite PRAGMAs to a raw DBAPI connection."""
cursor = dbapi_connection.cursor()
if DATABASE_ENABLE_SQLITE_WAL:
cursor.execute('PRAGMA journal_mode=WAL')
else:
cursor.execute('PRAGMA journal_mode=DELETE')
# Each PRAGMA is skipped when its env var is empty, allowing opt-out.
if DATABASE_SQLITE_PRAGMA_SYNCHRONOUS:
cursor.execute(f'PRAGMA synchronous={DATABASE_SQLITE_PRAGMA_SYNCHRONOUS}')
if DATABASE_SQLITE_PRAGMA_BUSY_TIMEOUT:
cursor.execute(f'PRAGMA busy_timeout={DATABASE_SQLITE_PRAGMA_BUSY_TIMEOUT}')
if DATABASE_SQLITE_PRAGMA_CACHE_SIZE:
cursor.execute(f'PRAGMA cache_size={DATABASE_SQLITE_PRAGMA_CACHE_SIZE}')
if DATABASE_SQLITE_PRAGMA_TEMP_STORE:
cursor.execute(f'PRAGMA temp_store={DATABASE_SQLITE_PRAGMA_TEMP_STORE}')
if DATABASE_SQLITE_PRAGMA_MMAP_SIZE:
cursor.execute(f'PRAGMA mmap_size={DATABASE_SQLITE_PRAGMA_MMAP_SIZE}')
if DATABASE_SQLITE_PRAGMA_JOURNAL_SIZE_LIMIT:
cursor.execute(f'PRAGMA journal_size_limit={DATABASE_SQLITE_PRAGMA_JOURNAL_SIZE_LIMIT}')
cursor.close()
elif "sqlite" in SQLALCHEMY_DATABASE_URL:
engine = create_engine(
SQLALCHEMY_DATABASE_URL, connect_args={"check_same_thread": False}
)
def on_connect(dbapi_connection, connection_record):
_apply_sqlite_pragmas(dbapi_connection)
cursor = dbapi_connection.cursor()
if DATABASE_ENABLE_SQLITE_WAL:
cursor.execute("PRAGMA journal_mode=WAL")
else:
cursor.execute("PRAGMA journal_mode=DELETE")
cursor.close()
event.listen(engine, 'connect', on_connect)
event.listen(engine, "connect", on_connect)
else:
if isinstance(DATABASE_POOL_SIZE, int):
if DATABASE_POOL_SIZE > 0:
@@ -264,20 +138,22 @@ else:
poolclass=QueuePool,
)
else:
engine = create_engine(SQLALCHEMY_DATABASE_URL, pool_pre_ping=True, poolclass=NullPool)
engine = create_engine(
SQLALCHEMY_DATABASE_URL, pool_pre_ping=True, poolclass=NullPool
)
else:
engine = create_engine(SQLALCHEMY_DATABASE_URL, pool_pre_ping=True)
# Sync session — used ONLY for startup config loading (config.py runs at import time)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine, expire_on_commit=False)
SessionLocal = sessionmaker(
autocommit=False, autoflush=False, bind=engine, expire_on_commit=False
)
metadata_obj = MetaData(schema=DATABASE_SCHEMA)
Base = declarative_base(metadata=metadata_obj)
ScopedSession = scoped_session(SessionLocal)
Session = scoped_session(SessionLocal)
def get_session():
"""Sync session generator — used ONLY for startup/config operations."""
db = SessionLocal()
try:
yield db
@@ -286,98 +162,3 @@ def get_session():
get_db = contextmanager(get_session)
# ============================================================
# ASYNC ENGINE (used for ALL runtime database operations)
# ============================================================
# psycopg (v3) speaks libpq natively — the full DATABASE_URL is passed
# through as-is. SSL params, ``options``, ``target_session_attrs``, etc.
# all work without any stripping or translation.
ASYNC_SQLALCHEMY_DATABASE_URL = _make_async_url(SQLALCHEMY_DATABASE_URL)
# psycopg v3 cannot run in async mode under Windows' default
# ProactorEventLoop — switch to SelectorEventLoop before creating
# the async engine. This runs at import time, which is early enough
# to cover every entry point (workers, reload, direct invocations).
if sys.platform == 'win32' and _is_postgres_url(DATABASE_URL):
import asyncio
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
if 'sqlite' in ASYNC_SQLALCHEMY_DATABASE_URL:
# Generous default — async coroutines + no session sharing = high connection demand.
_sqlite_pool_size = DATABASE_POOL_SIZE if isinstance(DATABASE_POOL_SIZE, int) and DATABASE_POOL_SIZE > 0 else 512
async_engine = create_async_engine(
ASYNC_SQLALCHEMY_DATABASE_URL,
connect_args={'check_same_thread': False},
pool_size=_sqlite_pool_size,
pool_timeout=DATABASE_POOL_TIMEOUT,
pool_recycle=DATABASE_POOL_RECYCLE,
pool_pre_ping=True,
)
@event.listens_for(async_engine.sync_engine, 'connect')
def _set_sqlite_pragmas(dbapi_connection, connection_record):
_apply_sqlite_pragmas(dbapi_connection)
else:
if isinstance(DATABASE_POOL_SIZE, int):
if DATABASE_POOL_SIZE > 0:
async_engine = create_async_engine(
ASYNC_SQLALCHEMY_DATABASE_URL,
pool_size=DATABASE_POOL_SIZE,
max_overflow=DATABASE_POOL_MAX_OVERFLOW,
pool_timeout=DATABASE_POOL_TIMEOUT,
pool_recycle=DATABASE_POOL_RECYCLE,
pool_pre_ping=True,
)
else:
async_engine = create_async_engine(
ASYNC_SQLALCHEMY_DATABASE_URL,
pool_pre_ping=True,
poolclass=NullPool,
)
else:
async_engine = create_async_engine(
ASYNC_SQLALCHEMY_DATABASE_URL,
pool_pre_ping=True,
)
AsyncSessionLocal = async_sessionmaker(
bind=async_engine,
class_=AsyncSession,
autocommit=False,
autoflush=False,
expire_on_commit=False,
)
async def get_async_session():
"""Async session generator for FastAPI Depends()."""
async with AsyncSessionLocal() as db:
try:
yield db
finally:
await db.close()
@asynccontextmanager
async def get_async_db():
"""Async context manager for use outside of FastAPI dependency injection."""
async with AsyncSessionLocal() as db:
try:
yield db
finally:
await db.close()
@asynccontextmanager
async def get_async_db_context(db: AsyncSession | None = None):
"""Async context manager that reuses an existing session if provided and session sharing is enabled."""
if isinstance(db, AsyncSession) and DATABASE_ENABLE_SESSION_SHARING:
yield db
else:
async with get_async_db() as session:
yield session
@@ -0,0 +1,254 @@
"""Peewee migrations -- 001_initial_schema.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
# We perform different migrations for SQLite and other databases
# This is because SQLite is very loose with enforcing its schema, and trying to migrate other databases like SQLite
# will require per-database SQL queries.
# Instead, we assume that because external DB support was added at a later date, it is safe to assume a newer base
# schema instead of trying to migrate from an older schema.
if isinstance(database, pw.SqliteDatabase):
migrate_sqlite(migrator, database, fake=fake)
else:
migrate_external(migrator, database, fake=fake)
def migrate_sqlite(migrator: Migrator, database: pw.Database, *, fake=False):
@migrator.create_model
class Auth(pw.Model):
id = pw.CharField(max_length=255, unique=True)
email = pw.CharField(max_length=255)
password = pw.CharField(max_length=255)
active = pw.BooleanField()
class Meta:
table_name = "auth"
@migrator.create_model
class Chat(pw.Model):
id = pw.CharField(max_length=255, unique=True)
user_id = pw.CharField(max_length=255)
title = pw.CharField()
chat = pw.TextField()
timestamp = pw.BigIntegerField()
class Meta:
table_name = "chat"
@migrator.create_model
class ChatIdTag(pw.Model):
id = pw.CharField(max_length=255, unique=True)
tag_name = pw.CharField(max_length=255)
chat_id = pw.CharField(max_length=255)
user_id = pw.CharField(max_length=255)
timestamp = pw.BigIntegerField()
class Meta:
table_name = "chatidtag"
@migrator.create_model
class Document(pw.Model):
id = pw.AutoField()
collection_name = pw.CharField(max_length=255, unique=True)
name = pw.CharField(max_length=255, unique=True)
title = pw.CharField()
filename = pw.CharField()
content = pw.TextField(null=True)
user_id = pw.CharField(max_length=255)
timestamp = pw.BigIntegerField()
class Meta:
table_name = "document"
@migrator.create_model
class Modelfile(pw.Model):
id = pw.AutoField()
tag_name = pw.CharField(max_length=255, unique=True)
user_id = pw.CharField(max_length=255)
modelfile = pw.TextField()
timestamp = pw.BigIntegerField()
class Meta:
table_name = "modelfile"
@migrator.create_model
class Prompt(pw.Model):
id = pw.AutoField()
command = pw.CharField(max_length=255, unique=True)
user_id = pw.CharField(max_length=255)
title = pw.CharField()
content = pw.TextField()
timestamp = pw.BigIntegerField()
class Meta:
table_name = "prompt"
@migrator.create_model
class Tag(pw.Model):
id = pw.CharField(max_length=255, unique=True)
name = pw.CharField(max_length=255)
user_id = pw.CharField(max_length=255)
data = pw.TextField(null=True)
class Meta:
table_name = "tag"
@migrator.create_model
class User(pw.Model):
id = pw.CharField(max_length=255, unique=True)
name = pw.CharField(max_length=255)
email = pw.CharField(max_length=255)
role = pw.CharField(max_length=255)
profile_image_url = pw.CharField(max_length=255)
timestamp = pw.BigIntegerField()
class Meta:
table_name = "user"
def migrate_external(migrator: Migrator, database: pw.Database, *, fake=False):
@migrator.create_model
class Auth(pw.Model):
id = pw.CharField(max_length=255, unique=True)
email = pw.CharField(max_length=255)
password = pw.TextField()
active = pw.BooleanField()
class Meta:
table_name = "auth"
@migrator.create_model
class Chat(pw.Model):
id = pw.CharField(max_length=255, unique=True)
user_id = pw.CharField(max_length=255)
title = pw.TextField()
chat = pw.TextField()
timestamp = pw.BigIntegerField()
class Meta:
table_name = "chat"
@migrator.create_model
class ChatIdTag(pw.Model):
id = pw.CharField(max_length=255, unique=True)
tag_name = pw.CharField(max_length=255)
chat_id = pw.CharField(max_length=255)
user_id = pw.CharField(max_length=255)
timestamp = pw.BigIntegerField()
class Meta:
table_name = "chatidtag"
@migrator.create_model
class Document(pw.Model):
id = pw.AutoField()
collection_name = pw.CharField(max_length=255, unique=True)
name = pw.CharField(max_length=255, unique=True)
title = pw.TextField()
filename = pw.TextField()
content = pw.TextField(null=True)
user_id = pw.CharField(max_length=255)
timestamp = pw.BigIntegerField()
class Meta:
table_name = "document"
@migrator.create_model
class Modelfile(pw.Model):
id = pw.AutoField()
tag_name = pw.CharField(max_length=255, unique=True)
user_id = pw.CharField(max_length=255)
modelfile = pw.TextField()
timestamp = pw.BigIntegerField()
class Meta:
table_name = "modelfile"
@migrator.create_model
class Prompt(pw.Model):
id = pw.AutoField()
command = pw.CharField(max_length=255, unique=True)
user_id = pw.CharField(max_length=255)
title = pw.TextField()
content = pw.TextField()
timestamp = pw.BigIntegerField()
class Meta:
table_name = "prompt"
@migrator.create_model
class Tag(pw.Model):
id = pw.CharField(max_length=255, unique=True)
name = pw.CharField(max_length=255)
user_id = pw.CharField(max_length=255)
data = pw.TextField(null=True)
class Meta:
table_name = "tag"
@migrator.create_model
class User(pw.Model):
id = pw.CharField(max_length=255, unique=True)
name = pw.CharField(max_length=255)
email = pw.CharField(max_length=255)
role = pw.CharField(max_length=255)
profile_image_url = pw.TextField()
timestamp = pw.BigIntegerField()
class Meta:
table_name = "user"
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_model("user")
migrator.remove_model("tag")
migrator.remove_model("prompt")
migrator.remove_model("modelfile")
migrator.remove_model("document")
migrator.remove_model("chatidtag")
migrator.remove_model("chat")
migrator.remove_model("auth")
@@ -0,0 +1,48 @@
"""Peewee migrations -- 002_add_local_sharing.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
migrator.add_fields(
"chat", share_id=pw.CharField(max_length=255, null=True, unique=True)
)
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_fields("chat", "share_id")
@@ -0,0 +1,48 @@
"""Peewee migrations -- 002_add_local_sharing.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
migrator.add_fields(
"user", api_key=pw.CharField(max_length=255, null=True, unique=True)
)
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_fields("user", "api_key")
@@ -0,0 +1,46 @@
"""Peewee migrations -- 002_add_local_sharing.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
migrator.add_fields("chat", archived=pw.BooleanField(default=False))
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_fields("chat", "archived")
@@ -0,0 +1,130 @@
"""Peewee migrations -- 002_add_local_sharing.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
if isinstance(database, pw.SqliteDatabase):
migrate_sqlite(migrator, database, fake=fake)
else:
migrate_external(migrator, database, fake=fake)
def migrate_sqlite(migrator: Migrator, database: pw.Database, *, fake=False):
# Adding fields created_at and updated_at to the 'chat' table
migrator.add_fields(
"chat",
created_at=pw.DateTimeField(null=True), # Allow null for transition
updated_at=pw.DateTimeField(null=True), # Allow null for transition
)
# Populate the new fields from an existing 'timestamp' field
migrator.sql(
"UPDATE chat SET created_at = timestamp, updated_at = timestamp WHERE timestamp IS NOT NULL"
)
# Now that the data has been copied, remove the original 'timestamp' field
migrator.remove_fields("chat", "timestamp")
# Update the fields to be not null now that they are populated
migrator.change_fields(
"chat",
created_at=pw.DateTimeField(null=False),
updated_at=pw.DateTimeField(null=False),
)
def migrate_external(migrator: Migrator, database: pw.Database, *, fake=False):
# Adding fields created_at and updated_at to the 'chat' table
migrator.add_fields(
"chat",
created_at=pw.BigIntegerField(null=True), # Allow null for transition
updated_at=pw.BigIntegerField(null=True), # Allow null for transition
)
# Populate the new fields from an existing 'timestamp' field
migrator.sql(
"UPDATE chat SET created_at = timestamp, updated_at = timestamp WHERE timestamp IS NOT NULL"
)
# Now that the data has been copied, remove the original 'timestamp' field
migrator.remove_fields("chat", "timestamp")
# Update the fields to be not null now that they are populated
migrator.change_fields(
"chat",
created_at=pw.BigIntegerField(null=False),
updated_at=pw.BigIntegerField(null=False),
)
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
if isinstance(database, pw.SqliteDatabase):
rollback_sqlite(migrator, database, fake=fake)
else:
rollback_external(migrator, database, fake=fake)
def rollback_sqlite(migrator: Migrator, database: pw.Database, *, fake=False):
# Recreate the timestamp field initially allowing null values for safe transition
migrator.add_fields("chat", timestamp=pw.DateTimeField(null=True))
# Copy the earliest created_at date back into the new timestamp field
# This assumes created_at was originally a copy of timestamp
migrator.sql("UPDATE chat SET timestamp = created_at")
# Remove the created_at and updated_at fields
migrator.remove_fields("chat", "created_at", "updated_at")
# Finally, alter the timestamp field to not allow nulls if that was the original setting
migrator.change_fields("chat", timestamp=pw.DateTimeField(null=False))
def rollback_external(migrator: Migrator, database: pw.Database, *, fake=False):
# Recreate the timestamp field initially allowing null values for safe transition
migrator.add_fields("chat", timestamp=pw.BigIntegerField(null=True))
# Copy the earliest created_at date back into the new timestamp field
# This assumes created_at was originally a copy of timestamp
migrator.sql("UPDATE chat SET timestamp = created_at")
# Remove the created_at and updated_at fields
migrator.remove_fields("chat", "created_at", "updated_at")
# Finally, alter the timestamp field to not allow nulls if that was the original setting
migrator.change_fields("chat", timestamp=pw.BigIntegerField(null=False))
@@ -0,0 +1,130 @@
"""Peewee migrations -- 006_migrate_timestamps_and_charfields.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
# Alter the tables with timestamps
migrator.change_fields(
"chatidtag",
timestamp=pw.BigIntegerField(),
)
migrator.change_fields(
"document",
timestamp=pw.BigIntegerField(),
)
migrator.change_fields(
"modelfile",
timestamp=pw.BigIntegerField(),
)
migrator.change_fields(
"prompt",
timestamp=pw.BigIntegerField(),
)
migrator.change_fields(
"user",
timestamp=pw.BigIntegerField(),
)
# Alter the tables with varchar to text where necessary
migrator.change_fields(
"auth",
password=pw.TextField(),
)
migrator.change_fields(
"chat",
title=pw.TextField(),
)
migrator.change_fields(
"document",
title=pw.TextField(),
filename=pw.TextField(),
)
migrator.change_fields(
"prompt",
title=pw.TextField(),
)
migrator.change_fields(
"user",
profile_image_url=pw.TextField(),
)
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
if isinstance(database, pw.SqliteDatabase):
# Alter the tables with timestamps
migrator.change_fields(
"chatidtag",
timestamp=pw.DateField(),
)
migrator.change_fields(
"document",
timestamp=pw.DateField(),
)
migrator.change_fields(
"modelfile",
timestamp=pw.DateField(),
)
migrator.change_fields(
"prompt",
timestamp=pw.DateField(),
)
migrator.change_fields(
"user",
timestamp=pw.DateField(),
)
migrator.change_fields(
"auth",
password=pw.CharField(max_length=255),
)
migrator.change_fields(
"chat",
title=pw.CharField(),
)
migrator.change_fields(
"document",
title=pw.CharField(),
filename=pw.CharField(),
)
migrator.change_fields(
"prompt",
title=pw.CharField(),
)
migrator.change_fields(
"user",
profile_image_url=pw.CharField(),
)
@@ -0,0 +1,79 @@
"""Peewee migrations -- 002_add_local_sharing.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
# Adding fields created_at and updated_at to the 'user' table
migrator.add_fields(
"user",
created_at=pw.BigIntegerField(null=True), # Allow null for transition
updated_at=pw.BigIntegerField(null=True), # Allow null for transition
last_active_at=pw.BigIntegerField(null=True), # Allow null for transition
)
# Populate the new fields from an existing 'timestamp' field
migrator.sql(
'UPDATE "user" SET created_at = timestamp, updated_at = timestamp, last_active_at = timestamp WHERE timestamp IS NOT NULL'
)
# Now that the data has been copied, remove the original 'timestamp' field
migrator.remove_fields("user", "timestamp")
# Update the fields to be not null now that they are populated
migrator.change_fields(
"user",
created_at=pw.BigIntegerField(null=False),
updated_at=pw.BigIntegerField(null=False),
last_active_at=pw.BigIntegerField(null=False),
)
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
# Recreate the timestamp field initially allowing null values for safe transition
migrator.add_fields("user", timestamp=pw.BigIntegerField(null=True))
# Copy the earliest created_at date back into the new timestamp field
# This assumes created_at was originally a copy of timestamp
migrator.sql('UPDATE "user" SET timestamp = created_at')
# Remove the created_at and updated_at fields
migrator.remove_fields("user", "created_at", "updated_at", "last_active_at")
# Finally, alter the timestamp field to not allow nulls if that was the original setting
migrator.change_fields("user", timestamp=pw.BigIntegerField(null=False))
@@ -0,0 +1,53 @@
"""Peewee migrations -- 002_add_local_sharing.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
@migrator.create_model
class Memory(pw.Model):
id = pw.CharField(max_length=255, unique=True)
user_id = pw.CharField(max_length=255)
content = pw.TextField(null=False)
updated_at = pw.BigIntegerField(null=False)
created_at = pw.BigIntegerField(null=False)
class Meta:
table_name = "memory"
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_model("memory")
@@ -0,0 +1,61 @@
"""Peewee migrations -- 009_add_models.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
@migrator.create_model
class Model(pw.Model):
id = pw.TextField(unique=True)
user_id = pw.TextField()
base_model_id = pw.TextField(null=True)
name = pw.TextField()
meta = pw.TextField()
params = pw.TextField()
created_at = pw.BigIntegerField(null=False)
updated_at = pw.BigIntegerField(null=False)
class Meta:
table_name = "model"
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_model("model")
@@ -0,0 +1,130 @@
"""Peewee migrations -- 009_add_models.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
import json
from open_webui.utils.misc import parse_ollama_modelfile
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
# Fetch data from 'modelfile' table and insert into 'model' table
migrate_modelfile_to_model(migrator, database)
# Drop the 'modelfile' table
migrator.remove_model("modelfile")
def migrate_modelfile_to_model(migrator: Migrator, database: pw.Database):
ModelFile = migrator.orm["modelfile"]
Model = migrator.orm["model"]
modelfiles = ModelFile.select()
for modelfile in modelfiles:
# Extract and transform data in Python
modelfile.modelfile = json.loads(modelfile.modelfile)
meta = json.dumps(
{
"description": modelfile.modelfile.get("desc"),
"profile_image_url": modelfile.modelfile.get("imageUrl"),
"ollama": {"modelfile": modelfile.modelfile.get("content")},
"suggestion_prompts": modelfile.modelfile.get("suggestionPrompts"),
"categories": modelfile.modelfile.get("categories"),
"user": {**modelfile.modelfile.get("user", {}), "community": True},
}
)
info = parse_ollama_modelfile(modelfile.modelfile.get("content"))
# Insert the processed data into the 'model' table
Model.create(
id=f"ollama-{modelfile.tag_name}",
user_id=modelfile.user_id,
base_model_id=info.get("base_model_id"),
name=modelfile.modelfile.get("title"),
meta=meta,
params=json.dumps(info.get("params", {})),
created_at=modelfile.timestamp,
updated_at=modelfile.timestamp,
)
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
recreate_modelfile_table(migrator, database)
move_data_back_to_modelfile(migrator, database)
migrator.remove_model("model")
def recreate_modelfile_table(migrator: Migrator, database: pw.Database):
query = """
CREATE TABLE IF NOT EXISTS modelfile (
user_id TEXT,
tag_name TEXT,
modelfile JSON,
timestamp BIGINT
)
"""
migrator.sql(query)
def move_data_back_to_modelfile(migrator: Migrator, database: pw.Database):
Model = migrator.orm["model"]
Modelfile = migrator.orm["modelfile"]
models = Model.select()
for model in models:
# Extract and transform data in Python
meta = json.loads(model.meta)
modelfile_data = {
"title": model.name,
"desc": meta.get("description"),
"imageUrl": meta.get("profile_image_url"),
"content": meta.get("ollama", {}).get("modelfile"),
"suggestionPrompts": meta.get("suggestion_prompts"),
"categories": meta.get("categories"),
"user": {k: v for k, v in meta.get("user", {}).items() if k != "community"},
}
# Insert the processed data back into the 'modelfile' table
Modelfile.create(
user_id=model.user_id,
tag_name=model.id,
modelfile=modelfile_data,
timestamp=model.created_at,
)
@@ -0,0 +1,48 @@
"""Peewee migrations -- 002_add_local_sharing.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
# Adding fields settings to the 'user' table
migrator.add_fields("user", settings=pw.TextField(null=True))
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
# Remove the settings field
migrator.remove_fields("user", "settings")
@@ -0,0 +1,61 @@
"""Peewee migrations -- 009_add_models.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
@migrator.create_model
class Tool(pw.Model):
id = pw.TextField(unique=True)
user_id = pw.TextField()
name = pw.TextField()
content = pw.TextField()
specs = pw.TextField()
meta = pw.TextField()
created_at = pw.BigIntegerField(null=False)
updated_at = pw.BigIntegerField(null=False)
class Meta:
table_name = "tool"
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_model("tool")
@@ -0,0 +1,48 @@
"""Peewee migrations -- 002_add_local_sharing.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
# Adding fields info to the 'user' table
migrator.add_fields("user", info=pw.TextField(null=True))
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
# Remove the settings field
migrator.remove_fields("user", "info")
@@ -0,0 +1,55 @@
"""Peewee migrations -- 009_add_models.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
@migrator.create_model
class File(pw.Model):
id = pw.TextField(unique=True)
user_id = pw.TextField()
filename = pw.TextField()
meta = pw.TextField()
created_at = pw.BigIntegerField(null=False)
class Meta:
table_name = "file"
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_model("file")
@@ -0,0 +1,61 @@
"""Peewee migrations -- 009_add_models.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
@migrator.create_model
class Function(pw.Model):
id = pw.TextField(unique=True)
user_id = pw.TextField()
name = pw.TextField()
type = pw.TextField()
content = pw.TextField()
meta = pw.TextField()
created_at = pw.BigIntegerField(null=False)
updated_at = pw.BigIntegerField(null=False)
class Meta:
table_name = "function"
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_model("function")
@@ -0,0 +1,50 @@
"""Peewee migrations -- 009_add_models.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
migrator.add_fields("tool", valves=pw.TextField(null=True))
migrator.add_fields("function", valves=pw.TextField(null=True))
migrator.add_fields("function", is_active=pw.BooleanField(default=False))
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_fields("tool", "valves")
migrator.remove_fields("function", "valves")
migrator.remove_fields("function", "is_active")
@@ -0,0 +1,45 @@
"""Peewee migrations -- 017_add_user_oauth_sub.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
migrator.add_fields(
"user",
oauth_sub=pw.TextField(null=True, unique=True),
)
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_fields("user", "oauth_sub")
@@ -0,0 +1,49 @@
"""Peewee migrations -- 017_add_user_oauth_sub.py.
Some examples (model - class or model name)::
> Model = migrator.orm['table_name'] # Return model in current state by name
> Model = migrator.ModelClass # Return model in current state by name
> migrator.sql(sql) # Run custom SQL
> migrator.run(func, *args, **kwargs) # Run python function with the given args
> migrator.create_model(Model) # Create a model (could be used as decorator)
> migrator.remove_model(model, cascade=True) # Remove a model
> migrator.add_fields(model, **fields) # Add fields to a model
> migrator.change_fields(model, **fields) # Change fields
> migrator.remove_fields(model, *field_names, cascade=True)
> migrator.rename_field(model, old_field_name, new_field_name)
> migrator.rename_table(model, new_table_name)
> migrator.add_index(model, *col_names, unique=False)
> migrator.add_not_null(model, *field_names)
> migrator.add_default(model, field_name, default)
> migrator.add_constraint(model, name, sql)
> migrator.drop_index(model, *col_names)
> migrator.drop_not_null(model, *field_names)
> migrator.drop_constraints(model, *constraints)
"""
from contextlib import suppress
import peewee as pw
from peewee_migrate import Migrator
with suppress(ImportError):
import playhouse.postgres_ext as pw_pext
def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your migrations here."""
migrator.add_fields(
"function",
is_global=pw.BooleanField(default=False),
)
def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
"""Write your rollback migrations here."""
migrator.remove_fields("function", "is_global")
+90
View File
@@ -0,0 +1,90 @@
import logging
import os
from contextvars import ContextVar
from open_webui.env import SRC_LOG_LEVELS
from peewee import *
from peewee import InterfaceError as PeeWeeInterfaceError
from peewee import PostgresqlDatabase
from playhouse.db_url import connect, parse
from playhouse.shortcuts import ReconnectMixin
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["DB"])
db_state_default = {"closed": None, "conn": None, "ctx": None, "transactions": None}
db_state = ContextVar("db_state", default=db_state_default.copy())
class PeeweeConnectionState(object):
def __init__(self, **kwargs):
super().__setattr__("_state", db_state)
super().__init__(**kwargs)
def __setattr__(self, name, value):
self._state.get()[name] = value
def __getattr__(self, name):
value = self._state.get()[name]
return value
class CustomReconnectMixin(ReconnectMixin):
reconnect_errors = (
# psycopg2
(OperationalError, "termin"),
(InterfaceError, "closed"),
# peewee
(PeeWeeInterfaceError, "closed"),
)
class ReconnectingPostgresqlDatabase(CustomReconnectMixin, PostgresqlDatabase):
pass
def register_connection(db_url):
# Check if using SQLCipher protocol
if db_url.startswith("sqlite+sqlcipher://"):
database_password = os.environ.get("DATABASE_PASSWORD")
if not database_password or database_password.strip() == "":
raise ValueError(
"DATABASE_PASSWORD is required when using sqlite+sqlcipher:// URLs"
)
from playhouse.sqlcipher_ext import SqlCipherDatabase
# Parse the database path from SQLCipher URL
# Convert sqlite+sqlcipher:///path/to/db.sqlite to /path/to/db.sqlite
db_path = db_url.replace("sqlite+sqlcipher://", "")
if db_path.startswith("/"):
db_path = db_path[1:] # Remove leading slash for relative paths
# Use Peewee's native SqlCipherDatabase with encryption
db = SqlCipherDatabase(db_path, passphrase=database_password)
db.autoconnect = True
db.reuse_if_open = True
log.info("Connected to encrypted SQLite database using SQLCipher")
else:
# Standard database connection (existing logic)
db = connect(db_url, unquote_user=True, unquote_password=True)
if isinstance(db, PostgresqlDatabase):
# Enable autoconnect for SQLite databases, managed by Peewee
db.autoconnect = True
db.reuse_if_open = True
log.info("Connected to PostgreSQL database")
# Get the connection details
connection = parse(db_url, unquote_user=True, unquote_password=True)
# Use our custom database class that supports reconnection
db = ReconnectingPostgresqlDatabase(**connection)
db.connect(reuse_if_open=True)
elif isinstance(db, SqliteDatabase):
# Enable autoconnect for SQLite databases, managed by Peewee
db.autoconnect = True
db.reuse_if_open = True
log.info("Connected to SQLite database")
else:
raise ValueError("Unsupported database connection")
return db
+1531 -1599
View File
File diff suppressed because it is too large Load Diff
+92 -68
View File
@@ -1,84 +1,108 @@
from __future__ import annotations
from logging.config import fileConfig
# Alembic environment configuration runner.
# Coordinates database migrations in both offline and online execution modes.
import logging.config
import logging
import alembic.context
from open_webui.env import DATABASE_PASSWORD, DATABASE_URL, LOG_FORMAT
from open_webui.internal.db import extract_ssl_params_from_url, reattach_ssl_params_to_url
from alembic import context
from open_webui.models.auths import Auth
from open_webui.models.calendar import Calendar, CalendarEvent, CalendarEventAttendee # noqa: F401
from sqlalchemy import create_engine, engine_from_config, pool
from open_webui.env import DATABASE_URL, DATABASE_PASSWORD
from sqlalchemy import engine_from_config, pool, create_engine
alembic_config = alembic.context.config
if alembic_config.config_file_name:
logging.config.fileConfig(alembic_config.config_file_name, disable_existing_loggers=False)
if LOG_FORMAT == 'json':
from open_webui.env import JSONFormatter
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
for log_handler in logging.root.handlers:
log_handler.setFormatter(JSONFormatter())
migration_metadata = Auth.metadata
target_db_url = DATABASE_URL
base_url, ssl_query_params = extract_ssl_params_from_url(target_db_url)
if ssl_query_params:
target_db_url = reattach_ssl_params_to_url(base_url, ssl_query_params)
if target_db_url:
alembic_config.set_main_option('sqlalchemy.url', target_db_url.replace('%', '%%'))
# Interpret the config file for Python logging.
# This line sets up loggers basically.
if config.config_file_name is not None:
fileConfig(config.config_file_name, disable_existing_loggers=False)
# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
# target_metadata = mymodel.Base.metadata
target_metadata = Auth.metadata
# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
DB_URL = DATABASE_URL
if DB_URL:
config.set_main_option("sqlalchemy.url", DB_URL.replace("%", "%%"))
def run_migrations_offline() -> None:
"""Execute Alembic migrations in offline mode (outputs raw SQL DDL)."""
db_connection_url = alembic_config.get_main_option('sqlalchemy.url')
alembic.context.configure(
url=db_connection_url,
target_metadata=migration_metadata,
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={'paramstyle': 'named'},
dialect_opts={"paramstyle": "named"},
)
with alembic.context.begin_transaction():
alembic.context.run_migrations()
def _get_engine_connectable():
"""Build the database engine based on target URL and authentication credentials."""
if target_db_url and target_db_url.startswith('sqlite+sqlcipher://'):
if not DATABASE_PASSWORD or not DATABASE_PASSWORD.strip():
raise ValueError('DATABASE_PASSWORD is required when using sqlite+sqlcipher:// URLs')
raw_db_path = target_db_url.replace('sqlite+sqlcipher://', '')
if raw_db_path.startswith('/'):
raw_db_path = raw_db_path[1:]
def _sqlite_cipher_creator():
import sqlcipher3
cipher_conn = sqlcipher3.connect(raw_db_path, check_same_thread=False)
cipher_conn.execute(f"PRAGMA key = '{DATABASE_PASSWORD}'")
return cipher_conn
return create_engine('sqlite://', creator=_sqlite_cipher_creator, echo=False)
return engine_from_config(
alembic_config.get_section(alembic_config.config_ini_section, {}),
prefix='sqlalchemy.',
poolclass=pool.NullPool,
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
"""Execute migrations against a live database connection."""
live_connectable = _get_engine_connectable()
with live_connectable.connect() as live_connection:
alembic.context.configure(
connection=live_connection,
target_metadata=migration_metadata,
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
# Handle SQLCipher URLs
if DB_URL and DB_URL.startswith("sqlite+sqlcipher://"):
if not DATABASE_PASSWORD or DATABASE_PASSWORD.strip() == "":
raise ValueError(
"DATABASE_PASSWORD is required when using sqlite+sqlcipher:// URLs"
)
# Extract database path from SQLCipher URL
db_path = DB_URL.replace("sqlite+sqlcipher://", "")
if db_path.startswith("/"):
db_path = db_path[1:] # Remove leading slash for relative paths
# Create a custom creator function that uses sqlcipher3
def create_sqlcipher_connection():
import sqlcipher3
conn = sqlcipher3.connect(db_path, check_same_thread=False)
conn.execute(f"PRAGMA key = '{DATABASE_PASSWORD}'")
return conn
connectable = create_engine(
"sqlite://", # Dummy URL since we're using creator
creator=create_sqlcipher_connection,
echo=False,
)
with alembic.context.begin_transaction():
alembic.context.run_migrations()
else:
# Standard database connection (existing logic)
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
# Alembic execution entrypoint branch
if alembic.context.is_offline_mode():
run_migrations_offline() # run in offline mode
if not alembic.context.is_offline_mode():
run_migrations_online() # run in online mode
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
+9 -14
View File
@@ -1,20 +1,15 @@
from __future__ import annotations
"""Alembic migration utilities."""
from alembic import op # noqa: E402 — alembic runtime context
from sqlalchemy import inspect # metadata inspection
from alembic import op
from sqlalchemy import Inspector
# --- database helper functions ---
def get_existing_tables() -> set[str]:
"""Return table names already present in the database."""
conn = op.get_bind()
return set(inspect(conn).get_table_names())
def get_existing_tables():
con = op.get_bind()
inspector = Inspector.from_engine(con)
tables = set(inspector.get_table_names())
return tables
def get_revision_id() -> str:
"""Generate a short random revision identifier."""
def get_revision_id():
import uuid
return uuid.uuid4().hex[:12]
return str(uuid.uuid4()).replace("-", "")[:12]
@@ -6,57 +6,41 @@ Create Date: 2025-08-13 03:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
revision = '018012973d35'
down_revision = 'd31026856c01'
revision = "018012973d35"
down_revision = "d31026856c01"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
def _idx_exists(table, idx_name):
return any(i['name'] == idx_name for i in inspector.get_indexes(table))
# Chat table indexes
if not _idx_exists('chat', 'folder_id_idx'):
op.create_index('folder_id_idx', 'chat', ['folder_id'])
if not _idx_exists('chat', 'user_id_pinned_idx'):
op.create_index('user_id_pinned_idx', 'chat', ['user_id', 'pinned'])
if not _idx_exists('chat', 'user_id_archived_idx'):
op.create_index('user_id_archived_idx', 'chat', ['user_id', 'archived'])
if not _idx_exists('chat', 'updated_at_user_id_idx'):
op.create_index('updated_at_user_id_idx', 'chat', ['updated_at', 'user_id'])
if not _idx_exists('chat', 'folder_id_user_id_idx'):
op.create_index('folder_id_user_id_idx', 'chat', ['folder_id', 'user_id'])
op.create_index("folder_id_idx", "chat", ["folder_id"])
op.create_index("user_id_pinned_idx", "chat", ["user_id", "pinned"])
op.create_index("user_id_archived_idx", "chat", ["user_id", "archived"])
op.create_index("updated_at_user_id_idx", "chat", ["updated_at", "user_id"])
op.create_index("folder_id_user_id_idx", "chat", ["folder_id", "user_id"])
# Tag table index
if not _idx_exists('tag', 'user_id_idx'):
op.create_index('user_id_idx', 'tag', ['user_id'])
op.create_index("user_id_idx", "tag", ["user_id"])
# Function table index (only if is_global column exists — added by a later migration)
conn = op.get_bind()
inspector = sa.inspect(conn)
func_cols = {c['name'] for c in inspector.get_columns('function')}
if 'is_global' in func_cols and not _idx_exists('function', 'is_global_idx'):
op.create_index('is_global_idx', 'function', ['is_global'])
# Function table index
op.create_index("is_global_idx", "function", ["is_global"])
def downgrade():
# Chat table indexes
op.drop_index('folder_id_idx', table_name='chat')
op.drop_index('user_id_pinned_idx', table_name='chat')
op.drop_index('user_id_archived_idx', table_name='chat')
op.drop_index('updated_at_user_id_idx', table_name='chat')
op.drop_index('folder_id_user_id_idx', table_name='chat')
op.drop_index("folder_id_idx", table_name="chat")
op.drop_index("user_id_pinned_idx", table_name="chat")
op.drop_index("user_id_archived_idx", table_name="chat")
op.drop_index("updated_at_user_id_idx", table_name="chat")
op.drop_index("folder_id_user_id_idx", table_name="chat")
# Tag table index
op.drop_index('user_id_idx', table_name='tag')
op.drop_index("user_id_idx", table_name="tag")
# Function table index
op.drop_index('is_global_idx', table_name='function')
op.drop_index("is_global_idx", table_name="function")
@@ -6,15 +6,15 @@ Create Date: 2024-10-09 21:02:35.241684
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.sql import table, select, update, column
from sqlalchemy.engine.reflection import Inspector
import json
import sqlalchemy as sa
from alembic import op
from sqlalchemy.engine.reflection import Inspector
from sqlalchemy.sql import column, select, table, update
revision = '1af9b942657b'
down_revision = '242a2047eae0'
revision = "1af9b942657b"
down_revision = "242a2047eae0"
branch_labels = None
depends_on = None
@@ -25,40 +25,43 @@ def upgrade():
inspector = Inspector.from_engine(conn)
# Clean up potential leftover temp table from previous failures
conn.execute(sa.text('DROP TABLE IF EXISTS _alembic_tmp_tag'))
conn.execute(sa.text("DROP TABLE IF EXISTS _alembic_tmp_tag"))
# Check if the 'tag' table exists
tables = inspector.get_table_names()
# Step 1: Modify Tag table using batch mode for SQLite support
if 'tag' in tables:
if "tag" in tables:
# Get the current columns in the 'tag' table
columns = [col['name'] for col in inspector.get_columns('tag')]
columns = [col["name"] for col in inspector.get_columns("tag")]
# Get any existing unique constraints on the 'tag' table
current_constraints = inspector.get_unique_constraints('tag')
current_constraints = inspector.get_unique_constraints("tag")
with op.batch_alter_table('tag', schema=None) as batch_op:
with op.batch_alter_table("tag", schema=None) as batch_op:
# Check if the unique constraint already exists
if not any(constraint['name'] == 'uq_id_user_id' for constraint in current_constraints):
if not any(
constraint["name"] == "uq_id_user_id"
for constraint in current_constraints
):
# Create unique constraint if it doesn't exist
batch_op.create_unique_constraint('uq_id_user_id', ['id', 'user_id'])
batch_op.create_unique_constraint("uq_id_user_id", ["id", "user_id"])
# Check if the 'data' column exists before trying to drop it
if 'data' in columns:
batch_op.drop_column('data')
if "data" in columns:
batch_op.drop_column("data")
# Check if the 'meta' column needs to be created
if 'meta' not in columns:
if "meta" not in columns:
# Add the 'meta' column if it doesn't already exist
batch_op.add_column(sa.Column('meta', sa.JSON(), nullable=True))
batch_op.add_column(sa.Column("meta", sa.JSON(), nullable=True))
tag = table(
'tag',
column('id', sa.String()),
column('name', sa.String()),
column('user_id', sa.String()),
column('meta', sa.JSON()),
"tag",
column("id", sa.String()),
column("name", sa.String()),
column("user_id", sa.String()),
column("meta", sa.JSON()),
)
# Step 2: Migrate tags
@@ -67,12 +70,12 @@ def upgrade():
tag_updates = {}
for row in result:
new_id = row.name.replace(' ', '_').lower()
new_id = row.name.replace(" ", "_").lower()
tag_updates[row.id] = new_id
for tag_id, new_tag_id in tag_updates.items():
print(f'Updating tag {tag_id} to {new_tag_id}')
if new_tag_id == 'pinned':
print(f"Updating tag {tag_id} to {new_tag_id}")
if new_tag_id == "pinned":
# delete tag
delete_stmt = sa.delete(tag).where(tag.c.id == tag_id)
conn.execute(delete_stmt)
@@ -83,7 +86,9 @@ def upgrade():
if existing_tag_result:
# Handle duplicate case: the new_tag_id already exists
print(f'Tag {new_tag_id} already exists. Removing current tag with ID {tag_id} to avoid duplicates.')
print(
f"Tag {new_tag_id} already exists. Removing current tag with ID {tag_id} to avoid duplicates."
)
# Option 1: Delete the current tag if an update to new_tag_id would cause duplication
delete_stmt = sa.delete(tag).where(tag.c.id == tag_id)
conn.execute(delete_stmt)
@@ -93,18 +98,19 @@ def upgrade():
conn.execute(update_stmt)
# Add columns `pinned` and `meta` to 'chat'
chat_columns = {c['name'] for c in inspector.get_columns('chat')}
if 'pinned' not in chat_columns:
op.add_column('chat', sa.Column('pinned', sa.Boolean(), nullable=True))
if 'meta' not in chat_columns:
op.add_column('chat', sa.Column('meta', sa.JSON(), nullable=False, server_default='{}'))
op.add_column("chat", sa.Column("pinned", sa.Boolean(), nullable=True))
op.add_column(
"chat", sa.Column("meta", sa.JSON(), nullable=False, server_default="{}")
)
chatidtag = table('chatidtag', column('chat_id', sa.String()), column('tag_name', sa.String()))
chatidtag = table(
"chatidtag", column("chat_id", sa.String()), column("tag_name", sa.String())
)
chat = table(
'chat',
column('id', sa.String()),
column('pinned', sa.Boolean()),
column('meta', sa.JSON()),
"chat",
column("id", sa.String()),
column("pinned", sa.Boolean()),
column("meta", sa.JSON()),
)
# Fetch existing tags
@@ -114,27 +120,29 @@ def upgrade():
chat_updates = {}
for row in result:
chat_id = row.chat_id
tag_name = row.tag_name.replace(' ', '_').lower()
tag_name = row.tag_name.replace(" ", "_").lower()
if tag_name == 'pinned':
if tag_name == "pinned":
# Specifically handle 'pinned' tag
if chat_id not in chat_updates:
chat_updates[chat_id] = {'pinned': True, 'meta': {}}
chat_updates[chat_id] = {"pinned": True, "meta": {}}
else:
chat_updates[chat_id]['pinned'] = True
chat_updates[chat_id]["pinned"] = True
else:
if chat_id not in chat_updates:
chat_updates[chat_id] = {'pinned': False, 'meta': {'tags': [tag_name]}}
chat_updates[chat_id] = {"pinned": False, "meta": {"tags": [tag_name]}}
else:
tags = chat_updates[chat_id]['meta'].get('tags', [])
tags = chat_updates[chat_id]["meta"].get("tags", [])
tags.append(tag_name)
chat_updates[chat_id]['meta']['tags'] = list(set(tags))
chat_updates[chat_id]["meta"]["tags"] = list(set(tags))
# Update chats based on accumulated changes
for chat_id, updates in chat_updates.items():
update_stmt = sa.update(chat).where(chat.c.id == chat_id)
update_stmt = update_stmt.values(meta=updates.get('meta', {}), pinned=updates.get('pinned', False))
update_stmt = update_stmt.values(
meta=updates.get("meta", {}), pinned=updates.get("pinned", False)
)
conn.execute(update_stmt)
pass
@@ -6,14 +6,14 @@ Create Date: 2024-10-09 21:02:35.241684
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.sql import table, select, update
import json
import sqlalchemy as sa
from alembic import op
from sqlalchemy.sql import select, table, update
revision = '242a2047eae0'
down_revision = '6a39f3d8e55c'
revision = "242a2047eae0"
down_revision = "6a39f3d8e55c"
branch_labels = None
depends_on = None
@@ -22,79 +22,86 @@ def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
columns = inspector.get_columns('chat')
column_dict = {col['name']: col for col in columns}
columns = inspector.get_columns("chat")
column_dict = {col["name"]: col for col in columns}
chat_column = column_dict.get('chat')
old_chat_exists = 'old_chat' in column_dict
chat_column = column_dict.get("chat")
old_chat_exists = "old_chat" in column_dict
if chat_column:
if isinstance(chat_column['type'], sa.Text):
if isinstance(chat_column["type"], sa.Text):
print("Converting 'chat' column to JSON")
if old_chat_exists:
print("Dropping old 'old_chat' column")
op.drop_column('chat', 'old_chat')
op.drop_column("chat", "old_chat")
# Step 1: Rename current 'chat' column to 'old_chat'
print("Renaming 'chat' column to 'old_chat'")
op.alter_column('chat', 'chat', new_column_name='old_chat', existing_type=sa.Text())
op.alter_column(
"chat", "chat", new_column_name="old_chat", existing_type=sa.Text()
)
# Step 2: Add new 'chat' column of type JSON
print("Adding new 'chat' column of type JSON")
op.add_column('chat', sa.Column('chat', sa.JSON(), nullable=True))
op.add_column("chat", sa.Column("chat", sa.JSON(), nullable=True))
else:
# If the column is already JSON, no need to do anything
pass
# Step 3: Migrate data from 'old_chat' to 'chat' (only if old_chat exists)
# Re-check columns after potential rename above
current_cols = {c['name'] for c in inspector.get_columns('chat')}
if 'old_chat' in current_cols:
chat_table = table(
'chat',
sa.Column('id', sa.String(), primary_key=True),
sa.Column('old_chat', sa.Text()),
sa.Column('chat', sa.JSON()),
# Step 3: Migrate data from 'old_chat' to 'chat'
chat_table = table(
"chat",
sa.Column("id", sa.String(), primary_key=True),
sa.Column("old_chat", sa.Text()),
sa.Column("chat", sa.JSON()),
)
# - Selecting all data from the table
connection = op.get_bind()
results = connection.execute(select(chat_table.c.id, chat_table.c.old_chat))
for row in results:
try:
# Convert text JSON to actual JSON object, assuming the text is in JSON format
json_data = json.loads(row.old_chat)
except json.JSONDecodeError:
json_data = None # Handle cases where the text cannot be converted to JSON
connection.execute(
sa.update(chat_table)
.where(chat_table.c.id == row.id)
.values(chat=json_data)
)
# - Selecting all data from the table
connection = op.get_bind()
results = connection.execute(select(chat_table.c.id, chat_table.c.old_chat))
for row in results:
try:
# Convert text JSON to actual JSON object, assuming the text is in JSON format
json_data = json.loads(row.old_chat)
except json.JSONDecodeError:
json_data = None # Handle cases where the text cannot be converted to JSON
connection.execute(sa.update(chat_table).where(chat_table.c.id == row.id).values(chat=json_data))
# Step 4: Drop 'old_chat' column
print("Dropping 'old_chat' column")
op.drop_column('chat', 'old_chat')
# Step 4: Drop 'old_chat' column
print("Dropping 'old_chat' column")
op.drop_column("chat", "old_chat")
def downgrade():
# Step 1: Add 'old_chat' column back as Text
op.add_column('chat', sa.Column('old_chat', sa.Text(), nullable=True))
op.add_column("chat", sa.Column("old_chat", sa.Text(), nullable=True))
# Step 2: Convert 'chat' JSON data back to text and store in 'old_chat'
chat_table = table(
'chat',
sa.Column('id', sa.String(), primary_key=True),
sa.Column('chat', sa.JSON()),
sa.Column('old_chat', sa.Text()),
"chat",
sa.Column("id", sa.String(), primary_key=True),
sa.Column("chat", sa.JSON()),
sa.Column("old_chat", sa.Text()),
)
connection = op.get_bind()
results = connection.execute(select(chat_table.c.id, chat_table.c.chat))
for row in results:
text_data = json.dumps(row.chat) if row.chat is not None else None
connection.execute(sa.update(chat_table).where(chat_table.c.id == row.id).values(old_chat=text_data))
connection.execute(
sa.update(chat_table)
.where(chat_table.c.id == row.id)
.values(old_chat=text_data)
)
# Step 3: Remove the new 'chat' JSON column
op.drop_column('chat', 'chat')
op.drop_column("chat", "chat")
# Step 4: Rename 'old_chat' back to 'chat'
op.alter_column('chat', 'old_chat', new_column_name='chat', existing_type=sa.Text())
op.alter_column("chat", "old_chat", new_column_name="chat", existing_type=sa.Text())
@@ -1,107 +0,0 @@
"""Update messages and channel member table
Revision ID: 2f1211949ecc
Revises: 37f288994c47
Create Date: 2025-11-27 03:07:56.200231
"""
from typing import Sequence, Union
import open_webui.internal.db
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = '2f1211949ecc'
down_revision: Union[str, None] = '37f288994c47'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
# New columns to be added to channel_member table
cm_cols = {c['name'] for c in inspector.get_columns('channel_member')}
if 'status' not in cm_cols:
op.add_column('channel_member', sa.Column('status', sa.Text(), nullable=True))
if 'is_active' not in cm_cols:
op.add_column(
'channel_member',
sa.Column(
'is_active',
sa.Boolean(),
nullable=False,
default=True,
server_default=sa.sql.expression.true(),
),
)
if 'is_channel_muted' not in cm_cols:
op.add_column(
'channel_member',
sa.Column(
'is_channel_muted',
sa.Boolean(),
nullable=False,
default=False,
server_default=sa.sql.expression.false(),
),
)
if 'is_channel_pinned' not in cm_cols:
op.add_column(
'channel_member',
sa.Column(
'is_channel_pinned',
sa.Boolean(),
nullable=False,
default=False,
server_default=sa.sql.expression.false(),
),
)
if 'data' not in cm_cols:
op.add_column('channel_member', sa.Column('data', sa.JSON(), nullable=True))
if 'meta' not in cm_cols:
op.add_column('channel_member', sa.Column('meta', sa.JSON(), nullable=True))
if 'joined_at' not in cm_cols:
op.add_column('channel_member', sa.Column('joined_at', sa.BigInteger(), nullable=False))
if 'left_at' not in cm_cols:
op.add_column('channel_member', sa.Column('left_at', sa.BigInteger(), nullable=True))
if 'last_read_at' not in cm_cols:
op.add_column('channel_member', sa.Column('last_read_at', sa.BigInteger(), nullable=True))
if 'updated_at' not in cm_cols:
op.add_column('channel_member', sa.Column('updated_at', sa.BigInteger(), nullable=True))
# New columns to be added to message table
msg_cols = {c['name'] for c in inspector.get_columns('message')}
if 'is_pinned' not in msg_cols:
op.add_column(
'message',
sa.Column(
'is_pinned',
sa.Boolean(),
nullable=False,
default=False,
server_default=sa.sql.expression.false(),
),
)
if 'pinned_at' not in msg_cols:
op.add_column('message', sa.Column('pinned_at', sa.BigInteger(), nullable=True))
if 'pinned_by' not in msg_cols:
op.add_column('message', sa.Column('pinned_by', sa.Text(), nullable=True))
def downgrade() -> None:
op.drop_column('channel_member', 'updated_at')
op.drop_column('channel_member', 'last_read_at')
op.drop_column('channel_member', 'meta')
op.drop_column('channel_member', 'data')
op.drop_column('channel_member', 'is_channel_pinned')
op.drop_column('channel_member', 'is_channel_muted')
op.drop_column('message', 'pinned_by')
op.drop_column('message', 'pinned_at')
op.drop_column('message', 'is_pinned')
@@ -1,262 +0,0 @@
"""Add prompt history table
Revision ID: 374d2f66af06
Revises: c440947495f3
Create Date: 2026-01-23 17:15:00.000000
"""
import uuid
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = '374d2f66af06'
down_revision: Union[str, None] = 'c440947495f3'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
# If the final state already exists (prompt has 'id' PK + prompt_history exists),
# the migration completed successfully on a prior run — nothing to do.
if 'prompt_history' in existing_tables and 'prompt_new' not in existing_tables:
# prompt_history exists and prompt_new was already renamed → done
prompt_cols = {c['name'] for c in inspector.get_columns('prompt')}
if 'id' in prompt_cols and 'version_id' in prompt_cols:
return
# Step 1: Read existing data from OLD table (schema: command as PK)
# Only read if the old-schema prompt table still exists (has 'command' but no 'version_id')
existing_prompts = []
if 'prompt' in existing_tables and 'prompt_new' not in existing_tables:
prompt_cols = {c['name'] for c in inspector.get_columns('prompt')}
if 'command' in prompt_cols and 'version_id' not in prompt_cols:
old_prompt_table = sa.table(
'prompt',
sa.column('command', sa.Text()),
sa.column('user_id', sa.Text()),
sa.column('title', sa.Text()),
sa.column('content', sa.Text()),
sa.column('timestamp', sa.BigInteger()),
sa.column('access_control', sa.JSON()),
)
try:
existing_prompts = conn.execute(
sa.select(
old_prompt_table.c.command,
old_prompt_table.c.user_id,
old_prompt_table.c.title,
old_prompt_table.c.content,
old_prompt_table.c.timestamp,
old_prompt_table.c.access_control,
)
).fetchall()
except Exception:
existing_prompts = []
# Step 2: Create new prompt table with 'id' as PRIMARY KEY (if not already created)
if 'prompt_new' not in existing_tables:
op.create_table(
'prompt_new',
sa.Column('id', sa.Text(), primary_key=True),
sa.Column('command', sa.String(), unique=True, index=True),
sa.Column('user_id', sa.String(), nullable=False),
sa.Column('name', sa.Text(), nullable=False),
sa.Column('content', sa.Text(), nullable=False),
sa.Column('data', sa.JSON(), nullable=True),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('access_control', sa.JSON(), nullable=True),
sa.Column('is_active', sa.Boolean(), nullable=False, server_default='1'),
sa.Column('version_id', sa.Text(), nullable=True),
sa.Column('tags', sa.JSON(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
)
# Step 3: Create prompt_history table (if not already created)
if 'prompt_history' not in existing_tables:
op.create_table(
'prompt_history',
sa.Column('id', sa.Text(), primary_key=True),
sa.Column('prompt_id', sa.Text(), nullable=False, index=True),
sa.Column('parent_id', sa.Text(), nullable=True),
sa.Column('snapshot', sa.JSON(), nullable=False),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column('commit_message', sa.Text(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=False),
)
# Step 4: Migrate data (only if we have old data to migrate)
if existing_prompts:
prompt_new_table = sa.table(
'prompt_new',
sa.column('id', sa.Text()),
sa.column('command', sa.String()),
sa.column('user_id', sa.String()),
sa.column('name', sa.Text()),
sa.column('content', sa.Text()),
sa.column('data', sa.JSON()),
sa.column('meta', sa.JSON()),
sa.column('access_control', sa.JSON()),
sa.column('is_active', sa.Boolean()),
sa.column('version_id', sa.Text()),
sa.column('tags', sa.JSON()),
sa.column('created_at', sa.BigInteger()),
sa.column('updated_at', sa.BigInteger()),
)
prompt_history_table = sa.table(
'prompt_history',
sa.column('id', sa.Text()),
sa.column('prompt_id', sa.Text()),
sa.column('parent_id', sa.Text()),
sa.column('snapshot', sa.JSON()),
sa.column('user_id', sa.Text()),
sa.column('commit_message', sa.Text()),
sa.column('created_at', sa.BigInteger()),
)
for row in existing_prompts:
command = row[0]
user_id = row[1]
title = row[2]
content = row[3]
timestamp = row[4]
access_control = row[5]
new_uuid = str(uuid.uuid4())
history_uuid = str(uuid.uuid4())
clean_command = command[1:] if command and command.startswith('/') else command
# Insert into prompt_new
conn.execute(
sa.insert(prompt_new_table).values(
id=new_uuid,
command=clean_command,
user_id=user_id,
name=title,
content=content,
data={},
meta={},
access_control=access_control,
is_active=True,
version_id=history_uuid,
tags=[],
created_at=timestamp,
updated_at=timestamp,
)
)
# Create initial history entry
conn.execute(
sa.insert(prompt_history_table).values(
id=history_uuid,
prompt_id=new_uuid,
parent_id=None,
snapshot={
'name': title,
'content': content,
'command': clean_command,
'data': {},
'meta': {},
'access_control': access_control,
},
user_id=user_id,
commit_message=None,
created_at=timestamp,
)
)
# Step 5: Replace old table with new one (only if prompt_new exists)
# Re-check tables after potential creation above
inspector.clear_cache()
current_tables = set(inspector.get_table_names())
if 'prompt_new' in current_tables:
if 'prompt' in current_tables:
op.drop_table('prompt')
op.rename_table('prompt_new', 'prompt')
def downgrade() -> None:
conn = op.get_bind()
# Step 1: Read new data
prompt_table = sa.table(
'prompt',
sa.column('command', sa.String()),
sa.column('name', sa.Text()),
sa.column('created_at', sa.BigInteger()),
sa.column('user_id', sa.Text()),
sa.column('content', sa.Text()),
sa.column('access_control', sa.JSON()),
)
try:
current_data = conn.execute(
sa.select(
prompt_table.c.command,
prompt_table.c.name,
prompt_table.c.created_at,
prompt_table.c.user_id,
prompt_table.c.content,
prompt_table.c.access_control,
)
).fetchall()
except Exception:
current_data = []
# Step 2: Drop history and table
op.drop_table('prompt_history')
op.drop_table('prompt')
# Step 3: Recreate old table (command as PK?)
# Assuming old schema:
op.create_table(
'prompt',
sa.Column('command', sa.String(), primary_key=True),
sa.Column('user_id', sa.String()),
sa.Column('title', sa.Text()),
sa.Column('content', sa.Text()),
sa.Column('timestamp', sa.BigInteger()),
sa.Column('access_control', sa.JSON()),
sa.Column('id', sa.Integer(), nullable=True),
)
# Step 4: Restore data
old_prompt_table = sa.table(
'prompt',
sa.column('command', sa.String()),
sa.column('user_id', sa.String()),
sa.column('title', sa.Text()),
sa.column('content', sa.Text()),
sa.column('timestamp', sa.BigInteger()),
sa.column('access_control', sa.JSON()),
)
for row in current_data:
command = row[0]
name = row[1]
created_at = row[2]
user_id = row[3]
content = row[4]
access_control = row[5]
# Restore leading /
old_command = '/' + command if command and not command.startswith('/') else command
conn.execute(
sa.insert(old_prompt_table).values(
command=old_command,
user_id=user_id,
title=name,
content=content,
timestamp=created_at,
access_control=access_control,
)
)
@@ -6,65 +6,65 @@ Create Date: 2024-12-30 03:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
revision = '3781e22d8b01'
down_revision = '7826ab40b532'
revision = "3781e22d8b01"
down_revision = "7826ab40b532"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
# Add 'type' column to the 'channel' table
channel_cols = {c['name'] for c in inspector.get_columns('channel')}
if 'type' not in channel_cols:
op.add_column(
'channel',
sa.Column(
'type',
sa.Text(),
nullable=True,
),
)
op.add_column(
"channel",
sa.Column(
"type",
sa.Text(),
nullable=True,
),
)
# Add 'parent_id' column to the 'message' table for threads
message_cols = {c['name'] for c in inspector.get_columns('message')}
if 'parent_id' not in message_cols:
op.add_column(
'message',
sa.Column('parent_id', sa.Text(), nullable=True),
)
op.add_column(
"message",
sa.Column("parent_id", sa.Text(), nullable=True),
)
if 'message_reaction' not in existing_tables:
op.create_table(
'message_reaction',
sa.Column('id', sa.Text(), nullable=False, primary_key=True, unique=True), # Unique reaction ID
sa.Column('user_id', sa.Text(), nullable=False), # User who reacted
sa.Column('message_id', sa.Text(), nullable=False), # Message that was reacted to
sa.Column('name', sa.Text(), nullable=False), # Reaction name (e.g. "thumbs_up")
sa.Column('created_at', sa.BigInteger(), nullable=True), # Timestamp of when the reaction was added
)
op.create_table(
"message_reaction",
sa.Column(
"id", sa.Text(), nullable=False, primary_key=True, unique=True
), # Unique reaction ID
sa.Column("user_id", sa.Text(), nullable=False), # User who reacted
sa.Column(
"message_id", sa.Text(), nullable=False
), # Message that was reacted to
sa.Column(
"name", sa.Text(), nullable=False
), # Reaction name (e.g. "thumbs_up")
sa.Column(
"created_at", sa.BigInteger(), nullable=True
), # Timestamp of when the reaction was added
)
if 'channel_member' not in existing_tables:
op.create_table(
'channel_member',
sa.Column(
'id', sa.Text(), nullable=False, primary_key=True, unique=True
), # Record ID for the membership row
sa.Column('channel_id', sa.Text(), nullable=False), # Associated channel
sa.Column('user_id', sa.Text(), nullable=False), # Associated user
sa.Column('created_at', sa.BigInteger(), nullable=True), # Timestamp of when the user joined the channel
)
op.create_table(
"channel_member",
sa.Column(
"id", sa.Text(), nullable=False, primary_key=True, unique=True
), # Record ID for the membership row
sa.Column("channel_id", sa.Text(), nullable=False), # Associated channel
sa.Column("user_id", sa.Text(), nullable=False), # Associated user
sa.Column(
"created_at", sa.BigInteger(), nullable=True
), # Timestamp of when the user joined the channel
)
def downgrade():
# Revert 'type' column addition to the 'channel' table
op.drop_column('channel', 'type')
op.drop_column('message', 'parent_id')
op.drop_table('message_reaction')
op.drop_table('channel_member')
op.drop_column("channel", "type")
op.drop_column("message", "parent_id")
op.drop_table("message_reaction")
op.drop_table("channel_member")
@@ -6,73 +6,69 @@ Create Date: 2025-11-17 03:45:25.123939
"""
import json
import time
import uuid
import time
import json
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '37f288994c47'
down_revision: Union[str, None] = 'a5c220713937'
revision: str = "37f288994c47"
down_revision: Union[str, None] = "a5c220713937"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'group_member' in existing_tables:
return # Already created — skip everything
# 1. Create new table
op.create_table(
'group_member',
sa.Column('id', sa.Text(), primary_key=True, unique=True, nullable=False),
"group_member",
sa.Column("id", sa.Text(), primary_key=True, unique=True, nullable=False),
sa.Column(
'group_id',
"group_id",
sa.Text(),
sa.ForeignKey('group.id', ondelete='CASCADE'),
sa.ForeignKey("group.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
'user_id',
"user_id",
sa.Text(),
sa.ForeignKey('user.id', ondelete='CASCADE'),
sa.ForeignKey("user.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column('created_at', sa.BigInteger(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
sa.UniqueConstraint('group_id', 'user_id', name='uq_group_member_group_user'),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
sa.UniqueConstraint("group_id", "user_id", name="uq_group_member_group_user"),
)
connection = op.get_bind()
# 2. Read existing group with user_ids JSON column
group_table = sa.Table(
'group',
"group",
sa.MetaData(),
sa.Column('id', sa.Text()),
sa.Column('user_ids', sa.JSON()), # JSON stored as text in SQLite + PG
sa.Column("id", sa.Text()),
sa.Column("user_ids", sa.JSON()), # JSON stored as text in SQLite + PG
)
results = connection.execute(sa.select(group_table.c.id, group_table.c.user_ids)).fetchall()
results = connection.execute(
sa.select(group_table.c.id, group_table.c.user_ids)
).fetchall()
print(results)
# 3. Insert members into group_member table
gm_table = sa.Table(
'group_member',
"group_member",
sa.MetaData(),
sa.Column('id', sa.Text()),
sa.Column('group_id', sa.Text()),
sa.Column('user_id', sa.Text()),
sa.Column('created_at', sa.BigInteger()),
sa.Column('updated_at', sa.BigInteger()),
sa.Column("id", sa.Text()),
sa.Column("group_id", sa.Text()),
sa.Column("user_id", sa.Text()),
sa.Column("created_at", sa.BigInteger()),
sa.Column("updated_at", sa.BigInteger()),
)
now = int(time.time())
@@ -91,11 +87,11 @@ def upgrade() -> None:
rows = [
{
'id': str(uuid.uuid4()),
'group_id': group_id,
'user_id': uid,
'created_at': now,
'updated_at': now,
"id": str(uuid.uuid4()),
"group_id": group_id,
"user_id": uid,
"created_at": now,
"updated_at": now,
}
for uid in user_ids
]
@@ -104,41 +100,47 @@ def upgrade() -> None:
connection.execute(gm_table.insert(), rows)
# 4. Optionally drop the old column
with op.batch_alter_table('group') as batch:
batch.drop_column('user_ids')
with op.batch_alter_table("group") as batch:
batch.drop_column("user_ids")
def downgrade():
# Reverse: restore user_ids column
with op.batch_alter_table('group') as batch:
batch.add_column(sa.Column('user_ids', sa.JSON()))
with op.batch_alter_table("group") as batch:
batch.add_column(sa.Column("user_ids", sa.JSON()))
connection = op.get_bind()
gm_table = sa.Table(
'group_member',
"group_member",
sa.MetaData(),
sa.Column('group_id', sa.Text()),
sa.Column('user_id', sa.Text()),
sa.Column('created_at', sa.BigInteger()),
sa.Column('updated_at', sa.BigInteger()),
sa.Column("group_id", sa.Text()),
sa.Column("user_id", sa.Text()),
sa.Column("created_at", sa.BigInteger()),
sa.Column("updated_at", sa.BigInteger()),
)
group_table = sa.Table(
'group',
"group",
sa.MetaData(),
sa.Column('id', sa.Text()),
sa.Column('user_ids', sa.JSON()),
sa.Column("id", sa.Text()),
sa.Column("user_ids", sa.JSON()),
)
# Build JSON arrays again
results = connection.execute(sa.select(group_table.c.id)).fetchall()
for (group_id,) in results:
members = connection.execute(sa.select(gm_table.c.user_id).where(gm_table.c.group_id == group_id)).fetchall()
members = connection.execute(
sa.select(gm_table.c.user_id).where(gm_table.c.group_id == group_id)
).fetchall()
member_ids = [m[0] for m in members]
connection.execute(group_table.update().where(group_table.c.id == group_id).values(user_ids=member_ids))
connection.execute(
group_table.update()
.where(group_table.c.id == group_id)
.values(user_ids=member_ids)
)
# Drop the new table
op.drop_table('group_member')
op.drop_table("group_member")
@@ -8,57 +8,45 @@ Create Date: 2025-09-08 14:19:59.583921
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '38d63c18f30f'
down_revision: Union[str, None] = '3af16a1c9fb6'
revision: str = "38d63c18f30f"
down_revision: Union[str, None] = "3af16a1c9fb6"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
existing_tables = set(inspector.get_table_names())
# ── Create oauth_session table (idempotent) ───────────────────────
if 'oauth_session' not in existing_tables:
op.create_table(
'oauth_session',
sa.Column('id', sa.Text(), primary_key=True, nullable=False, unique=True),
sa.Column(
'user_id',
sa.Text(),
sa.ForeignKey('user.id', ondelete='CASCADE'),
nullable=False,
),
sa.Column('provider', sa.Text(), nullable=False),
sa.Column('token', sa.Text(), nullable=False),
sa.Column('expires_at', sa.BigInteger(), nullable=False),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
)
# Create indexes (idempotent — no-ops when table was just created
# with the columns above, and safe to call if indexes already exist).
existing_indexes = (
{idx['name'] for idx in inspector.get_indexes('oauth_session')} if 'oauth_session' in existing_tables else set()
# Create oauth_session table
op.create_table(
"oauth_session",
sa.Column("id", sa.Text(), nullable=False),
sa.Column("user_id", sa.Text(), nullable=False),
sa.Column("provider", sa.Text(), nullable=False),
sa.Column("token", sa.Text(), nullable=False),
sa.Column("expires_at", sa.BigInteger(), nullable=False),
sa.Column("created_at", sa.BigInteger(), nullable=False),
sa.Column("updated_at", sa.BigInteger(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.ForeignKeyConstraint(["user_id"], ["user.id"], ondelete="CASCADE"),
)
if 'idx_oauth_session_user_id' not in existing_indexes:
op.create_index('idx_oauth_session_user_id', 'oauth_session', ['user_id'])
if 'idx_oauth_session_expires_at' not in existing_indexes:
op.create_index('idx_oauth_session_expires_at', 'oauth_session', ['expires_at'])
if 'idx_oauth_session_user_provider' not in existing_indexes:
op.create_index('idx_oauth_session_user_provider', 'oauth_session', ['user_id', 'provider'])
# Create indexes for better performance
op.create_index("idx_oauth_session_user_id", "oauth_session", ["user_id"])
op.create_index("idx_oauth_session_expires_at", "oauth_session", ["expires_at"])
op.create_index(
"idx_oauth_session_user_provider", "oauth_session", ["user_id", "provider"]
)
def downgrade() -> None:
# Drop indexes first
op.drop_index('idx_oauth_session_user_provider', table_name='oauth_session')
op.drop_index('idx_oauth_session_expires_at', table_name='oauth_session')
op.drop_index('idx_oauth_session_user_id', table_name='oauth_session')
op.drop_index("idx_oauth_session_user_provider", table_name="oauth_session")
op.drop_index("idx_oauth_session_expires_at", table_name="oauth_session")
op.drop_index("idx_oauth_session_user_id", table_name="oauth_session")
# Drop the table
op.drop_table('oauth_session')
op.drop_table("oauth_session")
@@ -6,15 +6,15 @@ Create Date: 2024-10-09 21:02:35.241684
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.sql import table, select, update, column
from sqlalchemy.engine.reflection import Inspector
import json
import sqlalchemy as sa
from alembic import op
from sqlalchemy.engine.reflection import Inspector
from sqlalchemy.sql import column, select, table, update
revision = '3ab32c4b8f59'
down_revision = '1af9b942657b'
revision = "3ab32c4b8f59"
down_revision = "1af9b942657b"
branch_labels = None
depends_on = None
@@ -24,55 +24,58 @@ def upgrade():
inspector = Inspector.from_engine(conn)
# Inspecting the 'tag' table constraints and structure
existing_pk = inspector.get_pk_constraint('tag')
unique_constraints = inspector.get_unique_constraints('tag')
existing_indexes = inspector.get_indexes('tag')
existing_pk = inspector.get_pk_constraint("tag")
unique_constraints = inspector.get_unique_constraints("tag")
existing_indexes = inspector.get_indexes("tag")
print(f'Primary Key: {existing_pk}')
print(f'Unique Constraints: {unique_constraints}')
print(f'Indexes: {existing_indexes}')
print(f"Primary Key: {existing_pk}")
print(f"Unique Constraints: {unique_constraints}")
print(f"Indexes: {existing_indexes}")
with op.batch_alter_table('tag', schema=None) as batch_op:
with op.batch_alter_table("tag", schema=None) as batch_op:
# Drop existing primary key constraint if it exists
if existing_pk and existing_pk.get('constrained_columns'):
pk_name = existing_pk.get('name')
if existing_pk and existing_pk.get("constrained_columns"):
pk_name = existing_pk.get("name")
if pk_name:
print(f'Dropping primary key constraint: {pk_name}')
batch_op.drop_constraint(pk_name, type_='primary')
print(f"Dropping primary key constraint: {pk_name}")
batch_op.drop_constraint(pk_name, type_="primary")
# Now create the new primary key with the combination of 'id' and 'user_id'
print("Creating new primary key with 'id' and 'user_id'.")
batch_op.create_primary_key('pk_id_user_id', ['id', 'user_id'])
batch_op.create_primary_key("pk_id_user_id", ["id", "user_id"])
# Drop unique constraints that could conflict with the new primary key
for constraint in unique_constraints:
if (
constraint['name'] == 'uq_id_user_id'
constraint["name"] == "uq_id_user_id"
): # Adjust this name according to what is actually returned by the inspector
print(f'Dropping unique constraint: {constraint["name"]}')
batch_op.drop_constraint(constraint['name'], type_='unique')
print(f"Dropping unique constraint: {constraint['name']}")
batch_op.drop_constraint(constraint["name"], type_="unique")
for index in existing_indexes:
if index['unique']:
if not any(constraint['name'] == index['name'] for constraint in unique_constraints):
if index["unique"]:
if not any(
constraint["name"] == index["name"]
for constraint in unique_constraints
):
# You are attempting to drop unique indexes
print(f'Dropping unique index: {index["name"]}')
batch_op.drop_index(index['name'])
print(f"Dropping unique index: {index['name']}")
batch_op.drop_index(index["name"])
def downgrade():
conn = op.get_bind()
inspector = Inspector.from_engine(conn)
current_pk = inspector.get_pk_constraint('tag')
current_pk = inspector.get_pk_constraint("tag")
with op.batch_alter_table('tag', schema=None) as batch_op:
with op.batch_alter_table("tag", schema=None) as batch_op:
# Drop the current primary key first, if it matches the one we know we added in upgrade
if current_pk and 'pk_id_user_id' == current_pk.get('name'):
batch_op.drop_constraint('pk_id_user_id', type_='primary')
if current_pk and "pk_id_user_id" == current_pk.get("name"):
batch_op.drop_constraint("pk_id_user_id", type_="primary")
# Restore the original primary key
batch_op.create_primary_key('pk_id', ['id'])
batch_op.create_primary_key("pk_id", ["id"])
# Since primary key on just 'id' is restored, we now add back any unique constraints if necessary
batch_op.create_unique_constraint('uq_id_user_id', ['id', 'user_id'])
batch_op.create_unique_constraint("uq_id_user_id", ["id", "user_id"])
@@ -8,33 +8,25 @@ Create Date: 2025-08-21 02:07:18.078283
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '3af16a1c9fb6'
down_revision: Union[str, None] = '018012973d35'
revision: str = "3af16a1c9fb6"
down_revision: Union[str, None] = "018012973d35"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
user_cols = {c['name'] for c in inspector.get_columns('user')}
if 'username' not in user_cols:
op.add_column('user', sa.Column('username', sa.String(length=50), nullable=True))
if 'bio' not in user_cols:
op.add_column('user', sa.Column('bio', sa.Text(), nullable=True))
if 'gender' not in user_cols:
op.add_column('user', sa.Column('gender', sa.Text(), nullable=True))
if 'date_of_birth' not in user_cols:
op.add_column('user', sa.Column('date_of_birth', sa.Date(), nullable=True))
op.add_column("user", sa.Column("username", sa.String(length=50), nullable=True))
op.add_column("user", sa.Column("bio", sa.Text(), nullable=True))
op.add_column("user", sa.Column("gender", sa.Text(), nullable=True))
op.add_column("user", sa.Column("date_of_birth", sa.Date(), nullable=True))
def downgrade() -> None:
op.drop_column('user', 'username')
op.drop_column('user', 'bio')
op.drop_column('user', 'gender')
op.drop_column('user', 'date_of_birth')
op.drop_column("user", "username")
op.drop_column("user", "bio")
op.drop_column("user", "gender")
op.drop_column("user", "date_of_birth")
@@ -1,72 +0,0 @@
"""add knowledge_directory table
Revision ID: 3c9b0ca343fd
Revises: a0b1c2d3e4f5
Create Date: 2026-05-13 21:58:40.832482
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '3c9b0ca343fd'
down_revision: Union[str, None] = 'a0b1c2d3e4f5'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'knowledge_directory' not in existing_tables:
# Create knowledge_directory table
op.create_table(
'knowledge_directory',
sa.Column('id', sa.Text(), nullable=False),
sa.Column('knowledge_id', sa.Text(), nullable=False),
sa.Column('parent_id', sa.Text(), nullable=True),
sa.Column('name', sa.Text(), nullable=False),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
sa.ForeignKeyConstraint(['knowledge_id'], ['knowledge.id'], ondelete='CASCADE'),
sa.ForeignKeyConstraint(['parent_id'], ['knowledge_directory.id'], ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id'),
sa.UniqueConstraint(
'knowledge_id', 'parent_id', 'name', name='uq_knowledge_directory_knowledge_parent_name'
),
)
op.create_index('ix_knowledge_directory_knowledge_id', 'knowledge_directory', ['knowledge_id'])
op.create_index('ix_knowledge_directory_parent_id', 'knowledge_directory', ['parent_id'])
# Add directory_id column to knowledge_file
kf_cols = {c['name'] for c in inspector.get_columns('knowledge_file')}
if 'directory_id' not in kf_cols:
with op.batch_alter_table('knowledge_file') as batch:
batch.add_column(sa.Column('directory_id', sa.Text(), nullable=True))
batch.create_foreign_key(
'fk_knowledge_file_directory_id',
'knowledge_directory',
['directory_id'],
['id'],
ondelete='SET NULL',
)
batch.create_index('ix_knowledge_file_directory_id', ['directory_id'])
def downgrade() -> None:
# Remove directory_id from knowledge_file
with op.batch_alter_table('knowledge_file') as batch:
batch.drop_index('ix_knowledge_file_directory_id')
batch.drop_constraint('fk_knowledge_file_directory_id', type_='foreignkey')
batch.drop_column('directory_id')
# Drop knowledge_directory table
op.drop_index('ix_knowledge_directory_parent_id', table_name='knowledge_directory')
op.drop_index('ix_knowledge_directory_knowledge_id', table_name='knowledge_directory')
op.drop_table('knowledge_directory')
@@ -1,167 +0,0 @@
"""Add knowledge_file table
Revision ID: 3e0e00844bb0
Revises: 90ef40d4714e
Create Date: 2025-12-02 06:54:19.401334
"""
import json
import time
import uuid
from typing import Sequence, Union
import open_webui.internal.db
import sqlalchemy as sa
from alembic import op
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision: str = '3e0e00844bb0'
down_revision: Union[str, None] = '90ef40d4714e'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'knowledge_file' in existing_tables:
return # Already created — skip everything
op.create_table(
'knowledge_file',
sa.Column('id', sa.Text(), primary_key=True),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column(
'knowledge_id',
sa.Text(),
sa.ForeignKey('knowledge.id', ondelete='CASCADE'),
nullable=False,
),
sa.Column(
'file_id',
sa.Text(),
sa.ForeignKey('file.id', ondelete='CASCADE'),
nullable=False,
),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
# indexes
sa.Index('ix_knowledge_file_knowledge_id', 'knowledge_id'),
sa.Index('ix_knowledge_file_file_id', 'file_id'),
sa.Index('ix_knowledge_file_user_id', 'user_id'),
# unique constraints
sa.UniqueConstraint(
'knowledge_id', 'file_id', name='uq_knowledge_file_knowledge_file'
), # prevent duplicate entries
)
connection = op.get_bind()
# 2. Read existing group with user_ids JSON column
knowledge_table = sa.Table(
'knowledge',
sa.MetaData(),
sa.Column('id', sa.Text()),
sa.Column('user_id', sa.Text()),
sa.Column('data', sa.JSON()), # JSON stored as text in SQLite + PG
)
results = connection.execute(
sa.select(knowledge_table.c.id, knowledge_table.c.user_id, knowledge_table.c.data)
).fetchall()
# 3. Insert members into group_member table
kf_table = sa.Table(
'knowledge_file',
sa.MetaData(),
sa.Column('id', sa.Text()),
sa.Column('user_id', sa.Text()),
sa.Column('knowledge_id', sa.Text()),
sa.Column('file_id', sa.Text()),
sa.Column('created_at', sa.BigInteger()),
sa.Column('updated_at', sa.BigInteger()),
)
file_table = sa.Table(
'file',
sa.MetaData(),
sa.Column('id', sa.Text()),
)
now = int(time.time())
for knowledge_id, user_id, data in results:
if not data:
continue
if isinstance(data, str):
try:
data = json.loads(data)
except Exception:
continue # skip invalid JSON
if not isinstance(data, dict):
continue
file_ids = data.get('file_ids', [])
for file_id in file_ids:
file_exists = connection.execute(sa.select(file_table.c.id).where(file_table.c.id == file_id)).fetchone()
if not file_exists:
continue # skip non-existing files
row = {
'id': str(uuid.uuid4()),
'user_id': user_id,
'knowledge_id': knowledge_id,
'file_id': file_id,
'created_at': now,
'updated_at': now,
}
connection.execute(kf_table.insert().values(**row))
with op.batch_alter_table('knowledge') as batch:
batch.drop_column('data')
def downgrade() -> None:
# 1. Add back the old data column
op.add_column('knowledge', sa.Column('data', sa.JSON(), nullable=True))
connection = op.get_bind()
# 2. Read knowledge_file entries and reconstruct data JSON
knowledge_table = sa.Table(
'knowledge',
sa.MetaData(),
sa.Column('id', sa.Text()),
sa.Column('data', sa.JSON()),
)
kf_table = sa.Table(
'knowledge_file',
sa.MetaData(),
sa.Column('id', sa.Text()),
sa.Column('knowledge_id', sa.Text()),
sa.Column('file_id', sa.Text()),
)
results = connection.execute(sa.select(knowledge_table.c.id)).fetchall()
for (knowledge_id,) in results:
file_ids = connection.execute(
sa.select(kf_table.c.file_id).where(kf_table.c.knowledge_id == knowledge_id)
).fetchall()
file_ids_list = [fid for (fid,) in file_ids]
data_json = {'file_ids': file_ids_list}
connection.execute(knowledge_table.update().where(knowledge_table.c.id == knowledge_id).values(data=data_json))
# 3. Drop the knowledge_file table
op.drop_table('knowledge_file')
@@ -1,580 +0,0 @@
"""reshape config to per key rows
Revision ID: 3ff2c63645b8
Revises: 461111b60977
Create Date: 2026-06-17 00:50:51.477073
"""
import json
import time
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = '3ff2c63645b8'
down_revision: Union[str, None] = '461111b60977'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
# Maps every dot-notation blob path to its legacy env/config key name.
# Built from the legacy persistent config declarations in config.py.
BLOB_PATH_TO_KEY = {
"audio.stt.allowed_extensions": "AUDIO_STT_ALLOWED_EXTENSIONS",
"audio.stt.azure.api_key": "AUDIO_STT_AZURE_API_KEY",
"audio.stt.azure.base_url": "AUDIO_STT_AZURE_BASE_URL",
"audio.stt.azure.locales": "AUDIO_STT_AZURE_LOCALES",
"audio.stt.azure.max_speakers": "AUDIO_STT_AZURE_MAX_SPEAKERS",
"audio.stt.azure.region": "AUDIO_STT_AZURE_REGION",
"audio.stt.deepgram.api_key": "DEEPGRAM_API_KEY",
"audio.stt.engine": "AUDIO_STT_ENGINE",
"audio.stt.mistral.api_base_url": "AUDIO_STT_MISTRAL_API_BASE_URL",
"audio.stt.mistral.api_key": "AUDIO_STT_MISTRAL_API_KEY",
"audio.stt.mistral.use_chat_completions": "AUDIO_STT_MISTRAL_USE_CHAT_COMPLETIONS",
"audio.stt.model": "AUDIO_STT_MODEL",
"audio.stt.openai.api_base_url": "AUDIO_STT_OPENAI_API_BASE_URL",
"audio.stt.openai.api_key": "AUDIO_STT_OPENAI_API_KEY",
"audio.stt.supported_content_types": "AUDIO_STT_SUPPORTED_CONTENT_TYPES",
"audio.stt.whisper_model": "WHISPER_MODEL",
"audio.tts.api_key": "AUDIO_TTS_API_KEY",
"audio.tts.azure.speech_base_url": "AUDIO_TTS_AZURE_SPEECH_BASE_URL",
"audio.tts.azure.speech_output_format": "AUDIO_TTS_AZURE_SPEECH_OUTPUT_FORMAT",
"audio.tts.azure.speech_region": "AUDIO_TTS_AZURE_SPEECH_REGION",
"audio.tts.engine": "AUDIO_TTS_ENGINE",
"audio.tts.mistral.api_base_url": "AUDIO_TTS_MISTRAL_API_BASE_URL",
"audio.tts.mistral.api_key": "AUDIO_TTS_MISTRAL_API_KEY",
"audio.tts.model": "AUDIO_TTS_MODEL",
"audio.tts.openai.api_base_url": "AUDIO_TTS_OPENAI_API_BASE_URL",
"audio.tts.openai.api_key": "AUDIO_TTS_OPENAI_API_KEY",
"audio.tts.openai.params": "AUDIO_TTS_OPENAI_PARAMS",
"audio.tts.split_on": "AUDIO_TTS_SPLIT_ON",
"audio.tts.voice": "AUDIO_TTS_VOICE",
"auth.admin.email": "ADMIN_EMAIL",
"auth.admin.show": "SHOW_ADMIN_DETAILS",
"auth.api_key.allowed_endpoints": "API_KEYS_ALLOWED_ENDPOINTS",
"auth.api_key.endpoint_restrictions": "ENABLE_API_KEYS_ENDPOINT_RESTRICTIONS",
"auth.enable_api_keys": "ENABLE_API_KEYS",
"auth.jwt_expiry": "JWT_EXPIRES_IN",
"automations.enable": "ENABLE_AUTOMATIONS",
"automations.max_count": "AUTOMATION_MAX_COUNT",
"automations.min_interval": "AUTOMATION_MIN_INTERVAL",
"calendar.enable": "ENABLE_CALENDAR",
"channels.enable": "ENABLE_CHANNELS",
"code_execution.enable": "ENABLE_CODE_EXECUTION",
"code_execution.engine": "CODE_EXECUTION_ENGINE",
"code_execution.jupyter.auth": "CODE_EXECUTION_JUPYTER_AUTH",
"code_execution.jupyter.auth_password": "CODE_EXECUTION_JUPYTER_AUTH_PASSWORD",
"code_execution.jupyter.auth_token": "CODE_EXECUTION_JUPYTER_AUTH_TOKEN",
"code_execution.jupyter.timeout": "CODE_EXECUTION_JUPYTER_TIMEOUT",
"code_execution.jupyter.url": "CODE_EXECUTION_JUPYTER_URL",
"code_interpreter.enable": "ENABLE_CODE_INTERPRETER",
"code_interpreter.engine": "CODE_INTERPRETER_ENGINE",
"code_interpreter.jupyter.auth": "CODE_INTERPRETER_JUPYTER_AUTH",
"code_interpreter.jupyter.auth_password": "CODE_INTERPRETER_JUPYTER_AUTH_PASSWORD",
"code_interpreter.jupyter.auth_token": "CODE_INTERPRETER_JUPYTER_AUTH_TOKEN",
"code_interpreter.jupyter.timeout": "CODE_INTERPRETER_JUPYTER_TIMEOUT",
"code_interpreter.jupyter.url": "CODE_INTERPRETER_JUPYTER_URL",
"code_interpreter.prompt_template": "CODE_INTERPRETER_PROMPT_TEMPLATE",
"direct.enable": "ENABLE_DIRECT_CONNECTIONS",
"evaluation.arena.enable": "ENABLE_EVALUATION_ARENA_MODELS",
"evaluation.arena.models": "EVALUATION_ARENA_MODELS",
"file.image_compression_height": "FILE_IMAGE_COMPRESSION_HEIGHT",
"file.image_compression_width": "FILE_IMAGE_COMPRESSION_WIDTH",
"folders.enable": "ENABLE_FOLDERS",
"folders.max_file_count": "FOLDER_MAX_FILE_COUNT",
"google_drive.api_key": "GOOGLE_DRIVE_API_KEY",
"google_drive.client_id": "GOOGLE_DRIVE_CLIENT_ID",
"google_drive.enable": "ENABLE_GOOGLE_DRIVE_INTEGRATION",
"image_generation.automatic1111.api_auth": "AUTOMATIC1111_API_AUTH",
"image_generation.automatic1111.api_params": "AUTOMATIC1111_PARAMS",
"image_generation.automatic1111.base_url": "AUTOMATIC1111_BASE_URL",
"image_generation.comfyui.api_key": "COMFYUI_API_KEY",
"image_generation.comfyui.base_url": "COMFYUI_BASE_URL",
"image_generation.comfyui.nodes": "COMFYUI_WORKFLOW_NODES",
"image_generation.comfyui.workflow": "COMFYUI_WORKFLOW",
"image_generation.enable": "ENABLE_IMAGE_GENERATION",
"image_generation.engine": "IMAGE_GENERATION_ENGINE",
"image_generation.gemini.api_base_url": "IMAGES_GEMINI_API_BASE_URL",
"image_generation.gemini.api_key": "IMAGES_GEMINI_API_KEY",
"image_generation.gemini.endpoint_method": "IMAGES_GEMINI_ENDPOINT_METHOD",
"image_generation.model": "IMAGE_GENERATION_MODEL",
"image_generation.openai.api_base_url": "IMAGES_OPENAI_API_BASE_URL",
"image_generation.openai.api_key": "IMAGES_OPENAI_API_KEY",
"image_generation.openai.api_version": "IMAGES_OPENAI_API_VERSION",
"image_generation.openai.params": "IMAGES_OPENAI_API_PARAMS",
"image_generation.prompt.enable": "ENABLE_IMAGE_PROMPT_GENERATION",
"image_generation.size": "IMAGE_SIZE",
"image_generation.steps": "IMAGE_STEPS",
"images.edit.comfyui.api_key": "IMAGES_EDIT_COMFYUI_API_KEY",
"images.edit.comfyui.base_url": "IMAGES_EDIT_COMFYUI_BASE_URL",
"images.edit.comfyui.nodes": "IMAGES_EDIT_COMFYUI_WORKFLOW_NODES",
"images.edit.comfyui.workflow": "IMAGES_EDIT_COMFYUI_WORKFLOW",
"images.edit.enable": "ENABLE_IMAGE_EDIT",
"images.edit.engine": "IMAGE_EDIT_ENGINE",
"images.edit.gemini.api_base_url": "IMAGES_EDIT_GEMINI_API_BASE_URL",
"images.edit.gemini.api_key": "IMAGES_EDIT_GEMINI_API_KEY",
"images.edit.model": "IMAGE_EDIT_MODEL",
"images.edit.openai.api_base_url": "IMAGES_EDIT_OPENAI_API_BASE_URL",
"images.edit.openai.api_key": "IMAGES_EDIT_OPENAI_API_KEY",
"images.edit.openai.api_version": "IMAGES_EDIT_OPENAI_API_VERSION",
"images.edit.size": "IMAGE_EDIT_SIZE",
"ldap.enable": "ENABLE_LDAP",
"ldap.group.enable_creation": "ENABLE_LDAP_GROUP_CREATION",
"ldap.group.enable_management": "ENABLE_LDAP_GROUP_MANAGEMENT",
"ldap.server.app_dn": "LDAP_APP_DN",
"ldap.server.app_password": "LDAP_APP_PASSWORD",
"ldap.server.attribute_for_groups": "LDAP_ATTRIBUTE_FOR_GROUPS",
"ldap.server.attribute_for_mail": "LDAP_ATTRIBUTE_FOR_MAIL",
"ldap.server.attribute_for_username": "LDAP_ATTRIBUTE_FOR_USERNAME",
"ldap.server.ca_cert_file": "LDAP_CA_CERT_FILE",
"ldap.server.ciphers": "LDAP_CIPHERS",
"ldap.server.host": "LDAP_SERVER_HOST",
"ldap.server.label": "LDAP_SERVER_LABEL",
"ldap.server.port": "LDAP_SERVER_PORT",
"ldap.server.search_filter": "LDAP_SEARCH_FILTER",
"ldap.server.use_tls": "LDAP_USE_TLS",
"ldap.server.users_dn": "LDAP_SEARCH_BASE",
"ldap.server.validate_cert": "LDAP_VALIDATE_CERT",
"memories.enable": "ENABLE_MEMORIES",
"models.base_models_cache": "ENABLE_BASE_MODELS_CACHE",
"models.default_metadata": "DEFAULT_MODEL_METADATA",
"models.default_params": "DEFAULT_MODEL_PARAMS",
"notes.enable": "ENABLE_NOTES",
# OAuth — direct paths
"oauth.admin_roles": "OAUTH_ADMIN_ROLES",
"oauth.allowed_domains": "OAUTH_ALLOWED_DOMAINS",
"oauth.allowed_roles": "OAUTH_ALLOWED_ROLES",
"oauth.audience": "OAUTH_AUDIENCE",
"oauth.auto_redirect": "OAUTH_AUTO_REDIRECT",
"oauth.blocked_groups": "OAUTH_BLOCKED_GROUPS",
"oauth.client.timeout": "OAUTH_CLIENT_TIMEOUT",
"oauth.enable_group_creation": "ENABLE_OAUTH_GROUP_CREATION",
"oauth.enable_group_mapping": "ENABLE_OAUTH_GROUP_MANAGEMENT",
"oauth.enable_role_mapping": "ENABLE_OAUTH_ROLE_MANAGEMENT",
"oauth.enable_signup": "ENABLE_OAUTH_SIGNUP",
"oauth.group_default_share": "OAUTH_GROUP_DEFAULT_SHARE",
"oauth.merge_accounts_by_email": "OAUTH_MERGE_ACCOUNTS_BY_EMAIL",
"oauth.refresh_token_include_scope": "OAUTH_REFRESH_TOKEN_INCLUDE_SCOPE",
"oauth.roles_claim": "OAUTH_ROLES_CLAIM",
"oauth.update_email_on_login": "OAUTH_UPDATE_EMAIL_ON_LOGIN",
"oauth.update_name_on_login": "OAUTH_UPDATE_NAME_ON_LOGIN",
"oauth.update_picture_on_login": "OAUTH_UPDATE_PICTURE_ON_LOGIN",
# OAuth — generic provider paths
"oauth.client_id": "OAUTH_CLIENT_ID",
"oauth.client_secret": "OAUTH_CLIENT_SECRET",
"oauth.code_challenge_method": "OAUTH_CODE_CHALLENGE_METHOD",
"oauth.email_claim": "OAUTH_EMAIL_CLAIM",
"oauth.end_session_endpoint": "OPENID_END_SESSION_ENDPOINT",
"oauth.group_claim": "OAUTH_GROUP_CLAIM",
"oauth.picture_claim": "OAUTH_PICTURE_CLAIM",
"oauth.provider_name": "OAUTH_PROVIDER_NAME",
"oauth.provider_url": "OPENID_PROVIDER_URL",
"oauth.redirect_uri": "OPENID_REDIRECT_URI",
"oauth.scopes": "OAUTH_SCOPES",
"oauth.sub_claim": "OAUTH_SUB_CLAIM",
"oauth.timeout": "OAUTH_TIMEOUT",
"oauth.token_endpoint_auth_method": "OAUTH_TOKEN_ENDPOINT_AUTH_METHOD",
"oauth.username_claim": "OAUTH_USERNAME_CLAIM",
# OAuth — OIDC nested paths (flattened)
"oauth.oidc.avatar_claim": "OAUTH_PICTURE_CLAIM",
"oauth.oidc.client_id": "OAUTH_CLIENT_ID",
"oauth.oidc.client_secret": "OAUTH_CLIENT_SECRET",
"oauth.oidc.code_challenge_method": "OAUTH_CODE_CHALLENGE_METHOD",
"oauth.oidc.email_claim": "OAUTH_EMAIL_CLAIM",
"oauth.oidc.end_session_endpoint": "OPENID_END_SESSION_ENDPOINT",
"oauth.oidc.group_claim": "OAUTH_GROUP_CLAIM", # renamed from OAUTH_GROUPS_CLAIM
"oauth.oidc.oauth_timeout": "OAUTH_TIMEOUT",
"oauth.oidc.provider_name": "OAUTH_PROVIDER_NAME",
"oauth.oidc.provider_url": "OPENID_PROVIDER_URL",
"oauth.oidc.redirect_uri": "OPENID_REDIRECT_URI",
"oauth.oidc.scopes": "OAUTH_SCOPES",
"oauth.oidc.sub_claim": "OAUTH_SUB_CLAIM",
"oauth.oidc.token_endpoint_auth_method": "OAUTH_TOKEN_ENDPOINT_AUTH_METHOD",
"oauth.oidc.username_claim": "OAUTH_USERNAME_CLAIM",
# OAuth — provider-specific
"oauth.feishu.client_id": "FEISHU_CLIENT_ID",
"oauth.feishu.client_secret": "FEISHU_CLIENT_SECRET",
"oauth.feishu.redirect_uri": "FEISHU_REDIRECT_URI",
"oauth.feishu.scope": "FEISHU_OAUTH_SCOPE",
"oauth.github.client_id": "GITHUB_CLIENT_ID",
"oauth.github.client_secret": "GITHUB_CLIENT_SECRET",
"oauth.github.redirect_uri": "GITHUB_CLIENT_REDIRECT_URI",
"oauth.github.scope": "GITHUB_CLIENT_SCOPE",
"oauth.google.client_id": "GOOGLE_CLIENT_ID",
"oauth.google.client_secret": "GOOGLE_CLIENT_SECRET",
"oauth.google.redirect_uri": "GOOGLE_REDIRECT_URI",
"oauth.google.scope": "GOOGLE_OAUTH_SCOPE",
"oauth.microsoft.client_id": "MICROSOFT_CLIENT_ID",
"oauth.microsoft.client_secret": "MICROSOFT_CLIENT_SECRET",
"oauth.microsoft.login_base_url": "MICROSOFT_CLIENT_LOGIN_BASE_URL",
"oauth.microsoft.picture_url": "MICROSOFT_CLIENT_PICTURE_URL",
"oauth.microsoft.redirect_uri": "MICROSOFT_REDIRECT_URI",
"oauth.microsoft.scope": "MICROSOFT_OAUTH_SCOPE",
"oauth.microsoft.tenant_id": "MICROSOFT_CLIENT_TENANT_ID",
# Ollama / OpenAI
"ollama.api_configs": "OLLAMA_API_CONFIGS",
"ollama.base_urls": "OLLAMA_BASE_URLS",
"ollama.enable": "ENABLE_OLLAMA_API",
"onedrive.enable": "ENABLE_ONEDRIVE_INTEGRATION",
"onedrive.sharepoint_tenant_id": "ONEDRIVE_SHAREPOINT_TENANT_ID",
"onedrive.sharepoint_url": "ONEDRIVE_SHAREPOINT_URL",
"openai.api_base_urls": "OPENAI_API_BASE_URLS",
"openai.api_configs": "OPENAI_API_CONFIGS",
"openai.api_keys": "OPENAI_API_KEYS",
"openai.enable": "ENABLE_OPENAI_API",
# RAG
"rag.content_extraction_engine": "CONTENT_EXTRACTION_ENGINE",
"rag.datalab_marker_use_llm": "DATALAB_MARKER_USE_LLM",
"rag.mistral_ocr_api_base_url": "MISTRAL_OCR_API_BASE_URL",
"rag.azure_openai.api_key": "RAG_AZURE_OPENAI_API_KEY",
"rag.azure_openai.api_version": "RAG_AZURE_OPENAI_API_VERSION",
"rag.azure_openai.base_url": "RAG_AZURE_OPENAI_BASE_URL",
"rag.bypass_embedding_and_retrieval": "BYPASS_EMBEDDING_AND_RETRIEVAL",
"rag.chunk_min_size_target": "CHUNK_MIN_SIZE_TARGET",
"rag.chunk_overlap": "CHUNK_OVERLAP",
"rag.chunk_size": "CHUNK_SIZE",
"rag.datalab_marker_additional_config": "DATALAB_MARKER_ADDITIONAL_CONFIG",
"rag.datalab_marker_api_base_url": "DATALAB_MARKER_API_BASE_URL",
"rag.datalab_marker_api_key": "DATALAB_MARKER_API_KEY",
"rag.datalab_marker_disable_image_extraction": "DATALAB_MARKER_DISABLE_IMAGE_EXTRACTION",
"rag.datalab_marker_force_ocr": "DATALAB_MARKER_FORCE_OCR",
"rag.datalab_marker_format_lines": "DATALAB_MARKER_FORMAT_LINES",
"rag.datalab_marker_output_format": "DATALAB_MARKER_OUTPUT_FORMAT",
"rag.datalab_marker_paginate": "DATALAB_MARKER_PAGINATE",
"rag.datalab_marker_skip_cache": "DATALAB_MARKER_SKIP_CACHE",
"rag.datalab_marker_strip_existing_ocr": "DATALAB_MARKER_STRIP_EXISTING_OCR",
"rag.docling_api_key": "DOCLING_API_KEY",
"rag.docling_params": "DOCLING_PARAMS",
"rag.docling_server_url": "DOCLING_SERVER_URL",
"rag.document_intelligence_endpoint": "DOCUMENT_INTELLIGENCE_ENDPOINT",
"rag.document_intelligence_key": "DOCUMENT_INTELLIGENCE_KEY",
"rag.document_intelligence_model": "DOCUMENT_INTELLIGENCE_MODEL",
"rag.embedding_batch_size": "RAG_EMBEDDING_BATCH_SIZE",
"rag.embedding_concurrent_requests": "RAG_EMBEDDING_CONCURRENT_REQUESTS",
"rag.embedding_engine": "RAG_EMBEDDING_ENGINE",
"rag.embedding_model": "RAG_EMBEDDING_MODEL",
"rag.enable_async_embedding": "ENABLE_ASYNC_EMBEDDING",
"rag.enable_hybrid_search": "ENABLE_RAG_HYBRID_SEARCH",
"rag.enable_hybrid_search_enriched_texts": "ENABLE_RAG_HYBRID_SEARCH_ENRICHED_TEXTS",
"rag.enable_markdown_header_text_splitter": "ENABLE_MARKDOWN_HEADER_TEXT_SPLITTER",
"rag.external_document_loader_api_key": "EXTERNAL_DOCUMENT_LOADER_API_KEY",
"rag.external_document_loader_url": "EXTERNAL_DOCUMENT_LOADER_URL",
"rag.external_reranker_api_key": "RAG_EXTERNAL_RERANKER_API_KEY",
"rag.external_reranker_timeout": "RAG_EXTERNAL_RERANKER_TIMEOUT",
"rag.external_reranker_url": "RAG_EXTERNAL_RERANKER_URL",
"rag.file.allowed_extensions": "RAG_ALLOWED_FILE_EXTENSIONS",
"rag.file.max_count": "RAG_FILE_MAX_COUNT",
"rag.file.max_size": "RAG_FILE_MAX_SIZE",
"rag.full_context": "RAG_FULL_CONTEXT",
"rag.hybrid_bm25_weight": "RAG_HYBRID_BM25_WEIGHT",
"rag.mineru_api_key": "MINERU_API_KEY",
"rag.mineru_api_mode": "MINERU_API_MODE",
"rag.mineru_api_timeout": "MINERU_API_TIMEOUT",
"rag.mineru_api_url": "MINERU_API_URL",
"rag.mineru_file_extensions": "MINERU_FILE_EXTENSIONS",
"rag.mineru_params": "MINERU_PARAMS",
"rag.mistral_ocr_api_key": "MISTRAL_OCR_API_KEY",
"rag.ollama.key": "RAG_OLLAMA_API_KEY",
"rag.ollama.url": "RAG_OLLAMA_BASE_URL",
"rag.openai_api_base_url": "RAG_OPENAI_API_BASE_URL",
"rag.openai_api_key": "RAG_OPENAI_API_KEY",
"rag.paddleocr_vl_base_url": "PADDLEOCR_VL_BASE_URL",
"rag.paddleocr_vl_token": "PADDLEOCR_VL_TOKEN",
"rag.pdf_extract_images": "PDF_EXTRACT_IMAGES",
"rag.pdf_loader_mode": "PDF_LOADER_MODE",
"rag.relevance_threshold": "RAG_RELEVANCE_THRESHOLD",
"rag.reranking_batch_size": "RAG_RERANKING_BATCH_SIZE",
"rag.reranking_engine": "RAG_RERANKING_ENGINE",
"rag.reranking_model": "RAG_RERANKING_MODEL",
"rag.template": "RAG_TEMPLATE",
"rag.text_splitter": "RAG_TEXT_SPLITTER",
"rag.tika_server_url": "TIKA_SERVER_URL",
"rag.tiktoken_encoding_name": "TIKTOKEN_ENCODING_NAME",
"rag.top_k": "RAG_TOP_K",
"rag.top_k_reranker": "RAG_TOP_K_RERANKER",
# RAG — Web
"rag.web.fetch.max_content_length": "WEB_FETCH_MAX_CONTENT_LENGTH",
"rag.web.loader.concurrent_requests": "WEB_LOADER_CONCURRENT_REQUESTS",
"rag.web.loader.engine": "WEB_LOADER_ENGINE",
"rag.web.loader.external_web_loader_api_key": "EXTERNAL_WEB_LOADER_API_KEY",
"rag.web.loader.external_web_loader_url": "EXTERNAL_WEB_LOADER_URL",
"rag.web.loader.firecrawl_api_key": "FIRECRAWL_API_KEY",
"rag.web.loader.firecrawl_api_url": "FIRECRAWL_API_BASE_URL",
"rag.web.loader.firecrawl_timeout": "FIRECRAWL_TIMEOUT",
"rag.web.loader.playwright_timeout": "PLAYWRIGHT_TIMEOUT",
"rag.web.loader.playwright_ws_url": "PLAYWRIGHT_WS_URL",
"rag.web.loader.ssl_verification": "ENABLE_WEB_LOADER_SSL_VERIFICATION",
"rag.web.loader.timeout": "WEB_LOADER_TIMEOUT",
"rag.web.search.azure_ai_search_api_key": "AZURE_AI_SEARCH_API_KEY",
"rag.web.search.azure_ai_search_endpoint": "AZURE_AI_SEARCH_ENDPOINT",
"rag.web.search.azure_ai_search_index_name": "AZURE_AI_SEARCH_INDEX_NAME",
"rag.web.search.bing_search_v7_endpoint": "BING_SEARCH_V7_ENDPOINT",
"rag.web.search.bing_search_v7_subscription_key": "BING_SEARCH_V7_SUBSCRIPTION_KEY",
"rag.web.search.bocha_search_api_key": "BOCHA_SEARCH_API_KEY",
"rag.web.search.brave_search_api_key": "BRAVE_SEARCH_API_KEY",
"rag.web.search.brave_search_context_tokens": "BRAVE_SEARCH_CONTEXT_TOKENS",
"rag.web.search.bypass_embedding_and_retrieval": "BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL",
"rag.web.search.bypass_web_loader": "BYPASS_WEB_SEARCH_WEB_LOADER",
"rag.web.search.concurrent_requests": "WEB_SEARCH_CONCURRENT_REQUESTS",
"rag.web.search.ddgs_backend": "DDGS_BACKEND",
"rag.web.search.domain.filter_list": "WEB_SEARCH_DOMAIN_FILTER_LIST",
"rag.web.search.enable": "ENABLE_WEB_SEARCH",
"rag.web.search.engine": "WEB_SEARCH_ENGINE",
"rag.web.search.exa_api_key": "EXA_API_KEY",
"rag.web.search.external_web_search_api_key": "EXTERNAL_WEB_SEARCH_API_KEY",
"rag.web.search.external_web_search_url": "EXTERNAL_WEB_SEARCH_URL",
"rag.web.search.google_pse_api_key": "GOOGLE_PSE_API_KEY",
"rag.web.search.google_pse_engine_id": "GOOGLE_PSE_ENGINE_ID",
"rag.web.search.jina_api_base_url": "JINA_API_BASE_URL",
"rag.web.search.jina_api_key": "JINA_API_KEY",
"rag.web.search.kagi_search_api_key": "KAGI_SEARCH_API_KEY",
"rag.web.search.linkup_api_key": "LINKUP_API_KEY",
"rag.web.search.linkup_search_params": "LINKUP_SEARCH_PARAMS",
"rag.web.search.mojeek_search_api_key": "MOJEEK_SEARCH_API_KEY",
"rag.web.search.ollama_cloud_api_key": "OLLAMA_CLOUD_WEB_SEARCH_API_KEY",
"rag.web.search.perplexity_api_key": "PERPLEXITY_API_KEY",
"rag.web.search.perplexity_model": "PERPLEXITY_MODEL",
"rag.web.search.perplexity_search_api_url": "PERPLEXITY_SEARCH_API_URL",
"rag.web.search.perplexity_search_context_usage": "PERPLEXITY_SEARCH_CONTEXT_USAGE",
"rag.web.search.result_count": "WEB_SEARCH_RESULT_COUNT",
"rag.web.search.searchapi_api_key": "SEARCHAPI_API_KEY",
"rag.web.search.searchapi_engine": "SEARCHAPI_ENGINE",
"rag.web.search.searxng_language": "SEARXNG_LANGUAGE",
"rag.web.search.searxng_query_url": "SEARXNG_QUERY_URL",
"rag.web.search.serpapi_api_key": "SERPAPI_API_KEY",
"rag.web.search.serpapi_engine": "SERPAPI_ENGINE",
"rag.web.search.serper_api_key": "SERPER_API_KEY",
"rag.web.search.serply_api_key": "SERPLY_API_KEY",
"rag.web.search.serpstack_api_key": "SERPSTACK_API_KEY",
"rag.web.search.serpstack_https": "SERPSTACK_HTTPS",
"rag.web.search.sougou_api_sid": "SOUGOU_API_SID",
"rag.web.search.sougou_api_sk": "SOUGOU_API_SK",
"rag.web.search.tavily_api_key": "TAVILY_API_KEY",
"rag.web.search.tavily_extract_depth": "TAVILY_EXTRACT_DEPTH",
"rag.web.search.trust_env": "WEB_SEARCH_TRUST_ENV",
"rag.web.search.yacy_password": "YACY_PASSWORD",
"rag.web.search.yacy_query_url": "YACY_QUERY_URL",
"rag.web.search.yacy_username": "YACY_USERNAME",
"rag.web.search.yandex_web_search_api_key": "YANDEX_WEB_SEARCH_API_KEY",
"rag.web.search.yandex_web_search_config": "YANDEX_WEB_SEARCH_CONFIG",
"rag.web.search.yandex_web_search_url": "YANDEX_WEB_SEARCH_URL",
"rag.web.search.youcom_api_key": "YOUCOM_API_KEY",
"rag.youtube_loader_language": "YOUTUBE_LOADER_LANGUAGE",
"rag.youtube_loader_proxy_url": "YOUTUBE_LOADER_PROXY_URL",
# Tasks
"task.autocomplete.enable": "ENABLE_AUTOCOMPLETE_GENERATION",
"task.autocomplete.input_max_length": "AUTOCOMPLETE_GENERATION_INPUT_MAX_LENGTH",
"task.autocomplete.prompt_template": "AUTOCOMPLETE_GENERATION_PROMPT_TEMPLATE",
"task.follow_up.enable": "ENABLE_FOLLOW_UP_GENERATION",
"task.follow_up.prompt_template": "FOLLOW_UP_GENERATION_PROMPT_TEMPLATE",
"task.image.prompt_template": "IMAGE_PROMPT_GENERATION_PROMPT_TEMPLATE",
"task.model.default": "TASK_MODEL",
"task.model.external": "TASK_MODEL_EXTERNAL",
"task.query.prompt_template": "QUERY_GENERATION_PROMPT_TEMPLATE",
"task.query.retrieval.enable": "ENABLE_RETRIEVAL_QUERY_GENERATION",
"task.query.search.enable": "ENABLE_SEARCH_QUERY_GENERATION",
"task.tags.enable": "ENABLE_TAGS_GENERATION",
"task.tags.prompt_template": "TAGS_GENERATION_PROMPT_TEMPLATE",
"task.title.enable": "ENABLE_TITLE_GENERATION",
"task.title.prompt_template": "TITLE_GENERATION_PROMPT_TEMPLATE",
"task.tools.prompt_template": "TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE",
"task.voice.prompt.enable": "ENABLE_VOICE_MODE_PROMPT",
"task.voice.prompt_template": "VOICE_MODE_PROMPT_TEMPLATE",
# Misc
"terminal_server.connections": "TERMINAL_SERVER_CONNECTIONS",
"tool_server.connections": "TOOL_SERVER_CONNECTIONS",
"ui.banners": "WEBUI_BANNERS",
"ui.default_group_id": "DEFAULT_GROUP_ID",
"ui.default_locale": "DEFAULT_LOCALE",
"ui.default_models": "DEFAULT_MODELS",
"ui.default_pinned_models": "DEFAULT_PINNED_MODELS",
"ui.default_user_role": "DEFAULT_USER_ROLE",
"ui.enable_community_sharing": "ENABLE_COMMUNITY_SHARING",
"ui.enable_login_form": "ENABLE_LOGIN_FORM",
"ui.enable_message_rating": "ENABLE_MESSAGE_RATING",
"ui.enable_password_change_form": "ENABLE_PASSWORD_CHANGE_FORM",
"ui.enable_signup": "ENABLE_SIGNUP",
"ui.enable_user_webhooks": "ENABLE_USER_WEBHOOKS",
"ui.model_order_list": "MODEL_ORDER_LIST",
"ui.pending_user_overlay_content": "PENDING_USER_OVERLAY_CONTENT",
"ui.pending_user_overlay_title": "PENDING_USER_OVERLAY_TITLE",
"ui.prompt_suggestions": "DEFAULT_PROMPT_SUGGESTIONS",
"ui.watermark": "RESPONSE_WATERMARK",
"user.permissions": "USER_PERMISSIONS",
"users.enable_status": "ENABLE_USER_STATUS",
"webhook_url": "WEBHOOK_URL",
"webui.url": "WEBUI_URL",
}
STORAGE_KEY_REWRITES = {
"oauth.refresh_token_include_scope": "oauth.refresh_token.include_scope",
"rag.openai_api_base_url": "rag.openai.api_base_url",
"rag.openai_api_key": "rag.openai.api_key",
"rag.ollama.url": "rag.ollama.base_url",
"rag.ollama.key": "rag.ollama.api_key",
"oauth.oidc.avatar_claim": "oauth.picture_claim",
"oauth.oidc.client_id": "oauth.client_id",
"oauth.oidc.client_secret": "oauth.client_secret",
"oauth.oidc.code_challenge_method": "oauth.code_challenge_method",
"oauth.oidc.email_claim": "oauth.email_claim",
"oauth.oidc.end_session_endpoint": "oauth.end_session_endpoint",
"oauth.oidc.group_claim": "oauth.group_claim",
"oauth.oidc.oauth_timeout": "oauth.timeout",
"oauth.oidc.provider_name": "oauth.provider_name",
"oauth.oidc.provider_url": "oauth.provider_url",
"oauth.oidc.redirect_uri": "oauth.redirect_uri",
"oauth.oidc.scopes": "oauth.scopes",
"oauth.oidc.sub_claim": "oauth.sub_claim",
"oauth.oidc.token_endpoint_auth_method": "oauth.token_endpoint_auth_method",
"oauth.oidc.username_claim": "oauth.username_claim",
}
LEGACY_KEY_TO_STORAGE_KEY = {
legacy_key: STORAGE_KEY_REWRITES.get(blob_path, blob_path)
for blob_path, legacy_key in BLOB_PATH_TO_KEY.items()
}
def _walk_blob(data: dict, prefix: str = '') -> dict:
"""Recursively walk a nested dict, yielding (dot.path, value) for leaf nodes."""
result = {}
for key, value in data.items():
path = f'{prefix}{key}' if not prefix else f'{prefix}.{key}'
if isinstance(value, dict):
result.update(_walk_blob(value, path))
else:
result[path] = value
return result
def upgrade() -> None:
"""Reshape config from single-row JSON blob to per-key rows."""
conn = op.get_bind()
inspector = sa.inspect(conn)
table_names = set(inspector.get_table_names())
config_columns = {column['name'] for column in inspector.get_columns('config')} if 'config' in table_names else set()
has_old_config = {'id', 'data'}.issubset(config_columns)
has_new_config = {'key', 'value'}.issubset(config_columns)
# Ad-hoc table reference for reading the old schema
old_config = sa.table(
'config',
sa.column('id', sa.Integer),
sa.column('data', sa.JSON),
)
# 1. Read existing blob
blob_data = {}
if has_old_config:
try:
result = conn.execute(
sa.select(old_config.c.data).order_by(old_config.c.id.desc()).limit(1)
)
row = result.fetchone()
if row and row[0]:
raw = row[0]
blob_data = json.loads(raw) if isinstance(raw, str) else raw
except Exception:
pass # Table might be partially migrated or empty
# 2. Preserve old blob table for rollback/inspection, then create per-key table.
if has_old_config:
if 'config_old' in table_names:
op.drop_table('config_old')
op.rename_table('config', 'config_old')
# 3. Create new per-key table
new_config = (
sa.table(
'config',
sa.column('key', sa.Text),
sa.column('value', sa.JSON()),
sa.column('updated_at', sa.BigInteger),
)
if has_new_config
else op.create_table(
'config',
sa.Column('key', sa.Text(), primary_key=True),
sa.Column('value', sa.JSON(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
)
)
# 4. Flatten blob and insert per-key rows
if blob_data:
flat = _walk_blob(blob_data)
# Keep stable dot-notation paths as the database keys.
# Known legacy env-style keys are rewritten to their dotted keys; unknown
# keys are still copied so custom/future config is not silently lost.
rows = {}
for blob_path, value in flat.items():
if blob_path in BLOB_PATH_TO_KEY:
storage_key = STORAGE_KEY_REWRITES.get(blob_path, blob_path)
elif blob_path in LEGACY_KEY_TO_STORAGE_KEY:
storage_key = LEGACY_KEY_TO_STORAGE_KEY[blob_path]
else:
storage_key = STORAGE_KEY_REWRITES.get(blob_path, blob_path)
if storage_key not in rows:
rows[storage_key] = value
# Batch insert via SQLAlchemy table reference
if rows:
now = int(time.time())
op.bulk_insert(
new_config,
[
{'key': k, 'value': v, 'updated_at': now}
for k, v in rows.items()
],
)
def downgrade() -> None:
"""Restore preserved old single-row config table when available."""
conn = op.get_bind()
inspector = sa.inspect(conn)
table_names = set(inspector.get_table_names())
if 'config_old' in table_names:
if 'config' in table_names:
op.drop_table('config')
op.rename_table('config_old', 'config')
return
config_columns = {column['name'] for column in inspector.get_columns('config')} if 'config' in table_names else set()
has_per_key_config = {'key', 'value'}.issubset(config_columns)
blob_data = {}
if has_per_key_config:
config = sa.table(
'config',
sa.column('key', sa.Text),
sa.column('value', sa.JSON),
)
for key, value in conn.execute(sa.select(config.c.key, config.c.value)):
blob_data[key] = json.loads(value) if isinstance(value, str) else value
op.drop_table('config')
if 'config' in table_names and not has_per_key_config:
return
old_config = op.create_table(
'config',
sa.Column('id', sa.Integer(), primary_key=True),
sa.Column('data', sa.JSON(), nullable=False),
sa.Column('version', sa.Integer(), nullable=False, server_default='0'),
sa.Column('created_at', sa.DateTime(), nullable=False, server_default=sa.func.now()),
sa.Column('updated_at', sa.DateTime(), nullable=True),
)
if blob_data:
op.bulk_insert(old_config, [{'data': blob_data, 'version': 0}])
@@ -1,74 +0,0 @@
"""add missing primary keys to legacy peewee tables
Revision ID: 461111b60977
Revises: 3c9b0ca343fd
Create Date: 2026-05-14 04:38:14.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = '461111b60977'
down_revision: Union[str, None] = '3c9b0ca343fd'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
# Tables bootstrapped by the old Peewee migration layer that may have
# UNIQUE(id) but no PRIMARY KEY constraint. Fresh Alembic installs
# already have correct PKs from 7e5b5dc7342b_init.py.
# 'tag' uses a composite PK since the same tag name can exist for multiple users.
LEGACY_TABLES = {
'auth': ['id'],
'chat': ['id'],
'chatidtag': ['id'],
'document': ['id'],
'file': ['id'],
'function': ['id'],
'memory': ['id'],
'model': ['id'],
'prompt': ['id'],
'tag': ['id', 'user_id'],
'tool': ['id'],
'user': ['id'],
}
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
for table_name, pk_columns in LEGACY_TABLES.items():
if table_name not in existing_tables:
continue
pk = inspector.get_pk_constraint(table_name)
pk_cols = pk.get('constrained_columns', [])
# Already has the correct PK — nothing to do
if sorted(pk_cols) == sorted(pk_columns):
continue
# Check that all PK columns exist
columns = {c['name'] for c in inspector.get_columns(table_name)}
if not all(c in columns for c in pk_columns):
continue
print(f"Promoting UNIQUE(id) -> PRIMARY KEY({', '.join(pk_columns)}) for '{table_name}'")
conn.execute(sa.text(f'DROP TABLE IF EXISTS _alembic_tmp_{table_name}'))
with op.batch_alter_table(table_name) as batch_op:
# Drop existing PK if any (e.g. on wrong column)
if pk_cols and pk.get('name'):
batch_op.drop_constraint(pk['name'], type_='primary')
batch_op.create_primary_key(f'pk_{table_name}', pk_columns)
def downgrade() -> None:
# Downgrade is a no-op — we don't want to remove PKs
pass
@@ -6,78 +6,62 @@ Create Date: 2024-10-23 03:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
revision = '4ace53fd72c8'
down_revision = 'af906e964978'
revision = "4ace53fd72c8"
down_revision = "af906e964978"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
columns = {c['name']: c for c in inspector.get_columns('folder')}
created_at_col = columns.get('created_at')
if not created_at_col:
return
# Only convert if still DateTime — skip if already BigInteger
if isinstance(created_at_col['type'], sa.BigInteger):
return
# Perform safe alterations using batch operation
with op.batch_alter_table('folder', schema=None) as batch_op:
with op.batch_alter_table("folder", schema=None) as batch_op:
# Step 1: Remove server defaults for created_at and updated_at
batch_op.alter_column(
'created_at',
"created_at",
server_default=None, # Removing server default
)
batch_op.alter_column(
'updated_at',
"updated_at",
server_default=None, # Removing server default
)
# Step 2: Change the column types to BigInteger for created_at
batch_op.alter_column(
'created_at',
"created_at",
type_=sa.BigInteger(),
existing_type=sa.DateTime(),
existing_nullable=False,
postgresql_using='extract(epoch from created_at)::bigint', # Conversion for PostgreSQL
postgresql_using="extract(epoch from created_at)::bigint", # Conversion for PostgreSQL
)
# Change the column types to BigInteger for updated_at
batch_op.alter_column(
'updated_at',
"updated_at",
type_=sa.BigInteger(),
existing_type=sa.DateTime(),
existing_nullable=False,
postgresql_using='extract(epoch from updated_at)::bigint', # Conversion for PostgreSQL
postgresql_using="extract(epoch from updated_at)::bigint", # Conversion for PostgreSQL
)
def downgrade():
# Convert columns back to DateTime and restore defaults. Mirrors the
# upgrade's postgresql_using cast — without it, Postgres can't
# auto-cast BigInteger → timestamp and aborts with DatatypeMismatch.
with op.batch_alter_table('folder', schema=None) as batch_op:
# Downgrade: Convert columns back to DateTime and restore defaults
with op.batch_alter_table("folder", schema=None) as batch_op:
batch_op.alter_column(
'created_at',
"created_at",
type_=sa.DateTime(),
existing_type=sa.BigInteger(),
existing_nullable=False,
server_default=sa.func.now(),
postgresql_using='to_timestamp(created_at)::timestamp without time zone',
server_default=sa.func.now(), # Restoring server default on downgrade
)
batch_op.alter_column(
'updated_at',
"updated_at",
type_=sa.DateTime(),
existing_type=sa.BigInteger(),
existing_nullable=False,
server_default=sa.func.now(),
onupdate=sa.func.now(),
postgresql_using='to_timestamp(updated_at)::timestamp without time zone',
server_default=sa.func.now(), # Restoring server default on downgrade
onupdate=sa.func.now(), # Restoring onupdate behavior if it was there
)
@@ -1,88 +0,0 @@
"""add pinned_note table
Revision ID: 4de81c2a3af1
Revises: 56359461a091
Create Date: 2026-05-09 04:29:27.651341
"""
from typing import Sequence, Union
import open_webui.internal.db
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = '4de81c2a3af1'
down_revision: Union[str, None] = '56359461a091'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
import time
import uuid
from sqlalchemy import insert, select, update
from sqlalchemy.sql import column, table
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'pinned_note' not in existing_tables:
op.create_table(
'pinned_note',
sa.Column('id', sa.Text(), nullable=False),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column('note_id', sa.Text(), sa.ForeignKey('note.id', ondelete='CASCADE'), nullable=False),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.PrimaryKeyConstraint('id'),
sa.UniqueConstraint('user_id', 'note_id', name='uq_pinned_note'),
)
note_table = table('note', column('id', sa.Text), column('user_id', sa.Text), column('is_pinned', sa.Boolean))
pinned_note_table = table(
'pinned_note',
column('id', sa.Text),
column('user_id', sa.Text),
column('note_id', sa.Text),
column('created_at', sa.BigInteger),
)
notes = conn.execute(
select(note_table.c.id, note_table.c.user_id).where(note_table.c.is_pinned == True)
).fetchall()
if notes:
now = int(time.time_ns())
conn.execute(
insert(pinned_note_table),
[
{'id': str(uuid.uuid4()), 'user_id': note[1], 'note_id': note[0], 'created_at': now}
for note in notes
],
)
with op.batch_alter_table('note', schema=None) as batch_op:
batch_op.drop_column('is_pinned')
def downgrade() -> None:
with op.batch_alter_table('note', schema=None) as batch_op:
batch_op.add_column(sa.Column('is_pinned', sa.Boolean(), nullable=True))
conn = op.get_bind()
note_table = table('note', column('id', sa.Text), column('is_pinned', sa.Boolean))
pinned_note_table = table('pinned_note', column('note_id', sa.Text))
notes = conn.execute(select(pinned_note_table.c.note_id)).fetchall()
for note in notes:
conn.execute(update(note_table).where(note_table.c.id == note[0]).values(is_pinned=True))
op.drop_table('pinned_note')
@@ -1,110 +0,0 @@
"""add calendar tables
Revision ID: 56359461a091
Revises: c1d2e3f4a5b6
Create Date: 2026-04-19 16:20:58.162045
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = '56359461a091'
down_revision: Union[str, None] = 'c1d2e3f4a5b6'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def _index_exists(inspector, index_name, table_name):
"""Check if an index already exists on the given table."""
indexes = inspector.get_indexes(table_name)
return any(idx['name'] == index_name for idx in indexes)
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
tables = inspector.get_table_names()
if 'calendar' not in tables:
op.create_table(
'calendar',
sa.Column('id', sa.Text(), nullable=False),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column('name', sa.Text(), nullable=False),
sa.Column('color', sa.Text(), nullable=True),
sa.Column('is_default', sa.Boolean(), nullable=False),
sa.Column('data', sa.JSON(), nullable=True),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
sa.PrimaryKeyConstraint('id'),
)
inspector.clear_cache()
if 'calendar' in inspector.get_table_names():
if not _index_exists(inspector, 'ix_calendar_user', 'calendar'):
op.create_index('ix_calendar_user', 'calendar', ['user_id'], unique=False)
if 'calendar_event' not in tables:
op.create_table(
'calendar_event',
sa.Column('id', sa.Text(), nullable=False),
sa.Column('calendar_id', sa.Text(), nullable=False),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column('title', sa.Text(), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('start_at', sa.BigInteger(), nullable=False),
sa.Column('end_at', sa.BigInteger(), nullable=True),
sa.Column('all_day', sa.Boolean(), nullable=False),
sa.Column('rrule', sa.Text(), nullable=True),
sa.Column('color', sa.Text(), nullable=True),
sa.Column('location', sa.Text(), nullable=True),
sa.Column('data', sa.JSON(), nullable=True),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('is_cancelled', sa.Boolean(), nullable=False),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
sa.PrimaryKeyConstraint('id'),
)
inspector.clear_cache()
if 'calendar_event' in inspector.get_table_names():
if not _index_exists(inspector, 'ix_calendar_event_calendar', 'calendar_event'):
op.create_index('ix_calendar_event_calendar', 'calendar_event', ['calendar_id', 'start_at'], unique=False)
if not _index_exists(inspector, 'ix_calendar_event_user_date', 'calendar_event'):
op.create_index('ix_calendar_event_user_date', 'calendar_event', ['user_id', 'start_at'], unique=False)
if 'calendar_event_attendee' not in tables:
op.create_table(
'calendar_event_attendee',
sa.Column('id', sa.Text(), nullable=False),
sa.Column('event_id', sa.Text(), nullable=False),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column('status', sa.Text(), nullable=False),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
sa.PrimaryKeyConstraint('id'),
sa.UniqueConstraint('event_id', 'user_id', name='uq_event_attendee'),
)
inspector.clear_cache()
if 'calendar_event_attendee' in inspector.get_table_names():
if not _index_exists(inspector, 'ix_calendar_event_attendee_user', 'calendar_event_attendee'):
op.create_index(
'ix_calendar_event_attendee_user', 'calendar_event_attendee', ['user_id', 'status'], unique=False
)
def downgrade() -> None:
op.drop_index('ix_calendar_event_attendee_user', table_name='calendar_event_attendee')
op.drop_table('calendar_event_attendee')
op.drop_index('ix_calendar_event_user_date', table_name='calendar_event')
op.drop_index('ix_calendar_event_calendar', table_name='calendar_event')
op.drop_table('calendar_event')
op.drop_index('ix_calendar_user', table_name='calendar')
op.drop_table('calendar')
@@ -6,49 +6,43 @@ Create Date: 2024-12-22 03:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
revision = '57c599a3cb57'
down_revision = '922e7a387820'
revision = "57c599a3cb57"
down_revision = "922e7a387820"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
op.create_table(
"channel",
sa.Column("id", sa.Text(), nullable=False, primary_key=True, unique=True),
sa.Column("user_id", sa.Text()),
sa.Column("name", sa.Text()),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("data", sa.JSON(), nullable=True),
sa.Column("meta", sa.JSON(), nullable=True),
sa.Column("access_control", sa.JSON(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
)
if 'channel' not in existing_tables:
op.create_table(
'channel',
sa.Column('id', sa.Text(), nullable=False, primary_key=True, unique=True),
sa.Column('user_id', sa.Text()),
sa.Column('name', sa.Text()),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('data', sa.JSON(), nullable=True),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('access_control', sa.JSON(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
)
if 'message' not in existing_tables:
op.create_table(
'message',
sa.Column('id', sa.Text(), nullable=False, primary_key=True, unique=True),
sa.Column('user_id', sa.Text()),
sa.Column('channel_id', sa.Text(), nullable=True),
sa.Column('content', sa.Text()),
sa.Column('data', sa.JSON(), nullable=True),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
)
op.create_table(
"message",
sa.Column("id", sa.Text(), nullable=False, primary_key=True, unique=True),
sa.Column("user_id", sa.Text()),
sa.Column("channel_id", sa.Text(), nullable=True),
sa.Column("content", sa.Text()),
sa.Column("data", sa.JSON(), nullable=True),
sa.Column("meta", sa.JSON(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
)
def downgrade():
op.drop_table('channel')
op.drop_table("channel")
op.drop_table('message')
op.drop_table("message")
@@ -1,58 +0,0 @@
"""Add channel file table
Revision ID: 6283dc0e4d8d
Revises: 3e0e00844bb0
Create Date: 2025-12-10 15:11:39.424601
"""
from typing import Sequence, Union
import open_webui.internal.db
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = '6283dc0e4d8d'
down_revision: Union[str, None] = '3e0e00844bb0'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'channel_file' not in existing_tables:
op.create_table(
'channel_file',
sa.Column('id', sa.Text(), primary_key=True),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column(
'channel_id',
sa.Text(),
sa.ForeignKey('channel.id', ondelete='CASCADE'),
nullable=False,
),
sa.Column(
'file_id',
sa.Text(),
sa.ForeignKey('file.id', ondelete='CASCADE'),
nullable=False,
),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
# indexes
sa.Index('ix_channel_file_channel_id', 'channel_id'),
sa.Index('ix_channel_file_file_id', 'file_id'),
sa.Index('ix_channel_file_user_id', 'user_id'),
# unique constraints
sa.UniqueConstraint(
'channel_id', 'file_id', name='uq_channel_file_channel_file'
), # prevent duplicate entries
)
def downgrade() -> None:
op.drop_table('channel_file')
@@ -6,80 +6,75 @@ Create Date: 2024-10-01 14:02:35.241684
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.sql import table, column, select
import json
import sqlalchemy as sa
from alembic import op
from sqlalchemy.sql import column, select, table
revision = '6a39f3d8e55c'
down_revision = 'c0fbf31ca0db'
revision = "6a39f3d8e55c"
down_revision = "c0fbf31ca0db"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
# Creating the 'knowledge' table
print("Creating knowledge table")
knowledge_table = op.create_table(
"knowledge",
sa.Column("id", sa.Text(), primary_key=True),
sa.Column("user_id", sa.Text(), nullable=False),
sa.Column("name", sa.Text(), nullable=False),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("data", sa.JSON(), nullable=True),
sa.Column("meta", sa.JSON(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=False),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
)
if 'knowledge' not in existing_tables:
# Creating the 'knowledge' table
print('Creating knowledge table')
knowledge_table = op.create_table(
'knowledge',
sa.Column('id', sa.Text(), primary_key=True),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column('name', sa.Text(), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('data', sa.JSON(), nullable=True),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
print("Migrating data from document table to knowledge table")
# Representation of the existing 'document' table
document_table = table(
"document",
column("collection_name", sa.String()),
column("user_id", sa.String()),
column("name", sa.String()),
column("title", sa.Text()),
column("content", sa.Text()),
column("timestamp", sa.BigInteger()),
)
# Select all from existing document table
documents = op.get_bind().execute(
select(
document_table.c.collection_name,
document_table.c.user_id,
document_table.c.name,
document_table.c.title,
document_table.c.content,
document_table.c.timestamp,
)
)
print('Migrating data from document table to knowledge table')
# Representation of the existing 'document' table
document_table = table(
'document',
column('collection_name', sa.String()),
column('user_id', sa.String()),
column('name', sa.String()),
column('title', sa.Text()),
column('content', sa.Text()),
column('timestamp', sa.BigInteger()),
)
# Select all from existing document table
documents = conn.execute(
select(
document_table.c.collection_name,
document_table.c.user_id,
document_table.c.name,
document_table.c.title,
document_table.c.content,
document_table.c.timestamp,
# Insert data into knowledge table from document table
for doc in documents:
op.get_bind().execute(
knowledge_table.insert().values(
id=doc.collection_name,
user_id=doc.user_id,
description=doc.name,
meta={
"legacy": True,
"document": True,
"tags": json.loads(doc.content or "{}").get("tags", []),
},
name=doc.title,
created_at=doc.timestamp,
updated_at=doc.timestamp, # using created_at for both created_at and updated_at in project
)
)
# Insert data into knowledge table from document table
for doc in documents:
conn.execute(
knowledge_table.insert().values(
id=doc.collection_name,
user_id=doc.user_id,
description=doc.name,
meta={
'legacy': True,
'document': True,
'tags': json.loads(doc.content or '{}').get('tags', []),
},
name=doc.title,
created_at=doc.timestamp,
updated_at=doc.timestamp,
)
)
def downgrade():
op.drop_table('knowledge')
op.drop_table("knowledge")
@@ -6,26 +6,21 @@ Create Date: 2024-12-23 03:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
revision = '7826ab40b532'
down_revision = '57c599a3cb57'
revision = "7826ab40b532"
down_revision = "57c599a3cb57"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
file_cols = {c['name'] for c in inspector.get_columns('file')}
if 'access_control' not in file_cols:
op.add_column(
'file',
sa.Column('access_control', sa.JSON(), nullable=True),
)
op.add_column(
"file",
sa.Column("access_control", sa.JSON(), nullable=True),
)
def downgrade():
op.drop_column('file', 'access_control')
op.drop_column("file", "access_control")
@@ -1,192 +1,204 @@
# Initial bootstrap migration version.
# Revision ID: 7e5b5dc7342b
# Revises: (none)
# Created on: 2024-06-24 13:15:33.808998
from __future__ import annotations
from typing import Sequence
import open_webui.internal.db # noqa: F401
"""init
Revision ID: 7e5b5dc7342b
Revises:
Create Date: 2024-06-24 13:15:33.808998
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
import open_webui.internal.db
from open_webui.internal.db import JSONField
from open_webui.migrations.util import get_existing_tables
revision: str = '7e5b5dc7342b'
down_revision: str | None = None
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
# Initial schema table declarations
_INITIAL_TABLES: list[tuple[str, list[sa.Column], list]] = [
(
'auth',
[
sa.Column('id', sa.String(), nullable=False),
sa.Column('email', sa.String(), nullable=True),
sa.Column('password', sa.Text(), nullable=True),
sa.Column('active', sa.Boolean(), nullable=True),
],
[sa.PrimaryKeyConstraint('id')],
),
(
'chat',
[
sa.Column('id', sa.String(), nullable=False),
sa.Column('user_id', sa.String(), nullable=True),
sa.Column('title', sa.Text(), nullable=True),
sa.Column('chat', sa.Text(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
sa.Column('share_id', sa.Text(), nullable=True),
sa.Column('archived', sa.Boolean(), nullable=True),
],
[sa.PrimaryKeyConstraint('id'), sa.UniqueConstraint('share_id')],
),
(
'chatidtag',
[
sa.Column('id', sa.String(), nullable=False),
sa.Column('tag_name', sa.String(), nullable=True),
sa.Column('chat_id', sa.String(), nullable=True),
sa.Column('user_id', sa.String(), nullable=True),
sa.Column('timestamp', sa.BigInteger(), nullable=True),
],
[sa.PrimaryKeyConstraint('id')],
),
(
'document',
[
sa.Column('collection_name', sa.String(), nullable=False),
sa.Column('name', sa.String(), nullable=True),
sa.Column('title', sa.Text(), nullable=True),
sa.Column('filename', sa.Text(), nullable=True),
sa.Column('content', sa.Text(), nullable=True),
sa.Column('user_id', sa.String(), nullable=True),
sa.Column('timestamp', sa.BigInteger(), nullable=True),
],
[sa.PrimaryKeyConstraint('collection_name'), sa.UniqueConstraint('name')],
),
(
'file',
[
sa.Column('id', sa.String(), nullable=False),
sa.Column('user_id', sa.String(), nullable=True),
sa.Column('filename', sa.Text(), nullable=True),
sa.Column('meta', JSONField(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
],
[sa.PrimaryKeyConstraint('id')],
),
(
'function',
[
sa.Column('id', sa.String(), nullable=False),
sa.Column('user_id', sa.String(), nullable=True),
sa.Column('name', sa.Text(), nullable=True),
sa.Column('type', sa.Text(), nullable=True),
sa.Column('content', sa.Text(), nullable=True),
sa.Column('meta', JSONField(), nullable=True),
sa.Column('valves', JSONField(), nullable=True),
sa.Column('is_active', sa.Boolean(), nullable=True),
sa.Column('is_global', sa.Boolean(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
],
[sa.PrimaryKeyConstraint('id')],
),
(
'memory',
[
sa.Column('id', sa.String(), nullable=False),
sa.Column('user_id', sa.String(), nullable=True),
sa.Column('content', sa.Text(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
],
[sa.PrimaryKeyConstraint('id')],
),
(
'model',
[
sa.Column('id', sa.Text(), nullable=False),
sa.Column('user_id', sa.Text(), nullable=True),
sa.Column('base_model_id', sa.Text(), nullable=True),
sa.Column('name', sa.Text(), nullable=True),
sa.Column('params', JSONField(), nullable=True),
sa.Column('meta', JSONField(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
],
[sa.PrimaryKeyConstraint('id')],
),
(
'prompt',
[
sa.Column('command', sa.String(), nullable=False),
sa.Column('user_id', sa.String(), nullable=True),
sa.Column('title', sa.Text(), nullable=True),
sa.Column('content', sa.Text(), nullable=True),
sa.Column('timestamp', sa.BigInteger(), nullable=True),
],
[sa.PrimaryKeyConstraint('command')],
),
(
'tag',
[
sa.Column('id', sa.String(), nullable=False),
sa.Column('name', sa.String(), nullable=True),
sa.Column('user_id', sa.String(), nullable=True),
sa.Column('data', sa.Text(), nullable=True),
],
[sa.PrimaryKeyConstraint('id')],
),
(
'tool',
[
sa.Column('id', sa.String(), nullable=False),
sa.Column('user_id', sa.String(), nullable=True),
sa.Column('name', sa.Text(), nullable=True),
sa.Column('content', sa.Text(), nullable=True),
sa.Column('specs', JSONField(), nullable=True),
sa.Column('meta', JSONField(), nullable=True),
sa.Column('valves', JSONField(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
],
[sa.PrimaryKeyConstraint('id')],
),
(
'user',
[
sa.Column('id', sa.String(), nullable=False),
sa.Column('name', sa.String(), nullable=True),
sa.Column('email', sa.String(), nullable=True),
sa.Column('role', sa.String(), nullable=True),
sa.Column('profile_image_url', sa.Text(), nullable=True),
sa.Column('last_active_at', sa.BigInteger(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
sa.Column('api_key', sa.String(), nullable=True),
sa.Column('settings', JSONField(), nullable=True),
sa.Column('info', JSONField(), nullable=True),
sa.Column('oauth_sub', sa.Text(), nullable=True),
],
[
sa.PrimaryKeyConstraint('id'),
sa.UniqueConstraint('api_key'),
sa.UniqueConstraint('oauth_sub'),
],
),
]
# revision identifiers, used by Alembic.
revision: str = "7e5b5dc7342b"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
# --- migration execution ---
def upgrade() -> None: # deploy initial schema tables
def upgrade() -> None:
existing_tables = set(get_existing_tables())
for name, columns, constraints in _INITIAL_TABLES:
if name not in existing_tables:
op.create_table(name, *columns, *constraints)
# ### commands auto generated by Alembic - please adjust! ###
if "auth" not in existing_tables:
op.create_table(
"auth",
sa.Column("id", sa.String(), nullable=False),
sa.Column("email", sa.String(), nullable=True),
sa.Column("password", sa.Text(), nullable=True),
sa.Column("active", sa.Boolean(), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
if "chat" not in existing_tables:
op.create_table(
"chat",
sa.Column("id", sa.String(), nullable=False),
sa.Column("user_id", sa.String(), nullable=True),
sa.Column("title", sa.Text(), nullable=True),
sa.Column("chat", sa.Text(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
sa.Column("share_id", sa.Text(), nullable=True),
sa.Column("archived", sa.Boolean(), nullable=True),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("share_id"),
)
if "chatidtag" not in existing_tables:
op.create_table(
"chatidtag",
sa.Column("id", sa.String(), nullable=False),
sa.Column("tag_name", sa.String(), nullable=True),
sa.Column("chat_id", sa.String(), nullable=True),
sa.Column("user_id", sa.String(), nullable=True),
sa.Column("timestamp", sa.BigInteger(), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
if "document" not in existing_tables:
op.create_table(
"document",
sa.Column("collection_name", sa.String(), nullable=False),
sa.Column("name", sa.String(), nullable=True),
sa.Column("title", sa.Text(), nullable=True),
sa.Column("filename", sa.Text(), nullable=True),
sa.Column("content", sa.Text(), nullable=True),
sa.Column("user_id", sa.String(), nullable=True),
sa.Column("timestamp", sa.BigInteger(), nullable=True),
sa.PrimaryKeyConstraint("collection_name"),
sa.UniqueConstraint("name"),
)
if "file" not in existing_tables:
op.create_table(
"file",
sa.Column("id", sa.String(), nullable=False),
sa.Column("user_id", sa.String(), nullable=True),
sa.Column("filename", sa.Text(), nullable=True),
sa.Column("meta", JSONField(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
if "function" not in existing_tables:
op.create_table(
"function",
sa.Column("id", sa.String(), nullable=False),
sa.Column("user_id", sa.String(), nullable=True),
sa.Column("name", sa.Text(), nullable=True),
sa.Column("type", sa.Text(), nullable=True),
sa.Column("content", sa.Text(), nullable=True),
sa.Column("meta", JSONField(), nullable=True),
sa.Column("valves", JSONField(), nullable=True),
sa.Column("is_active", sa.Boolean(), nullable=True),
sa.Column("is_global", sa.Boolean(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
if "memory" not in existing_tables:
op.create_table(
"memory",
sa.Column("id", sa.String(), nullable=False),
sa.Column("user_id", sa.String(), nullable=True),
sa.Column("content", sa.Text(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
if "model" not in existing_tables:
op.create_table(
"model",
sa.Column("id", sa.Text(), nullable=False),
sa.Column("user_id", sa.Text(), nullable=True),
sa.Column("base_model_id", sa.Text(), nullable=True),
sa.Column("name", sa.Text(), nullable=True),
sa.Column("params", JSONField(), nullable=True),
sa.Column("meta", JSONField(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
if "prompt" not in existing_tables:
op.create_table(
"prompt",
sa.Column("command", sa.String(), nullable=False),
sa.Column("user_id", sa.String(), nullable=True),
sa.Column("title", sa.Text(), nullable=True),
sa.Column("content", sa.Text(), nullable=True),
sa.Column("timestamp", sa.BigInteger(), nullable=True),
sa.PrimaryKeyConstraint("command"),
)
if "tag" not in existing_tables:
op.create_table(
"tag",
sa.Column("id", sa.String(), nullable=False),
sa.Column("name", sa.String(), nullable=True),
sa.Column("user_id", sa.String(), nullable=True),
sa.Column("data", sa.Text(), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
if "tool" not in existing_tables:
op.create_table(
"tool",
sa.Column("id", sa.String(), nullable=False),
sa.Column("user_id", sa.String(), nullable=True),
sa.Column("name", sa.Text(), nullable=True),
sa.Column("content", sa.Text(), nullable=True),
sa.Column("specs", JSONField(), nullable=True),
sa.Column("meta", JSONField(), nullable=True),
sa.Column("valves", JSONField(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
if "user" not in existing_tables:
op.create_table(
"user",
sa.Column("id", sa.String(), nullable=False),
sa.Column("name", sa.String(), nullable=True),
sa.Column("email", sa.String(), nullable=True),
sa.Column("role", sa.String(), nullable=True),
sa.Column("profile_image_url", sa.Text(), nullable=True),
sa.Column("last_active_at", sa.BigInteger(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.Column("api_key", sa.String(), nullable=True),
sa.Column("settings", JSONField(), nullable=True),
sa.Column("info", JSONField(), nullable=True),
sa.Column("oauth_sub", sa.Text(), nullable=True),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("api_key"),
sa.UniqueConstraint("oauth_sub"),
)
# ### end Alembic commands ###
# --- rollback function ---
def downgrade() -> None: # rollback initial schema tables
for table_name, _, _ in reversed(_INITIAL_TABLES):
op.drop_table(table_name)
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.drop_table("user")
op.drop_table("tool")
op.drop_table("tag")
op.drop_table("prompt")
op.drop_table("model")
op.drop_table("memory")
op.drop_table("function")
op.drop_table("file")
op.drop_table("document")
op.drop_table("chatidtag")
op.drop_table("chat")
op.drop_table("auth")
# ### end Alembic commands ###
@@ -1,53 +0,0 @@
"""Update channel file and knowledge table
Revision ID: 81cc2ce44d79
Revises: 6283dc0e4d8d
Create Date: 2025-12-10 16:07:58.001282
"""
from typing import Sequence, Union
import open_webui.internal.db
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = '81cc2ce44d79'
down_revision: Union[str, None] = '6283dc0e4d8d'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
# Add message_id column to channel_file table
cf_cols = {c['name'] for c in inspector.get_columns('channel_file')}
if 'message_id' not in cf_cols:
with op.batch_alter_table('channel_file', schema=None) as batch_op:
batch_op.add_column(
sa.Column(
'message_id',
sa.Text(),
sa.ForeignKey('message.id', ondelete='CASCADE', name='fk_channel_file_message_id'),
nullable=True,
)
)
# Add data column to knowledge table
k_cols = {c['name'] for c in inspector.get_columns('knowledge')}
if 'data' not in k_cols:
with op.batch_alter_table('knowledge', schema=None) as batch_op:
batch_op.add_column(sa.Column('data', sa.JSON(), nullable=True))
def downgrade() -> None:
# Remove message_id column from channel_file table
with op.batch_alter_table('channel_file', schema=None) as batch_op:
batch_op.drop_column('message_id')
# Remove data column from knowledge table
with op.batch_alter_table('knowledge', schema=None) as batch_op:
batch_op.drop_column('data')
@@ -1,226 +0,0 @@
"""Add chat_message table
Revision ID: 8452d01d26d7
Revises: 374d2f66af06
Create Date: 2026-02-01 04:00:00.000000
"""
import json
import logging
import time
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
log = logging.getLogger(__name__)
revision: str = '8452d01d26d7'
down_revision: Union[str, None] = '374d2f66af06'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
BATCH_SIZE = 5000
def _flush_batch(conn, table, batch):
"""
Insert a batch of messages, falling back to row-by-row on error.
Tries a single bulk insert first (fast path). If that fails (e.g. due to
a duplicate key), falls back to individual inserts wrapped in savepoints
so the rest of the batch can still succeed.
"""
savepoint = conn.begin_nested()
try:
conn.execute(sa.insert(table), batch)
savepoint.commit()
return len(batch), 0
except Exception:
savepoint.rollback()
# Batch failed - insert one-by-one to isolate the bad row(s)
inserted = 0
failed = 0
for msg in batch:
sp = conn.begin_nested()
try:
conn.execute(sa.insert(table).values(**msg))
sp.commit()
inserted += 1
except Exception as e:
sp.rollback()
failed += 1
log.warning(f'Failed to insert message {msg["id"]}: {e}')
return inserted, failed
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'chat_message' in existing_tables:
return # Already created — skip everything
# Step 1: Create table
op.create_table(
'chat_message',
sa.Column('id', sa.Text(), primary_key=True),
sa.Column('chat_id', sa.Text(), nullable=False, index=True),
sa.Column('user_id', sa.Text(), index=True),
sa.Column('role', sa.Text(), nullable=False),
sa.Column('parent_id', sa.Text(), nullable=True),
sa.Column('content', sa.JSON(), nullable=True),
sa.Column('output', sa.JSON(), nullable=True),
sa.Column('model_id', sa.Text(), nullable=True, index=True),
sa.Column('files', sa.JSON(), nullable=True),
sa.Column('sources', sa.JSON(), nullable=True),
sa.Column('embeds', sa.JSON(), nullable=True),
sa.Column('done', sa.Boolean(), default=True),
sa.Column('status_history', sa.JSON(), nullable=True),
sa.Column('error', sa.JSON(), nullable=True),
sa.Column('usage', sa.JSON(), nullable=True),
sa.Column('created_at', sa.BigInteger(), index=True),
sa.Column('updated_at', sa.BigInteger()),
sa.ForeignKeyConstraint(['chat_id'], ['chat.id'], ondelete='CASCADE'),
)
# Create composite indexes
op.create_index('chat_message_chat_parent_idx', 'chat_message', ['chat_id', 'parent_id'])
op.create_index('chat_message_model_created_idx', 'chat_message', ['model_id', 'created_at'])
op.create_index('chat_message_user_created_idx', 'chat_message', ['user_id', 'created_at'])
# Step 2: Backfill from existing chats
chat_table = sa.table(
'chat',
sa.column('id', sa.Text()),
sa.column('user_id', sa.Text()),
sa.column('chat', sa.JSON()),
)
chat_message_table = sa.table(
'chat_message',
sa.column('id', sa.Text()),
sa.column('chat_id', sa.Text()),
sa.column('user_id', sa.Text()),
sa.column('role', sa.Text()),
sa.column('parent_id', sa.Text()),
sa.column('content', sa.JSON()),
sa.column('output', sa.JSON()),
sa.column('model_id', sa.Text()),
sa.column('files', sa.JSON()),
sa.column('sources', sa.JSON()),
sa.column('embeds', sa.JSON()),
sa.column('done', sa.Boolean()),
sa.column('status_history', sa.JSON()),
sa.column('error', sa.JSON()),
sa.column('usage', sa.JSON()),
sa.column('created_at', sa.BigInteger()),
sa.column('updated_at', sa.BigInteger()),
)
# Stream rows instead of loading all into memory:
# - yield_per: fetches rows in chunks via cursor.fetchmany() (all backends)
# - stream_results: enables server-side cursors on PostgreSQL (no-op on SQLite)
result = conn.execute(
sa.select(chat_table.c.id, chat_table.c.user_id, chat_table.c.chat)
.where(~chat_table.c.user_id.like('shared-%'))
.execution_options(yield_per=1000, stream_results=True)
)
now = int(time.time())
messages_batch = []
total_inserted = 0
total_failed = 0
for chat_row in result:
chat_id = chat_row[0]
user_id = chat_row[1]
chat_data = chat_row[2]
if not chat_data:
continue
# Handle both string and dict chat data
if isinstance(chat_data, str):
try:
chat_data = json.loads(chat_data)
except Exception:
continue
history = chat_data.get('history', {})
if not isinstance(history, dict):
continue
messages = history.get('messages', {})
if not isinstance(messages, dict):
continue
for message_id, message in messages.items():
if not isinstance(message, dict):
continue
role = message.get('role')
if not role:
continue
timestamp = message.get('timestamp', now)
try:
timestamp = int(float(timestamp))
except Exception as e:
timestamp = now
# Normalize timestamp: convert ms to seconds, validate range
if timestamp > 10_000_000_000:
timestamp = timestamp // 1000
# Must be after 2020 and not too far in the future
if timestamp < 1577836800 or timestamp > now + 86400:
timestamp = now
messages_batch.append(
{
'id': f'{chat_id}-{message_id}',
'chat_id': chat_id,
'user_id': user_id,
'role': role,
'parent_id': message.get('parentId'),
'content': message.get('content'),
'output': message.get('output'),
'model_id': message.get('model'),
'files': message.get('files'),
'sources': message.get('sources'),
'embeds': message.get('embeds'),
'done': message.get('done', True),
'status_history': message.get('statusHistory'),
'error': message.get('error'),
'usage': message.get('usage'),
'created_at': timestamp,
'updated_at': timestamp,
}
)
# Flush batch when full
if len(messages_batch) >= BATCH_SIZE:
inserted, failed = _flush_batch(conn, chat_message_table, messages_batch)
total_inserted += inserted
total_failed += failed
if total_inserted % 50000 < BATCH_SIZE:
log.info(f'Migration progress: {total_inserted} messages inserted...')
messages_batch.clear()
# Flush remaining messages
if messages_batch:
inserted, failed = _flush_batch(conn, chat_message_table, messages_batch)
total_inserted += inserted
total_failed += failed
log.info(f'Backfilled {total_inserted} messages into chat_message table ({total_failed} failed)')
def downgrade() -> None:
op.drop_index('chat_message_user_created_idx', table_name='chat_message')
op.drop_index('chat_message_model_created_idx', table_name='chat_message')
op.drop_index('chat_message_chat_parent_idx', table_name='chat_message')
op.drop_table('chat_message')
@@ -1,87 +0,0 @@
"""Update channel and channel members table
Revision ID: 90ef40d4714e
Revises: b10670c03dd5
Create Date: 2025-11-30 06:33:38.790341
"""
from typing import Sequence, Union
import open_webui.internal.db
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = '90ef40d4714e'
down_revision: Union[str, None] = 'b10670c03dd5'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
# Update 'channel' table
channel_cols = {c['name'] for c in inspector.get_columns('channel')}
if 'is_private' not in channel_cols:
op.add_column('channel', sa.Column('is_private', sa.Boolean(), nullable=True))
if 'archived_at' not in channel_cols:
op.add_column('channel', sa.Column('archived_at', sa.BigInteger(), nullable=True))
if 'archived_by' not in channel_cols:
op.add_column('channel', sa.Column('archived_by', sa.Text(), nullable=True))
if 'deleted_at' not in channel_cols:
op.add_column('channel', sa.Column('deleted_at', sa.BigInteger(), nullable=True))
if 'deleted_by' not in channel_cols:
op.add_column('channel', sa.Column('deleted_by', sa.Text(), nullable=True))
if 'updated_by' not in channel_cols:
op.add_column('channel', sa.Column('updated_by', sa.Text(), nullable=True))
# Update 'channel_member' table
cm_cols = {c['name'] for c in inspector.get_columns('channel_member')}
if 'role' not in cm_cols:
op.add_column('channel_member', sa.Column('role', sa.Text(), nullable=True))
if 'invited_by' not in cm_cols:
op.add_column('channel_member', sa.Column('invited_by', sa.Text(), nullable=True))
if 'invited_at' not in cm_cols:
op.add_column('channel_member', sa.Column('invited_at', sa.BigInteger(), nullable=True))
# Create 'channel_webhook' table
if 'channel_webhook' not in existing_tables:
op.create_table(
'channel_webhook',
sa.Column('id', sa.Text(), primary_key=True, unique=True, nullable=False),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column(
'channel_id',
sa.Text(),
sa.ForeignKey('channel.id', ondelete='CASCADE'),
nullable=False,
),
sa.Column('name', sa.Text(), nullable=False),
sa.Column('profile_image_url', sa.Text(), nullable=True),
sa.Column('token', sa.Text(), nullable=False),
sa.Column('last_used_at', sa.BigInteger(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
)
def downgrade() -> None:
# Downgrade 'channel' table
op.drop_column('channel', 'is_private')
op.drop_column('channel', 'archived_at')
op.drop_column('channel', 'archived_by')
op.drop_column('channel', 'deleted_at')
op.drop_column('channel', 'deleted_by')
op.drop_column('channel', 'updated_by')
# Downgrade 'channel_member' table
op.drop_column('channel_member', 'role')
op.drop_column('channel_member', 'invited_by')
op.drop_column('channel_member', 'invited_at')
# Drop 'channel_webhook' table
op.drop_table('channel_webhook')
@@ -6,65 +6,80 @@ Create Date: 2024-11-14 03:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
revision = '922e7a387820'
down_revision = '4ace53fd72c8'
revision = "922e7a387820"
down_revision = "4ace53fd72c8"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'group' not in existing_tables:
op.create_table(
'group',
sa.Column('id', sa.Text(), nullable=False, primary_key=True, unique=True),
sa.Column('user_id', sa.Text(), nullable=True),
sa.Column('name', sa.Text(), nullable=True),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('data', sa.JSON(), nullable=True),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('permissions', sa.JSON(), nullable=True),
sa.Column('user_ids', sa.JSON(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
)
op.create_table(
"group",
sa.Column("id", sa.Text(), nullable=False, primary_key=True, unique=True),
sa.Column("user_id", sa.Text(), nullable=True),
sa.Column("name", sa.Text(), nullable=True),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("data", sa.JSON(), nullable=True),
sa.Column("meta", sa.JSON(), nullable=True),
sa.Column("permissions", sa.JSON(), nullable=True),
sa.Column("user_ids", sa.JSON(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
)
# Add 'access_control' column to 'model' table
model_cols = {c['name'] for c in inspector.get_columns('model')}
if 'access_control' not in model_cols:
op.add_column('model', sa.Column('access_control', sa.JSON(), nullable=True))
if 'is_active' not in model_cols:
op.add_column(
'model',
sa.Column('is_active', sa.Boolean(), nullable=False, server_default=sa.sql.expression.true()),
)
op.add_column(
"model",
sa.Column("access_control", sa.JSON(), nullable=True),
)
# Add 'is_active' column to 'model' table
op.add_column(
"model",
sa.Column(
"is_active",
sa.Boolean(),
nullable=False,
server_default=sa.sql.expression.true(),
),
)
# Add 'access_control' column to 'knowledge' table
knowledge_cols = {c['name'] for c in inspector.get_columns('knowledge')}
if 'access_control' not in knowledge_cols:
op.add_column('knowledge', sa.Column('access_control', sa.JSON(), nullable=True))
op.add_column(
"knowledge",
sa.Column("access_control", sa.JSON(), nullable=True),
)
# Add 'access_control' column to 'prompt' table
prompt_cols = {c['name'] for c in inspector.get_columns('prompt')}
if 'access_control' not in prompt_cols:
op.add_column('prompt', sa.Column('access_control', sa.JSON(), nullable=True))
op.add_column(
"prompt",
sa.Column("access_control", sa.JSON(), nullable=True),
)
# Add 'access_control' column to 'tools' table
tool_cols = {c['name'] for c in inspector.get_columns('tool')}
if 'access_control' not in tool_cols:
op.add_column('tool', sa.Column('access_control', sa.JSON(), nullable=True))
op.add_column(
"tool",
sa.Column("access_control", sa.JSON(), nullable=True),
)
def downgrade():
op.drop_table('group')
op.drop_column('model', 'access_control')
op.drop_column('model', 'is_active')
op.drop_column('knowledge', 'access_control')
op.drop_column('prompt', 'access_control')
op.drop_column('tool', 'access_control')
op.drop_table("group")
# Drop 'access_control' column from 'model' table
op.drop_column("model", "access_control")
# Drop 'is_active' column from 'model' table
op.drop_column("model", "is_active")
# Drop 'access_control' column from 'knowledge' table
op.drop_column("knowledge", "access_control")
# Drop 'access_control' column from 'prompt' table
op.drop_column("prompt", "access_control")
# Drop 'access_control' column from 'tools' table
op.drop_column("tool", "access_control")
@@ -6,33 +6,28 @@ Create Date: 2025-05-03 03:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
revision = '9f0c9cd09105'
down_revision = '3781e22d8b01'
revision = "9f0c9cd09105"
down_revision = "3781e22d8b01"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'note' not in existing_tables:
op.create_table(
'note',
sa.Column('id', sa.Text(), nullable=False, primary_key=True, unique=True),
sa.Column('user_id', sa.Text(), nullable=True),
sa.Column('title', sa.Text(), nullable=True),
sa.Column('data', sa.JSON(), nullable=True),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('access_control', sa.JSON(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
)
op.create_table(
"note",
sa.Column("id", sa.Text(), nullable=False, primary_key=True, unique=True),
sa.Column("user_id", sa.Text(), nullable=True),
sa.Column("title", sa.Text(), nullable=True),
sa.Column("data", sa.JSON(), nullable=True),
sa.Column("meta", sa.JSON(), nullable=True),
sa.Column("access_control", sa.JSON(), nullable=True),
sa.Column("created_at", sa.BigInteger(), nullable=True),
sa.Column("updated_at", sa.BigInteger(), nullable=True),
)
def downgrade():
op.drop_table('note')
op.drop_table("note")
@@ -1,28 +0,0 @@
"""Add memory user_id index
Revision ID: a0b1c2d3e4f5
Revises: 4de81c2a3af1
Create Date: 2025-09-15 03:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
revision = 'a0b1c2d3e4f5'
down_revision = '4de81c2a3af1'
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_indexes = {idx['name'] for idx in inspector.get_indexes('memory')}
if 'ix_memory_user_id' not in existing_indexes:
op.create_index('ix_memory_user_id', 'memory', ['user_id'])
def downgrade():
op.drop_index('ix_memory_user_id', table_name='memory')
@@ -1,44 +0,0 @@
"""Add skill table
Revision ID: a1b2c3d4e5f6
Revises: f1e2d3c4b5a6
Create Date: 2026-02-11 09:30:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from open_webui.migrations.util import get_existing_tables
revision: str = 'a1b2c3d4e5f6'
down_revision: Union[str, None] = 'f1e2d3c4b5a6'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
existing_tables = set(get_existing_tables())
if 'skill' not in existing_tables:
op.create_table(
'skill',
sa.Column('id', sa.String(), nullable=False, primary_key=True),
sa.Column('user_id', sa.String(), nullable=False),
sa.Column('name', sa.Text(), nullable=False, unique=True),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('content', sa.Text(), nullable=False),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('is_active', sa.Boolean(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
sa.Column('created_at', sa.BigInteger(), nullable=False),
)
op.create_index('idx_skill_user_id', 'skill', ['user_id'])
op.create_index('idx_skill_updated_at', 'skill', ['updated_at'])
def downgrade() -> None:
op.drop_index('idx_skill_updated_at', table_name='skill')
op.drop_index('idx_skill_user_id', table_name='skill')
op.drop_table('skill')
@@ -1,34 +0,0 @@
"""Add tasks and summary columns to chat table
Revision ID: a3dd5bedd151
Revises: b2c3d4e5f6a7
Create Date: 2026-03-29 22:15:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = 'a3dd5bedd151'
down_revision: Union[str, None] = 'b2c3d4e5f6a7'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
columns = [col['name'] for col in inspector.get_columns('chat')]
if 'tasks' not in columns:
op.add_column('chat', sa.Column('tasks', sa.JSON(), nullable=True))
if 'summary' not in columns:
op.add_column('chat', sa.Column('summary', sa.Text(), nullable=True))
def downgrade() -> None:
op.drop_column('chat', 'summary')
op.drop_column('chat', 'tasks')
@@ -8,29 +8,27 @@ Create Date: 2025-09-27 02:24:18.058455
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'a5c220713937'
down_revision: Union[str, None] = '38d63c18f30f'
revision: str = "a5c220713937"
down_revision: Union[str, None] = "38d63c18f30f"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
msg_cols = {c['name'] for c in inspector.get_columns('message')}
# Add 'reply_to_id' column to the 'message' table for replying to messages
if 'reply_to_id' not in msg_cols:
op.add_column(
'message',
sa.Column('reply_to_id', sa.Text(), nullable=True),
)
op.add_column(
"message",
sa.Column("reply_to_id", sa.Text(), nullable=True),
)
pass
def downgrade() -> None:
# Remove 'reply_to_id' column from the 'message' table
op.drop_column('message', 'reply_to_id')
op.drop_column("message", "reply_to_id")
pass
@@ -6,41 +6,46 @@ Create Date: 2024-10-20 17:02:35.241684
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
# Revision identifiers, used by Alembic.
revision = 'af906e964978'
down_revision = 'c29facfe716b'
revision = "af906e964978"
down_revision = "c29facfe716b"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'feedback' not in existing_tables:
# ### Create feedback table ###
op.create_table(
'feedback',
sa.Column('id', sa.Text(), primary_key=True), # Unique identifier for each feedback (TEXT type)
sa.Column('user_id', sa.Text(), nullable=True), # ID of the user providing the feedback (TEXT type)
sa.Column('version', sa.BigInteger(), default=0), # Version of feedback (BIGINT type)
sa.Column('type', sa.Text(), nullable=True), # Type of feedback (TEXT type)
sa.Column('data', sa.JSON(), nullable=True), # Feedback data (JSON type)
sa.Column('meta', sa.JSON(), nullable=True), # Metadata for feedback (JSON type)
sa.Column('snapshot', sa.JSON(), nullable=True), # snapshot data for feedback (JSON type)
sa.Column(
'created_at', sa.BigInteger(), nullable=False
), # Feedback creation timestamp (BIGINT representing epoch)
sa.Column(
'updated_at', sa.BigInteger(), nullable=False
), # Feedback update timestamp (BIGINT representing epoch)
)
# ### Create feedback table ###
op.create_table(
"feedback",
sa.Column(
"id", sa.Text(), primary_key=True
), # Unique identifier for each feedback (TEXT type)
sa.Column(
"user_id", sa.Text(), nullable=True
), # ID of the user providing the feedback (TEXT type)
sa.Column(
"version", sa.BigInteger(), default=0
), # Version of feedback (BIGINT type)
sa.Column("type", sa.Text(), nullable=True), # Type of feedback (TEXT type)
sa.Column("data", sa.JSON(), nullable=True), # Feedback data (JSON type)
sa.Column(
"meta", sa.JSON(), nullable=True
), # Metadata for feedback (JSON type)
sa.Column(
"snapshot", sa.JSON(), nullable=True
), # snapshot data for feedback (JSON type)
sa.Column(
"created_at", sa.BigInteger(), nullable=False
), # Feedback creation timestamp (BIGINT representing epoch)
sa.Column(
"updated_at", sa.BigInteger(), nullable=False
), # Feedback update timestamp (BIGINT representing epoch)
)
def downgrade():
# ### Drop feedback table ###
op.drop_table('feedback')
op.drop_table("feedback")
@@ -1,263 +0,0 @@
"""Update user table
Revision ID: b10670c03dd5
Revises: 2f1211949ecc
Create Date: 2025-11-28 04:55:31.737538
"""
import json
import time
from typing import Sequence, Union
import open_webui.internal.db
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = 'b10670c03dd5'
down_revision: Union[str, None] = '2f1211949ecc'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
# ── Ad-hoc table references for Core DML ─────────────────────────────────
# These are lightweight table() / column() references used only inside this
# migration for SELECT / UPDATE / INSERT — they do NOT create or alter
# anything on disk.
_user = sa.table(
'user',
sa.column('id', sa.Text),
sa.column('oauth_sub', sa.Text),
sa.column('oauth', sa.JSON),
sa.column('api_key', sa.Text),
sa.column('info', sa.Text),
sa.column('settings', sa.Text),
)
_api_key = sa.table(
'api_key',
sa.column('id', sa.Text),
sa.column('user_id', sa.Text),
sa.column('key', sa.Text),
sa.column('created_at', sa.BigInteger),
sa.column('updated_at', sa.BigInteger),
)
def _drop_sqlite_indexes_for_column(table_name, column_name, conn):
"""
SQLite requires manual removal of any user-created indexes referencing
a column before ALTER TABLE ... DROP COLUMN can succeed.
NOTE: PRAGMAs have no Core equivalent raw text is unavoidable here.
"""
indexes = conn.execute(sa.text(f"PRAGMA index_list('{table_name}')")).fetchall()
for idx in indexes:
index_name = idx[1] # index name
# Skip system-managed autoindexes (PK / UNIQUE constraints) — they
# cannot be dropped directly and will disappear when the column is
# removed via batch_alter_table.
if index_name.startswith('sqlite_autoindex_'):
continue
idx_info = conn.execute(sa.text(f"PRAGMA index_info('{index_name}')")).fetchall()
indexed_cols = [row[2] for row in idx_info]
if column_name in indexed_cols:
conn.execute(sa.text(f'DROP INDEX IF EXISTS {index_name}'))
def _convert_column_to_json(table: str, column: str):
conn = op.get_bind()
dialect = conn.dialect.name
t = sa.table(table, sa.column('id', sa.Text), sa.column(column, sa.Text))
t_json = sa.column(f'{column}_json', sa.JSON)
# SQLite cannot ALTER COLUMN → must recreate column
if dialect == 'sqlite':
op.add_column(table, sa.Column(f'{column}_json', sa.JSON(), nullable=True))
rows = conn.execute(sa.select(t.c.id, t.c[column])).fetchall()
for uid, raw in rows:
if raw is None:
parsed = None
else:
try:
parsed = json.loads(raw)
except Exception:
parsed = None
conn.execute(
sa.update(sa.table(table, sa.column('id'), t_json))
.where(sa.column('id') == uid)
.values({f'{column}_json': json.dumps(parsed) if parsed else None})
)
op.drop_column(table, column)
op.alter_column(table, f'{column}_json', new_column_name=column)
else:
# PostgreSQL supports direct CAST
op.alter_column(
table,
column,
type_=sa.JSON(),
postgresql_using=f'{column}::json',
)
def _convert_column_to_text(table: str, column: str):
conn = op.get_bind()
dialect = conn.dialect.name
t = sa.table(table, sa.column('id', sa.Text), sa.column(column))
t_text = sa.column(f'{column}_text', sa.Text)
if dialect == 'sqlite':
op.add_column(table, sa.Column(f'{column}_text', sa.Text(), nullable=True))
rows = conn.execute(sa.select(t.c.id, t.c[column])).fetchall()
for uid, raw in rows:
conn.execute(
sa.update(sa.table(table, sa.column('id'), t_text))
.where(sa.column('id') == uid)
.values({f'{column}_text': json.dumps(raw) if raw else None})
)
op.drop_column(table, column)
op.alter_column(table, f'{column}_text', new_column_name=column)
else:
op.alter_column(
table,
column,
type_=sa.Text(),
postgresql_using=f'to_json({column})::text',
)
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
user_columns = {c['name'] for c in inspector.get_columns('user')}
# ── Add new columns (idempotent) ──────────────────────────────────
for col_name, col_type in [
('profile_banner_image_url', sa.Text()),
('timezone', sa.String()),
('presence_state', sa.String()),
('status_emoji', sa.String()),
('status_message', sa.Text()),
('status_expires_at', sa.BigInteger()),
('oauth', sa.JSON()),
]:
if col_name not in user_columns:
op.add_column('user', sa.Column(col_name, col_type, nullable=True))
# Convert info (TEXT/JSONField) → JSON (skip if already JSON)
user_col_types = {c['name']: c['type'] for c in inspector.get_columns('user')}
if isinstance(user_col_types.get('info'), sa.Text):
_convert_column_to_json('user', 'info')
# Convert settings (TEXT/JSONField) → JSON (skip if already JSON)
if isinstance(user_col_types.get('settings'), sa.Text):
_convert_column_to_json('user', 'settings')
# ── Create api_key table (idempotent) ─────────────────────────────
if 'api_key' not in existing_tables:
op.create_table(
'api_key',
sa.Column('id', sa.Text(), primary_key=True, unique=True),
sa.Column('user_id', sa.Text(), sa.ForeignKey('user.id', ondelete='CASCADE')),
sa.Column('key', sa.Text(), unique=True, nullable=False),
sa.Column('data', sa.JSON(), nullable=True),
sa.Column('expires_at', sa.BigInteger(), nullable=True),
sa.Column('last_used_at', sa.BigInteger(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
)
# ── Migrate oauth_sub → oauth JSON (only if old column still exists)
if 'oauth_sub' in user_columns:
rows = conn.execute(sa.select(_user.c.id, _user.c.oauth_sub).where(_user.c.oauth_sub.is_not(None))).fetchall()
for uid, oauth_sub in rows:
if oauth_sub:
provider, sub = oauth_sub.split('@', 1) if '@' in oauth_sub else ('oidc', oauth_sub)
conn.execute(
sa.update(_user).where(_user.c.id == uid).values(oauth=json.dumps({provider: {'sub': sub}}))
)
# ── Migrate api_key column → api_key table (only if old column still exists)
if 'api_key' in user_columns:
rows = conn.execute(sa.select(_user.c.id, _user.c.api_key).where(_user.c.api_key.is_not(None))).fetchall()
now = int(time.time())
for uid, key_val in rows:
if key_val:
conn.execute(
sa.insert(_api_key).values(
id=f'key_{uid}',
user_id=uid,
key=key_val,
created_at=now,
updated_at=now,
)
)
# ── Drop legacy columns (idempotent) ──────────────────────────────
cols_to_drop = {'api_key', 'oauth_sub'} & user_columns
if cols_to_drop:
if conn.dialect.name == 'sqlite':
for col in cols_to_drop:
_drop_sqlite_indexes_for_column('user', col, conn)
with op.batch_alter_table('user') as batch_op:
for col in cols_to_drop:
batch_op.drop_column(col)
def downgrade() -> None:
op.add_column('user', sa.Column('oauth_sub', sa.Text(), nullable=True))
conn = op.get_bind()
rows = conn.execute(sa.select(_user.c.id, _user.c.oauth).where(_user.c.oauth.is_not(None))).fetchall()
for uid, oauth in rows:
try:
data = json.loads(oauth)
provider = list(data.keys())[0]
sub = data[provider].get('sub')
oauth_sub = f'{provider}@{sub}'
except Exception:
oauth_sub = None
conn.execute(sa.update(_user).where(_user.c.id == uid).values(oauth_sub=oauth_sub))
op.drop_column('user', 'oauth')
# --- Restore api_key field ---
op.add_column('user', sa.Column('api_key', sa.String(), nullable=True))
keys = conn.execute(sa.select(_api_key.c.user_id, _api_key.c.key)).fetchall()
for uid, key in keys:
conn.execute(sa.update(_user).where(_user.c.id == uid).values(api_key=key))
op.drop_table('api_key')
with op.batch_alter_table('user') as batch_op:
batch_op.drop_column('profile_banner_image_url')
batch_op.drop_column('timezone')
batch_op.drop_column('presence_state')
batch_op.drop_column('status_emoji')
batch_op.drop_column('status_message')
batch_op.drop_column('status_expires_at')
# Convert info (JSON) → TEXT
_convert_column_to_text('user', 'info')
# Convert settings (JSON) → TEXT
_convert_column_to_text('user', 'settings')
@@ -1,31 +0,0 @@
"""add scim column to user table
Revision ID: b2c3d4e5f6a7
Revises: a1b2c3d4e5f6
Create Date: 2026-02-13 14:19:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = 'b2c3d4e5f6a7'
down_revision: Union[str, None] = 'a1b2c3d4e5f6'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
user_cols = {c['name'] for c in inspector.get_columns('user')}
if 'scim' not in user_cols:
op.add_column('user', sa.Column('scim', sa.JSON(), nullable=True))
def downgrade() -> None:
op.drop_column('user', 'scim')
@@ -1,31 +0,0 @@
"""add last_read_at to chat
Revision ID: b7c8d9e0f1a2
Revises: d4e5f6a7b8c9
Create Date: 2026-04-01 04:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = 'b7c8d9e0f1a2'
down_revision = 'd4e5f6a7b8c9'
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
columns = [col['name'] for col in inspector.get_columns('chat')]
if 'last_read_at' not in columns:
op.add_column('chat', sa.Column('last_read_at', sa.BigInteger(), nullable=True))
# Set existing chats to be marked as read
op.execute('UPDATE chat SET last_read_at = updated_at')
def downgrade():
op.drop_column('chat', 'last_read_at')
@@ -12,28 +12,21 @@ import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = 'c0fbf31ca0db'
down_revision: Union[str, None] = 'ca81bd47c050'
revision: str = "c0fbf31ca0db"
down_revision: Union[str, None] = "ca81bd47c050"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
file_cols = {c['name'] for c in inspector.get_columns('file')}
# ### commands auto generated by Alembic - please adjust! ###
if 'hash' not in file_cols:
op.add_column('file', sa.Column('hash', sa.Text(), nullable=True))
if 'data' not in file_cols:
op.add_column('file', sa.Column('data', sa.JSON(), nullable=True))
if 'updated_at' not in file_cols:
op.add_column('file', sa.Column('updated_at', sa.BigInteger(), nullable=True))
op.add_column("file", sa.Column("hash", sa.Text(), nullable=True))
op.add_column("file", sa.Column("data", sa.JSON(), nullable=True))
op.add_column("file", sa.Column("updated_at", sa.BigInteger(), nullable=True))
def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.drop_column('file', 'updated_at')
op.drop_column('file', 'data')
op.drop_column('file', 'hash')
op.drop_column("file", "updated_at")
op.drop_column("file", "data")
op.drop_column("file", "hash")
@@ -1,187 +0,0 @@
"""Add shared_chat table and migrate existing shares
Revision ID: c1d2e3f4a5b6
Revises: e1f2a3b4c5d6
Create Date: 2026-04-16 23:00:00.000000
"""
import time
import uuid
import sqlalchemy as sa
from alembic import op
revision = 'c1d2e3f4a5b6'
down_revision = 'e1f2a3b4c5d6'
branch_labels = None
depends_on = None
# Lightweight table references for data migration (no ORM models needed)
chat_t = sa.table(
'chat',
sa.column('id', sa.Text),
sa.column('user_id', sa.Text),
sa.column('title', sa.Text),
sa.column('chat', sa.JSON),
sa.column('share_id', sa.Text),
sa.column('created_at', sa.BigInteger),
sa.column('updated_at', sa.BigInteger),
sa.column('archived', sa.Boolean),
sa.column('meta', sa.JSON),
)
shared_chat_t = sa.table(
'shared_chat',
sa.column('id', sa.Text),
sa.column('chat_id', sa.Text),
sa.column('user_id', sa.Text),
sa.column('title', sa.Text),
sa.column('chat', sa.JSON),
sa.column('created_at', sa.BigInteger),
sa.column('updated_at', sa.BigInteger),
)
chat_message_t = sa.table(
'chat_message',
sa.column('chat_id', sa.Text),
)
access_grant_t = sa.table(
'access_grant',
sa.column('id', sa.Text),
sa.column('resource_type', sa.Text),
sa.column('resource_id', sa.Text),
sa.column('principal_type', sa.Text),
sa.column('principal_id', sa.Text),
sa.column('permission', sa.Text),
sa.column('created_at', sa.BigInteger),
)
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
tables = inspector.get_table_names()
# 1. Create shared_chat table (idempotent)
if 'shared_chat' not in tables:
op.create_table(
'shared_chat',
sa.Column('id', sa.Text(), primary_key=True),
sa.Column('chat_id', sa.Text(), sa.ForeignKey('chat.id', ondelete='CASCADE'), nullable=False),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column('title', sa.Text(), nullable=True),
sa.Column('chat', sa.JSON(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=True),
sa.Column('updated_at', sa.BigInteger(), nullable=True),
)
# 2. Migrate existing shared-* rows
shared_rows = conn.execute(
sa.select(
chat_t.c.id,
chat_t.c.user_id,
chat_t.c.title,
chat_t.c.chat,
chat_t.c.created_at,
chat_t.c.updated_at,
).where(chat_t.c.user_id.like('shared-%'))
).fetchall()
for row in shared_rows:
share_token = row.id
original_chat_id = row.user_id.replace('shared-', '', 1)
# Verify original chat still exists
original = conn.execute(sa.select(chat_t.c.user_id).where(chat_t.c.id == original_chat_id)).fetchone()
if not original:
continue
# Check if shared_chat record already exists (idempotent)
existing_shared = conn.execute(
sa.select(shared_chat_t.c.id).where(shared_chat_t.c.id == share_token)
).fetchone()
if not existing_shared:
# Insert snapshot into shared_chat
conn.execute(
shared_chat_t.insert().values(
id=share_token,
chat_id=original_chat_id,
user_id=original.user_id,
title=row.title,
chat=row.chat,
created_at=row.created_at,
updated_at=row.updated_at,
)
)
# Check if access_grant record already exists (idempotent)
existing_grant = conn.execute(
sa.select(access_grant_t.c.id).where(
sa.and_(
access_grant_t.c.resource_type == 'shared_chat',
access_grant_t.c.resource_id == original_chat_id,
access_grant_t.c.principal_type == 'user',
access_grant_t.c.principal_id == '*',
access_grant_t.c.permission == 'read',
)
)
).fetchone()
if not existing_grant:
# Create user:*:read grant for backward compat
conn.execute(
access_grant_t.insert().values(
id=str(uuid.uuid4()),
resource_type='shared_chat',
resource_id=original_chat_id,
principal_type='user',
principal_id='*',
permission='read',
created_at=row.created_at or int(time.time()),
)
)
# 3. Clean up old phantom rows
conn.execute(
chat_message_t.delete().where(
chat_message_t.c.chat_id.in_(sa.select(chat_t.c.id).where(chat_t.c.user_id.like('shared-%')))
)
)
conn.execute(chat_t.delete().where(chat_t.c.user_id.like('shared-%')))
def downgrade():
conn = op.get_bind()
shared_rows = conn.execute(
sa.select(
shared_chat_t.c.id,
shared_chat_t.c.chat_id,
shared_chat_t.c.user_id,
shared_chat_t.c.title,
shared_chat_t.c.chat,
shared_chat_t.c.created_at,
shared_chat_t.c.updated_at,
)
).fetchall()
for row in shared_rows:
conn.execute(
chat_t.insert().values(
id=row.id,
user_id=f'shared-{row.chat_id}',
title=row.title,
chat=row.chat,
created_at=row.created_at,
updated_at=row.updated_at,
archived=False,
meta={},
)
)
conn.execute(access_grant_t.delete().where(access_grant_t.c.resource_type == 'shared_chat'))
op.drop_table('shared_chat')
@@ -6,45 +6,42 @@ Create Date: 2024-10-20 17:02:35.241684
"""
import json
import sqlalchemy as sa
from alembic import op
from sqlalchemy import JSON, String, Text, and_
from sqlalchemy.sql import column, table
import sqlalchemy as sa
import json
from sqlalchemy.sql import table, column
from sqlalchemy import String, Text, JSON, and_
revision = 'c29facfe716b'
down_revision = 'c69f45358db4'
revision = "c29facfe716b"
down_revision = "c69f45358db4"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
file_cols = {c['name'] for c in inspector.get_columns('file')}
# 1. Add the `path` column to the "file" table.
if 'path' not in file_cols:
op.add_column('file', sa.Column('path', sa.Text(), nullable=True))
op.add_column("file", sa.Column("path", sa.Text(), nullable=True))
# 2. Convert the `meta` column from Text/JSONField to `JSON()`
# Use Alembic's default batch_op for dialect compatibility.
with op.batch_alter_table('file', schema=None) as batch_op:
with op.batch_alter_table("file", schema=None) as batch_op:
batch_op.alter_column(
'meta',
"meta",
type_=sa.JSON(),
existing_type=sa.Text(),
existing_nullable=True,
nullable=True,
postgresql_using='meta::json',
postgresql_using="meta::json",
)
# 3. Migrate legacy data from `meta` JSONField
# Fetch and process `meta` data from the table, add values to the new `path` column as necessary.
# We will use SQLAlchemy core bindings to ensure safety across different databases.
file_table = table('file', column('id', String), column('meta', JSON), column('path', Text))
file_table = table(
"file", column("id", String), column("meta", JSON), column("path", Text)
)
# Create connection to the database
connection = op.get_bind()
@@ -59,18 +56,24 @@ def upgrade():
# Iterate over each row to extract and update the `path` from `meta` column
for row in results:
if 'path' in row.meta:
if "path" in row.meta:
# Extract the `path` field from the `meta` JSON
path = row.meta.get('path')
path = row.meta.get("path")
# Update the `file` table with the new `path` value
connection.execute(file_table.update().where(file_table.c.id == row.id).values({'path': path}))
connection.execute(
file_table.update()
.where(file_table.c.id == row.id)
.values({"path": path})
)
def downgrade():
# 1. Remove the `path` column
op.drop_column('file', 'path')
op.drop_column("file", "path")
# 2. Revert the `meta` column back to Text/JSONField
with op.batch_alter_table('file', schema=None) as batch_op:
batch_op.alter_column('meta', type_=sa.Text(), existing_type=sa.JSON(), existing_nullable=True)
with op.batch_alter_table("file", schema=None) as batch_op:
batch_op.alter_column(
"meta", type_=sa.Text(), existing_type=sa.JSON(), existing_nullable=True
)
@@ -1,57 +0,0 @@
"""Add chat_file table
Revision ID: c440947495f3
Revises: 81cc2ce44d79
Create Date: 2025-12-21 20:27:41.694897
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = 'c440947495f3'
down_revision: Union[str, None] = '81cc2ce44d79'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'chat_file' not in existing_tables:
op.create_table(
'chat_file',
sa.Column('id', sa.Text(), primary_key=True),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column(
'chat_id',
sa.Text(),
sa.ForeignKey('chat.id', ondelete='CASCADE'),
nullable=False,
),
sa.Column(
'file_id',
sa.Text(),
sa.ForeignKey('file.id', ondelete='CASCADE'),
nullable=False,
),
sa.Column('message_id', sa.Text(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
# indexes
sa.Index('ix_chat_file_chat_id', 'chat_id'),
sa.Index('ix_chat_file_file_id', 'file_id'),
sa.Index('ix_chat_file_message_id', 'message_id'),
sa.Index('ix_chat_file_user_id', 'user_id'),
# unique constraints
sa.UniqueConstraint('chat_id', 'file_id', name='uq_chat_file_chat_file'), # prevent duplicate entries
)
def downgrade() -> None:
op.drop_table('chat_file')
@@ -6,50 +6,45 @@ Create Date: 2024-10-16 02:02:35.241684
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
revision = 'c69f45358db4'
down_revision = '3ab32c4b8f59'
revision = "c69f45358db4"
down_revision = "3ab32c4b8f59"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
op.create_table(
"folder",
sa.Column("id", sa.Text(), nullable=False),
sa.Column("parent_id", sa.Text(), nullable=True),
sa.Column("user_id", sa.Text(), nullable=False),
sa.Column("name", sa.Text(), nullable=False),
sa.Column("items", sa.JSON(), nullable=True),
sa.Column("meta", sa.JSON(), nullable=True),
sa.Column("is_expanded", sa.Boolean(), default=False, nullable=False),
sa.Column(
"created_at", sa.DateTime(), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at",
sa.DateTime(),
nullable=False,
server_default=sa.func.now(),
onupdate=sa.func.now(),
),
sa.PrimaryKeyConstraint("id", "user_id"),
)
if 'folder' not in existing_tables:
op.create_table(
'folder',
sa.Column('id', sa.Text(), nullable=False),
sa.Column('parent_id', sa.Text(), nullable=True),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column('name', sa.Text(), nullable=False),
sa.Column('items', sa.JSON(), nullable=True),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('is_expanded', sa.Boolean(), default=False, nullable=False),
sa.Column('created_at', sa.DateTime(), server_default=sa.func.now(), nullable=False),
sa.Column(
'updated_at',
sa.DateTime(),
nullable=False,
server_default=sa.func.now(),
onupdate=sa.func.now(),
),
sa.PrimaryKeyConstraint('id', 'user_id'),
)
chat_cols = {c['name'] for c in inspector.get_columns('chat')}
if 'folder_id' not in chat_cols:
op.add_column(
'chat',
sa.Column('folder_id', sa.Text(), nullable=True),
)
op.add_column(
"chat",
sa.Column("folder_id", sa.Text(), nullable=True),
)
def downgrade():
op.drop_column('chat', 'folder_id')
op.drop_column("chat", "folder_id")
op.drop_table('folder')
op.drop_table("folder")
@@ -1,8 +1,9 @@
"""Add config table.
"""Add config table
Revision ID: ca81bd47c050
Revises: 7e5b5dc7342b
Create Date: 2024-08-25 15:26:35.241684
"""
from typing import Sequence, Union
@@ -10,40 +11,31 @@ from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = 'ca81bd47c050'
down_revision: Union[str, None] = '7e5b5dc7342b'
# revision identifiers, used by Alembic.
revision: str = "ca81bd47c050"
down_revision: Union[str, None] = "7e5b5dc7342b"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Create a key-value config table with versioning."""
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if 'config' not in existing_tables:
op.create_table(
'config',
sa.Column('id', sa.Integer, primary_key=True),
sa.Column('data', sa.JSON(), nullable=False),
sa.Column('version', sa.Integer, nullable=False),
sa.Column(
'created_at',
sa.DateTime(),
nullable=False,
server_default=sa.func.now(),
),
sa.Column(
'updated_at',
sa.DateTime(),
nullable=True,
server_default=sa.func.now(),
onupdate=sa.func.now(),
),
)
def upgrade():
op.create_table(
"config",
sa.Column("id", sa.Integer, primary_key=True),
sa.Column("data", sa.JSON(), nullable=False),
sa.Column("version", sa.Integer, nullable=False),
sa.Column(
"created_at", sa.DateTime(), nullable=False, server_default=sa.func.now()
),
sa.Column(
"updated_at",
sa.DateTime(),
nullable=True,
server_default=sa.func.now(),
onupdate=sa.func.now(),
),
)
def downgrade() -> None:
"""Drop the config table."""
op.drop_table('config')
def downgrade():
op.drop_table("config")
@@ -6,23 +6,18 @@ Create Date: 2025-07-13 03:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
import sqlalchemy as sa
revision = 'd31026856c01'
down_revision = '9f0c9cd09105'
revision = "d31026856c01"
down_revision = "9f0c9cd09105"
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
folder_cols = {c['name'] for c in inspector.get_columns('folder')}
if 'data' not in folder_cols:
op.add_column('folder', sa.Column('data', sa.JSON(), nullable=True))
op.add_column("folder", sa.Column("data", sa.JSON(), nullable=True))
def downgrade():
op.drop_column('folder', 'data')
op.drop_column("folder", "data")
@@ -1,75 +0,0 @@
"""add automation tables
Revision ID: d4e5f6a7b8c9
Revises: f1e2d3c4b5a6
Create Date: 2026-03-30
"""
from typing import Union
import sqlalchemy as sa
from alembic import op
revision: str = 'd4e5f6a7b8c9'
down_revision: Union[str, None] = 'a3dd5bedd151'
branch_labels = None
depends_on = None
def _index_exists(inspector, index_name, table_name):
"""Check if an index already exists on the given table (works for both SQLite and PostgreSQL)."""
indexes = inspector.get_indexes(table_name)
return any(idx['name'] == index_name for idx in indexes)
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
tables = inspector.get_table_names()
if 'automation' not in tables:
op.create_table(
'automation',
sa.Column('id', sa.Text(), primary_key=True),
sa.Column('user_id', sa.Text(), nullable=False),
sa.Column('name', sa.Text(), nullable=False),
sa.Column('data', sa.JSON(), nullable=False),
sa.Column('meta', sa.JSON(), nullable=True),
sa.Column('is_active', sa.Boolean(), nullable=False, default=True),
sa.Column('last_run_at', sa.BigInteger(), nullable=True),
sa.Column('next_run_at', sa.BigInteger(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.Column('updated_at', sa.BigInteger(), nullable=False),
)
inspector.clear_cache()
if 'automation' in inspector.get_table_names():
if not _index_exists(inspector, 'ix_automation_next_run', 'automation'):
op.create_index('ix_automation_next_run', 'automation', ['next_run_at'])
if 'automation_run' not in tables:
op.create_table(
'automation_run',
sa.Column('id', sa.Text(), primary_key=True),
sa.Column('automation_id', sa.Text(), nullable=False),
sa.Column('chat_id', sa.Text(), nullable=True),
sa.Column('status', sa.Text(), nullable=False),
sa.Column('error', sa.Text(), nullable=True),
sa.Column('created_at', sa.BigInteger(), nullable=False),
)
inspector.clear_cache()
if 'automation_run' in inspector.get_table_names():
if not _index_exists(inspector, 'ix_automation_run_automation_id', 'automation_run'):
op.create_index(
'ix_automation_run_automation_id',
'automation_run',
['automation_id'],
)
def downgrade():
op.drop_index('ix_automation_run_automation_id')
op.drop_table('automation_run')
op.drop_index('ix_automation_next_run')
op.drop_table('automation')
@@ -1,28 +0,0 @@
"""Add is_pinned to note table
Revision ID: e1f2a3b4c5d6
Revises: b7c8d9e0f1a2
Create Date: 2026-04-14 22:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
revision = 'e1f2a3b4c5d6'
down_revision = 'b7c8d9e0f1a2'
branch_labels = None
depends_on = None
def upgrade():
conn = op.get_bind()
inspector = sa.inspect(conn)
columns = [col['name'] for col in inspector.get_columns('note')]
if 'is_pinned' not in columns:
op.add_column('note', sa.Column('is_pinned', sa.Boolean(), nullable=True))
def downgrade():
op.drop_column('note', 'is_pinned')
@@ -1,348 +0,0 @@
"""Add access_grant table
Revision ID: f1e2d3c4b5a6
Revises: 8452d01d26d7
Create Date: 2026-02-05 10:00:00.000000
Migrates from JSON access_control columns to normalized access_grant table.
Access control semantics:
- NULL: Public access (all users can read) -> insert user:* for read
- {}: Private/owner-only (no grants) -> insert nothing
- {read: {...}, write: {...}}: Custom permissions -> insert specific grants
"""
import time
import uuid
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from open_webui.migrations.util import get_existing_tables
revision: str = 'f1e2d3c4b5a6'
down_revision: Union[str, None] = '8452d01d26d7'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
existing_tables = set(get_existing_tables())
# Create access_grant table
if 'access_grant' not in existing_tables:
op.create_table(
'access_grant',
sa.Column('id', sa.Text(), nullable=False, primary_key=True),
sa.Column('resource_type', sa.Text(), nullable=False),
sa.Column('resource_id', sa.Text(), nullable=False),
sa.Column('principal_type', sa.Text(), nullable=False),
sa.Column('principal_id', sa.Text(), nullable=False),
sa.Column('permission', sa.Text(), nullable=False),
sa.Column('created_at', sa.BigInteger(), nullable=False),
sa.UniqueConstraint(
'resource_type',
'resource_id',
'principal_type',
'principal_id',
'permission',
name='uq_access_grant_grant',
),
)
op.create_index(
'idx_access_grant_resource',
'access_grant',
['resource_type', 'resource_id'],
)
op.create_index(
'idx_access_grant_principal',
'access_grant',
['principal_type', 'principal_id'],
)
# Backfill existing access_control JSON data
conn = op.get_bind()
# Tables with access_control JSON columns: (table_name, resource_type)
resource_tables = [
('knowledge', 'knowledge'),
('prompt', 'prompt'),
('tool', 'tool'),
('model', 'model'),
('note', 'note'),
('channel', 'channel'),
('file', 'file'),
]
now = int(time.time())
inserted = set()
for table_name, resource_type in resource_tables:
if table_name not in existing_tables:
continue
# Check if access_control and id columns exist (may already be dropped on re-run,
# or table may have been rebuilt without id during intermediate migration states)
insp = sa.inspect(conn)
insp.clear_cache() # Ensure fresh metadata after prior migrations that rebuild tables
table_cols = {c['name'] for c in insp.get_columns(table_name)}
if 'access_control' not in table_cols or 'id' not in table_cols:
continue
# Query all rows
result = conn.execute(sa.text(f'SELECT id, access_control FROM "{table_name}"'))
rows = result.fetchall()
for row in rows:
resource_id = row[0]
access_control_json = row[1]
# Handle NULL or JSON "null" = public access (user:* for read)
# Could be Python None (SQL NULL) or string "null" (JSON null)
# EXCEPTION: files with NULL are PRIVATE (owner-only), not public
is_null = (
access_control_json is None
or access_control_json == 'null'
or (isinstance(access_control_json, str) and access_control_json.strip().lower() == 'null')
)
if is_null:
# Files: NULL = private (no entry needed, owner has implicit access)
# Other resources: NULL = public (insert user:* for read)
if resource_type == 'file':
continue # Private - no entry needed
key = (resource_type, resource_id, 'user', '*', 'read')
if key not in inserted:
try:
conn.execute(
sa.text("""
INSERT INTO access_grant (id, resource_type, resource_id, principal_type, principal_id, permission, created_at)
VALUES (:id, :resource_type, :resource_id, :principal_type, :principal_id, :permission, :created_at)
"""),
{
'id': str(uuid.uuid4()),
'resource_type': resource_type,
'resource_id': resource_id,
'principal_type': 'user',
'principal_id': '*',
'permission': 'read',
'created_at': now,
},
)
inserted.add(key)
except Exception:
pass
continue
# Handle JSON parsing
if isinstance(access_control_json, str):
import json
try:
access_control_json = json.loads(access_control_json)
except Exception:
continue
# Handle {} = private/owner-only - NO entries needed
# Owner access is implicit, no grants to store
if not access_control_json or not isinstance(access_control_json, dict):
continue
# Check if it's effectively empty (no read/write keys with content)
read_data = access_control_json.get('read', {})
write_data = access_control_json.get('write', {})
has_read_grants = read_data.get('group_ids', []) or read_data.get('user_ids', [])
has_write_grants = write_data.get('group_ids', []) or write_data.get('user_ids', [])
if not has_read_grants and not has_write_grants:
# Empty permissions = private, no grants needed
continue
# Extract permissions and insert into access_grant table
for permission in ['read', 'write']:
perm_data = access_control_json.get(permission, {})
if not perm_data:
continue
for group_id in perm_data.get('group_ids', []):
key = (resource_type, resource_id, 'group', group_id, permission)
if key in inserted:
continue
try:
conn.execute(
sa.text("""
INSERT INTO access_grant (id, resource_type, resource_id, principal_type, principal_id, permission, created_at)
VALUES (:id, :resource_type, :resource_id, :principal_type, :principal_id, :permission, :created_at)
"""),
{
'id': str(uuid.uuid4()),
'resource_type': resource_type,
'resource_id': resource_id,
'principal_type': 'group',
'principal_id': group_id,
'permission': permission,
'created_at': now,
},
)
inserted.add(key)
except Exception:
pass
for user_id in perm_data.get('user_ids', []):
key = (resource_type, resource_id, 'user', user_id, permission)
if key in inserted:
continue
try:
conn.execute(
sa.text("""
INSERT INTO access_grant (id, resource_type, resource_id, principal_type, principal_id, permission, created_at)
VALUES (:id, :resource_type, :resource_id, :principal_type, :principal_id, :permission, :created_at)
"""),
{
'id': str(uuid.uuid4()),
'resource_type': resource_type,
'resource_id': resource_id,
'principal_type': 'user',
'principal_id': user_id,
'permission': permission,
'created_at': now,
},
)
inserted.add(key)
except Exception:
pass
# Drop access_control columns from resource tables (only if column still exists)
inspector = sa.inspect(conn)
for table_name, _ in resource_tables:
if table_name not in existing_tables:
continue
cols = {c['name'] for c in inspector.get_columns(table_name)}
if 'access_control' in cols:
with op.batch_alter_table(table_name) as batch:
batch.drop_column('access_control')
def downgrade() -> None:
import json
conn = op.get_bind()
# Resource tables mapping: (table_name, resource_type)
resource_tables = [
('knowledge', 'knowledge'),
('prompt', 'prompt'),
('tool', 'tool'),
('model', 'model'),
('note', 'note'),
('channel', 'channel'),
('file', 'file'),
]
# Step 1: Re-add access_control columns to resource tables
for table_name, _ in resource_tables:
try:
with op.batch_alter_table(table_name) as batch:
batch.add_column(sa.Column('access_control', sa.JSON(), nullable=True))
except Exception:
pass
# Step 2: Query access_grant table and reconstruct JSON for each resource
for table_name, resource_type in resource_tables:
try:
# Get all grants for this resource type
result = conn.execute(
sa.text("""
SELECT resource_id, principal_type, principal_id, permission
FROM access_grant
WHERE resource_type = :resource_type
"""),
{'resource_type': resource_type},
)
rows = result.fetchall()
except Exception:
continue
# Group by resource_id and reconstruct JSON structure
resource_grants = {}
for row in rows:
resource_id = row[0]
principal_type = row[1]
principal_id = row[2]
permission = row[3]
if resource_id not in resource_grants:
resource_grants[resource_id] = {
'is_public': False,
'read': {'group_ids': [], 'user_ids': []},
'write': {'group_ids': [], 'user_ids': []},
}
# Handle public access (user:* for read)
if principal_type == 'user' and principal_id == '*' and permission == 'read':
resource_grants[resource_id]['is_public'] = True
continue
# Add to appropriate list
if permission in ['read', 'write']:
if principal_type == 'group':
if principal_id not in resource_grants[resource_id][permission]['group_ids']:
resource_grants[resource_id][permission]['group_ids'].append(principal_id)
elif principal_type == 'user':
if principal_id not in resource_grants[resource_id][permission]['user_ids']:
resource_grants[resource_id][permission]['user_ids'].append(principal_id)
# Step 3: Update each resource with reconstructed JSON
for resource_id, grants in resource_grants.items():
if grants['is_public']:
# Public = NULL
access_control_value = None
elif (
not grants['read']['group_ids']
and not grants['read']['user_ids']
and not grants['write']['group_ids']
and not grants['write']['user_ids']
):
# No grants = should not happen (would mean no entries), default to {}
access_control_value = json.dumps({})
else:
# Custom permissions
access_control_value = json.dumps(
{
'read': grants['read'],
'write': grants['write'],
}
)
try:
conn.execute(
sa.text(f'UPDATE "{table_name}" SET access_control = :access_control WHERE id = :id'),
{'access_control': access_control_value, 'id': resource_id},
)
except Exception:
pass
# Step 4: Set all resources WITHOUT entries to private
# For files: NULL means private (owner-only), so leave as NULL
# For other resources: {} means private, so update to {}
if resource_type != 'file':
try:
conn.execute(
sa.text(f"""
UPDATE "{table_name}"
SET access_control = :private_value
WHERE id NOT IN (
SELECT DISTINCT resource_id FROM access_grant WHERE resource_type = :resource_type
)
AND access_control IS NULL
"""),
{'private_value': json.dumps({}), 'resource_type': resource_type},
)
except Exception:
pass
# For files, NULL stays NULL - no action needed
# Step 5: Drop the access_grant table
op.drop_index('idx_access_grant_principal', table_name='access_grant')
op.drop_index('idx_access_grant_resource', table_name='access_grant')
op.drop_table('access_grant')
-880
View File
@@ -1,880 +0,0 @@
import logging
import time
import uuid
from typing import Optional
from open_webui.internal.db import Base, get_async_db_context
from pydantic import BaseModel, ConfigDict
from sqlalchemy import BigInteger, Column, Text, UniqueConstraint, and_, delete, or_, select
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.ext.asyncio import AsyncSession
log = logging.getLogger(__name__)
####################
# AccessGrant DB Schema
####################
class AccessGrant(Base):
__tablename__ = 'access_grant'
id = Column(Text, primary_key=True)
resource_type = Column(Text, nullable=False) # "knowledge", "model", "prompt", "tool", "note", "channel", "file"
resource_id = Column(Text, nullable=False)
principal_type = Column(Text, nullable=False) # "user" or "group"
principal_id = Column(Text, nullable=False) # user_id, group_id, or "*" (wildcard for public)
permission = Column(Text, nullable=False) # "read" or "write"
created_at = Column(BigInteger, nullable=False)
__table_args__ = (
UniqueConstraint(
'resource_type',
'resource_id',
'principal_type',
'principal_id',
'permission',
name='uq_access_grant_grant',
),
)
class AccessGrantModel(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: str
resource_type: str
resource_id: str
principal_type: str
principal_id: str
permission: str
created_at: int
class AccessGrantResponse(BaseModel):
"""Slim grant model for API responses — resource context is implicit from the parent."""
id: str
principal_type: str
principal_id: str
permission: str
@classmethod
def from_grant(cls, grant: 'AccessGrantModel') -> 'AccessGrantResponse':
return cls(
id=grant.id,
principal_type=grant.principal_type,
principal_id=grant.principal_id,
permission=grant.permission,
)
####################
# Conversion utilities
####################
def access_control_to_grants(
resource_type: str,
resource_id: str,
access_control: Optional[dict],
) -> list[dict]:
"""
Convert an old-style access_control JSON dict to a flat list of grant dicts.
Semantics:
- None public read (user:* read) except files which are private
- {} private/owner-only (no grants)
- {read: {group_ids, user_ids}, write: {group_ids, user_ids}} specific grants
Returns a list of dicts with keys: resource_type, resource_id, principal_type, principal_id, permission
"""
grants = []
if access_control is None:
# NULL → public read (user:* for read)
# Exception: files with NULL are private (owner-only), no grants needed
if resource_type != 'file':
grants.append(
{
'resource_type': resource_type,
'resource_id': resource_id,
'principal_type': 'user',
'principal_id': '*',
'permission': 'read',
}
)
return grants
# {} → private/owner-only, no grants
if not access_control:
return grants
# Parse structured permissions
for permission in ['read', 'write']:
perm_data = access_control.get(permission, {})
if not perm_data:
continue
for group_id in perm_data.get('group_ids', []):
grants.append(
{
'resource_type': resource_type,
'resource_id': resource_id,
'principal_type': 'group',
'principal_id': group_id,
'permission': permission,
}
)
for user_id in perm_data.get('user_ids', []):
grants.append(
{
'resource_type': resource_type,
'resource_id': resource_id,
'principal_type': 'user',
'principal_id': user_id,
'permission': permission,
}
)
return grants
def normalize_access_grants(access_grants: Optional[list]) -> list[dict]:
"""
Normalize direct access_grants payloads from API forms.
Keeps only valid grants and removes duplicates by
(principal_type, principal_id, permission).
"""
if not access_grants:
return []
deduped = {}
for grant in access_grants:
if isinstance(grant, BaseModel):
grant = grant.model_dump()
if not isinstance(grant, dict):
continue
principal_type = grant.get('principal_type')
principal_id = grant.get('principal_id')
permission = grant.get('permission')
if principal_type not in ('user', 'group'):
continue
if permission not in ('read', 'write'):
continue
if not isinstance(principal_id, str) or not principal_id:
continue
key = (principal_type, principal_id, permission)
deduped[key] = {
'id': (grant.get('id') if isinstance(grant.get('id'), str) and grant.get('id') else str(uuid.uuid4())),
'principal_type': principal_type,
'principal_id': principal_id,
'permission': permission,
}
return list(deduped.values())
def has_public_read_access_grant(access_grants: Optional[list]) -> bool:
"""
Returns True when a direct grant list includes wildcard public-read.
"""
for grant in normalize_access_grants(access_grants):
if grant['principal_type'] == 'user' and grant['principal_id'] == '*' and grant['permission'] == 'read':
return True
return False
def has_public_write_access_grant(access_grants: Optional[list]) -> bool:
"""
Returns True when a direct grant list includes wildcard public-write.
"""
for grant in normalize_access_grants(access_grants):
if grant['principal_type'] == 'user' and grant['principal_id'] == '*' and grant['permission'] == 'write':
return True
return False
def has_user_access_grant(access_grants: Optional[list]) -> bool:
"""
Returns True when a direct grant list includes any non-wildcard user grant.
"""
for grant in normalize_access_grants(access_grants):
if grant['principal_type'] == 'user' and grant['principal_id'] != '*':
return True
return False
def strip_user_access_grants(access_grants: Optional[list]) -> list:
"""
Remove all non-wildcard user grants from the list.
Keeps group grants and the public wildcard (user:*) intact.
"""
if not access_grants:
return []
return [
grant
for grant in access_grants
if not (
(grant.get('principal_type') if isinstance(grant, dict) else getattr(grant, 'principal_type', None))
== 'user'
and (grant.get('principal_id') if isinstance(grant, dict) else getattr(grant, 'principal_id', None)) != '*'
)
]
def grants_to_access_control(grants: list) -> Optional[dict]:
"""
Convert a list of grant objects (AccessGrantModel or AccessGrantResponse)
back to the old-style access_control JSON dict for backward compatibility.
Semantics:
- [] (empty) {} (private/owner-only)
- Contains user:*:read None (public), but write grants are preserved
- Otherwise {read: {group_ids, user_ids}, write: {group_ids, user_ids}}
Note: "public" (user:*:read) still allows additional write permissions
to coexist. When the wildcard read is present the function returns None
for the legacy dict, so callers that need write info should inspect the
grants list directly.
"""
if not grants:
return {} # No grants = private/owner-only
result = {
'read': {'group_ids': [], 'user_ids': []},
'write': {'group_ids': [], 'user_ids': []},
}
is_public = False
for grant in grants:
if grant.principal_type == 'user' and grant.principal_id == '*' and grant.permission == 'read':
is_public = True
continue # Don't add wildcard to user_ids list
if grant.permission not in ('read', 'write'):
continue
if grant.principal_type == 'group':
if grant.principal_id not in result[grant.permission]['group_ids']:
result[grant.permission]['group_ids'].append(grant.principal_id)
elif grant.principal_type == 'user':
if grant.principal_id not in result[grant.permission]['user_ids']:
result[grant.permission]['user_ids'].append(grant.principal_id)
if is_public:
return None # Public read access
return result
####################
# Table Operations
####################
class AccessGrantsTable:
async def grant_access(
self,
resource_type: str,
resource_id: str,
principal_type: str,
principal_id: str,
permission: str,
db: Optional[AsyncSession] = None,
) -> Optional[AccessGrantModel]:
"""Add a single access grant. Idempotent (ignores duplicates)."""
async with get_async_db_context(db) as db:
# Check for existing grant
result = await db.execute(
select(AccessGrant).filter_by(
resource_type=resource_type,
resource_id=resource_id,
principal_type=principal_type,
principal_id=principal_id,
permission=permission,
)
)
existing = result.scalars().first()
if existing:
return AccessGrantModel.model_validate(existing)
grant = AccessGrant(
id=str(uuid.uuid4()),
resource_type=resource_type,
resource_id=resource_id,
principal_type=principal_type,
principal_id=principal_id,
permission=permission,
created_at=int(time.time()),
)
db.add(grant)
await db.commit()
await db.refresh(grant)
return AccessGrantModel.model_validate(grant)
async def revoke_access(
self,
resource_type: str,
resource_id: str,
principal_type: str,
principal_id: str,
permission: str,
db: Optional[AsyncSession] = None,
) -> bool:
"""Remove a single access grant."""
async with get_async_db_context(db) as db:
result = await db.execute(
delete(AccessGrant).filter_by(
resource_type=resource_type,
resource_id=resource_id,
principal_type=principal_type,
principal_id=principal_id,
permission=permission,
)
)
await db.commit()
return result.rowcount > 0
async def revoke_all_access(
self,
resource_type: str,
resource_id: str,
db: Optional[AsyncSession] = None,
) -> int:
"""Remove all access grants for a resource."""
async with get_async_db_context(db) as db:
result = await db.execute(
delete(AccessGrant).filter_by(
resource_type=resource_type,
resource_id=resource_id,
)
)
await db.commit()
return result.rowcount
async def set_access_control(
self,
resource_type: str,
resource_id: str,
access_control: Optional[dict],
db: Optional[AsyncSession] = None,
) -> list[AccessGrantModel]:
"""
Replace all grants for a resource from an access_control JSON dict.
This is the primary bridge for backward compat with the frontend.
"""
async with get_async_db_context(db) as db:
# Delete all existing grants for this resource
await db.execute(
delete(AccessGrant).filter_by(
resource_type=resource_type,
resource_id=resource_id,
)
)
# Convert JSON to grant dicts
grant_dicts = access_control_to_grants(resource_type, resource_id, access_control)
# Insert new grants
results = []
for grant_dict in grant_dicts:
grant = AccessGrant(
id=str(uuid.uuid4()),
**grant_dict,
created_at=int(time.time()),
)
db.add(grant)
results.append(grant)
await db.commit()
return [AccessGrantModel.model_validate(g) for g in results]
async def set_access_grants(
self,
resource_type: str,
resource_id: str,
access_grants: Optional[list],
db: Optional[AsyncSession] = None,
) -> list[AccessGrantModel]:
"""
Replace all grants for a resource from a direct access_grants list.
"""
async with get_async_db_context(db) as db:
await db.execute(
delete(AccessGrant).filter_by(
resource_type=resource_type,
resource_id=resource_id,
)
)
normalized_grants = normalize_access_grants(access_grants)
results = []
for grant_dict in normalized_grants:
grant = AccessGrant(
id=str(uuid.uuid4()),
resource_type=resource_type,
resource_id=resource_id,
principal_type=grant_dict['principal_type'],
principal_id=grant_dict['principal_id'],
permission=grant_dict['permission'],
created_at=int(time.time()),
)
db.add(grant)
results.append(grant)
await db.commit()
return [AccessGrantModel.model_validate(g) for g in results]
async def get_access_control(
self,
resource_type: str,
resource_id: str,
db: Optional[AsyncSession] = None,
) -> Optional[dict]:
"""
Reconstruct the old-style access_control JSON dict from grants.
For backward compat with the frontend.
"""
async with get_async_db_context(db) as db:
result = await db.execute(
select(AccessGrant).filter_by(
resource_type=resource_type,
resource_id=resource_id,
)
)
grants = result.scalars().all()
grant_models = [AccessGrantModel.model_validate(g) for g in grants]
return grants_to_access_control(grant_models)
async def get_grants_by_resource(
self,
resource_type: str,
resource_id: str,
db: Optional[AsyncSession] = None,
) -> list[AccessGrantModel]:
"""Get all grants for a specific resource."""
async with get_async_db_context(db) as db:
result = await db.execute(
select(AccessGrant).filter_by(
resource_type=resource_type,
resource_id=resource_id,
)
)
grants = result.scalars().all()
return [AccessGrantModel.model_validate(g) for g in grants]
async def get_grants_by_resources(
self,
resource_type: str,
resource_ids: list[str],
db: Optional[AsyncSession] = None,
) -> dict[str, list[AccessGrantModel]]:
"""Batch-fetch grants for multiple resources. Returns {resource_id: [grants]}."""
if not resource_ids:
return {}
async with get_async_db_context(db) as db:
result = await db.execute(
select(AccessGrant).filter(
AccessGrant.resource_type == resource_type,
AccessGrant.resource_id.in_(resource_ids),
)
)
grants = result.scalars().all()
result_dict: dict[str, list[AccessGrantModel]] = {rid: [] for rid in resource_ids}
for g in grants:
result_dict[g.resource_id].append(AccessGrantModel.model_validate(g))
return result_dict
async def has_access(
self,
user_id: str,
resource_type: str,
resource_id: str,
permission: str = 'read',
user_group_ids: Optional[set[str]] = None,
db: Optional[AsyncSession] = None,
) -> bool:
"""
Check if a user has the specified permission on a resource.
Access is granted if any of the following is true:
- There's a grant for user:* (public) with the requested permission
- There's a grant for the specific user with the requested permission
- There's a grant for any of the user's groups with the requested permission
"""
async with get_async_db_context(db) as db:
# Build conditions for matching grants
conditions = [
# Public access
and_(
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == '*',
),
# Direct user access
and_(
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == user_id,
),
]
# Group access
if user_group_ids is None:
from open_webui.models.groups import Groups
user_groups = await Groups.get_groups_by_member_id(user_id, db=db)
user_group_ids = {group.id for group in user_groups}
if user_group_ids:
conditions.append(
and_(
AccessGrant.principal_type == 'group',
AccessGrant.principal_id.in_(user_group_ids),
)
)
result = await db.execute(
select(AccessGrant)
.filter(
AccessGrant.resource_type == resource_type,
AccessGrant.resource_id == resource_id,
AccessGrant.permission == permission,
or_(*conditions),
)
.limit(1)
)
grant = result.scalars().first()
return grant is not None
async def get_accessible_resource_ids(
self,
user_id: str,
resource_type: str,
resource_ids: list[str],
permission: str = 'read',
user_group_ids: Optional[set[str]] = None,
db: Optional[AsyncSession] = None,
) -> set[str]:
"""
Batch check: return the subset of resource_ids that the user can access.
This replaces calling has_access() in a loop (N+1) with a single query.
"""
if not resource_ids:
return set()
async with get_async_db_context(db) as db:
conditions = [
and_(
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == '*',
),
and_(
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == user_id,
),
]
if user_group_ids is None:
from open_webui.models.groups import Groups
user_groups = await Groups.get_groups_by_member_id(user_id, db=db)
user_group_ids = {group.id for group in user_groups}
if user_group_ids:
conditions.append(
and_(
AccessGrant.principal_type == 'group',
AccessGrant.principal_id.in_(user_group_ids),
)
)
result = await db.execute(
select(AccessGrant.resource_id)
.filter(
AccessGrant.resource_type == resource_type,
AccessGrant.resource_id.in_(resource_ids),
AccessGrant.permission == permission,
or_(*conditions),
)
.distinct()
)
rows = result.all()
return {row[0] for row in rows}
async def get_users_with_access(
self,
resource_type: str,
resource_id: str,
permission: str = 'read',
db: Optional[AsyncSession] = None,
) -> list:
"""
Get all users who have the specified permission on a resource.
Returns a list of UserModel instances.
"""
from open_webui.models.groups import Groups
from open_webui.models.users import UserModel, Users
async with get_async_db_context(db) as db:
result = await db.execute(
select(AccessGrant).filter_by(
resource_type=resource_type,
resource_id=resource_id,
permission=permission,
)
)
grants = result.scalars().all()
# Check for public access
for grant in grants:
if grant.principal_type == 'user' and grant.principal_id == '*':
result = await Users.get_users(filter={'roles': ['!pending']}, db=db)
return result.get('users', [])
user_ids_with_access = set()
for grant in grants:
if grant.principal_type == 'user':
user_ids_with_access.add(grant.principal_id)
elif grant.principal_type == 'group':
group_user_ids = await Groups.get_group_user_ids_by_id(grant.principal_id, db=db)
if group_user_ids:
user_ids_with_access.update(group_user_ids)
if not user_ids_with_access:
return []
return await Users.get_users_by_user_ids(list(user_ids_with_access), db=db)
def has_permission_filter(
self,
db,
query,
DocumentModel,
filter: dict,
resource_type: str,
permission: str = 'read',
):
"""
Apply access control filtering to a SQLAlchemy query by JOINing with access_grant.
This replaces the old JSON-column-based filtering with a proper relational JOIN.
Note: This method builds SQLAlchemy expressions and does NOT perform I/O itself,
so it remains synchronous. The caller is responsible for executing the query
asynchronously with `await db.execute(...)`.
"""
group_ids = filter.get('group_ids', [])
user_id = filter.get('user_id')
if permission == 'read_only':
return self._has_read_only_permission_filter(db, query, DocumentModel, filter, resource_type)
# Build principal conditions
principal_conditions = []
if group_ids or user_id:
# Public access: user:* read
principal_conditions.append(
and_(
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == '*',
)
)
if user_id:
# Owner always has access
principal_conditions.append(DocumentModel.user_id == user_id)
# Direct user grant
principal_conditions.append(
and_(
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == user_id,
)
)
if group_ids:
# Group grants
principal_conditions.append(
and_(
AccessGrant.principal_type == 'group',
AccessGrant.principal_id.in_(group_ids),
)
)
if not principal_conditions:
return query
# LEFT JOIN access_grant and filter
# We use a subquery approach to avoid duplicates from multiple matching grants
from sqlalchemy import exists as sa_exists
grant_exists = (
select(AccessGrant.id)
.where(
AccessGrant.resource_type == resource_type,
AccessGrant.resource_id == DocumentModel.id,
AccessGrant.permission == permission,
or_(
and_(
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == '*',
),
*(
[
and_(
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == user_id,
)
]
if user_id
else []
),
*(
[
and_(
AccessGrant.principal_type == 'group',
AccessGrant.principal_id.in_(group_ids),
)
]
if group_ids
else []
),
),
)
.correlate(DocumentModel)
.exists()
)
# Owner OR has a matching grant
owner_or_grant = [grant_exists]
if user_id:
owner_or_grant.append(DocumentModel.user_id == user_id)
query = query.filter(or_(*owner_or_grant))
return query
def _has_read_only_permission_filter(
self,
db,
query,
DocumentModel,
filter: dict,
resource_type: str,
):
"""
Filter for items where user has read BUT NOT write access.
Public items are NOT considered read_only.
Note: This method builds SQLAlchemy expressions and does NOT perform I/O itself,
so it remains synchronous. The caller is responsible for executing the query
asynchronously with `await db.execute(...)`.
"""
group_ids = filter.get('group_ids', [])
user_id = filter.get('user_id')
from sqlalchemy import exists as sa_exists
# Has read grant (not public)
read_grant_exists = (
select(AccessGrant.id)
.where(
AccessGrant.resource_type == resource_type,
AccessGrant.resource_id == DocumentModel.id,
AccessGrant.permission == 'read',
or_(
*(
[
and_(
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == user_id,
)
]
if user_id
else []
),
*(
[
and_(
AccessGrant.principal_type == 'group',
AccessGrant.principal_id.in_(group_ids),
)
]
if group_ids
else []
),
),
)
.correlate(DocumentModel)
.exists()
)
# Does NOT have write grant
write_grant_exists = (
select(AccessGrant.id)
.where(
AccessGrant.resource_type == resource_type,
AccessGrant.resource_id == DocumentModel.id,
AccessGrant.permission == 'write',
or_(
*(
[
and_(
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == user_id,
)
]
if user_id
else []
),
*(
[
and_(
AccessGrant.principal_type == 'group',
AccessGrant.principal_id.in_(group_ids),
)
]
if group_ids
else []
),
),
)
.correlate(DocumentModel)
.exists()
)
# Is NOT public
public_grant_exists = (
select(AccessGrant.id)
.where(
AccessGrant.resource_type == resource_type,
AccessGrant.resource_id == DocumentModel.id,
AccessGrant.permission == 'read',
AccessGrant.principal_type == 'user',
AccessGrant.principal_id == '*',
)
.correlate(DocumentModel)
.exists()
)
conditions = [read_grant_exists, ~write_grant_exists, ~public_grant_exists]
# Not owner
if user_id:
conditions.append(DocumentModel.user_id != user_id)
query = query.filter(and_(*conditions))
return query
AccessGrants = AccessGrantsTable()
+129 -162
View File
@@ -1,59 +1,60 @@
"""Auth credential models and data-access layer."""
from __future__ import annotations
import logging
import uuid
from typing import Optional
import bcrypt
from open_webui.internal.db import Base, JSONField, get_async_db_context
from open_webui.models.users import User, UserModel, UserProfileImageResponse, Users
from open_webui.utils.validate import validate_profile_image_url
from pydantic import BaseModel, field_validator
from sqlalchemy import Boolean, Column, String, Text, delete, select, update
from sqlalchemy.ext.asyncio import AsyncSession
from open_webui.internal.db import Base, get_db
from open_webui.models.users import UserModel, Users
from open_webui.env import SRC_LOG_LEVELS
from pydantic import BaseModel
from sqlalchemy import Boolean, Column, String, Text
log = logging.getLogger(__name__)
log.setLevel(SRC_LOG_LEVELS["MODELS"])
# Pre-computed hash verified on signin paths that lack a real credential
# (unknown user, inactive account) so response timing cannot reveal
# whether an account exists (CWE-208).
PLACEHOLDER_HASH = bcrypt.hashpw(b'placeholder', bcrypt.gensalt()).decode('utf-8')
####################
# DB MODEL
####################
class Auth(Base): # credential ↔ user linkage
"""Maps a user ID to an email/password pair with an active flag."""
class Auth(Base):
__tablename__ = "auth"
__tablename__ = 'auth'
id = Column(String, primary_key=True, unique=True) # mirrors User.id
email = Column(String) # login address, kept in sync with User.email
password = Column(Text) # argon2 / bcrypt hash
active = Column(Boolean) # account soft-disable toggle
id = Column(String, primary_key=True, unique=True)
email = Column(String)
password = Column(Text)
active = Column(Boolean)
class AuthModel(BaseModel):
"""Pydantic mirror of the ``auth`` table row."""
id: str
email: str
password: str
active: bool = True
class Token(BaseModel):
"""JWT bearer-token response wrapper."""
####################
# Forms
####################
class Token(BaseModel):
token: str
token_type: str
class ApiKey(BaseModel):
api_key: str | None = None
api_key: Optional[str] = None
class SigninResponse(Token, UserProfileImageResponse):
class UserResponse(BaseModel):
id: str
email: str
name: str
role: str
profile_image_url: str
class SigninResponse(Token, UserResponse):
pass
@@ -80,160 +81,126 @@ class SignupForm(BaseModel):
name: str
email: str
password: str
profile_image_url: str | None = '/user.png'
@field_validator('profile_image_url')
@classmethod
def check_profile_image_url(cls, v: str | None) -> str | None:
if v is not None:
return validate_profile_image_url(v)
return v
profile_image_url: Optional[str] = "/user.png"
class AddUserForm(SignupForm):
role: str | None = 'pending'
# --- data-access layer ---
role: Optional[str] = "pending"
class AuthsTable:
"""Provides CRUD operations for the Auth ↔ User lifecycle."""
async def insert_new_auth(
def insert_new_auth(
self,
email: str,
password: str,
name: str,
profile_image_url: str = '/user.png',
role: str = 'pending',
oauth: dict | None = None,
db: AsyncSession | None = None,
) -> UserModel | None:
"""Create an Auth + User pair inside a single transaction."""
async with get_async_db_context(db) as session:
log.info('insert_new_auth')
profile_image_url: str = "/user.png",
role: str = "pending",
oauth_sub: Optional[str] = None,
) -> Optional[UserModel]:
with get_db() as db:
log.info("insert_new_auth")
new_id = str(uuid.uuid4())
id = str(uuid.uuid4())
credential = Auth(
id=new_id,
email=email,
password=password,
active=True,
auth = AuthModel(
**{"id": id, "email": email, "password": password, "active": True}
)
session.add(credential)
result = Auth(**auth.model_dump())
db.add(result)
created_user = await Users.insert_new_user(
new_id,
name,
email,
profile_image_url,
role,
oauth=oauth,
db=session,
user = Users.insert_new_user(
id, name, email, profile_image_url, role, oauth_sub
)
# persist both records and reload generated defaults
await session.commit()
await session.refresh(credential)
return created_user if credential and created_user else None
async def authenticate_user(
self,
email: str,
verify_password: callable,
db: AsyncSession | None = None,
) -> UserModel | None:
"""Verify email + password credentials and return the matching user."""
log.info('authenticate_user: %s', email)
resolved = await Users.get_user_by_email(email, db=db)
if not resolved:
verify_password(PLACEHOLDER_HASH)
return
# load the credential row and verify the password hash
async with get_async_db_context(db) as session:
credential = await session.get(Auth, resolved.id)
if not credential or not credential.active:
verify_password(PLACEHOLDER_HASH)
return
if not verify_password(credential.password):
return
return resolved
db.commit()
db.refresh(result)
async def authenticate_user_by_api_key(
self,
api_key: str,
db: AsyncSession | None = None,
) -> UserModel | None:
"""Look up the user that owns the given API key."""
log.info('authenticate_user_by_api_key')
if result and user:
return user
else:
return None
def authenticate_user(
self, email: str, verify_password: callable
) -> Optional[UserModel]:
log.info(f"authenticate_user: {email}")
user = Users.get_user_by_email(email)
if not user:
return None
try:
with get_db() as db:
auth = db.query(Auth).filter_by(id=user.id, active=True).first()
if auth:
if verify_password(auth.password):
return user
else:
return None
else:
return None
except Exception:
return None
def authenticate_user_by_api_key(self, api_key: str) -> Optional[UserModel]:
log.info(f"authenticate_user_by_api_key: {api_key}")
# if no api_key, return None
if not api_key:
return
# delegate to the Users model for the actual lookup
return await Users.get_user_by_api_key(api_key, db=db)
return None
async def authenticate_user_by_email(
self,
email: str,
db: AsyncSession | None = None,
) -> UserModel | None:
"""Single-query auth via JOIN on Auth ↔ User, filtered by active flag."""
log.info('authenticate_user_by_email: %s', email)
# single JOIN avoids N+1 — returns (Auth, User) tuple or None
async with get_async_db_context(db) as session:
joined_query = (
select(Auth, User).join(User, Auth.id == User.id).where(Auth.email == email, Auth.active.is_(True))
)
match = (await session.execute(joined_query)).first()
if not match:
return
_, found_user = match
return UserModel.model_validate(found_user)
try:
user = Users.get_user_by_api_key(api_key)
return user if user else None
except Exception:
return False
async def update_email_by_id(
self,
user_id: str,
email: str,
db: AsyncSession | None = None,
) -> bool:
"""Set a new email on the auth record and propagate to the user row."""
async with get_async_db_context(db) as session:
auth_row = await session.get(Auth, user_id)
if auth_row is None:
return False
auth_row.email = email
await session.commit()
await Users.update_user_by_id(user_id, {'email': email}, db=session)
return True
# --- password modification ---
def authenticate_user_by_email(self, email: str) -> Optional[UserModel]:
log.info(f"authenticate_user_by_email: {email}")
try:
with get_db() as db:
auth = db.query(Auth).filter_by(email=email, active=True).first()
if auth:
user = Users.get_user_by_id(auth.id)
return user
except Exception:
return None
async def update_user_password_by_id(
self,
user_id: str,
new_password: str,
db: AsyncSession | None = None,
) -> bool:
"""Set a new password hash for an existing user."""
async with get_async_db_context(db) as session:
auth_row = await session.get(Auth, user_id)
if auth_row is None:
return False
auth_row.password = new_password
await session.commit()
return True
def update_user_password_by_id(self, id: str, new_password: str) -> bool:
try:
with get_db() as db:
result = (
db.query(Auth).filter_by(id=id).update({"password": new_password})
)
db.commit()
return True if result == 1 else False
except Exception:
return False
async def delete_auth_by_id(
self,
id: str,
db: AsyncSession | None = None,
) -> bool:
"""Remove a user and their auth credential in one transaction."""
async with get_async_db_context(db) as session:
if not await Users.delete_user_by_id(id, db=session):
return False
await session.execute(delete(Auth).where(Auth.id == id))
await session.commit()
return True
def update_email_by_id(self, id: str, email: str) -> bool:
try:
with get_db() as db:
result = db.query(Auth).filter_by(id=id).update({"email": email})
db.commit()
return True if result == 1 else False
except Exception:
return False
def delete_auth_by_id(self, id: str) -> bool:
try:
with get_db() as db:
# Delete User
result = Users.delete_user_by_id(id)
if result:
db.query(Auth).filter_by(id=id).delete()
db.commit()
return True
else:
return False
except Exception:
return False
Auths = AuthsTable() # singleton — module-level instance
Auths = AuthsTable()
-420
View File
@@ -1,420 +0,0 @@
import logging
import time
from typing import Optional
from uuid import uuid4
from open_webui.internal.db import Base, get_async_db_context
from pydantic import BaseModel, ConfigDict
from sqlalchemy import JSON, BigInteger, Boolean, Column, Index, String, Text, cast, delete, func, or_, select, update
from sqlalchemy.ext.asyncio import AsyncSession
log = logging.getLogger(__name__)
####################
# Automation DB Schema
####################
class Automation(Base):
__tablename__ = 'automation'
id = Column(Text, primary_key=True)
user_id = Column(Text, nullable=False)
name = Column(Text, nullable=False)
data = Column(JSON, nullable=False) # {prompt, model_id, rrule}
meta = Column(JSON, nullable=True)
is_active = Column(Boolean, nullable=False, default=True)
last_run_at = Column(BigInteger, nullable=True)
next_run_at = Column(BigInteger, nullable=True)
created_at = Column(BigInteger, nullable=False)
updated_at = Column(BigInteger, nullable=False)
__table_args__ = (Index('ix_automation_next_run', 'next_run_at'),)
class AutomationRun(Base):
__tablename__ = 'automation_run'
id = Column(Text, primary_key=True)
automation_id = Column(Text, nullable=False)
chat_id = Column(Text, nullable=True)
status = Column(Text, nullable=False) # success | error
error = Column(Text, nullable=True)
created_at = Column(BigInteger, nullable=False)
__table_args__ = (
Index('ix_automation_run_automation_id', 'automation_id'),
Index('ix_automation_run_aid_created', 'automation_id', 'created_at'),
)
####################
# Pydantic Models
####################
class AutomationTerminalConfig(BaseModel):
server_id: str
cwd: Optional[str] = None
class AutomationData(BaseModel):
prompt: str
model_id: str
rrule: str
terminal: Optional[AutomationTerminalConfig] = None
class AutomationModel(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: str
user_id: str
name: str
data: dict
meta: Optional[dict] = None
is_active: bool
last_run_at: Optional[int] = None
next_run_at: Optional[int] = None
created_at: int
updated_at: int
class AutomationRunModel(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: str
automation_id: str
chat_id: Optional[str] = None
status: str
error: Optional[str] = None
created_at: int
class AutomationForm(BaseModel):
name: str
data: AutomationData
meta: Optional[dict] = None
is_active: Optional[bool] = True
class AutomationResponse(AutomationModel):
last_run: Optional[AutomationRunModel] = None
next_runs: Optional[list[int]] = None
class AutomationListResponse(BaseModel):
items: list[AutomationModel]
total: int
####################
# AutomationTable
####################
class AutomationTable:
async def insert(
self,
user_id: str,
form: AutomationForm,
next_run_at: int,
db: Optional[AsyncSession] = None,
) -> AutomationModel:
async with get_async_db_context(db) as db:
now = int(time.time_ns())
row = Automation(
id=str(uuid4()),
user_id=user_id,
name=form.name,
data=form.data.model_dump(),
meta=form.meta,
is_active=form.is_active,
next_run_at=next_run_at,
created_at=now,
updated_at=now,
)
db.add(row)
await db.commit()
await db.refresh(row)
return AutomationModel.model_validate(row)
async def count_by_user(self, user_id: str, db: Optional[AsyncSession] = None) -> int:
async with get_async_db_context(db) as db:
result = await db.execute(select(func.count()).select_from(Automation).filter_by(user_id=user_id))
return result.scalar()
async def get_by_id(self, id: str, db: Optional[AsyncSession] = None) -> Optional[AutomationModel]:
async with get_async_db_context(db) as db:
row = await db.get(Automation, id)
return AutomationModel.model_validate(row) if row else None
async def get_active_by_user(self, user_id: str, db: Optional[AsyncSession] = None) -> list[AutomationModel]:
"""Get active automations for a user (for calendar RRULE expansion)."""
async with get_async_db_context(db) as db:
result = await db.execute(
select(Automation).filter_by(user_id=user_id, is_active=True).order_by(Automation.created_at.desc())
)
return [AutomationModel.model_validate(r) for r in result.scalars().all()]
async def search_automations(
self,
user_id: str,
query: Optional[str] = None,
status: Optional[str] = None,
skip: int = 0,
limit: int = 30,
db: Optional[AsyncSession] = None,
) -> 'AutomationListResponse':
async with get_async_db_context(db) as db:
stmt = select(Automation).filter_by(user_id=user_id)
if query:
search = f'%{query}%'
# Search in name and prompt inside JSON data
stmt = stmt.filter(
or_(
Automation.name.ilike(search),
cast(Automation.data, String).ilike(search),
)
)
if status == 'active':
stmt = stmt.filter(Automation.is_active == True)
elif status == 'paused':
stmt = stmt.filter(Automation.is_active == False)
stmt = stmt.order_by(Automation.created_at.desc())
# Get total count
count_result = await db.execute(select(func.count()).select_from(stmt.subquery()))
total = count_result.scalar()
if skip:
stmt = stmt.offset(skip)
if limit:
stmt = stmt.limit(limit)
result = await db.execute(stmt)
rows = result.scalars().all()
return AutomationListResponse(
items=[AutomationModel.model_validate(r) for r in rows],
total=total,
)
async def update_by_id(
self,
id: str,
form: AutomationForm,
next_run_at: int,
db: Optional[AsyncSession] = None,
) -> Optional[AutomationModel]:
async with get_async_db_context(db) as db:
row = await db.get(Automation, id)
if not row:
return None
row.name = form.name
row.data = form.data.model_dump()
row.meta = form.meta
if form.is_active is not None:
row.is_active = form.is_active
row.next_run_at = next_run_at
row.updated_at = int(time.time_ns())
await db.commit()
await db.refresh(row)
return AutomationModel.model_validate(row)
async def toggle(
self,
id: str,
next_run_at: Optional[int],
db: Optional[AsyncSession] = None,
) -> Optional[AutomationModel]:
async with get_async_db_context(db) as db:
row = await db.get(Automation, id)
if not row:
return None
row.is_active = not row.is_active
row.next_run_at = next_run_at if row.is_active else None
row.updated_at = int(time.time_ns())
await db.commit()
await db.refresh(row)
return AutomationModel.model_validate(row)
async def delete(self, id: str, db: Optional[AsyncSession] = None) -> bool:
async with get_async_db_context(db) as db:
row = await db.get(Automation, id)
if not row:
return False
await db.delete(row)
await db.commit()
return True
async def claim_due(self, now_ns: int, limit: int = 10, db: Optional[AsyncSession] = None) -> list[AutomationModel]:
"""
Atomically claim due automations for execution.
Advances next_run_at immediately so the row can never be
double-claimed. On PostgreSQL, uses FOR UPDATE SKIP LOCKED
for zero-contention distributed work claiming.
"""
async with get_async_db_context(db) as db:
stmt = (
select(Automation)
.where(
Automation.is_active == True,
Automation.next_run_at <= now_ns,
)
.order_by(Automation.next_run_at)
.limit(limit)
)
if db.bind.dialect.name == 'postgresql':
stmt = stmt.with_for_update(skip_locked=True)
result = await db.execute(stmt)
rows = result.scalars().all()
from open_webui.utils.automations import next_run_ns
# Batch-fetch user timezones so rescheduling respects each
# user's local timezone instead of falling back to server time.
user_ids = list({row.user_id for row in rows})
timezone_by_user_id: dict[str, Optional[str]] = {}
if user_ids:
from open_webui.models.users import User
tz_result = await db.execute(select(User.id, User.timezone).where(User.id.in_(user_ids)))
timezone_by_user_id = {uid: tz for uid, tz in tz_result.all()}
for row in rows:
row.last_run_at = now_ns
row.next_run_at = next_run_ns(row.data.get('rrule', ''), tz=timezone_by_user_id.get(row.user_id))
await db.commit()
return [AutomationModel.model_validate(r) for r in rows]
####################
# AutomationRunTable
####################
class AutomationRunTable:
async def insert(
self,
automation_id: str,
status: str,
chat_id: Optional[str] = None,
error: Optional[str] = None,
db: Optional[AsyncSession] = None,
) -> AutomationRunModel:
async with get_async_db_context(db) as db:
row = AutomationRun(
id=str(uuid4()),
automation_id=automation_id,
chat_id=chat_id,
status=status,
error=error,
created_at=int(time.time_ns()),
)
db.add(row)
await db.commit()
await db.refresh(row)
return AutomationRunModel.model_validate(row)
async def get_latest(self, automation_id: str, db: Optional[AsyncSession] = None) -> Optional[AutomationRunModel]:
async with get_async_db_context(db) as db:
result = await db.execute(
select(AutomationRun)
.filter_by(automation_id=automation_id)
.order_by(AutomationRun.created_at.desc())
.limit(1)
)
row = result.scalars().first()
return AutomationRunModel.model_validate(row) if row else None
async def get_latest_batch(
self, automation_ids: list[str], db: Optional[AsyncSession] = None
) -> dict[str, AutomationRunModel]:
"""Fetch the latest run for each automation in a single query."""
if not automation_ids:
return {}
async with get_async_db_context(db) as db:
# Subquery: max created_at per automation_id
subq = (
select(
AutomationRun.automation_id,
func.max(AutomationRun.created_at).label('max_created'),
)
.filter(AutomationRun.automation_id.in_(automation_ids))
.group_by(AutomationRun.automation_id)
.subquery()
)
result = await db.execute(
select(AutomationRun).join(
subq,
(AutomationRun.automation_id == subq.c.automation_id)
& (AutomationRun.created_at == subq.c.max_created),
)
)
rows = result.scalars().all()
return {row.automation_id: AutomationRunModel.model_validate(row) for row in rows}
async def get_by_automation(
self,
automation_id: str,
skip: int = 0,
limit: int = 50,
db: Optional[AsyncSession] = None,
) -> list[AutomationRunModel]:
async with get_async_db_context(db) as db:
result = await db.execute(
select(AutomationRun)
.filter_by(automation_id=automation_id)
.order_by(AutomationRun.created_at.desc())
.offset(skip)
.limit(limit)
)
rows = result.scalars().all()
return [AutomationRunModel.model_validate(r) for r in rows]
async def delete_by_automation(self, automation_id: str, db: Optional[AsyncSession] = None) -> int:
async with get_async_db_context(db) as db:
result = await db.execute(delete(AutomationRun).filter_by(automation_id=automation_id))
await db.commit()
return result.rowcount
async def get_runs_by_user_range(
self,
user_id: str,
start_ns: int,
end_ns: int,
limit: int = 500,
db: Optional[AsyncSession] = None,
) -> list[tuple['AutomationRunModel', 'AutomationModel']]:
"""Get runs within a date range for a user, joined with parent automation."""
async with get_async_db_context(db) as db:
result = await db.execute(
select(AutomationRun, Automation)
.join(Automation, Automation.id == AutomationRun.automation_id)
.filter(
Automation.user_id == user_id,
AutomationRun.created_at >= start_ns,
AutomationRun.created_at < end_ns,
)
.order_by(AutomationRun.created_at.desc())
.limit(limit)
)
return [
(AutomationRunModel.model_validate(run), AutomationModel.model_validate(auto))
for run, auto in result.all()
]
Automations = AutomationTable()
AutomationRuns = AutomationRunTable()

Some files were not shown because too many files have changed in this diff Show More