diff --git a/docs/guide/17-txsql_architecture.md b/docs/guide/17-txsql_architecture.md index 3495a05..80f87f7 100644 --- a/docs/guide/17-txsql_architecture.md +++ b/docs/guide/17-txsql_architecture.md @@ -99,7 +99,19 @@ InnoDB的底层结构主要由两部分组成:内存结构和物理文件。 ### 物理文件 TXSQL的物理文件主要有如下集中类型: + + + - 数据文件(data file)是存储引擎存储数据的主要文件,它是由一个或多个固定大小的数据页(data page)组成的文件,每个数据页的大小默认为16KB。数据文件可以是单个文件(ibdata1)或多个文件(ibdata1, ibdata2, …),也可以是按表分割的文件(table_name.ibd)。数据文件中存储了表的数据、索引、元数据等信息。 + + + - 日志文件(log file)是存储日志的主要文件,它是由两个或多个固定大小的日志段(log segment)组成的循环文件,每个日志段的大小默认为48MB。日志文件可以是单个文件(ib_logfile0)或多个文件(ib_logfile0, ib_logfile1, …)。日志文件中存储了事务的修改记录,用于保证事务的持久性和原子性。 + + + - 索引文件(index file)是存储索引的辅助文件,它是由一个或多个固定大小的索引页(index page)组成的文件,每个索引页的大小默认为16KB。索引文件可以是单个文件(table_name.MYI)或多个文件(table_name.MYI, table_name.MYI.1, …)。索引文件中存储了表的辅助索引,用于加速数据查询和排序等操作。 + + + - 临时文件(temp file)是存储临时数据的辅助文件,它是由一个或多个固定大小的临时页(temp page)组成的文件,每个临时页的大小默认为16KB。临时文件可以是单个文件(ibtmp1)或多个文件(ibtmp1, ibtmp2, …)。临时文件中存储了查询过程中产生的临时表、排序结果、分组结果等信息。 \ No newline at end of file diff --git a/docs/release/v5-0.en.md b/docs/release/v5-0.en.md new file mode 100644 index 0000000..9487f46 --- /dev/null +++ b/docs/release/v5-0.en.md @@ -0,0 +1,36 @@ +# V5.0-release + +The main changes in V5.0-release focus on the following areas: + +## 1. Distributed Architecture and Performance Optimization + +- Upgraded the transaction mechanism by eliminating the ProcArray performance bottleneck and replacing clog with csnlog, improving performance by more than 50% in high-concurrency TP scenarios. +- Upgraded the distributed execution architecture with a CN Coordination + DOP + Pipeline execution model, reducing redundant processes and blocking across data shards during distributed query execution and improving parallel query execution efficiency. +- Optimized the data forwarding layer through mechanisms such as control flow optimization and data flow reuse, reducing data forwarding overhead in large-scale clusters and improving query performance under HTAP mixed workloads. + +## 2. Enhanced Oracle Compatibility + +- Enhanced Oracle database compatibility by implementing isolation between PG and Oracle modes at the syntax, metadata, and view layers. +- Enhanced compatibility with Oracle syntax, data types, system views, functions, PLSQL, and other features. +- Supported the creation of both PG-mode and Oracle-mode databases within the same cluster while reusing the underlying distributed storage and computing framework, reducing the cost of Oracle application migration and adaptation. + +## 3. Enhanced Resource Management + +- Added SQL firewall capabilities to identify and block abnormal and high-resource-consumption SQL statements. +- Enhanced real-time database resource monitoring and management, with support for managing resources such as concurrency, memory, and CPU. +- Added a circuit-breaking mechanism for high-resource-consumption SQL statements, reducing the impact of abnormal SQL on overall cluster workload and stability. + +## 4. Enhanced AI and Vector Database Capabilities + +- Added distributed adaptation for pgvector, enhancing vector data storage, query, and indexing capabilities in distributed environments. +- Supported parallel creation of vector indexes, improving indexing efficiency for embedding data. +- Supported building RAG knowledge bases based on vector retrieval, enabling unified storage and querying of relational and vector data. +- Added the opentenbase_ai extension, enabling direct invocation of large language model capabilities through SQL. +- Supported large language models including OpenAI, DeepSeek, Tencent Hunyuan, and Alibaba Tongyi Qianwen, as well as integration with other model services through custom interfaces. +- Supported AI capabilities including text generation, text summarization, translation, sentiment analysis, Q&A extraction, embedding generation, and image analysis. + +## 5. Enhanced Multimodal Data Processing Capabilities + +- Enhanced unified processing capabilities for structured, semi-structured, and unstructured data. +- Supported the storage and analysis of text, images, vectors, and relational data within the same database. +- Supported joint analysis of relational and multimodal data through SQL, extending OpenTenBase capabilities for scenarios such as intelligent retrieval, knowledge bases, and multimodal data analytics. \ No newline at end of file diff --git a/docs/release/v5-0.md b/docs/release/v5-0.md new file mode 100644 index 0000000..bb4c72a --- /dev/null +++ b/docs/release/v5-0.md @@ -0,0 +1,36 @@ +# V5.0-release + +V5.0-release 版本主要修改集中在: + +## 1. 分布式架构及性能优化 + +- 事务机制升级,去除 ProcArray 性能瓶颈,使用 csnlog 替代 clog,高并发 TP 场景性能提升 50% 以上。 +- 分布式执行架构升级,采用 CN 协调 + DOP + Pipeline 执行模式,减少分布式查询执行过程中的冗余进程及分片阻塞,提升并行查询执行效率。 +- 数据转发层优化,通过控制流优化、数据流复用等机制,降低大规模集群下的数据转发开销,提升 HTAP 混合负载场景下的查询性能。 + +## 2. Oracle兼容性增强 + +- 增强 Oracle 数据库兼容能力,在语法层、元数据层及视图层实现 PG/Oracle 模式隔离。 +- 增强 Oracle 语法、数据类型、系统视图、函数及 PLSQL 等兼容能力。 +- 支持在同一集群中创建 PG 模式和 Oracle 模式数据库,并复用底层分布式存储及计算框架,降低 Oracle 应用迁移及改造成本。 + +## 3. 资源管控能力增强 + +- 增加 SQL 防火墙能力,支持对异常及高资源消耗 SQL 进行识别和拦截。 +- 增强数据库资源实时监控及管控能力,支持对并发数、内存、CPU 等资源进行管理。 +- 增加高消耗 SQL 熔断机制,降低异常 SQL 对集群整体负载及稳定性的影响。 + +## 4. AI及向量数据库能力增强 + +- 支持 pgvector 分布式适配,增强分布式环境下的向量数据存储、查询及索引能力。 +- 支持向量索引并行创建,提升 embedding 数据的索引构建效率。 +- 支持基于向量检索构建 RAG 知识库,实现关系数据与向量数据的统一存储和查询。 +- 新增 opentenbase_ai 插件,支持直接通过 SQL 调用大语言模型能力。 +- 支持 OpenAI、DeepSeek、腾讯混元、阿里通义千问等大模型,并支持通过自定义接口接入其他模型服务。 +- 支持文本生成、文本摘要、翻译、情感分析、问答提取、Embedding 生成及图像分析等 AI 能力。 + +## 5. 多模态数据处理能力增强 + +- 增强结构化、半结构化及非结构化数据的统一处理能力。 +- 支持文本、图像、向量及关系数据在同一数据库中进行存储和分析。 +- 支持通过 SQL 对关系数据与多模态数据进行关联分析,扩展 OpenTenBase 在智能检索、知识库及多模态数据分析等场景下的应用能力。 \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 8d67ce0..bd67df2 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -113,6 +113,7 @@ nav: - 使用docker构建一个简单集群: guide/15-docker-deploy.md - 版本说明: - OpenTenBase: + - v5.0: release/v5-0.md - v2.6.0: release/v2-6-0.md - v2.5.0: release/v2-5-0.md - v2.4.0: release/v2-4-0.md